topic Re: Meger certain shapefiles within sub folders with path name in attributes in Python Questions

Meger certain shapefiles within sub folders with path name in attributes

2Quiker — Thu, 22 Apr 2021 21:51:06 GMT

I need to merge certain shapefiles with a specific name in sub directories. I can do the merge but where I am stuck on is including each shapefile specific root path to each shapefile 'Path' field before the merge. I have the following.

rootpath will like below, Shapefile.shp will always be the same
C:\Tem\blah\2020 blah\Case1\Shapefile.shp
C:\Tem\blah\2020 blah\Case2\Shapefile.shp
C:\Tem\blah\2020 blah\Notice\Shapefile.shp
C:\Tem\blah\2020 blah\Use\Shapefile.shp

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this counter = 0 matches = [] for root, dirs, files in os.walk(rootPath): for filename in files: if filename == "shapefile.shp": match = ( os.path.join(root, filename)) matches.append (match) counter = counter + 1 arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

Re: Meger certain shapefiles within sub folders with path name in attributes

DanPatterson — Thu, 22 Apr 2021 22:28:59 GMT

add "dirs" into your loop then join root and dir, then that to filename

sort of like the second code example

Walk—ArcGIS Pro | Documentation

Re: Meger certain shapefiles within sub folders with path name in attributes

2Quiker — Fri, 23 Apr 2021 14:51:25 GMT

I see but I am not sure how put it together, do I put the "shapefile.shp" in a directory then use with arcpy.da.UpdateCursor to update the "Path" field with the dirnames? I guess I am uncertain on how to put it together.

Re: Meger certain shapefiles within sub folders with path name in attributes

2Quiker — Mon, 26 Apr 2021 21:35:08 GMT

With the following I can get to work with one single shapefile to a specific folder location but not with a sub folder. It also only works if the "Path' field is already there. It won't let me add a field with line 13, ERROR 000622: Failed to execute (Add Field). Parameters are not valid.
ERROR 000628: Cannot set input into parameter field_precision.

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this #rootPath = r'C:\Temo\blah\2020 blah\case\yoyo' #works with specific folder location counter = 0 matches = [] field = 'Path' for root, dirs, files in os.walk(rootPath): for filename in files: if filename == "shapefile.shp": match = ( os.path.join(root, filename)) matches.append (match) counter = counter + 1 #arcpy.AddField_management(os.path.join(root,match),field, "Path", 'TEXT') #Erros with ERROR 000622: Failed to execute (Add Field). Parameters are not valid. ERROR 000628: Cannot set input into parameter field_precision. with arcpy.da.UpdateCursor(os.path.join(root,match),field) as cur: for row in cur: row[0]= rootPath cur.updateRow(row) arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

Re: Meger certain shapefiles within sub folders with path name in attributes

DanPatterson — Mon, 26 Apr 2021 22:46:37 GMT

Add Field (Data Management)—ArcGIS Pro | Documentation

arcpy.management.AddField(

in_table,

field_name, # field, "Path", which is it?

field_type, # 'TEXT'

{field_precision}, {field_scale}, {field_length}, {field_alias}, {field_is_nullable}, {field_is_required}, {field_domain}

but putting "field" and "Path" in, you pushed "Text" into the field precision place

Re: Meger certain shapefiles within sub folders with path name in attributes

2Quiker — Tue, 27 Apr 2021 15:10:24 GMT

Oh no, my bad I have been messing with it for a few hours and must have missed it.

I have the updated code and I don't get an error but it doesn't populate all of the features 'Path' field and on the ones it does populate its not the path to shapefile, it is to another folder. For example the a good path would be r'C:\Temo\blah\2020 blah\case\yoyo' but some of them are populated with r'C:\Temo\blah\2020 blah\case\Notifications' even though the shapefile is not in that 'Notifications' folder.

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this #rootPath = r'C:\Temo\blah\2020 blah\case\yoyo' #works with specific folder location counter = 0 matches = [] field = 'Path' for root, dirs, files in os.walk(rootPath): for filename in files: if filename == "shapefile.shp": match = ( os.path.join(root, filename)) matches.append (match) counter = counter + 1 arcpy.AddField_management(os.path.join(root,match),field, 'TEXT') with arcpy.da.UpdateCursor(os.path.join(root,match),field) as cur: for row in cur: row[0]= os.path.join(root, filename) cur.updateRow(row) arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

Re: Meger certain shapefiles within sub folders with path name in attributes

DanPatterson — Tue, 27 Apr 2021 16:22:35 GMT

print("you got = {}".format(filename)) if filename == "shapefile.shp": match = ( os.path.join(root, filename)) matches.append (match)

Are you reluctant to thrown in print statements to see what you got?

and this line

arcpy.AddField_management(os.path.join(root,match),field, 'TEXT')

will just use your last match and not cycle through your list of matches.

Re: Meger certain shapefiles within sub folders with path name in attributes

2Quiker — Wed, 28 Apr 2021 17:31:40 GMT

I was some what able to get what I need with the following.

How do I slice the path to stop at the second to last folder ? for example,

C:\Cases\blah\2020 blah\yoyo\folder1 --> with the current code I get all of this.

C:\Cases\blah\2020 blah\ yoyo --> I needs this populated in the path field

rootPath = r'C:\Cases\2020 blah' shps = [] field = 'Path' for dirpath, dirnames, files in os.walk(rootPath): for x in files: if x == "Shapefile.shp": shps.append(os.path.join(dirpath, x)) for fc in shps: arcpy.AddField_management(fc, field, 'TEXT') dirname = os.path.dirname(fc) #just the name of the folder dirname = os.path.basename(os.path.dirname(fc)) with arcpy.da.UpdateCursor(fc,field) as cur: for row in cur: row[0]= dirname cur.updateRow(row) arcpy.Merge_management(shps, r"C:\Temp\MergedTest.shp")

Re: Meger certain shapefiles within sub folders with path name in attributes

JoshuaBixby — Wed, 28 Apr 2021 17:50:38 GMT

When working with file system paths in Python 3.4+, I strongly encourage people to switch to using 11.1. pathlib — Object-oriented filesystem paths — Python 3.6.13 documentation.

>>> from pathlib import Path >>> >>> shp_path = r"C:\Cases\blah\2020 blah\yoyo\folder1\shapefile.shp" >>> path = Path(shp_path) >>> print(path.parents[1]) C:\Cases\blah\2020 blah\yoyo >>>

Re: Meger certain shapefiles within sub folders with path name in attributes

2Quiker — Wed, 28 Apr 2021 21:32:31 GMT

Thanks for the suggestion.