Meger certain shapefiles within sub folders with path name in attributes

646
9
Jump to solution
04-22-2021 02:51 PM
2Quiker
Occasional Contributor II

I need to merge certain shapefiles with a specific name in sub directories. I can do the merge but where I am stuck on is including each shapefile specific root path to each shapefile 'Path' field before the merge. I have the following.

rootpath will like below, Shapefile.shp will always be the same
C:\Tem\blah\2020 blah\Case1\Shapefile.shp
C:\Tem\blah\2020 blah\Case2\Shapefile.shp
C:\Tem\blah\2020 blah\Notice\Shapefile.shp
C:\Tem\blah\2020 blah\Use\Shapefile.shp

 

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this
counter = 0
matches = []

for root, dirs, files in os.walk(rootPath):
    for filename in files:
        if filename == "shapefile.shp":
            match = ( os.path.join(root, filename))
            matches.append (match)
            counter = counter + 1

arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

 

0 Kudos
2 Solutions

Accepted Solutions
DanPatterson
MVP Esteemed Contributor

add "dirs" into your loop then join root and dir, then that to filename

sort of like the second code example

Walk—ArcGIS Pro | Documentation


... sort of retired...

View solution in original post

JoshuaBixby
MVP Esteemed Contributor

When working with file system paths in Python 3.4+, I strongly encourage people to switch to using 11.1. pathlib — Object-oriented filesystem paths — Python 3.6.13 documentation.

>>> from pathlib import Path
>>>
>>> shp_path = r"C:\Cases\blah\2020 blah\yoyo\folder1\shapefile.shp"
>>> path = Path(shp_path)
>>> print(path.parents[1])
C:\Cases\blah\2020 blah\yoyo
>>>

View solution in original post

9 Replies
DanPatterson
MVP Esteemed Contributor

add "dirs" into your loop then join root and dir, then that to filename

sort of like the second code example

Walk—ArcGIS Pro | Documentation


... sort of retired...
2Quiker
Occasional Contributor II

I see but I am not sure how put it together,  do I put the "shapefile.shp" in a directory then use with arcpy.da.UpdateCursor to update the "Path" field with the dirnames? I guess I am uncertain on how to put it together.

0 Kudos
2Quiker
Occasional Contributor II

With the following I can get to work with one single shapefile to a specific folder location but not with a sub folder. It also only works if the "Path' field is already there. It won't let me add a field with line 13, ERROR 000622: Failed to execute (Add Field). Parameters are not valid.
ERROR 000628: Cannot set input into parameter field_precision.

 

 

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this
#rootPath = r'C:\Temo\blah\2020 blah\case\yoyo' #works with specific folder location
counter = 0
matches = []

field = 'Path'
for root, dirs, files in os.walk(rootPath):
    for filename in files:
        if filename == "shapefile.shp":
            match = ( os.path.join(root, filename))
            matches.append (match)
            counter = counter + 1

#arcpy.AddField_management(os.path.join(root,match),field, "Path", 'TEXT') #Erros with ERROR 000622: Failed to execute (Add Field). Parameters are not valid.
ERROR 000628: Cannot set input into parameter field_precision.
with arcpy.da.UpdateCursor(os.path.join(root,match),field) as cur:
    for row in cur:
        row[0]= rootPath
        cur.updateRow(row)
arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

 

 

 

0 Kudos
DanPatterson
MVP Esteemed Contributor

Add Field (Data Management)—ArcGIS Pro | Documentation

arcpy.management.AddField(

     in_table,

    field_name,    # field, "Path",   which is it?

    field_type,      #  'TEXT'

    {field_precision}, {field_scale}, {field_length}, {field_alias}, {field_is_nullable}, {field_is_required}, {field_domain}

but putting "field" and "Path" in, you pushed "Text" into  the field precision place


... sort of retired...
0 Kudos
2Quiker
Occasional Contributor II

Oh no, my bad I have been messing with it for a few hours and must have missed it.

 I have the updated code and I don't get an error but it doesn't populate all of the features 'Path' field and on the ones it does populate its not the path to shapefile, it is to another folder.  For example the a good path would be r'C:\Temo\blah\2020 blah\case\yoyo' but some of them are populated with r'C:\Temo\blah\2020 blah\case\Notifications' even though the shapefile is not in that 'Notifications' folder.

 

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this
#rootPath = r'C:\Temo\blah\2020 blah\case\yoyo' #works with specific folder location
counter = 0
matches = []

field = 'Path'
for root, dirs, files in os.walk(rootPath):
    for filename in files:
        if filename == "shapefile.shp":
            match = ( os.path.join(root, filename))
            matches.append (match)
            counter = counter + 1

arcpy.AddField_management(os.path.join(root,match),field, 'TEXT') 
with arcpy.da.UpdateCursor(os.path.join(root,match),field) as cur:
    for row in cur:
        row[0]= os.path.join(root, filename)
        cur.updateRow(row)
arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

 

0 Kudos
DanPatterson
MVP Esteemed Contributor
print("you got = {}".format(filename))
if filename == "shapefile.shp":
    match = ( os.path.join(root, filename))
    matches.append (match)

Are you reluctant to thrown in print statements to see what you got?

and this line 

arcpy.AddField_management(os.path.join(root,match),field, 'TEXT')

will just use your last match and not cycle through your list of matches.

 


... sort of retired...
0 Kudos
2Quiker
Occasional Contributor II

I was some what able to get what I need with the following.

How do I slice the path to stop at the second to last folder ? for example,

C:\Cases\blah\2020 blah\yoyo\folder1 --> with the current code I get all of this.

C:\Cases\blah\2020 blah\ yoyo --> I needs this populated in the path field

 

rootPath = r'C:\Cases\2020 blah'

shps = []
field = 'Path'
for dirpath, dirnames, files in os.walk(rootPath):
    for x in files:
        if x == "Shapefile.shp":
            shps.append(os.path.join(dirpath, x))

for fc in shps:
    arcpy.AddField_management(fc, field, 'TEXT')
    dirname = os.path.dirname(fc) #just the name of the folder dirname = os.path.basename(os.path.dirname(fc))
    with arcpy.da.UpdateCursor(fc,field) as cur:
        for row in cur:
            row[0]= dirname
            cur.updateRow(row)
        
arcpy.Merge_management(shps, r"C:\Temp\MergedTest.shp")

 

 

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

When working with file system paths in Python 3.4+, I strongly encourage people to switch to using 11.1. pathlib — Object-oriented filesystem paths — Python 3.6.13 documentation.

>>> from pathlib import Path
>>>
>>> shp_path = r"C:\Cases\blah\2020 blah\yoyo\folder1\shapefile.shp"
>>> path = Path(shp_path)
>>> print(path.parents[1])
C:\Cases\blah\2020 blah\yoyo
>>>
2Quiker
Occasional Contributor II

Thanks for the suggestion.

0 Kudos