Meger certain shapefiles within sub folders with path name in attributes

[2Quiker]

I need to merge certain shapefiles with a specific name in sub directories. I can do the merge but where I am stuck on is including each shapefile specific root path to each shapefile 'Path' field before the merge. I have the following.

rootpath will like below, Shapefile.shp will always be the same
C:\Tem\blah\2020 blah\Case1\Shapefile.shp
C:\Tem\blah\2020 blah\Case2\Shapefile.shp
C:\Tem\blah\2020 blah\Notice\Shapefile.shp
C:\Tem\blah\2020 blah\Use\Shapefile.shp

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this
counter = 0
matches = []

for root, dirs, files in os.walk(rootPath):
    for filename in files:
        if filename == "shapefile.shp":
            match = ( os.path.join(root, filename))
            matches.append (match)
            counter = counter + 1

arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

[DanPatterson · Accepted solution]

add "dirs" into your loop then join root and dir, then that to filename

sort of like the second code example

Walk—ArcGIS Pro | Documentation

[2Quiker]

I see but I am not sure how put it together,  do I put the "shapefile.shp" in a directory then use with arcpy.da.UpdateCursor to update the "Path" field with the dirnames? I guess I am uncertain on how to put it together.

[2Quiker]

With the following I can get to work with one single shapefile to a specific folder location but not with a sub folder. It also only works if the "Path' field is already there. It won't let me add a field with line 13, ERROR 000622: Failed to execute (Add Field). Parameters are not valid.
ERROR 000628: Cannot set input into parameter field_precision.

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this
#rootPath = r'C:\Temo\blah\2020 blah\case\yoyo' #works with specific folder location
counter = 0
matches = []

field = 'Path'
for root, dirs, files in os.walk(rootPath):
    for filename in files:
        if filename == "shapefile.shp":
            match = ( os.path.join(root, filename))
            matches.append (match)
            counter = counter + 1

#arcpy.AddField_management(os.path.join(root,match),field, "Path", 'TEXT') #Erros with ERROR 000622: Failed to execute (Add Field). Parameters are not valid.
ERROR 000628: Cannot set input into parameter field_precision.
with arcpy.da.UpdateCursor(os.path.join(root,match),field) as cur:
    for row in cur:
        row[0]= rootPath
        cur.updateRow(row)
arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

 

[DanPatterson]

Add Field (Data Management)—ArcGIS Pro | Documentation

arcpy.management.AddField(

     in_table,

    field_name,    # field, "Path",   which is it?

    field_type,      #  'TEXT'

    {field_precision}, {field_scale}, {field_length}, {field_alias}, {field_is_nullable}, {field_is_required}, {field_domain}

but putting "field" and "Path" in, you pushed "Text" into  the field precision place

[2Quiker]

Oh no, my bad I have been messing with it for a few hours and must have missed it.

 I have the updated code and I don't get an error but it doesn't populate all of the features 'Path' field and on the ones it does populate its not the path to shapefile, it is to another folder.  For example the a good path would be r'C:\Temo\blah\2020 blah\case\yoyo' but some of them are populated with r'C:\Temo\blah\2020 blah\case\Notifications' even though the shapefile is not in that 'Notifications' folder.

rootPath = r'C:\Temo\blah\2020 blah' # different folders aftre this
#rootPath = r'C:\Temo\blah\2020 blah\case\yoyo' #works with specific folder location
counter = 0
matches = []

field = 'Path'
for root, dirs, files in os.walk(rootPath):
    for filename in files:
        if filename == "shapefile.shp":
            match = ( os.path.join(root, filename))
            matches.append (match)
            counter = counter + 1

arcpy.AddField_management(os.path.join(root,match),field, 'TEXT') 
with arcpy.da.UpdateCursor(os.path.join(root,match),field) as cur:
    for row in cur:
        row[0]= os.path.join(root, filename)
        cur.updateRow(row)
arcpy.Merge_management(matches, r"C:\Temp\MergedTest.shp")

[DanPatterson]

print("you got = {}".format(filename))
if filename == "shapefile.shp":
    match = ( os.path.join(root, filename))
    matches.append (match)

Are you reluctant to thrown in print statements to see what you got?

and this line 

arcpy.AddField_management(os.path.join(root,match),field, 'TEXT')

will just use your last match and not cycle through your list of matches.

[2Quiker]

I was some what able to get what I need with the following.

How do I slice the path to stop at the second to last folder ? for example,

C:\Cases\blah\2020 blah\yoyo\folder1 --> with the current code I get all of this.

C:\Cases\blah\2020 blah\ yoyo --> I needs this populated in the path field

rootPath = r'C:\Cases\2020 blah'

shps = []
field = 'Path'
for dirpath, dirnames, files in os.walk(rootPath):
    for x in files:
        if x == "Shapefile.shp":
            shps.append(os.path.join(dirpath, x))

for fc in shps:
    arcpy.AddField_management(fc, field, 'TEXT')
    dirname = os.path.dirname(fc) #just the name of the folder dirname = os.path.basename(os.path.dirname(fc))
    with arcpy.da.UpdateCursor(fc,field) as cur:
        for row in cur:
            row[0]= dirname
            cur.updateRow(row)
        
arcpy.Merge_management(shps, r"C:\Temp\MergedTest.shp")

[JoshuaBixby · Accepted solution]

When working with file system paths in Python 3.4+, I strongly encourage people to switch to using 11.1. pathlib — Object-oriented filesystem paths — Python 3.6.13 documentation.

>>> from pathlib import Path
>>>
>>> shp_path = r"C:\Cases\blah\2020 blah\yoyo\folder1\shapefile.shp"
>>> path = Path(shp_path)
>>> print(path.parents[1])
C:\Cases\blah\2020 blah\yoyo
>>>

[2Quiker]

Thanks for the suggestion.