Looping through sub folders in a python script

HugoMercier · ‎03-04-2011

Hi everyone,

I'm using a script to define the projection of unprojected shapefiles. The shapefiles are grouped in different directories and the script works well except with two folders. The shapefiles in those folders are in sub-folders and the script skips them.

Now I would like to find a way to make the script loop through those sub folders.

I should mention that I'm not that experienced with scripting and most of what I have was taken from other scripts and adapted to what I wanted to accomplish. If you have any comments that could help me make this script better, let me know.

Thank you in advance for your help.

Here's the script:

# Import system modules
import sys, string, os, arcgisscripting

# Create the Geoprocessor object
gp = arcgisscripting.create()

# Load required toolboxes...
gp.AddToolbox("C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/Data Management Tools.tbx")

# Local variables...
dirname = "Z:\\01_Data\\04_Vector\\04_World\\DAFIF\\AVDAFIF"
datum = "GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]]"
folder = os.listdir(dirname)

X = 0

# Process: Define Projection...
for i in folder:
    print "working in the", folder, "folder"
    gp.workspace = "Z:\\01_Data\\04_Vector\\04_World\\DAFIF\\AVDAFIF\\" + str(folder)
    shape = gp.ListFeatureClasses()
    shapefileName = shape.next()

    while shapefileName:
        workShape = gp.workspace + "\\" + shapefileName
        print "defining projection for " + shapefileName + "..."
        gp.DefineProjection_management(workShape, datum)
        shapefileName = shape.next()
    else:
        print "Next Folder"

    X = X + 1

print "Done"

RuthEmerick · ‎03-04-2011

You can use something like os.path.isdir(path) to check if the file in the for loop definition is a directory, and if it is, then look inside that folder for shapefiles; repeat.

NiklasNorrthon · ‎03-04-2011

Untested, might need some debugging:

import os

def define_projection_recursive(root, spatial_ref):
    for path, subdirs, files in os.walk(root):
        gp.Workspace = path
        fc_list = gp.ListFeatureclasses()
        for fc in fc_list:
            gp.DefineProjection_management(fc, spatial_ref)

HugoMercier · ‎03-11-2011

Untested, might need some debugging:

import os

def define_projection_recursive(root, spatial_ref):
    for path, subdirs, files in os.walk(root):
        gp.Workspace = path
        fc_list = gp.ListFeatureclasses()
        for fc in fc_list:
            gp.DefineProjection_management(fc, spatial_ref)

Tried this but I got the following error, not too sure what to do to fix this:

Traceback (most recent call last):
File "D:\Hugo\Scripting\Define Projection Script\Define_Projection_script_AVDAFIF_Test.py", line 28, in <module>
DefineProjectionRecursive(root, datum);
File "D:\Hugo\Scripting\Define Projection Script\Define_Projection_script_AVDAFIF_Test.py", line 24, in DefineProjectionRecursive
for fc in fc_list:
TypeError: 'geoprocessing list object' object is not iterable

Thanks anyways 😉

ChrisSnyder · ‎03-11-2011

The os.walk command is the way to loop through subfolders (and files) from a top level directory. Example #1 is how to deal with folders, and example #2 is how to deal with the files within the folders.

Example #1: This code will attempt to compress any FGDB (a .gdb folder) under the rootDir

import os, fnmatch, arcgisscripting
gp = arcgisscripting.create(9.3)
rootDir = r"C:\temp"
gdbList = []
for dirPath, dirNames, fileNames in os.walk(rootDir, topdown=False):
    if fnmatch.fnmatch(dirPath, "*.gdb") == True:
        gdbList.append(dirPath)
gdbCount = len(gdbList)
if gdbCount == 0:
    print "No matching FGDBs found!"
else:
    print "Attempting to compress " + str(gdbCount) + " geodatabases..."
    for gdb in gdbList:
        try:
            gp.CompressFileGeodatabaseData_management(gdb)
            print "Compressed " + gdb + "..."
        except:
            print "ERROR: Failed to compress " + gdb + "!"

Example #2: This code will attempt to compress any PGDB (an .mdb file) under the rootDir

import os, fnmatch, arcgisscripting
gp = arcgisscripting.create(9.3)
rootDir = r"C:\temp"
gdbList = []
for dirPath, dirNames, fileNames in os.walk(rootDir, topdown=False):
    for file in fileNames:
        if fnmatch.fnmatch(file, "*.mdb") == True:
            gdbList.append(dirPath + "\\" + file)
gdbCount = len(gdbList)
if gdbCount == 0:
    print "No matching PGDBs found!"
else:
    print "Attempting to compact " + str(gdbCount) + " geodatabases..."
    for gdb in gdbList:
        try:
            gp.Compact_management(gdb)
            print "Compacted " + gdb + "..."
        except:
            print "ERROR: Failed to compact " + gdb + "!"

HugoMercier · ‎03-11-2011

Tried this but I got the following error, not too sure what to do to fix this:

Traceback (most recent call last):
File "D:\Hugo\Scripting\Define Projection Script\Define_Projection_script_AVDAFIF_Test.py", line 28, in <module>
DefineProjectionRecursive(root, datum);
File "D:\Hugo\Scripting\Define Projection Script\Define_Projection_script_AVDAFIF_Test.py", line 24, in DefineProjectionRecursive
for fc in fc_list:
TypeError: 'geoprocessing list object' object is not iterable

Thanks anyways 😉

Ok I got it to work (please don't ask me how), I was just trying something and it worked. If someone can explain to me how please do so.

Here's the final script:

# Import system modules
import sys, string, os, arcgisscripting

# Create the Geoprocessor object
gp = arcgisscripting.create()

# Load required toolboxes...
gp.AddToolbox("C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/Data Management Tools.tbx")

# Local variables...
root = "Z:\\01_Data\\04_Vector\\04_World\\DAFIF\\AVDAFIF"
datum = "GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]]"

def DefineProjectionRecursive(root, datum):
    for path, subdirs, files in os.walk(root):
        gp.workspace = path
        print "working in " + path
        shapefile = gp.ListFeatureClasses()
        shapefileName = shapefile.next()

        while shapefileName:
            gp.DefineProjection_Management(shapefileName, datum)
            print "Defining projection for " + str(shapefileName)
            shapefileName = shapefile.next()
        else:
            print "No more shapefiles, next folder..."

DefineProjectionRecursive(root, datum);

print "Done"

Thank you all for your help!

MarkWiygul · ‎03-11-2011

Hey, glad you got everything operational!

I'm studying up on Python. You've got the actual 'Define Projection' statement commented out, and so as far as I can see the script below is not using the geoprocessing tool at all. You must have already defined the projections beforehand ...or maybe running the tool without the "#" comment.

>>   #gp.DefineProjection_Management(shapefileName, datum)   <<

Otherwise you must have debugged the os.walk method to work in your script correctly (and it recursed all the subdirectories) and processed the data without the "#" on the geoprocessing line.

I hope someone posts something else if that's not the case.

HugoMercier · ‎03-11-2011

Yeah I commented out the define projection statement to test the sript. I forgot to take it out before I posted it. I'll fix it right away.