Help: looping funtion to creat a list for the files from different folders

[ZiaAhmed]

I am very new in Python.  I am trying to import several featue classess into file geodatabse (name: IA.gdb)  using a script. These files are in located in diffrent folders. For example: 
K:\SUURGO_Data\IA\soil_ia001\soil_ia001\spatial\soilmu_a_ia001.shp
K:\SUURGO_Data\IA\soil_ia001\soil_ia003\spatial\soilmu_a_ia003.shp
K:\SUURGO_Data\IA\soil_ia001\soil_ia005\spatial\soilmu_a_ia005.shp
K:\SUURGO_Data\IA\soil_ia001\soil_ia053\spatial\soilmu_a_ia053.shp
K:\SUURGO_Data\IA\soil_ia001\soil_ia193\spatial\soilmu_a_ia193.shp
.... so on

I used model bulider to do this for a  few number of files. But I want to do this with a loop to select (list)  all of these files  and import to a filegeodata base. I do not know how to write a loop function get the list of files located in different folders and subfolders.   Help will be highly appreciated.  Below find a python script - saved from Model Bulder.

Thanks
Zia

--------------------------------------------------------------------------
# ImprtFetureGeodatabse.py
# Created on: 2011-08-11 17:44:46.00000
#   (generated by ArcGIS/ModelBuilder)
# Description:
# ---------------------------------------------------------------------------

# Import arcpy module
import arcpy


# Local variables:
soilmu_a_ia001_shp = "K:\\SUURGO_Data\\IA\\soil_ia001\\soil_ia001\\spatial\\soilmu_a_ia001.shp"
soilmu_a_ia003_shp = "K:\\SUURGO_Data\\IA\\soil_ia003\\soil_ia003\\spatial\\soilmu_a_ia003.shp"
soilmu_a_ia005_shp = "K:\\SUURGO_Data\\IA\\soil_ia005\\soil_ia005\\spatial\\soilmu_a_ia005.shp"
soilmu_a_ia197_shp = "K:\\SUURGO_Data\\IA\\soil_ia197\\soil_ia197\\spatial\\soilmu_a_ia197.shp"
soilmu_a_ia053_shp = "K:\\SUURGO_Data\\IA\\soil_ia053\\soil_ia053\\spatial\\soilmu_a_ia053.shp"
IA_gdb__2_ = "K:\\COUNTY_MUKEY\\IA.gdb"

# Process: Feature Class to Geodatabase (multiple)
arcpy.FeatureClassToGeodatabase_conversion("K:\\SUURGO_Data\\IA\\soil_ia001\\soil_ia001\\spatial\\soilmu_a_ia001.shp;K:\\SUURGO_Data\\IA\\soil_ia003\\soil_ia003\\spatial\\soilmu_a_ia003.shp;K:\\SUURGO_Data\\IA\\soil_ia005\\soil_ia005\\spatial\\soilmu_a_ia005.shp;K:\\SUURGO_Data\\IA\\soil_ia197\\soil_ia197\\spatial\\soilmu_a_ia197.shp;K:\\SUURGO_Data\\IA\\soil_ia053\\soil_ia053\\spatial\\soilmu_a_ia053.shp", IA_gdb__2_)
###############################################################

[MathewCoyle]

Is there some kind of processing you need to do that requires you to loop through a list? You can simply create a list of shapefiles and bulk load them as you have done.

You could try just making it a list to be more readable

shape_list = ["K:\\SUURGO_Data\\IA\\soil_ia001\\soil_ia001\\spatial\\soilmu_a_ia001.shp",
              "K:\\SUURGO_Data\\IA\\soil_ia003\\soil_ia003\\spatial\\soilmu_a_ia003.shp",
              "K:\\SUURGO_Data\\IA\\soil_ia005\\soil_ia005\\spatial\\soilmu_a_ia005.shp",
              "K:\\SUURGO_Data\\IA\\soil_ia197\\soil_ia197\\spatial\\soilmu_a_ia197.shp",
              "K:\\SUURGO_Data\\IA\\soil_ia053\\soil_ia053\\spatial\\soilmu_a_ia053.shp"]
arcpy.FeatureClassToGeodatabase_conversion(shape_list, IA_gdb__2_)

[ZiaAhmed]

Hi mzcole:
Thanks for your reply. For example, i just mentioned for exporting  5 shape files in a geodatabse. I have to export 100s of files to a geodatabase. I want to write a loop that can allow me to select  soilmu_iacountyCode.shp from respective subdirectories (soil_iacountyCode\\soil_iacountyCode\\spatial) of K:\\SUURGO_Data\\IA directory.

countyCode = 001, 003, 015......so on. For example, my directory structure like this:

K:\SUURGO_Data\IA\soil_ia003\soil_ia003\spatial\soilmu_a_ia003.shp
...........................\soil_ia005\soil_ia005\spatial\soilmu_a_ia005.shp
........................... \soil_ia022\soil_ia022\spatial\soilmu_a_ia022.shp
.
.
............................ \soil_ia160\soil_ia160\spatial\soilmu_a_ia160.shp

Thanks
Zia

Is there some kind of processing you need to do that requires you to loop through a list? You can simply create a list of shapefiles and bulk load them as you have done.

You could try just making it a list to be more readable

shape_list = ["K:\\SUURGO_Data\\IA\\soil_ia001\\soil_ia001\\spatial\\soilmu_a_ia001.shp",
              "K:\\SUURGO_Data\\IA\\soil_ia003\\soil_ia003\\spatial\\soilmu_a_ia003.shp",
              "K:\\SUURGO_Data\\IA\\soil_ia005\\soil_ia005\\spatial\\soilmu_a_ia005.shp",
              "K:\\SUURGO_Data\\IA\\soil_ia197\\soil_ia197\\spatial\\soilmu_a_ia197.shp",
              "K:\\SUURGO_Data\\IA\\soil_ia053\\soil_ia053\\spatial\\soilmu_a_ia053.shp"]
arcpy.FeatureClassToGeodatabase_conversion(shape_list, IA_gdb__2_)

[RDHarles]

This will find EVERY shapefile in EVERY directory in the path you choose to start from (c:/shapefiles in the example below).
Then you can do whatever you want with the shapefile.

import os

for root, dirs, files in os.walk("c:/shapefiles"):
    for file in files:        
        if file.endswith(".shp"):            
            path = os.path.abspath(os.path.join(root, file))
            print path

[ZiaAhmed]

Thanks! If I want to select one particular type of shape file  ending with _a_ia001.shp  from a group of shape files from K:\SUURGO_Data\IA\  directory .. is following code is ok for do this? 


for root, dirs, files in os.walk("K/SUURGO_Data/IA/:shapefiles"):
    for file in files:       
        if file.endswith("_a_ia001.shp"):           
            path = os.path.abspath(os.path.join(root, file))
            print path
 

This will find EVERY shapefile in EVERY directory in the path you choose to start from (c:/shapefiles in the example below).
Then you can do whatever you want with the shapefile.

import os

for root, dirs, files in os.walk("c:/shapefiles"):
    for file in files:        
        if file.endswith(".shp"):            
            path = os.path.abspath(os.path.join(root, file))
            print path

[RDHarles]

Sure, that will work.
You can combine things as well.

# Examples:

if file.startswith("rd-") and file.endswith(".dbf"):

[ZiaAhmed]

I am using following code after adding a loop function for creating a list of  of al feature classes. But got some error. I think i miss somthing.. zia

ArcGIs 10 Help
#####################
#Code Sample
#ListFeatureClasses example
#Copy shapefiles to a geodatabase.

import arcpy
from arcpy import env
import os

# Set the workspace for the ListFeatureClass function
#
env.workspace = "c:/base"

# Use the ListFeatureClasses function to return a list of
#  all shapefiles.
#
fcList = arcpy.ListFeatureClasses()

# Copy shapefiles to a file geodatabase
#
for fc in fcList:
    arcpy.CopyFeatures_management(fc, "d:/base/output.gdb" + os.sep + fc.rstrip(".shp"))
#--------------------------------------------------------------------------------------

# This is my script adding a loop funtion
############################
import arcpy
from arcpy import env
import os


# Set the workspace for the ListFeatureClass function

env.workspace= "K:\SUURGO_Data\IA"

# To create list files from directories and subdirectories

for root, dirs, files in os.walk(env.workspace):
     for name in files:
       if file.startswith("soilmu_a_") and file.endswith(".shp"):
              path = os.path.abspath(os.path.join(root, file))     
              print path

fcList = arcpy.ListFeatureClasses()


# Process: Feature Class to Geodatabase (multiple)

for fc in fcList:
arcpy.CopyFeatures_management(fc, "K:/COUNTY_MUKEY/IA.gdb"" + os.sep + fc.rstrip(".shp"))

[RDHarles]

I changed quite a bit of the code, this should work now.

import arcpy, os

# Set the workspace
arcpy.env.workspace= "K:/SUURGO_Data/IA"

# To create list files from directories and subdirectories
for root, dirs, files in os.walk(arcpy.env.workspace):
    for name in files:        
        if name.startswith("soilmu_a_") and name.endswith(".shp"):
            path = os.path.abspath(os.path.join(root, name))
            # Remove the path from the shapefile name
            base = os.path.basename(path)
            print "Processing "+base
            # CopyFeatures
            arcpy.CopyFeatures_management(base, "IA.gdb/"+base.rstrip(".shp"))            
            print arcpy.GetMessages()

[ZiaAhmed]

When run this script in ArcTool box, it did not work-  it showed "Failed to execute (CopyFeatures). Then I tried in Python2.6Python GUI it showd following error:

Processing soilmu_a_ia001.shp

Traceback (most recent call last):
  File "K:\Script\Python_Script\ImprtFetureGeodatabase_1.py", line 22, in <module>
    arcpy.CopyFeatures_management(base, "IA.gdb/"+base.rstrip(".shp"))
  File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\management.py", line 1943, in CopyFeatures
    raise e
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000732: Input Features: Dataset soilmu_a_ia001.shp does not exist or is not supported
Failed to execute (CopyFeatures).

I am very new in python scripting in ArcGIS, I am not able to figure out what wrong here. Thanks again

I changed quite a bit of the code, this should work now.

import arcpy, os

# Set the workspace
arcpy.env.workspace= "K:/SUURGO_Data/IA"

# To create list files from directories and subdirectories
for root, dirs, files in os.walk(arcpy.env.workspace):
    for name in files:        
        if name.startswith("soilmu_a_") and name.endswith(".shp"):
            path = os.path.abspath(os.path.join(root, name))
            # Remove the path from the shapefile name
            base = os.path.basename(path)
            print "Processing "+base
            # CopyFeatures
            arcpy.CopyFeatures_management(base, "IA.gdb/"+base.rstrip(".shp"))            
            print arcpy.GetMessages()

[RDHarles]

My mistake, my environment is set up differently from yours.
Change the word 'base' to 'path' like this:

arcpy.CopyFeatures_management(path, "IA.gdb/"+base.rstrip(".shp"))