Help required - Script for merging shapefiles

AlisonBowyer · ‎06-25-2010

Does anyone have Python code lines that i could use to merge shapefiles within subdirectories? Ideally I only want to merge shapefiles with the same names (i.e. merge all shapefile1 from folder1, folder2, folder3 into a new shapefile, all shapefile2 from folder1, folder2 etc) I have point, polygon and line features that I need to merge.

I did find some code at http://forums.esri.com/thread.asp?t=231065&f=1729&c=93 but this merges all polygon or point features within the directories, and doesn't include line features.

Any help people can give me will be much appreciated as I don't understand the coding!

RDHarles · ‎12-17-2010

thanks Kristya

Saldy iam now getting an error on Line 4:

'Parsing error <type 'exceptions.SyntaxError'>: invalid syntax (line 4)'

i take this to be ' gp.workspace = ("C:\LINZ_Topo_Data\South_island\")' where all my subfolder are stored...

where you get the same problem?

You have the slashes the wrong way...
Try it like this:
gp.workspace = "C:/LINZ_Topo_Data/South_island"

LoganPugh · ‎12-17-2010

In Python, backslashes either have to be escaped (by another backslash), or put an r in front of the string to mark it as a raw string. I use raw strings since on Windows you can just copy and paste paths instead of having to tweak them.

Escaping backslashes:

gp.workspace = "C:\\LINZ_Topo_Data\\South_island\\"

Raw string:

gp.workspace = r"C:\LINZ_Topo_Data\South_island\"

EricWeber · ‎12-21-2010

Eric, I don't see an option to "execute code from the clipboard" when I right click in the interactive window. See attached .jpg...

What version of PythonWin are you using? I have v2.5 (build 210).

P.S. Yes I realize this topic was hijacked... Sorry.

Chris, I should have specified...I'm using build 212. It shows up right between "copy" and "paste."

ChrisSnyder · ‎12-21-2010

Yes - Eric I verified this is a new feature in build 212 (PythonWin for v2.6), that wasn't present in build 210.

Hooray - I have been wanting this functionality in PythonWin for years!!!

KristinaGrace · ‎01-26-2011

Hi, Everyone,

I am new to Python. I've copied the code you have posted here and modified it:

1. set the product to ArcEditor
2. commented out the code for polygon shp merging (I only need to merge points)
3.
I saved it in the C:\Python26\ArcGIS10.0\Scripts directory as MergeSHP.py

I am getting the following error when run the module:

Traceback (most recent call last):
File "C:\Python26\ArcGIS10.0\Scripts\MergeSHP.py", line 62, in <module>
pointmergedlist = pointshpList(pointPath)
NameError: name 'pointPath' is not defined

Apparently the line

def pointshpList(pointPath):

doesn't work...

Any help? Thank you

# This script will merge all point shapefiles in subfolders
# of the workspace specified below

# Import modules and create the geoprocessor object

try:
    # 9.2 and beyond
    import arcgisscripting, collections, arcpy, sys, os
    gp = arcgisscripting.create()
except:
    # 9.1 and before
    import win32com.client, collections, arcpy, sys, os
    gp = win32com.client.Dispatch("esriGeoprocessing.GpDispatch.1")

# Set version
gp.SetProduct("ArcEditor")

# Set the workspace
gp.workspace = ("W:\GPS\Trimble_Jobs\Trimble_Geomatics_Office\Projects")

# Start a blank list for Polygon files
# polyList = []

# Start a blank list for Point files
pointList = []

# For each subdirectory
for dir in os.listdir(gp.workspace):
    if os.path.isdir(gp.workspace+"/"+dir):
        # Get a list of the files in each directory
        files = os.listdir(gp.workspace+"/"+dir)
        # For each file in a given directory
        for file in files:
            # Get only files that end with .shp
            if (file.endswith(".shp")):
                print file
                # Describe feature class
                desc = gp.Describe(gp.workspace+"/"+dir+"/"+file)
                type = desc.ShapeType
                #print dir+"/"+file + " type is: " + type
                #if type == "Polygon":
                 #   print dir+"/"+file + " is Polygon"
                  # polyPath = polyList.append(dir+"/"+file)
                if type == "Point":
                    print dir+"/"+file + " is Point"
                    pointPath = pointList.append(dir+"/"+file)


            # Hard-code the output merged shapefile names
            # polyshapefile = "Mergedpoly.shp"
            pointshapefile = "Mergedpoint.shp"

            # Given a list of shapefiles, separate each by a ";"
            # and put quotes around the whole thing
            # def polyshpList(polyPath):
              # return '"%s"' % ';'.join(polyList)
            def pointshpList(pointPath):
                return '"%s"' % ';'.join(pointList)

# Set the variable to the newly formatted list of shapefiles
# polymergedlist = polyshpList(polyPath)
pointmergedlist = pointshpList(pointPath)

# Polygons
# try:
# print "\nMerging " + polymergedlist + " to get " + polyshapefile + "...\n"
   # gp.merge_management(polymergedlist, polyshapefile)
    #print gp.getMessages()
#except:
#   print gp.getMessages()
# print "\n *** ERROR: Shapefiles (Polygon) failed to merge *** \n"

# Points
try:
    print "\nMerging " + pointmergedlist + " to get " + pointshapefile + "...\n"
    gp.merge_management(pointmergedlist, pointshapefile)
    print gp.getMessages()
except:
    print gp.getMessages()
    print "\n *** ERROR: Shapefiles (Point) failed to merge *** \n"

print "\nDone."

RDHarles · ‎01-26-2011

The variable "pointPath" is not getting set here:

if type == "Point":
    print dir+"/"+file + " is Point"
    pointPath = pointList.append(dir+"/"+file)

That could be because:
1.) gp.workspace needs forward slashes (or another correct form) - you can't use backslashes.
2.) you need a full path to the file instead of dir+"/"+file (depending on where your running the script from).
3.) the shapefile is not "Point".

KristinaGrace · ‎01-26-2011

I thought so, too!

1.) gp.workspace needs forward slashes (or another correct form) - you can't use backslashes - did not make a difference when I changed it
2.) you need a full path to the file instead of dir+"/"+file (depending on where your running the script from).
3.) the shapefile is not "Point". shapefiles are points

Could it be because the shapefiles are buried in folders as shown in the attached image?

RDHarles · ‎01-26-2011

Yup, the script is only looking one directory deep.
In other words, you're pointing the workspace to "projects", so the script is looking for shapefiles in directory "1-4-10" and others in directory "projects".

KristinaGrace · ‎01-26-2011

Thank you, Harles.

I did get the code to work somewhat (added +"/Export/" after every dir+"/") but apparently there are some limitations to field entries... That is disappointing. It starts OK, and this is the error I get at the end:

Start Time: Wed Jan 26 12:01:16 2011
ERROR 000278: Failed on input OID 31, could not write value â??22.5degelbowâ?? to output field Feat_Code
Failed to execute (Merge).
Failed at Wed Jan 26 12:01:24 2011 (Elapsed Time: 8.00 seconds)

*** ERROR: Shapefiles (Point) failed to merge ***

Here is the code that somewhat works:

# This script will merge all point shapefiles in subfolders
# of the workspace specified below

# Import modules and create the geoprocessor object

try:
    # 9.2 and beyond
    import arcgisscripting, collections, arcpy, sys, os
    gp = arcgisscripting.create()
except:
    # 9.1 and before
    import win32com.client, collections, arcpy, sys, os
    gp = win32com.client.Dispatch("esriGeoprocessing.GpDispatch.1")

# Set version
gp.SetProduct("ArcEditor")

# Set the workspace
gp.workspace = ("C:/Trimble Geomatics Office/Projects")

# Start a blank list for Polygon files
# polyList = []

# Start a blank list for Point files
pointList = []

# For each subdirectory
for dir in os.listdir(gp.workspace):
    if os.path.isdir(gp.workspace+"/"+dir+"/Export/"):
        # Get a list of the files in each directory
        files = os.listdir(gp.workspace+"/"+dir+"/Export/")    
        # For each file in a given directory
        for file in files:        
            # Get only files that end with .shp
            if (file.endswith(".shp")):
                print file
                # Describe feature class
                desc = gp.Describe(gp.workspace+"/"+dir+"/"+"/Export/"+file)
                type = desc.ShapeType
                #print dir+"/"+file + " type is: " + type
                #if type == "Polygon":
                 #   print dir+"/"+file + " is Polygon"
                  #  polyPath = polyList.append(dir+"/"+file)
                if type == "Point":
                    print dir+"/"+file + " is Point"
                    pointPath = pointList.append(dir+"/"+"/Export/"+file)                                  
            

            # Hard-code the output merged shapefile names
            # polyshapefile = "Mergedpoly.shp"
            pointshapefile = "Mergedpoint2.shp"    

            # Given a list of shapefiles, separate each by a ";"
            # and put quotes around the whole thing
            # def polyshpList(polyPath):
              #  return '"%s"' % ';'.join(polyList)
            def pointshpList(pointPath):
                return '"%s"' % ';'.join(pointList)

# Set the variable to the newly formatted list of shapefiles            
# polymergedlist = polyshpList(polyPath)
pointmergedlist = pointshpList(pointPath)

# Polygons
# try:
  #  print "\nMerging " + polymergedlist + " to get " + polyshapefile + "...\n"    
   # gp.merge_management(polymergedlist, polyshapefile)
    #print gp.getMessages()    
#except:
 #   print gp.getMessages()
  #  print "\n *** ERROR: Shapefiles (Polygon) failed to merge *** \n"

# Points
try:
    print "\nMerging " + pointmergedlist + " to get " + pointshapefile + "...\n"    
    gp.merge_management(pointmergedlist, pointshapefile)
    print gp.getMessages()    
except:
    print gp.getMessages()
    print "\n *** ERROR: Shapefiles (Point) failed to merge *** \n"    
    
print "\nDone."

KristinaGrace · ‎01-26-2011

How can add a different field size for the Feat_Code field in the output? I think the problem here is that the entry I mentioned above - �??22.5degelbow�?? - is 12 characters long, and the output file has size = 11