csny490

A better way to run large Append/Merge jobs

Discussion created by csny490 on Sep 7, 2012
Latest reply on Oct 30, 2013 by lbrannon
Being frustrated at the unnecessarily long processing times of the Merge and/or Append tools for large datasets, I wrote some simple Python code (using the data access module in v10.1) to emulate the same functionality. Took my 2 hour Merge processing time to 11 minutes :) I hope someone else can benefit from this code. Right now it is designed to just run on polygon FCs with the same schema, although it could be enhanced to do field mapping as well (not on my time though). It relies on a simple search and insert cursor using the arcpy.da module. BTW, ESRI surely gets some huge props from me for the new cursor model...

import arcpy
fcList = [a list of FCs that you want to merge together]
outputFC = r"C:\temp\test.gdb\merge"
for fc in fcList:
    if fcList.index(fc) == 0:
        arcpy.CopyFeatures_management(fc, outputFC)
        insertRows = arcpy.da.InsertCursor(outputFC, ["SHAPE@","*"])
    else:
        searchRows = arcpy.da.SearchCursor(fc, ["SHAPE@","*"])
        for searchRow in searchRows:
            insertRows.insertRow(searchRow)
        del searchRow, searchRows
    print "Appended " + str(fc) + "..."
del insertRows

Outcomes