topic Re: CopyFeatures_management function working very slow in Python Questions

CopyFeatures_management function working very slow

MuraliChoudari — Thu, 19 Apr 2012 08:38:28 GMT

Hi , I am trying to use CopyFeatures_management function in ArcGIS 10 using python script and its working too slow. My dataset which it is copying is around 1 GB in size. It is taking 2 days to copy features to finish. Everything works fine like merging , joining ..but the problem is with CopyFeatures_management function. Here's my sample code for the join process.

def processJoin(lyr):
    # Join logistics tables to network fc
    gp.AddMessage("Joining lrs table...")
    logFile.write("Joining lrs table...\n")
    gp.AddJoin_management(lyr, JOIN_FLD, lrs, JOIN_FLD)
    gp.AddMessage("Joining ltr table...")
    logFile.write("Joining ltr table...\n")
    gp.AddJoin_management(lyr, tempFC + "." + JOIN_FLD, ltr, JOIN_FLD)

    # Copy layer to fc
    gp.AddMessage("Copying features to feature class...")
    logFile.write("Copying features to feature class...\n")
    gp.CopyFeatures_management(lyr, ExportRoadnetwork)

As you can see from the code, we are just calling CopyFeatures_management to make a copy of the newly joined feature class. But its taking 2 days to finish. Size of the joined dataset is around 1 GB for state data of DC,VA,PA,DE,MD and NJ. We are only using *.*nw data with lrs and ltr for restrictions. Although merge is just taking 20-25 minutes , copying the features is itself taking too long to complete. Can anybody please provide some suggestions where I am doing wrong or how can I improve performance in the script.

Re: CopyFeatures_management function working very slow

JakeSkinner — Thu, 19 Apr 2012 10:22:35 GMT

Hi Murali,

Have you tried using the Join Field tool? This will join the table directly to the other table without the need of copying the feature class to maintain the join.

Re: CopyFeatures_management function working very slow

AlexMackie — Fri, 29 Apr 2016 12:33:31 GMT

My experience is that CopyFeatures_management is fairly broken if you use it to overwrite an existing feature class. It takes 10-100 times longer than when there is no existing file at the destination.

Re: CopyFeatures_management function working very slow

DuncanHornby — Thu, 26 May 2016 11:39:47 GMT

Are there attribute indices on the two fields in the tables that you are joining?

Re: CopyFeatures_management function working very slow

RebeccaStrauch__GISP — Sat, 11 Dec 2021 07:05:03 GMT

If you intent is to have a new copy of the original FC, with the fields of the other two tables added, maybe try copying the FC first, then using the JoinField command ArcGIS Desktop

Summary

Joins the contents of a table to another table based on a common attribute field. The input table is updated to contain the fields from the join table. You can select which fields from the join table will be added to the input table.
The records in the Input Table are matched to the records in the Join Table based on the values of Input Join Field and the Output Join Field. Optionally, only desired fields can be selected from the Join Table and appended to the Input Table during the join.

The reason I say make a copy first...and work with the copy...is this is a permanent join, vs the add join being temporary. From the sample is 10.0 help..

# PermanentJoin.py
# Purpose: Join two fields from a table to a feature class 
# Author: ESRI

# Import system modules
import arcpy
from arcpy import env

# Set the current workspace 
env.workspace = "c:/data/data.gdb"

# Set the local parameters
inFeatures = "zion_park"
joinField = "zonecode"
joinTable = "zion_zoning"
fieldList = ["land_use", "land_cover"]

# Join two feature classes by the zonecode field and only carry 
# over the land use and land cover fields
arcpy.JoinField_management (inFeatures, joinField, joinTable, joinField, fieldList)