CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox

[LoganPugh]

Dismayed by how long a Python script I've developed was taking to copy features to a shapefile, I decided to do some testing. When run from ArcToolbox, the Copy Features (management) tool takes about 12 minutes to copy a large point feature class from SDE to a local shapefile. I wrote a small test script that does the exact same thing, using CopyFeatures_management to copy the same SDE feature class to a local shapefile, and it took over 2 hours!

What could possibly explain this discrepancy? I've seen a few mentions of similar problems with the CopyFeatures tool on the old forums but no real explanation or solution.

[BBicking1]

Dismayed by how long a Python script I've developed was taking to copy features to a shapefile, I decided to do some testing. When run from ArcToolbox, the Copy Features (management) tool takes about 12 minutes to copy a large point feature class from SDE to a local shapefile. I wrote a small test script that does the exact same thing, using CopyFeatures_management to copy the same SDE feature class to a local shapefile, and it took over 2 hours!

What could possibly explain this discrepancy? I've seen a few mentions of similar problems with the CopyFeatures tool on the old forums but no real explanation or solution.

Hi Logan,

can you include your script here, for us to look at?  Thanks.

Barbara Bicking
ESRI Geoprocessing Team

[LoganPugh]

Sure, here it is (very simple -- just a test script):

import arcgisscripting, os

in_feat = r"Database Connections\LPUGH@GISPROD.sde\SDE.ABBE\SDE.ADDRESSES"
out_shp = r"C:\temp\addr_test.shp"

gp = arcgisscripting.create(9.3)
try:
    gp.copyfeatures_management(in_feat, out_shp)
finally:
    print gp.GetMessages()

Thanks for the reply!

[BBicking1]

Sure, here it is (very simple -- just a test script):

import arcgisscripting, os

in_feat = r"Database Connections\LPUGH@GISPROD.sde\SDE.ABBE\SDE.ADDRESSES"
out_shp = r"C:\temp\addr_test.shp"

gp = arcgisscripting.create(9.3)
try:
    gp.copyfeatures_management(in_feat, out_shp)
finally:
    print gp.GetMessages()

Thanks for the reply!

Hi Logan,

I spoke with a bunch of people here and the consensus is that it's very difficult to say why your Python script takes so long. It can be the SDE database you're using; how many people are accessing it at the same time; the data itself; other settings you have.
Test with a smaller set of your data and see what time you get then.
For us to help, we'll need your data and details of your work environment.

Barbara Bicking

[LoganPugh]

Thanks for looking into it. I do not think it's an SDE problem because the same issue occurs on large file geodatabase feature classes as well. I will do some more testing on different sizes/types of feature classes and in the meantime will submit a bug report through our usual channel.

[LouJacoby]

Were you able to resolve what was causing the problem?  I am having a similar issue in v10, trying to use CopyFeatures from a layer file to an SDE feature class.  Running it in model builder it takes 14 minutes, but when I export that to a python script it takes 4.5 hours.  Thanks.

[LoganPugh]

This appears to be fixed for me at ArcGIS 10 SP2. Are you using SP2 as well?

Two workarounds were mentioned:

1. Use Feature Class to Feature Class instead of Copy Features (no improvement, at the time)


2. Call a model that uses CopyFeatures from the Python script (IIRC this was fast but not very practical).

You might check on the status of the following Nimbus IDs that were mentioned:
#NIM041113  The Python script to export shapefile from ArcSDE is extremely slow
#NIM043129  Performance of Feature Class to Shapefile when used in Python is slow