topic Re: CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox in Geoprocessing Questions

CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox

LoganPugh — Thu, 29 Jul 2010 19:18:59 GMT

Dismayed by how long a Python script I've developed was taking to copy features to a shapefile, I decided to do some testing. When run from ArcToolbox, the Copy Features (management) tool takes about 12 minutes to copy a large point feature class from SDE to a local shapefile. I wrote a small test script that does the exact same thing, using CopyFeatures_management to copy the same SDE feature class to a local shapefile, and it took over 2 hours!

What could possibly explain this discrepancy? I've seen a few mentions of similar problems with the CopyFeatures tool on the old forums but no real explanation or solution.

Re: CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox

BBicking1 — Thu, 29 Jul 2010 20:49:15 GMT

Dismayed by how long a Python script I've developed was taking to copy features to a shapefile, I decided to do some testing. When run from ArcToolbox, the Copy Features (management) tool takes about 12 minutes to copy a large point feature class from SDE to a local shapefile. I wrote a small test script that does the exact same thing, using CopyFeatures_management to copy the same SDE feature class to a local shapefile, and it took over 2 hours!

What could possibly explain this discrepancy? I've seen a few mentions of similar problems with the CopyFeatures tool on the old forums but no real explanation or solution.

Hi Logan,

can you include your script here, for us to look at? Thanks.

Barbara Bicking
ESRI Geoprocessing Team

Re: CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox

LoganPugh — Sat, 11 Dec 2021 07:46:57 GMT

Sure, here it is (very simple -- just a test script):

import arcgisscripting, os

in_feat = r"Database Connections\LPUGH@GISPROD.sde\SDE.ABBE\SDE.ADDRESSES"
out_shp = r"C:\temp\addr_test.shp"

gp = arcgisscripting.create(9.3)
try:
    gp.copyfeatures_management(in_feat, out_shp)
finally:
    print gp.GetMessages()

Thanks for the reply!

Re: CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox

BBicking1 — Sat, 11 Dec 2021 07:47:00 GMT

Sure, here it is (very simple -- just a test script):

import arcgisscripting, os

in_feat = r"Database Connections\LPUGH@GISPROD.sde\SDE.ABBE\SDE.ADDRESSES"
out_shp = r"C:\temp\addr_test.shp"

gp = arcgisscripting.create(9.3)
try:
    gp.copyfeatures_management(in_feat, out_shp)
finally:
    print gp.GetMessages()

Thanks for the reply!

Hi Logan,

I spoke with a bunch of people here and the consensus is that it's very difficult to say why your Python script takes so long. It can be the SDE database you're using; how many people are accessing it at the same time; the data itself; other settings you have.
Test with a smaller set of your data and see what time you get then.
For us to help, we'll need your data and details of your work environment.

Barbara Bicking

Re: CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox

LoganPugh — Fri, 30 Jul 2010 20:57:09 GMT

Thanks for looking into it. I do not think it's an SDE problem because the same issue occurs on large file geodatabase feature classes as well. I will do some more testing on different sizes/types of feature classes and in the meantime will submit a bug report through our usual channel.

Re: CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox

LouJacoby — Thu, 11 Aug 2011 19:01:32 GMT

Were you able to resolve what was causing the problem? I am having a similar issue in v10, trying to use CopyFeatures from a layer file to an SDE feature class. Running it in model builder it takes 14 minutes, but when I export that to a python script it takes 4.5 hours. Thanks.

Re: CopyFeatures tool runs 10 times slower in a Python script than from ArcToolbox

LoganPugh — Thu, 11 Aug 2011 20:56:52 GMT

This appears to be fixed for me at ArcGIS 10 SP2. Are you using SP2 as well?

Two workarounds were mentioned:

Use Feature Class to Feature Class instead of Copy Features (no improvement, at the time)

Call a model that uses CopyFeatures from the Python script (IIRC this was fast but not very practical).

You might check on the status of the following Nimbus IDs that were mentioned:
#NIM041113 The Python script to export shapefile from ArcSDE is extremely slow
#NIM043129 Performance of Feature Class to Shapefile when used in Python is slow