CopyFeatures from Excel data source much slower in 10.1 than 10.0

[MoniqueDawson]

Hi Everyone,

I have updated my ArcGIS from ArcGIS 10.0 to ArcGIS 10.1 and now my python code is SOOOO SLOW. And worked fine in ArcGIC 10.0!!  What is the difference between to two??

Python Code:

### Import arcpy module 
import arcpy
# Local variables:
dir = "E:\\Dawson_Monique\\Weather_Grid\\Louis_Excel_2\\Prov_MB\\"
for nodes in range(1,152):
  name = "nodes_gp_" + str(nodes) + ".csv"
  xyfile = dir + name
  eventfile = dir + "layer\\nodes_gp_"+ str(nodes) + "_layer.shp"
  arcpy.MakeXYEventLayer_management(xyfile, "lon", "lat", xyfile, "", "")
  arcpy.CopyFeatures_management(xyfile, eventfile, "", "0", "0", "0")
  print nodes
print "done"

The Copy Feature portion is Slow. Why? Use to take 5 min now all day.

Thanks

MO

Message was edited by: Curtis Price (edited title, formatted code)

[DanPatterson_Retired]

If it is truly taking all day, then I wouldn't blame the python code.  Try it on another machine if possible.

[JohnBrodnicki]

Just a shot in the dark but you may want to make sure the paths to the ArcGIS python install location are correct.

[DavidWynne · Accepted solution]

I believe the performance issue is likely due to the heavy use of csv files in your code. There is an outstanding issue NIM103059 that would seem to fit your case.

-Dave

[curtvprice]

A really good workaround (that, David, I think should be added to the NIM) is to use numpy. Monique Dawson‌ if you are processing hundreds of files should be much faster even after the bug is fixed (and would have been faster in 10.0 too).

from numpy import genfromtxt
my_data = genfromtxt('my_file.csv', delimiter=',')
arcpy.da.NumPyArrayToTable(array, "in_memory/tmp_xytable")
lyrTemp = arcpy.MakeXYEventLayer_management(tmp_xytable, "lon", "lat", xyfile, "", "")
arcpy.CopyFeatures_management(lyrTemp, eventfile, "", "0", "0", "0")

[DavidWynne]

Good idea

[DuncanHornby]

Have the input csv files massively changed in size? Or as Dan suggests has something changed with your machine? Are you getting low on disk space, when did you last run a disk clean up or defrag on your system?