topic Re: Making my script run faster in Python Questions

Making my script run faster

AleydisG__Pere — Tue, 08 Nov 2011 13:59:30 GMT

Hello,
I'm trying to get the fastest and most efficient script to take each feature in a shapefile (it's a 60x60m grid with 10 million features) as a mask to cut a DTM and apply the surface volume tool to each resulting raster.
This script needs 3 seconds per feature but bearing in mind that I have 10 million features this is too slow.
Any idea on how to improve it?
Thanks!

import arcpy

arcpy.CheckOutExtension("3D")
arcpy.CheckOutExtension("spatial")

Fishnet60x60 = arcpy.GetParameterAsText(0)
MDT15 = arcpy.GetParameterAsText(1)
Area2D3D_txt = arcpy.GetParameterAsText(2)

try:

    rows = arcpy.SearchCursor(Fishnet60x60)
    for row in rows:
        idvalue = row.getValue("ID_GRID")
        arcpy.MakeFeatureLayer_management(Fishnet60x60, Fishnet60x60 + "_layer", "\"ID_GRID\" = '" + idvalue + "'", "", "")
        arcpy.gp.ExtractByMask_sa(MDT15, Fishnet60x60 + "_layer", idvalue)
        arcpy.SurfaceVolume_3d(idvalue, Area2D3D_txt, "ABOVE", "0", "1", "0")

except:

    arcpy.AddMessage(arcpy.GetMessages(2))

Re: Making my script run faster

AndrewChapkowski — Tue, 08 Nov 2011 14:13:38 GMT

You can try using Split Raster to break up your analysis into smaller parts instead of the mask tool.

See: http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//00170000009v000000

Re: Making my script run faster

DuncanHornby — Tue, 08 Nov 2011 14:39:50 GMT

Aleydis,

I use PyScripter as my Python development environment. I find running a script from the Python command line window in ArcMap slower than running it from the IDE environment. Also if you have a multicore processor maybe you could try and utilise them all? There is a great blog article showing a simple example here.

Duncan

Re: Making my script run faster

KimOllivier — Wed, 09 Nov 2011 09:23:51 GMT

Quite right to worry about the performance.
You certainly don't want 10 million featureclasses output, that will kill any file system.

The cup of coffee rule states:

"If any single process takes longer than a cup of coffee then interrupt it and find a better way."

Just don't run a geoprocessing tool inside a cursor.
Why do you have to call it for each feature?

Maybe just run the Polygon_Volume_3D tool once?