Is there a way to speedup the following code

281
7
Jump to solution
03-13-2019 09:30 PM
AndrewNeedham1
New Contributor

Hi,

Back story, this code is part of a much larger script which loops a few thousand times and takes a few days to finish

I'm currently going through the code and trying to it speed up where possible. Can the following code be speed up?

I know arcpy.da curses are fast but can they be used here?

Edit: This is using ArcGIS 10.3.1 and can't be transferred to ArcPro 

 

# select Lines
whereClauseSort = """{} IN (1,2,3) AND {} = 'Pass' """.format('Sorting', 'Group2')
selectionLineFC = r"in_memory\line"
arcpy.Select_analysis(selectionFC, selectionLineFC, whereClauseSort)
# Dissolve selected lines
Dissolve1 = r"in_memory\Dissolve1"
arcpy.Dissolve_management(
selectionLineFC, Dissolve1, "Angle;BEARING",
"Sorting COUNT;Seam First", "MULTI_PART", "DISSOLVE_LINES")
# add attributes
arcpy.AddField_management(
Dissolve1, "Length", "LONG", "", "", "", "Length", "NULLABLE", "NON_REQUIRED")
arcpy.CalculateField_management(
Dissolve1, "Length", "round(!shape.length@meters!,10)", "PYTHON_9.3", "")

# Sort Dissolve selected lines
sort = r"in_memory\Sort1"
arcpy.Sort_management(
'Dissolve1', sort,
'Length DESCENDING;Angle ASCENDING;COUNT_Sorting ASCENDING', 'UR')

# Select first record
sort2 = r"in_memory\Sort2"
sel = arcpy.Select_analysis(sort, sort2, 'OID = 1')

# Buffer first record
convexHullBuff = r"in_memory\convexHullBuff"
convexHullbuff = arcpy.Buffer_analysis(
sel, convexHullBuff, '200 Meters', 'FULL', 'ROUND', 'NONE', '#', 'PLANAR')

# CONVEX_HULL of Buffered first record
ConvexHull = r"in_memory\ConvexHull"
convexHull = arcpy.MinimumBoundingGeometry_management(
convexHullbuff, ConvexHull, "CONVEX_HULL", 'NONE', '#', 'NO_MBG_FIELDS')
0 Kudos
1 Solution

Accepted Solutions
curtvprice
MVP Esteemed Contributor

Your physical sort is really slow, setting up a sorted cursor is a much faster way to get the first-sorted record. Here's my suggested fix for this:

# Sort Dissolve selected lines
sort = r"in_memory\Sort1"
arcpy.Sort_management(
'Dissolve1', sort,
'Length DESCENDING;Angle ASCENDING;COUNT_Sorting ASCENDING', 'UR')
# Select first record
sort2 = r"in_memory\Sort2"
sel = arcpy.Select_analysis(sort, sort2, 'OID = 1')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

if the data are in a file geodatabase, you can use ORDER BY with a search cursor instead of a (slow) sort to find the first OID (sorted)

xdis = arcpy.CopyFeatures_management(Dissolve1, arcpy.env.scratchGDB + "/xdis")
oid = arcpy.da.SearchCursor(xdis, "*", "", "",
["", "ORDER BY Length DESC, ANGLE ASC, COUNT_Sorting ASC"]).next()[0]
sort2 = arcpy.Select_analysis(xdis, 'in_memory/sort2',
'OID = {}'.format(oid))‍‍‍‍‍‍‍‍‍‍‍‍‍‍

View solution in original post

7 Replies
DanPatterson_Retired
MVP Esteemed Contributor

What is your end goal? Can you explain the purpose of those blocks of code?

Do you get different results for instance, if you determine the convex hull, then buffer vs the way you have it?

If you have complex shapes, the more points you have the longer the buffer is going to take

Also, In some cases, just working with the arcpy geometry methods and properties is faster than calling the arctoolbox tool that does the same thing (eg, like a dissolve).  Other times using numpy for geometry functions is orders of magnitude faster.

0 Kudos
AndrewNeedham1
New Contributor

The end goal is to run the script faster that it currently is. The order needs to be maintained such as, if you don't buffer the selected line\s (multipart) the convex hull will fail on a single line.

Yes the goal would be to use arcpy geometry methods, but I am unaware of the syntax/code for arcpy geometry methods to do a dissolve, do you have an example of this?

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

For the polygon class for instance, Polygon—ArcPy classes | ArcGIS Desktop 

you can find the geometry functions like convex hull in there

similarly for length etc.

You can create arrays do quick calculations resulting in the equivalent of a field, then use arcpy.ExtendTable, which effectively, allows you to do vectorized calculations, and add the result to a table.

You can accomplish many tasks at once.  For example, you need the perimeter of a buffered polyline's convex hull.

Your worst case scenario, a 2 point polyline parallel to the x-axis

p0 = arcpy.Point(0,0)
p1 = arcpy.Point(10,0)
poly = arcpy.Polyline(arcpy.Array([p0,p1]))
polybuff = poly.buffer(2)
ch = polybuff.convexHull()
ch.length
Out[171]: 32.55081089915268

of course this doesn't have to be done line-by-line, but you read the geometry, buffer (line 4) do the convex hull (line 5), get its length and add it to a field.  That whole sequence can be a 'def' for use in the field calculator for example

AndrewNeedham1
New Contributor

Thanks. I will look into this, it will take some time to get my head around this code  

0 Kudos
curtvprice
MVP Esteemed Contributor

Your physical sort is really slow, setting up a sorted cursor is a much faster way to get the first-sorted record. Here's my suggested fix for this:

# Sort Dissolve selected lines
sort = r"in_memory\Sort1"
arcpy.Sort_management(
'Dissolve1', sort,
'Length DESCENDING;Angle ASCENDING;COUNT_Sorting ASCENDING', 'UR')
# Select first record
sort2 = r"in_memory\Sort2"
sel = arcpy.Select_analysis(sort, sort2, 'OID = 1')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

if the data are in a file geodatabase, you can use ORDER BY with a search cursor instead of a (slow) sort to find the first OID (sorted)

xdis = arcpy.CopyFeatures_management(Dissolve1, arcpy.env.scratchGDB + "/xdis")
oid = arcpy.da.SearchCursor(xdis, "*", "", "",
["", "ORDER BY Length DESC, ANGLE ASC, COUNT_Sorting ASC"]).next()[0]
sort2 = arcpy.Select_analysis(xdis, 'in_memory/sort2',
'OID = {}'.format(oid))‍‍‍‍‍‍‍‍‍‍‍‍‍‍

View solution in original post

AndrewNeedham1
New Contributor

Awesome thanks, I haven't seen a search cursor used in that way before. I would assume the Dissolve FC could be used directly into the Search Cursor instead of using the Copy Features. Thus eliminating an extra step? 

0 Kudos
curtvprice
MVP Esteemed Contributor

I would assume the Dissolve FC could be used directly into the Search Cursor instead of using the Copy Features. Thus eliminating an extra step?

I don't believe the ORDER BY will work using an in_memory feature class.

0 Kudos