Problem with near analysis within an update cursor for loop

[JohnBowers]

I am new to python and trying to create a script that will help individuate trees from a LiDAR point cloud.  

I'm starting with a point feature class of LiDAR returns and using it as a template for another point feature class for spatial comparison.  An update cursor is then created for the initial feature class and a variable declared as zero to assign a tree ID number.  Using a for loop, each row is assigned a tree ID number based on the results of a near analysis(proximity toolset).  The row is assigned a unique tree number(if no near feature was found) or given the tree ID number of the near feature(if one was returned by the near analysis).  Then a copy of the current row is inserted into the comparison feature class and the loop iterates again until all features have been assigned a tree ID number.

That's the concept, but at the moment my script will simply assign every row a unique tree ID number rather than several rows sharing tree ID numbers.  Here is what my script looks like now:



#import required modules
import arcpy
from arcpy import env


#specify workspace
env.workspace = "C:/Columbus_OH_LiDAR_2011/Canopy.gdb"


#specify source geometry
points = "C:/Columbus_OH_LiDAR_2011/Canopy.gdb/Extract_BS868661"


#create comparison geometry feature class
ind_trees = arcpy.CreateFeatureclass_management(env.workspace, "ind_trees", "POINT", points, "DISABLED", "SAME_AS_TEMPLATE", points)


#create update cursor
Ucur = arcpy.updateCursor(points, "", "", "", "HEIGHT D")


#declare variable used to assign tree ID number
x = 0


for row_u in Ucur:

[INDENT]#find nearest feature within a search distance[/INDENT]
[INDENT]arcpy.Near_analysis(points, ind_trees, row_u.HEIGHT / 4)[/INDENT]

[INDENT]#tree ID assigned where no near feature was found[/INDENT]
[INDENT]if row_u.NEAR_FID == -1:[/INDENT]
[INDENT=2]row_u.TREE_NUM = x[/INDENT]
[INDENT=2]Ucur.updateRow(row_u)[/INDENT]
[INDENT=2]x = x + 1[/INDENT]

[INDENT]#assign tree ID from near feature[/INDENT]
[INDENT]else:[/INDENT]
[INDENT=2]Scur = arcpy.SearchCursor(ind_trees)[/INDENT]
[INDENT=2]for row_s in Scur:[/INDENT]
[INDENT=3]if row_s.OBJECTID == row_u.NEAR_FID:[/INDENT]
[INDENT=4]row_u.TREE_NUM = row_s.TREE_NUM[/INDENT]
[INDENT=4]Ucur.updateRow(row_u)[/INDENT]
[INDENT=2]del row_s[/INDENT]
[INDENT=2]del Scur[/INDENT]

[INDENT]#insert a copy of current update cursor row object to comparison geometry feature class[/INDENT]
[INDENT]Icur = arcpy.InsertCursor(ind_trees)[/INDENT]
[INDENT]Nrow = Icur.newRow()[/INDENT]
[INDENT]Nrow.Shape = row_u.Shape[/INDENT]
[INDENT]Nrow.PointCount = row_u.PointCount[/INDENT]
[INDENT]Nrow.ORIG_FID = row_u.ORIG_FID[/INDENT]
[INDENT]Nrow.Z = row_u.Z[/INDENT]
[INDENT]Nrow.RASTERVALU = row_u.RASTERVALU[/INDENT]
[INDENT]Nrow.HEIGHT = row_u.HEIGHT[/INDENT]
[INDENT]Nrow.TREE_NUM = row_u.TREE_NUM[/INDENT]
[INDENT]Nrow.NEAR_FID = row_u.NEAR_FID[/INDENT]
[INDENT]Nrow.NEAR_DIST = row_u.NEAR_DIST[/INDENT]
[INDENT]Icur.insertRow(Nrow)[/INDENT]
[INDENT]del Icur[/INDENT]


del row_u
del Ucur


It appears that the update cursor does not recognize the values generated by the near analysis after it was created, causing the loop never to fall to the else statement.  Due to my inexperience, I'm sure there are many things wrong with this script and there's probably a more optimal method to accomplish this task.  Any advice would be a huge help. Thanks!

[JoshuaChisholm · Accepted solution]

Hello John,

First off, nice script and documentation!

I think you're right, the problem is with the near tool.The near tool was running on all your points for each iteration of the loop. This could cause problems (I think only the last run near tool would be maintained). It's also very slow.

I moved the near tool to right before where you set up the update cursor. Since it's outside the for loop I couldn't use the row_u.HEIGHT / 4 for the search radius. Instead I used row_u.HEIGHT / 4 in the loop and compared it to row_u.NEAR_DIST (the distance to the nearest point).

#import required modules import arcpy from arcpy import env  #specify workspace env.workspace = "C:/Columbus_OH_LiDAR_2011/Canopy.gdb"  #specify source geometry points = "C:/Columbus_OH_LiDAR_2011/Canopy.gdb/Extract_BS868661"  #create comparison geometry feature class ind_trees = arcpy.CreateFeatureclass_management(env.workspace, "ind_trees", "POINT", points, "DISABLED", "SAME_AS_TEMPLATE", points)  #find nearest features arcpy.Near_analysis(points, ind_trees)  #create update cursor Ucur = arcpy.updateCursor(points, "", "", "", "HEIGHT D")  #declare variable used to assign tree ID number x = 0  for row_u in Ucur:     #tree ID assigned where no near feature was found within search radius     if row_u.NEAR_DIST > row_u.HEIGHT / 4:         row_u.TREE_NUM = x         Ucur.updateRow(row_u)         x = x + 1     #assign tree ID from near feature     else:         Scur = arcpy.SearchCursor(ind_trees)         for row_s in Scur:             if row_s.OBJECTID == row_u.NEAR_FID:                 row_u.TREE_NUM = row_s.TREE_NUM                 Ucur.updateRow(row_u)         del row_s         del Scur     #insert a copy of current update cursor row object to comparison geometry feature class     Icur = arcpy.InsertCursor(ind_trees)     Nrow = Icur.newRow()     Nrow.Shape = row_u.Shape     Nrow.PointCount = row_u.PointCount     Nrow.ORIG_FID = row_u.ORIG_FID     Nrow.Z = row_u.Z     Nrow.RASTERVALU = row_u.RASTERVALU     Nrow.HEIGHT = row_u.HEIGHT     Nrow.TREE_NUM = row_u.TREE_NUM     Nrow.NEAR_FID = row_u.NEAR_FID     Nrow.NEAR_DIST = row_u.NEAR_DIST     Icur.insertRow(Nrow)     del Icur  del row_u del Ucur

I think this might fix your issue, but I didn't have a chance to test it out.

Let me know if it works!
Thank you,
~Josh

[JohnBowers]

Thanks Josh,
this was good advice.  Unfortunately I do need the near tool to recalculate each iteration of the loop.  My tentative solution is to nest the entire script(except the initial variable declaration) within a where loop so that the update cursor can be deleted for the next near analysis to be run.  I am still working out the appropriate syntax for the where clause.  I appreciate the help!

[JoshuaChisholm]

Hello John,

I'm happy my first post was partially successful. I'm a little unclear as to why you need to run the near tool inside the loop. Would it work if you ran an extra loop to run the near tool and then did everything else in the second loop? This way all the fields would be ready (like NEAR_FID) before you initiate the updateCursor.

Maybe something like this (which I haven't had a chance to test yet):

#import required modules
import arcpy
from arcpy import env


#specify workspace
env.workspace = "C:/Columbus_OH_LiDAR_2011/Canopy.gdb"


#specify source geometry
points = "C:/Columbus_OH_LiDAR_2011/Canopy.gdb/Extract_BS868661"


#create comparison geometry feature class
ind_trees = arcpy.CreateFeatureclass_management(env.workspace, "ind_trees", "POINT", points, "DISABLED", "SAME_AS_TEMPLATE", points)

#declare variable used to assign tree ID number
x = 0

sCur2=arcpy.SearchCursor(points, "", "", "", "HEIGHT D")
#get nearest for each point
for row in Scur2:
    #Create feature layer for current point:
    arcpy.MakeFeatureLayer_management(points, "TargetPoint_lyr", '"FID" = '+str(row.FID))
    #find nearest feature within a search distance (for current point only)
    arcpy.Near_analysis("TargetPoint_lyr", ind_trees, row.HEIGHT / 4)
del row
del Scur2

#create update cursor
Ucur = arcpy.updateCursor(points, "", "", "", "HEIGHT D")

for row_u in Ucur:
    #tree ID assigned where no near feature was found
    if row_u.NEAR_FID == -1:
        row_u.TREE_NUM = x
        Ucur.updateRow(row_u)
        x = x + 1
    #assign tree ID from near feature
    else:
        Scur = arcpy.SearchCursor(ind_trees)
        for row_s in Scur:
            if row_s.OBJECTID == row_u.NEAR_FID:
                row_u.TREE_NUM = row_s.TREE_NUM
                Ucur.updateRow(row_u)
        del row_s
        del Scur
    #insert a copy of current update cursor row object to comparison geometry feature class
    Icur = arcpy.InsertCursor(ind_trees)
    Nrow = Icur.newRow()
    Nrow.Shape = row_u.Shape
    Nrow.PointCount = row_u.PointCount
    Nrow.ORIG_FID = row_u.ORIG_FID
    Nrow.Z = row_u.Z
    Nrow.RASTERVALU = row_u.RASTERVALU
    Nrow.HEIGHT = row_u.HEIGHT
    Nrow.TREE_NUM = row_u.TREE_NUM
    Nrow.NEAR_FID = row_u.NEAR_FID
    Nrow.NEAR_DIST = row_u.NEAR_DIST
    Icur.insertRow(Nrow)
    del Icur

del row_u
del Ucur

Let me know about any updates!