Issue with geometry not updating properly within multiple cursors

3388
11
Jump to solution
04-26-2016 05:45 PM
RebeccaStrauch__GISP
MVP Emeritus

I'm pretty sure I just have a syntax or indentation issue, but I've been trying to track this down a good chunk of the day, so time to "rubber duck".  The full program is a bit more than anyone will be able to deal with, but if anyone really wants to look at the full code, I can upload it.    Typing is out is sometime helpful for me too....so here's hoping.

Basic (and simplified) description: given a random point with a "transtype" Riparian or Contour,

  • if riparian, grabs river, creates route, snaps point to route.
    • if river < desired length, grabs full river
    • if river >= desired length, tries to create segment with length/2 in each direction
      • if first half < length/2, shifts both points, and vice versa (i.e., if snaped point is too close to end)
    • write to array...with a bunch of other stuff
  • if contour, creates contour as route, snaps point to route (only grabs segments > 5000m)
    • if contour < desire length, grabs full contour
    • if contour >= desired length, tries to create segment
    • with length/2 in each direction
      • if first half < length/2, shifts both points, and vice versa (i.e., if snaped point is too close to end)
    • write to array...with a bunch of other stuff
  • take the array and populates a FC with all the info.

For 2000 random points, it finished with no error, but for some reason about 380 points (or 279 of those 380 points) are duplicating the transect above it.  I do have multiple cursors, which for the most part seem to be fine, but one of the cursors has the "for aRoute2 in route2:" being bypassed.  I believe this is the issue.

So, with that all said, here is the part that I think is the issue.  Not sure that this will be of any used, but in case someone sees something obvious...    I'm on a very short deadline....i.e., here tonight until I get this fixed, most likely.  So any help is appreciated, but may not respond right away.

Dan PattersonCurtis PriceDarren WiensJake SkinnerJoshua Bixby​   any of you seeing anything obvious?  I know cursors within cursors is n necessarily a good idea, and have no time to change it in a major way (another time), I'm in triage mode right now.  Thanks for any help you might offer.  I'll by you a free drink if I ever see you at the dev summit or UC. 

NOTE: trying to fix the indentation in geonet is messing it up more....the first else: and everything else until the final else is indented....

# snippet...too much needed to send all and expect it to work
else:  # transtype == 'contour'
    # ##############
    #   Contours
    # ##############
    print(" -Create contour for elev {0} m".format(elevm))
    ContourList(inDEM, fullContour, elevm)
    #print(" -Clipping full contour by study area")
    arcpy.Clip_analysis(fullContour, inStudy, clipContour)
    #print("       making clipped contour FL")
    arcpy.MakeFeatureLayer_management(clipContour, "tmpClipContour")
    print(" -Selecting contour segs > 5000 m")
    #arcpy.SelectLayerByAttribute_management("tmpClipContour", "NEW_SELECTION", where_clause="Shape_Length >= 5000")  
    arcpy.SelectLayerByAttribute_management("tmpClipContour", "NEW_SELECTION", "Shape_Length >= 5000")

    # copies selection to new FC
    #print(" - Copying selection to new FC")  # here 4/4/2016
    arcpy.CopyFeatures_management("tmpClipContour", tmpContourFC)
    arcpy.MakeFeatureLayer_management(tmpContourFC, "tmpContourFL") 
    # # wrong? tmpContourFL = arcpy.mapping.Layer("tmpClipContour") 
    #print("\n**New selection of nearest contour segment to pt")
    #print(" - Creating new feature layer from origPtFL")
    arcpy.SelectLayerByLocation_management("tmpContourFL", nearType, "origPtFL", nearDist)    
    #print(" - Copy selection to new tmp FC contourSel")
    arcpy.CopyFeatures_management("tmpContourFL", contourSel)

    # Creates route of the selected contour segment
    print(" -Create temp route; create and process cursor")
    arcpy.CreateRoutes_lr(contourSel, contourField, tmpRoute)
    #print(" - Create SearchCursor from route")

    routeSCur2 = arcpy.da.SearchCursor(tmpRoute, ["SHAPE@", "Contour"])            
    print("getting here")

    with routeSCur2 as route2:
        print("getting inside cursor")   # will get to here
        for aRoute2 in route2:
            print("getting here2")     # #######   will not always get to here   #######
            geo = aRoute2[0]          # get all contour elevation for the route
            theElev = aRoute2[1]      # get elevation, actual contour as a route
            #snapPt = (geo.snapToLine(ptGeom)).centroid # shp_fld
            arcpy.LocateFeaturesAlongRoutes_lr(ptGeom, tmpRoute, contourField, nearDist, tmpRoutePtLoc, rid)
            lenTrans = geo.length  
            print("  **Contour elev {0}:  {1} m".format(theElev, RoundUp(lenTrans)))
            flyFullContour = lenTrans <= int(targetLength)
            if flyFullContour:
                print(" **Taking full contour: {0}m ".format(lenTrans))
                startx  = geo.firstPoint.X
                starty  = geo.firstPoint.Y
                endx = geo.lastPoint.X
                endy = geo.lastPoint.Y

                startPtdd = arcpy.PointGeometry(arcpy.Point(startx, starty) , initialSR).projectAs(geoSR)
                startxdd = startPtdd.centroid.X
                startydd = startPtdd.centroid.Y

                #print(" -Start point {0} {1}".format(startxdd, startydd))
                endPtdd = arcpy.PointGeometry(arcpy.Point(endx, endy) , initialSR).projectAs(geoSR)
                endxdd = endPtdd.centroid.X
                endydd = endPtdd.centroid.Y

                #print(" -End point {0} {1}".format(endxdd, endydd))
                tmpTrans = geo # taking full length.segmentAlongLine(startDist, endDist)
                #tlength = tmpTrans.length
                #ripNotes = "short"
                #print("{0} transect #  {1}".format(transtype, transID))
                print("{0} transect #  {1}  length: {2}  Notes: {3}".format(transtype, transID, tlength, ripNotes))
            elif lenTrans >= int(targetLength):    #else:

                ptSCue = arcpy.SearchCursor(tmpRoutePtLoc)
                field_name = 'MEAS'
                for routeMeas in ptSCue:
                    snapPtDist =  routeMeas.getValue(field_name)
                    print("  ->Measurement along route: {0}".format(snapPtDist))
                    startDist = snapPtDist - halfTrans
                    endDist = snapPtDist + halfTrans
                    print("\n -Initial startDist: {0}  endDist: {1}".format(startDist, endDist))
                    initialSegment = geo.segmentAlongLine(endDist, startDist)
                    initialSegmentLength = initialSegment.length
                    print(" -->Initial length: {0}".format(initialSegmentLength))
                    if RoundUp(initialSegmentLength) < int(targetLength):
                        # Check each side to make sure distance is = halfTrans
                        #  if not, shift as needed

                        # check length of the segment from the snapPtDist to startDist
                        firstSegment = geo.segmentAlongLine(snapPtDist, startDist)
                        firstSegmentLength = firstSegment.length
                        #print(" -First Length: {0}".format(firstSegmentLength))

                        #check first
                        if RoundUp(firstSegmentLength) < (halfTrans):
                            print(" -First half is short: {0}m".format(firstSegmentLength))
                            # calculate the distance it is short
                            shiftDist = halfTrans - firstSegmentLength
                            print("   ..shift distance for first segment: {0}m".format(shiftDist))
                            #startDist = startDist - shiftDist
                            # short from start to mid, so extend end distance
                            endDist = endDist + shiftDist
                            print("  ..new endDist: {0}m".format(endDist))
                        else:  # first segment ok, must be second part
                            endSegment = geo.segmentAlongLine(snapPtDist, endDist)
                            endSegmentLength = endSegment.length
                            print(" -Second half is short: {0}m".format(endSegmentLength))
                            # calculate the distance it is short
                            shiftDist = halfTrans - endSegmentLength
                            print("  ..shift distance for first segment: {0}m".format(shiftDist))
                            # short from end to mid, so start earlier on the line, it a minus measure
                            startDist = startDist - shiftDist
                            #endDist = endDist - shiftDist
                            print("  ..new startDist: {0}m".format(startDist))                  

                    # use the startDist and endDist to create the point geometries
                    #startPt = geo.positionAlongLine(startDist)
                    #endPt = geo.positionAlongLine(endDist)

                    startPt = geo.positionAlongLine(startDist)
                    startx = startPt.centroid.X    #need DD
                    starty = startPt.centroid.Y            
                    #print("Start Dist: {0}".format(startDist))
                    # midPt = geo.positionAlongLine(snapPtDist) #not using mid points
                    endPt = geo.positionAlongLine(endDist)
                    endx = endPt.centroid.X
                    endy = endPt.centroid.Y    
                    tmpTrans = geo.segmentAlongLine(startDist, endDist)
                    tlength = tmpTrans.length

            del ptSCue
            # Processing pts to geographic
            startPtdd = arcpy.PointGeometry(arcpy.Point(startx, starty) , initialSR).projectAs(geoSR)
            startxdd = startPtdd.centroid.X
            startydd = startPtdd.centroid.Y                 
            #print("start point {0} {1}".format(startxdd, startydd))
            #midPtdd = arcpy.PointGeometry(arcpy.Point(midPt.centroid.X, midPt.centroid.Y) , initialSR).projectAs(geoSR)   #not using mid points
            endPtdd = arcpy.PointGeometry(arcpy.Point(endx, endy) , initialSR).projectAs(geoSR)
            endxdd = endPtdd.centroid.X
            endydd = endPtdd.centroid.Y                            

            #tmpTrans = geo.segmentAlongLine(startDist, endDist)

            #tlength = tmpTrans.length
            shortLength = int(targetLength) - 500
            if tlength < shortLength:   # add A to ID for short trans...add in field.
                transID = "{0}A".format(transID)

            if tmpTrans.isMultipart:
                ripNotes = "contourParts"
                print("  ** Transect in multiple parts **")
            else:
                ripNotes = "contour"
            #print("{0} trans # {1}".format(transtype, transID))
            print("{0} transect #  {1}  length: {2}  Notes: {3}".format(transtype, transID, tlength, ripNotes))

else:
    break 

Edit:  added not above the python script..reformatting was not good.

0 Kudos
1 Solution

Accepted Solutions
DarrenWiens2
MVP Honored Contributor

ptScue is an old-style cursor, which behave unpredictably (or predictably bad) when nested. Does it help to switch it to a da cursor? In general, I also suggest using ' with ' syntax on all cursors to avoid them hanging in the void if your run ends in an error.

View solution in original post

11 Replies
DanPatterson_Retired
MVP Emeritus

can you confirm that the else statements on line 2 and 153 are supposed to be indented at the same level?

RebeccaStrauch__GISP
MVP Emeritus

Good catch on the indentation Dan...I'll fix it.  Such is copying and pasting snippets.  Actual code have the first else indented four levels. 

I should also add that, running an individual point manually works and does create the correct transect, so I know it is not the input random points, DEM, or riparian layers that are an issue.  I also should mention it does not always skip that missing that one cursor.  There is some combination from the previous point that causes the issue, but looking at the data and output, their is no  obvious pattern.  I just isolated 3 of the "problem" points and ran them thru the script and they worked fine.  That is why I think it might be an issue with the cursors not being lined up correctly, or the actual geometry field "tmpTrans" not being updated to over write the previous values....which then of course make all the coordinates, lengths, etc calcd from it off.

just fyi, thie is what I am appending

transList.append([tmpTrans, ptid, ptX, ptY, transtype, tlength, tlengthKm, transID, elevft, elevm, startxdd, startydd, endxdd, endydd, startxdms, startydms, endxdms, endydms, ripNotes, offset, displayID])

0 Kudos
DanPatterson_Retired
MVP Emeritus

I am kindof with Darren on this I am finding it hard to figure out the cursor stuff in lines 32 to 38.  But I am not a cursor person.  Darren and Xander have posted some good examples but I don't know how much reshaping would be needed. 

Is there anything you can 'def' to consolidate the code now? or is this for a non-deadline time?

0 Kudos
DarrenWiens2
MVP Honored Contributor

ptScue is an old-style cursor, which behave unpredictably (or predictably bad) when nested. Does it help to switch it to a da cursor? In general, I also suggest using ' with ' syntax on all cursors to avoid them hanging in the void if your run ends in an error.

RebeccaStrauch__GISP
MVP Emeritus

Good catch Darren.  I usually always use da cursors.  Not sure what I did there.  I had noticed I wasn't using the "with" for that one, and it didn't work last week when I was trying...I was "cursor" blind on that one and hadn't noticed.  I'll give that a try and be back...

0 Kudos
RebeccaStrauch__GISP
MVP Emeritus

Change those to da cursors helped me get "into" that cursor now, so that is good.  I actually rolled back to a version before I made the numerous changes I did this afternoon, made those the changes to with .da, and renamed a few of my cursors, so I wasn't repeating names.    Also got rid of my "del cursor" statements, which I had added when trouble shooting.

Anyway, still having issues, but may be able to track things for a bit again.  I'll update as I have more.

Thanks again for your feedback Dan and Darren.

0 Kudos
RebeccaStrauch__GISP
MVP Emeritus

I'm giving Darren credit for the answer at this point in time.  Even though the da.SearchCursor did not resolve the issue entirely, and is actually working the same,  it did help me find where it was having the issue and bandage it for now. (i.e. by adding the "with.." AND a "for..", found it was the "for..." that had the problem....i.e. zero records in cursor).  So thanks for finding that Darren.

0 Kudos
DanPatterson_Retired
MVP Emeritus

line 70 and 170 ... is the deletion of the cursor lined up correctly?    maybe I will wait until the while stuff is done

0 Kudos
RebeccaStrauch__GISP
MVP Emeritus

Still having issues, but figured out partially what the issue is, but not why.

I did switch the one cursor to a .da version.  It took me quite a while to figure out what I was trying to do.  The table (not FC) actually only has one record (or is supposed to have 1), and I was just trying to pull a the value from the "MEAS" field.  It is coming down to the input to my cursor occasionally had zero records.  What is strange is, if I do it manually, I get my one record, even on the problem points, but my program is skipping it for some points for some reason.

The command I am running (sorry about all the variables)

arcpy.LocateFeaturesAlongRoutes_lr(ptGeom, tmpRoute, contourField, nearDist, tmpRoutePtLoc, rid)

Although manually (i.e. in the python window), I do get one record with my nearDist value, and in most cases it works in the script, for some records it doesn't return anything.  So now I am testing for the number of records, and although I haven't fixed my duplication problem, I'm at least setting a flag in my notes field for my output line segments...that will help me pull those out for further testing and to fix them.

myCount = int(arcpy.GetCount_management(tmpRoutePtLoc).getOutput(0))
print("the count from Locate Along Route {0}".format(myCount))
if myCount == 0:
    ripNotes = "ERRORcon"

Giving up for the night, going to run it overnight, maybe with a new set of points, and deal with the problem points tomorrow...which is in about  40m. (sigh)

0 Kudos