I'm pretty sure I just have a syntax or indentation issue, but I've been trying to track this down a good chunk of the day, so time to "rubber duck". The full program is a bit more than anyone will be able to deal with, but if anyone really wants to look at the full code, I can upload it. Typing is out is sometime helpful for me too....so here's hoping.
Basic (and simplified) description: given a random point with a "transtype" Riparian or Contour,
For 2000 random points, it finished with no error, but for some reason about 380 points (or 279 of those 380 points) are duplicating the transect above it. I do have multiple cursors, which for the most part seem to be fine, but one of the cursors has the "for aRoute2 in route2:" being bypassed. I believe this is the issue.
So, with that all said, here is the part that I think is the issue. Not sure that this will be of any used, but in case someone sees something obvious... I'm on a very short deadline....i.e., here tonight until I get this fixed, most likely. So any help is appreciated, but may not respond right away.
Dan Patterson Curtis Price Darren Wiens Jake Skinner Joshua Bixby any of you seeing anything obvious? I know cursors within cursors is n necessarily a good idea, and have no time to change it in a major way (another time), I'm in triage mode right now. Thanks for any help you might offer. I'll by you a free drink if I ever see you at the dev summit or UC.
NOTE: trying to fix the indentation in geonet is messing it up more....the first else: and everything else until the final else is indented....
# snippet...too much needed to send all and expect it to work else: # transtype == 'contour' # ############## # Contours # ############## print(" -Create contour for elev {0} m".format(elevm)) ContourList(inDEM, fullContour, elevm) #print(" -Clipping full contour by study area") arcpy.Clip_analysis(fullContour, inStudy, clipContour) #print(" making clipped contour FL") arcpy.MakeFeatureLayer_management(clipContour, "tmpClipContour") print(" -Selecting contour segs > 5000 m") #arcpy.SelectLayerByAttribute_management("tmpClipContour", "NEW_SELECTION", where_clause="Shape_Length >= 5000") arcpy.SelectLayerByAttribute_management("tmpClipContour", "NEW_SELECTION", "Shape_Length >= 5000") # copies selection to new FC #print(" - Copying selection to new FC") # here 4/4/2016 arcpy.CopyFeatures_management("tmpClipContour", tmpContourFC) arcpy.MakeFeatureLayer_management(tmpContourFC, "tmpContourFL") # # wrong? tmpContourFL = arcpy.mapping.Layer("tmpClipContour") #print("\n**New selection of nearest contour segment to pt") #print(" - Creating new feature layer from origPtFL") arcpy.SelectLayerByLocation_management("tmpContourFL", nearType, "origPtFL", nearDist) #print(" - Copy selection to new tmp FC contourSel") arcpy.CopyFeatures_management("tmpContourFL", contourSel) # Creates route of the selected contour segment print(" -Create temp route; create and process cursor") arcpy.CreateRoutes_lr(contourSel, contourField, tmpRoute) #print(" - Create SearchCursor from route") routeSCur2 = arcpy.da.SearchCursor(tmpRoute, ["SHAPE@", "Contour"]) print("getting here") with routeSCur2 as route2: print("getting inside cursor") # will get to here for aRoute2 in route2: print("getting here2") # ####### will not always get to here ####### geo = aRoute2[0] # get all contour elevation for the route theElev = aRoute2[1] # get elevation, actual contour as a route #snapPt = (geo.snapToLine(ptGeom)).centroid # shp_fld arcpy.LocateFeaturesAlongRoutes_lr(ptGeom, tmpRoute, contourField, nearDist, tmpRoutePtLoc, rid) lenTrans = geo.length print(" **Contour elev {0}: {1} m".format(theElev, RoundUp(lenTrans))) flyFullContour = lenTrans <= int(targetLength) if flyFullContour: print(" **Taking full contour: {0}m ".format(lenTrans)) startx = geo.firstPoint.X starty = geo.firstPoint.Y endx = geo.lastPoint.X endy = geo.lastPoint.Y startPtdd = arcpy.PointGeometry(arcpy.Point(startx, starty) , initialSR).projectAs(geoSR) startxdd = startPtdd.centroid.X startydd = startPtdd.centroid.Y #print(" -Start point {0} {1}".format(startxdd, startydd)) endPtdd = arcpy.PointGeometry(arcpy.Point(endx, endy) , initialSR).projectAs(geoSR) endxdd = endPtdd.centroid.X endydd = endPtdd.centroid.Y #print(" -End point {0} {1}".format(endxdd, endydd)) tmpTrans = geo # taking full length.segmentAlongLine(startDist, endDist) #tlength = tmpTrans.length #ripNotes = "short" #print("{0} transect # {1}".format(transtype, transID)) print("{0} transect # {1} length: {2} Notes: {3}".format(transtype, transID, tlength, ripNotes)) elif lenTrans >= int(targetLength): #else: ptSCue = arcpy.SearchCursor(tmpRoutePtLoc) field_name = 'MEAS' for routeMeas in ptSCue: snapPtDist = routeMeas.getValue(field_name) print(" ->Measurement along route: {0}".format(snapPtDist)) startDist = snapPtDist - halfTrans endDist = snapPtDist + halfTrans print("\n -Initial startDist: {0} endDist: {1}".format(startDist, endDist)) initialSegment = geo.segmentAlongLine(endDist, startDist) initialSegmentLength = initialSegment.length print(" -->Initial length: {0}".format(initialSegmentLength)) if RoundUp(initialSegmentLength) < int(targetLength): # Check each side to make sure distance is = halfTrans # if not, shift as needed # check length of the segment from the snapPtDist to startDist firstSegment = geo.segmentAlongLine(snapPtDist, startDist) firstSegmentLength = firstSegment.length #print(" -First Length: {0}".format(firstSegmentLength)) #check first if RoundUp(firstSegmentLength) < (halfTrans): print(" -First half is short: {0}m".format(firstSegmentLength)) # calculate the distance it is short shiftDist = halfTrans - firstSegmentLength print(" ..shift distance for first segment: {0}m".format(shiftDist)) #startDist = startDist - shiftDist # short from start to mid, so extend end distance endDist = endDist + shiftDist print(" ..new endDist: {0}m".format(endDist)) else: # first segment ok, must be second part endSegment = geo.segmentAlongLine(snapPtDist, endDist) endSegmentLength = endSegment.length print(" -Second half is short: {0}m".format(endSegmentLength)) # calculate the distance it is short shiftDist = halfTrans - endSegmentLength print(" ..shift distance for first segment: {0}m".format(shiftDist)) # short from end to mid, so start earlier on the line, it a minus measure startDist = startDist - shiftDist #endDist = endDist - shiftDist print(" ..new startDist: {0}m".format(startDist)) # use the startDist and endDist to create the point geometries #startPt = geo.positionAlongLine(startDist) #endPt = geo.positionAlongLine(endDist) startPt = geo.positionAlongLine(startDist) startx = startPt.centroid.X #need DD starty = startPt.centroid.Y #print("Start Dist: {0}".format(startDist)) # midPt = geo.positionAlongLine(snapPtDist) #not using mid points endPt = geo.positionAlongLine(endDist) endx = endPt.centroid.X endy = endPt.centroid.Y tmpTrans = geo.segmentAlongLine(startDist, endDist) tlength = tmpTrans.length del ptSCue # Processing pts to geographic startPtdd = arcpy.PointGeometry(arcpy.Point(startx, starty) , initialSR).projectAs(geoSR) startxdd = startPtdd.centroid.X startydd = startPtdd.centroid.Y #print("start point {0} {1}".format(startxdd, startydd)) #midPtdd = arcpy.PointGeometry(arcpy.Point(midPt.centroid.X, midPt.centroid.Y) , initialSR).projectAs(geoSR) #not using mid points endPtdd = arcpy.PointGeometry(arcpy.Point(endx, endy) , initialSR).projectAs(geoSR) endxdd = endPtdd.centroid.X endydd = endPtdd.centroid.Y #tmpTrans = geo.segmentAlongLine(startDist, endDist) #tlength = tmpTrans.length shortLength = int(targetLength) - 500 if tlength < shortLength: # add A to ID for short trans...add in field. transID = "{0}A".format(transID) if tmpTrans.isMultipart: ripNotes = "contourParts" print(" ** Transect in multiple parts **") else: ripNotes = "contour" #print("{0} trans # {1}".format(transtype, transID)) print("{0} transect # {1} length: {2} Notes: {3}".format(transtype, transID, tlength, ripNotes)) else: break
Edit: added not above the python script..reformatting was not good.
Solved! Go to Solution.
ptScue is an old-style cursor, which behave unpredictably (or predictably bad) when nested. Does it help to switch it to a da cursor? In general, I also suggest using ' with ' syntax on all cursors to avoid them hanging in the void if your run ends in an error.
can you confirm that the else statements on line 2 and 153 are supposed to be indented at the same level?
Good catch on the indentation Dan...I'll fix it. Such is copying and pasting snippets. Actual code have the first else indented four levels.
I should also add that, running an individual point manually works and does create the correct transect, so I know it is not the input random points, DEM, or riparian layers that are an issue. I also should mention it does not always skip that missing that one cursor. There is some combination from the previous point that causes the issue, but looking at the data and output, their is no obvious pattern. I just isolated 3 of the "problem" points and ran them thru the script and they worked fine. That is why I think it might be an issue with the cursors not being lined up correctly, or the actual geometry field "tmpTrans" not being updated to over write the previous values....which then of course make all the coordinates, lengths, etc calcd from it off.
just fyi, thie is what I am appending
transList.append([tmpTrans, ptid, ptX, ptY, transtype, tlength, tlengthKm, transID, elevft, elevm, startxdd, startydd, endxdd, endydd, startxdms, startydms, endxdms, endydms, ripNotes, offset, displayID])
I am kindof with Darren on this I am finding it hard to figure out the cursor stuff in lines 32 to 38. But I am not a cursor person. Darren and Xander have posted some good examples but I don't know how much reshaping would be needed.
Is there anything you can 'def' to consolidate the code now? or is this for a non-deadline time?
ptScue is an old-style cursor, which behave unpredictably (or predictably bad) when nested. Does it help to switch it to a da cursor? In general, I also suggest using ' with ' syntax on all cursors to avoid them hanging in the void if your run ends in an error.
Good catch Darren. I usually always use da cursors. Not sure what I did there. I had noticed I wasn't using the "with" for that one, and it didn't work last week when I was trying...I was "cursor" blind on that one and hadn't noticed. I'll give that a try and be back...
Change those to da cursors helped me get "into" that cursor now, so that is good. I actually rolled back to a version before I made the numerous changes I did this afternoon, made those the changes to with .da, and renamed a few of my cursors, so I wasn't repeating names. Also got rid of my "del cursor" statements, which I had added when trouble shooting.
Anyway, still having issues, but may be able to track things for a bit again. I'll update as I have more.
Thanks again for your feedback Dan and Darren.
I'm giving Darren credit for the answer at this point in time. Even though the da.SearchCursor did not resolve the issue entirely, and is actually working the same, it did help me find where it was having the issue and bandage it for now. (i.e. by adding the "with.." AND a "for..", found it was the "for..." that had the problem....i.e. zero records in cursor). So thanks for finding that Darren.
line 70 and 170 ... is the deletion of the cursor lined up correctly? maybe I will wait until the while stuff is done
Still having issues, but figured out partially what the issue is, but not why.
I did switch the one cursor to a .da version. It took me quite a while to figure out what I was trying to do. The table (not FC) actually only has one record (or is supposed to have 1), and I was just trying to pull a the value from the "MEAS" field. It is coming down to the input to my cursor occasionally had zero records. What is strange is, if I do it manually, I get my one record, even on the problem points, but my program is skipping it for some points for some reason.
The command I am running (sorry about all the variables)
arcpy.LocateFeaturesAlongRoutes_lr(ptGeom, tmpRoute, contourField, nearDist, tmpRoutePtLoc, rid)
Although manually (i.e. in the python window), I do get one record with my nearDist value, and in most cases it works in the script, for some records it doesn't return anything. So now I am testing for the number of records, and although I haven't fixed my duplication problem, I'm at least setting a flag in my notes field for my output line segments...that will help me pull those out for further testing and to fix them.
myCount = int(arcpy.GetCount_management(tmpRoutePtLoc).getOutput(0)) print("the count from Locate Along Route {0}".format(myCount)) if myCount == 0: ripNotes = "ERRORcon"
Giving up for the night, going to run it overnight, maybe with a new set of points, and deal with the problem points tomorrow...which is in about 40m. (sigh)