I'm pretty sure I just have a syntax or indentation issue, but I've been trying to track this down a good chunk of the day, so time to "rubber duck". The full program is a bit more than anyone will be able to deal with, but if anyone really wants to look at the full code, I can upload it. Typing is out is sometime helpful for me too....so here's hoping.
Basic (and simplified) description: given a random point with a "transtype" Riparian or Contour,
For 2000 random points, it finished with no error, but for some reason about 380 points (or 279 of those 380 points) are duplicating the transect above it. I do have multiple cursors, which for the most part seem to be fine, but one of the cursors has the "for aRoute2 in route2:" being bypassed. I believe this is the issue.
So, with that all said, here is the part that I think is the issue. Not sure that this will be of any used, but in case someone sees something obvious... I'm on a very short deadline....i.e., here tonight until I get this fixed, most likely. So any help is appreciated, but may not respond right away.
Dan Patterson Curtis Price Darren Wiens Jake Skinner Joshua Bixby any of you seeing anything obvious? I know cursors within cursors is n necessarily a good idea, and have no time to change it in a major way (another time), I'm in triage mode right now. Thanks for any help you might offer. I'll by you a free drink if I ever see you at the dev summit or UC.
NOTE: trying to fix the indentation in geonet is messing it up more....the first else: and everything else until the final else is indented....
# snippet...too much needed to send all and expect it to work else: # transtype == 'contour' # ############## # Contours # ############## print(" -Create contour for elev {0} m".format(elevm)) ContourList(inDEM, fullContour, elevm) #print(" -Clipping full contour by study area") arcpy.Clip_analysis(fullContour, inStudy, clipContour) #print(" making clipped contour FL") arcpy.MakeFeatureLayer_management(clipContour, "tmpClipContour") print(" -Selecting contour segs > 5000 m") #arcpy.SelectLayerByAttribute_management("tmpClipContour", "NEW_SELECTION", where_clause="Shape_Length >= 5000") arcpy.SelectLayerByAttribute_management("tmpClipContour", "NEW_SELECTION", "Shape_Length >= 5000") # copies selection to new FC #print(" - Copying selection to new FC") # here 4/4/2016 arcpy.CopyFeatures_management("tmpClipContour", tmpContourFC) arcpy.MakeFeatureLayer_management(tmpContourFC, "tmpContourFL") # # wrong? tmpContourFL = arcpy.mapping.Layer("tmpClipContour") #print("\n**New selection of nearest contour segment to pt") #print(" - Creating new feature layer from origPtFL") arcpy.SelectLayerByLocation_management("tmpContourFL", nearType, "origPtFL", nearDist) #print(" - Copy selection to new tmp FC contourSel") arcpy.CopyFeatures_management("tmpContourFL", contourSel) # Creates route of the selected contour segment print(" -Create temp route; create and process cursor") arcpy.CreateRoutes_lr(contourSel, contourField, tmpRoute) #print(" - Create SearchCursor from route") routeSCur2 = arcpy.da.SearchCursor(tmpRoute, ["SHAPE@", "Contour"]) print("getting here") with routeSCur2 as route2: print("getting inside cursor") # will get to here for aRoute2 in route2: print("getting here2") # ####### will not always get to here ####### geo = aRoute2[0] # get all contour elevation for the route theElev = aRoute2[1] # get elevation, actual contour as a route #snapPt = (geo.snapToLine(ptGeom)).centroid # shp_fld arcpy.LocateFeaturesAlongRoutes_lr(ptGeom, tmpRoute, contourField, nearDist, tmpRoutePtLoc, rid) lenTrans = geo.length print(" **Contour elev {0}: {1} m".format(theElev, RoundUp(lenTrans))) flyFullContour = lenTrans <= int(targetLength) if flyFullContour: print(" **Taking full contour: {0}m ".format(lenTrans)) startx = geo.firstPoint.X starty = geo.firstPoint.Y endx = geo.lastPoint.X endy = geo.lastPoint.Y startPtdd = arcpy.PointGeometry(arcpy.Point(startx, starty) , initialSR).projectAs(geoSR) startxdd = startPtdd.centroid.X startydd = startPtdd.centroid.Y #print(" -Start point {0} {1}".format(startxdd, startydd)) endPtdd = arcpy.PointGeometry(arcpy.Point(endx, endy) , initialSR).projectAs(geoSR) endxdd = endPtdd.centroid.X endydd = endPtdd.centroid.Y #print(" -End point {0} {1}".format(endxdd, endydd)) tmpTrans = geo # taking full length.segmentAlongLine(startDist, endDist) #tlength = tmpTrans.length #ripNotes = "short" #print("{0} transect # {1}".format(transtype, transID)) print("{0} transect # {1} length: {2} Notes: {3}".format(transtype, transID, tlength, ripNotes)) elif lenTrans >= int(targetLength): #else: ptSCue = arcpy.SearchCursor(tmpRoutePtLoc) field_name = 'MEAS' for routeMeas in ptSCue: snapPtDist = routeMeas.getValue(field_name) print(" ->Measurement along route: {0}".format(snapPtDist)) startDist = snapPtDist - halfTrans endDist = snapPtDist + halfTrans print("\n -Initial startDist: {0} endDist: {1}".format(startDist, endDist)) initialSegment = geo.segmentAlongLine(endDist, startDist) initialSegmentLength = initialSegment.length print(" -->Initial length: {0}".format(initialSegmentLength)) if RoundUp(initialSegmentLength) < int(targetLength): # Check each side to make sure distance is = halfTrans # if not, shift as needed # check length of the segment from the snapPtDist to startDist firstSegment = geo.segmentAlongLine(snapPtDist, startDist) firstSegmentLength = firstSegment.length #print(" -First Length: {0}".format(firstSegmentLength)) #check first if RoundUp(firstSegmentLength) < (halfTrans): print(" -First half is short: {0}m".format(firstSegmentLength)) # calculate the distance it is short shiftDist = halfTrans - firstSegmentLength print(" ..shift distance for first segment: {0}m".format(shiftDist)) #startDist = startDist - shiftDist # short from start to mid, so extend end distance endDist = endDist + shiftDist print(" ..new endDist: {0}m".format(endDist)) else: # first segment ok, must be second part endSegment = geo.segmentAlongLine(snapPtDist, endDist) endSegmentLength = endSegment.length print(" -Second half is short: {0}m".format(endSegmentLength)) # calculate the distance it is short shiftDist = halfTrans - endSegmentLength print(" ..shift distance for first segment: {0}m".format(shiftDist)) # short from end to mid, so start earlier on the line, it a minus measure startDist = startDist - shiftDist #endDist = endDist - shiftDist print(" ..new startDist: {0}m".format(startDist)) # use the startDist and endDist to create the point geometries #startPt = geo.positionAlongLine(startDist) #endPt = geo.positionAlongLine(endDist) startPt = geo.positionAlongLine(startDist) startx = startPt.centroid.X #need DD starty = startPt.centroid.Y #print("Start Dist: {0}".format(startDist)) # midPt = geo.positionAlongLine(snapPtDist) #not using mid points endPt = geo.positionAlongLine(endDist) endx = endPt.centroid.X endy = endPt.centroid.Y tmpTrans = geo.segmentAlongLine(startDist, endDist) tlength = tmpTrans.length del ptSCue # Processing pts to geographic startPtdd = arcpy.PointGeometry(arcpy.Point(startx, starty) , initialSR).projectAs(geoSR) startxdd = startPtdd.centroid.X startydd = startPtdd.centroid.Y #print("start point {0} {1}".format(startxdd, startydd)) #midPtdd = arcpy.PointGeometry(arcpy.Point(midPt.centroid.X, midPt.centroid.Y) , initialSR).projectAs(geoSR) #not using mid points endPtdd = arcpy.PointGeometry(arcpy.Point(endx, endy) , initialSR).projectAs(geoSR) endxdd = endPtdd.centroid.X endydd = endPtdd.centroid.Y #tmpTrans = geo.segmentAlongLine(startDist, endDist) #tlength = tmpTrans.length shortLength = int(targetLength) - 500 if tlength < shortLength: # add A to ID for short trans...add in field. transID = "{0}A".format(transID) if tmpTrans.isMultipart: ripNotes = "contourParts" print(" ** Transect in multiple parts **") else: ripNotes = "contour" #print("{0} trans # {1}".format(transtype, transID)) print("{0} transect # {1} length: {2} Notes: {3}".format(transtype, transID, tlength, ripNotes)) else: break
Edit: added not above the python script..reformatting was not good.
Solved! Go to Solution.
Rebecca:
It appears you have created code that uses embedded cursors. The code must be rewritten to eliminate that configuration if you want to make it process efficiently, since embedded cursors are the absolute worst way to approach any problem. Instead one of the cursors should load the data to a dictionary for any look up of data and only one update cursor should be processed at a time. The code revision can result in a 500 fold improvement in speed if you are processing 10,000 records against 10,000 records, since my approach would only have to process 20,000 record reads while an embedded cursor would process up to 10,000,000 record reads..
Hi Richard,
Thanks for the comments, and I know there may be a more efficient ways to do this, but the embedded cursors in-and-of-themselves are not the issue, unless I have an indent off somewhere, i.e. the updating of my "tmpTrans" variable in the incorrect place. The issue seem to be that with some points, because I eliminate contour-parts that are < 5000m before making it into a route, it is no longer finding a "location along the route" within my search distance, which should be adequate. (BTW the contour is created on the fly from the point's elevm value),
Now that I fixed the two SearchCursor to the da.SearchCursor that I had missed, I was able to see that it was still crashing at the "for row in cursor:", but was making into the "with..." for the cursor. That lead me to check what was being created in the table feeding the cursor, and then setting a flag if the count=0. I need to back track further to see why it is not getting any records, since stepping thru it manually works, but I'm sure I am just missing something. At that point, I will need to set a "fix" by maybe dropping the elevm value a couple meters and trying again. Our DEM is not as accurate as lower-48, and these are traversed via plane (SuperCub) so a couple meters will not matter in the scheme of things. Also, the missed points typically are near the top of a hill/mountain.
BTW - we used to have many more of these issues in river valleys, so I added a "riparian" option that grabs the stream and gives and offset distance and direction from the stream...that is working. We labelled these as "skipped points" and dealt with them in the field.
After the crunch/field time is over, and for my own satisfaction, I will look into using dictionaries etc., and to resolve this issue fully. But since I was working on my logic from Avenue and AML programs, and the lead was changing/adding new requirements often, I did what I understood, and what I think is important, what the next person might be able to read and understand, if they are new to python. With that said, dictionaries and other solutions may be the way to go on a rewrite....but that will be all on my time....so a fun project.
Just an FYI - I ran this on 1250 points last night...74 came up with the flag. At this point, these would be similar to the older "skipped points" and I can put them and deal/test them directly.
I know this is more than anyone cares about fo this project, but it is good for me to think it out by typing.