Hello,
This is a going to be a long shot, but I am looking for advice or website material anything to help with an issue I have. This is a stormwater ArcGIS Pro issue.
The issue:
Research question:
Worst case I manually input the feature ID for each pipe segment, but I am sure there must be something to help jumpstart the process.
Thank you an advance 🙂
I think we're talking slightly tangential things, here:
1. What linkage IDs are worth having
At minimum, I'd generally recommend what you already have: Upstream ID and Downstream ID for your Line Features (i.e., Pipes). For probably 90% of use-cases, this is probably all you need. You might also consider an Outlet ID from your Polygon Features, like a detention pond. That's not handled at all by the code I provided below, though.
In our organization, we've also experimented with a suite of "helper" fields on our Point Feature, because it helps some of our older-school field people when they're locating new Assets. The goal was to not make them digitize a line feature from the field, so we needed a place to put the most critical datapoints for a newly-discovered pipe, like Material & Diameter. That evolved outward into back-populating the ID of the Pipe into one of those "helper" fields, once it's created. Downside is that it massively bloats your Point Feature's schema. It's a headache that I don't recommend doing lightly.
2. How to update linkage IDs for new features, as they're created:
Go with the code @RhettZufelt linked elsewhere in this thread, and build it as an Attribute Rule. It might or might not need a little retooling, depending on what linkage IDs you end up using. At a glance, it will work pretty much as-is, if all you have is Upstream ID / Downstream ID for a line feature.
(Sidebar: Thanks for finding that, Rhett—that step was in my near future and it's nice to not have to reinvent the wheel!)
3. How to update linkage IDs for existing features, right now:
A few assumptions/caveats/notes on this code, before I get to it:
See the SPOILER below for the code, and happy automating!
"""Stormwater Upstream/Downstream Locator Tool
Author: MErikReedAugusta (ESRI Community)
Python: 3.9
ArcGIS Pro: 3.1.3
Scans the provided Storm Structure feature to generate a list of nodes, then matches those
nodes to the up/downstream ends of the provided Storm Pipe feature, and logs the relevant
Storm Structure ID.
"""
# IMPORTS Standard Library
# IMPORTS 3rd Party
import arcpy
##############################################################################################################################
# INPUTS
##############################################################################################################################
# The Feature Classes you intend to target with this tool
structure_Feature = None
pipe_Feature = None
# These are the ID Fields for each Asset ON THEIR RESPECTIVE FEATURE (This is their "self" ID)
structure_IDField = None
pipe_IDField = None
# These are the Upstream/Downstream ID Fields on the Pipe Feature
upstream_IDField = None
downstream_IDField = None
##############################################################################################################################
# CLASSES & EXCEPTIONS
##############################################################################################################################
class NodeNotFoundError(KeyError):
"""Exception raised when a specified Node couldn't be found in the NodeMap."""
pass
class DuplicateNodeError(KeyError):
"""Exception raised when a specified Node already exists in the NodeMap."""
pass
class NodeObj():
"""Defines a geographic point of significance for the geometry of an Asset in 2-dimensional space."""
def __init__(self, x, y):
self.X = x
self.Y = y
self.ID_Structure = None
class NodeMap():
"""A searchable list-like database of all Nodes encountered."""
def __init__(self):
self._data = []
#################################################################################################
# NodeMap Navigation
#################################################################################################
def HasNode(self, nodeX, nodeY):
"""Returns True if the provided X/Y Coordinate is found in the NodeMap."""
# Get a shortened list of just the nodes that match your coordinates.
matchingNodes = [node for node in self._data if node.X == nodeX and node.Y == nodeY]
# If the list is 0, then those coordinates aren't here. If it's bigger, then they are.
return len(matchingNodes) > 0
def GetNode(self, nodeX, nodeY):
"""Given a provided coordinate pair, this finds and returns the first matching NodeObj in the NodeMap."""
for node in self._data:
if node.X == nodeX and node.Y == nodeY:
return node
# If you're still here, then no nodes have matched. Probably should've checked first! Have an Exception, instead.
raise NodeNotFoundError()
def AddNode(self, nodeObj):
"""Adds the provided Node Object to the NodeMap."""
if self.HasNode(nodeObj.X, nodeObj.Y):
raise DuplicateNodeError()
self._data.append(nodeObj)
#################################################################################################
# Read/Write Assets
#################################################################################################
def ReadPoints(self):
"""Reads the Point Feature Class provided above and converts it into Node Objects."""
# Sets up an ArcGIS Progressor, if you're running this in a Tool
totalRows = int(arcpy.management.GetCount(structure_Feature)[0])
stepFrag = 10 ** (len(str(totalRows)) - 2)
arcpy.SetProgressor('step',
message='Populating Nodes from Structures...',
min_range=0,
max_range=totalRows,
step_value=stepFrag,
)
progCount = 0
# Iterates through the Structure Table, reading Nodes
with arcpy.da.SearchCursor(structure_Feature, ['OID@', structure_IDField, 'SHAPE@']) as cursor:
for oid, assetID, geo in cursor:
# Update the Progressor as we go, if this is in a Tool
progCount += 1
if progCount % stepFrag == 0:
arcpy.SetProgressorPosition()
arcpy.SetProgressorLabel(f'Reading Feature Class for Assets: {progCount: >{len(str(totalRows))}}/{totalRows}...')
# Do the actual work of pulling the Node
node = NodeObj(geo.firstPoint.X, geo.firstPoint.Y)
node.ID_Structure = assetID
self.AddNode(node)
# Clean up the node object(s) for safety & paranoia, so nothing gets forwarded where it shouldn't
del node
def WriteLines(self):
"""Reads the endpoints of the Line Feature Class provided above and retrieves the specified Node, if applicable."""
# Sets up an ArcGIS Progressor, if you're running this in a Tool
totalRows = int(arcpy.management.GetCount(pipe_Feature)[0])
stepFrag = 10 ** (len(str(totalRows)) - 2)
arcpy.SetProgressor('step',
message='Populating Nodes from Structures...',
min_range=0,
max_range=totalRows,
step_value=stepFrag,
)
progCount = 0
# Iterates through the Structure Table, reading Nodes
with arcpy.da.UpdateCursor(pipe_Feature, ['OID@', pipe_IDField, upstream_IDField, downstream_IDField, 'SHAPE@']) as cursor:
for oid, assetID, frID, toID, geo in cursor:
# Update the Progressor as we go, if this is in a Tool
progCount += 1
if progCount % stepFrag == 0:
arcpy.SetProgressorPosition()
arcpy.SetProgressorLabel(f'Reading Feature Class for Assets: {progCount: >{len(str(totalRows))}}/{totalRows}...')
# Look for the Upstream Node and read the ID there; write a sentinel value if it fails
try:
nodeObj_UP = self.GetNode(geo.firstPoint.X, geo.firstPoint.Y)
frID = nodeObj_UP.ID_Structure
except NodeNotFoundError():
frID = 'ID_Not_Found'
# Look for the Downstream Node and read the ID there; write a sentinel value if it fails
try:
nodeObj_DN = self.GetNode(geo.lastPoint.X, geo.lastPoint.Y)
toID = nodeObj_DN.ID_Structure
except NodeNotFoundError():
toID = 'ID_Not_Found'
cursor.updateRow((oid, assetID, frID, toID, geo))
# Clean up the node object(s) for safety & paranoia, so nothing gets forwarded where it shouldn't
del nodeObj_UP, nodeObj_DN
##############################################################################################################################
# MAIN
##############################################################################################################################
def Main():
arcpy.AddMessage('Begin.')
nodeMap = NodeMap()
arcpy.AddMessage('Populating Nodes from Structures...')
nodeMap.ReadPoints()
arcpy.AddMessage('Updating Linkage Fields in Pipes...')
nodeMap.WriteLines()
if __name__ == '__main__':
Main()
I have done this in the past with using a searchcursor to get the SHAPE@ field. With the SHAPE@ object you can get firstpoint/lastpoint of the line. ( line[1].lastPoint.X,line[1].lastPoint.Y )
I then iterate through each line and grab the first point (from the line) and use that point to make a feature layer so that I can use it to Select by Location the appropriate manhole ad get it's ID. Then I use an update cursor to assign this MH id to the UpstreamMH field.
Then, Select by Location using the last point, grab that MH id and update cursor to the DownstreamMH field.
I would think you could do this fairly easy with Arcade as well using the paths of the line, but have not looked into it. The main benefit of using Arcade is that you could also use it to make an attribute rule that will populate these values automagically in the future when editing/adding new features.
R_
To add a little more, my data has separate featureclass for mahnoles or cleanouts on the sewer, and manholes, cleanouts, inlets, outlets, discharge, etc. on the storm.
So, to make the script easier so I only have to intersect with one layer, I run an append operation that copies over ALL the respective point features to a single temporary featureclass with the feature ID.
R_
I was going to suggest the same thing! Glad I’m on the right track.
It looks like this post shows how to do that with Arcade. Is for an attribute rule, but shows the syntax that could be modified run in field calculator (or use as a rule 🙂
R_