topic Re: identify the nearest gages downstream from dams in Water Resources Questions

identify the nearest gages downstream from dams

XianyingFan — Wed, 15 Jun 2016 14:55:49 GMT

Help please:

Task: identify the nearest gages downstream from dams

Data: Stream Flowline (line feature class), streamgage point feature class, dam point feature class (both point feature classes contain thousands of points)

What I did: Using geometric network analysis tools to first create a geometric network, then set flow direction and finally trace geometric network, I got the output of a group layer.

My questions: I am hoping to add a new field which contains the ID of the nearest gage point for each dam point in my dam point attribute table, is that possible to realize this using the output of tracing geometric network? How can I realize it?

Thank you so much!

Maggie

Re: identify the nearest gages downstream from dams

BillDaigle — Wed, 15 Jun 2016 15:14:56 GMT

Been a while since I have done this. It is certainly possible. Conceptually, you will want to add all of your dams as flags (starting points) and gauges as barriers (end points). When you run the analysis, you will get an output of lines for each set of start and end points. I believe you can join the line output back to the dam feature class if you want to populate the downstream gauge id.

Re: identify the nearest gages downstream from dams

XianyingFan — Wed, 15 Jun 2016 15:27:38 GMT

Thank you so much for your reply Bill!

Do you mean to join the line output with the original dam feature class? How can I get the closest downstream gauge id and add it to dam's attribute table?

So you've done this to identify the nearest downstream points for many upstream points in one time and we don't need to use python to automate?

Re: identify the nearest gages downstream from dams

BillDaigle — Wed, 15 Jun 2016 15:40:02 GMT

Yes, you should be able to do this without python. I think it is quite simple once you figure out how to add all of the flags and barriers at one time. I think if you select all of the dams there is an option to add then all as flags, then you can do the same with gauges as barriers.

The table for the output lines should have a should have a field with the OID (or FID) for the dam and the OID (or FID) for the associated gauge. You may need to fiddle with analysis setting so it only find the first (closest) barrier.

Re: identify the nearest gages downstream from dams

XianyingFan — Wed, 15 Jun 2016 15:55:06 GMT

Yes, I am able to select all of the dams points and gauges points with tracing geometric network tool. Do you mean all the information are contained in the output lines attribution table, including the ID(OID or FID) of dams as well as gauges? The computer I am using now has no ArcMap, I will try later. Can I ask you if I have further questions later? Thank you so much Bill!!

Re: identify the nearest gages downstream from dams

BillDaigle — Wed, 15 Jun 2016 15:57:38 GMT

Yes, the info you’ll need should be in the attribution table. I will do what I can to help. I haven’t done it in a while.

Re: identify the nearest gages downstream from dams

XianyingFan — Thu, 16 Jun 2016 21:27:11 GMT

Hi Bill, I use "trace geometric network" tool and set the dam points as flags, the gage points as barriers, I also disable gage point feature class so that it will stop trace when there is gage point. Then I limit the results to features stopping the trace. The output I get are the layers with selection features. Then I create layers from the selected features for both dam and gage point feature classes. But from the result I don't know which gage point are paired with dam point(nearest along the streamline) and what should I do next so that the nearest ID of gage point can appear in dam point attribution table.

Can you please recall which tools you used to realize the task? Is there any record available as my reference? May I write email to you or call you directly if possible? Thank you so much!

Maggie

Re: identify the nearest gages downstream from dams

BillDaigle — Thu, 16 Jun 2016 22:37:58 GMT

Hey Maggie

Now it’s coming back to me. It’s the network analyst toolset (instead of the geometric network tools…very confusing) that return an attribute table instead of a selection. The network analyst tools are designed for working with road networks, but also work for stream networks.

From here it seems like you have 2 options:

1. Use python or model builder to iteratively select each dam and run a the trace tool with one dam selected at a time.

2. Build a network dataset and use those tools.

From what I remember, setting up the other type of network dataset was significantly more complicated, but the tools are little more robust. Personally, I think I would lean towards option 1.

Re: identify the nearest gages downstream from dams

XianyingFan — Thu, 16 Jun 2016 22:58:35 GMT

Thanks a lot, Bill!

If using network analyst toolset, how can we set the flow direction? Is there a tracing tool under network analyst toolset? I can't find it.

Which option did you do to get the result? Option 1? Do you have more details? Can you share with me the python script? Since there are many dam points, we have to use loop, right? I just started learning python recently and it's so difficult for me to write script with no bugs....Thank you so much for your help!

Maggie

Re: identify the nearest gages downstream from dams

XianyingFan — Fri, 17 Jun 2016 20:15:59 GMT

Hi Bill, did you see my last comment? I tried to build network dataset, however, I encounter an error: "Network was build, but with some errors. The network can still be used in analysis, however the invalid features will not participate in the network", when I opened the "show build errors", I found that all my dam and gage points are listed there, so those points weren't able to participate in the network. Did you encounter this problem before?

Re: identify the nearest gages downstream from dams

BillDaigle — Wed, 22 Jun 2016 22:38:08 GMT

Hi Maggie,

Sorry for the lapse in response. I've been out for a few days. If you are still struggling, could you zip up a portion of you're network dataset and share it? It has been a while since I've worked with one and I don't seem to have any good examples for you.

Re: identify the nearest gages downstream from dams

XianyingFan — Wed, 22 Jun 2016 22:52:48 GMT

Hi Bill,

Excited to see you're back! How can I share the zip file? I saw insert image, insert video, insert link, but no insert files. Can I send directly to your email? Many thanks!

Re: identify the nearest gages downstream from dams

BillDaigle — Thu, 23 Jun 2016 14:01:25 GMT

Use the "Advanced Editor" option, then you will see an "Attach" link in the lower right hand corner.

Re: identify the nearest gages downstream from dams

XianyingFan — Thu, 23 Jun 2016 14:59:49 GMT

Thank you very much Bill! Here is the data.

For the two options that you mention:

Option1: I am able to trace downstream of one dam point using the Geometric Network Analysis Toolset and then join the selective line features with gage points to get the closest gage point info. My difficulty is to write the python code of the tracing process and automate to other points;

Option 2: I accept the logic of treating the flowline as one way street and use the Network Analysis Toolset (specifically: find closest facilities tool) to find the closest points for all dam point at one time, however I failed to get the result because I can't find a way to set the direction so that it will find the closest facilities downstream along the flowline.

Re: identify the nearest gages downstream from dams

BillDaigle — Thu, 23 Jun 2016 18:39:56 GMT

Let’s focus on option 1 first. Can you send me a python snippet from the geoprocessing result? If you run the tool, then find the geoprocessing task in the Results window, you can get a python snippet by right clicking on the result object. I think I can get you started on a python script without actually setting up the network on my end.

Re: identify the nearest gages downstream from dams

XianyingFan — Thu, 23 Jun 2016 20:53:20 GMT

Hi Bill, with the help of an incredibly nice friend's help, we have a draft python code to work with.

The attached includes a test geodatabase & a test script (traceGeometric.py). Unfortunately, the script works, so far, on the first feature (dam) only. It does assign a closest gage ID. By the way, there are two fields with reachcode info. of gage points because I snapped it to flowline feature class and I think using REACHCODE is the right field as gages ID instead of REACHCODE2.

Here is what she wrote about the current issue of the code:

---------------------------

After the first dam, "TraceGeometricNetwork" tool fails. The problem is, I think, the output Network/layer group. In m script, the output has to be the same as the input network, Hydro_Net. I've seen other examples and they seem to be able to create a new output. I think it would let the script to run against the rest of the dams if I can use a different output (but not so sure). The tool seems to be very picky, unfortunately. Anyway, I will share what I have so far and let you play with the script.

PS: my script runs directly from Python shell (2 not version 3, because ArcGIS doesn't use 3..), without opening ArcGIS (but you have to have ArcGIS on your local machine) and it's on Windows - below is the screenshot of the output.

-----------------------------

Can you please take a look at our data and python code how can we address the issue? Many thanks!

Re: identify the nearest gages downstream from dams

BillDaigle — Thu, 23 Jun 2016 22:01:33 GMT

Can you add the python script? It didn’t get included with the toolbox.

Re: identify the nearest gages downstream from dams

BillDaigle — Sat, 11 Dec 2021 14:39:59 GMT

I think this is probably what you need. You can run it independent of ArcMap. It should scale up pretty easily.

import arcpy, os

pathToGdb =  r'C:\temp\HydroNet\TraceGeometric\test.gdb'
geoNetwork = os.path.join(pathToGdb,'Hydro','Hydro_Net')
damsFC = os.path.join(pathToGdb,'Hydro','DamPoints')
gagesFC = os.path.join(pathToGdb,'Hydro','GagePoints')

temporaryOutputGroupLayerLoc = r'in_memory\outGroup'

#loop through each dam
with arcpy.da.SearchCursor(damsFC,['SHAPE@','OBJECTID']) as cursor_Dam:
    for row_Dam in cursor_Dam:
        print 'Current Dam ObjectID:',row_Dam[1]
        startGeom = row_Dam[0]

        #run the trace on a single dam
        arcpy.TraceGeometricNetwork_management(
            geoNetwork,
            temporaryOutputGroupLayerLoc,                                # using an in_memory workspace lets you access it via python
            startGeom,
            "TRACE_DOWNSTREAM",
            gagesFC,
            "#","#","#","#",
            "TRACE_ENDS")

        #loop through the output group layer to find the gages layer
        for layer in arcpy.mapping.Layer(temporaryOutputGroupLayerLoc): #referencing Group Layer in_memory
            if layer.name == 'GagePoints':
                #loop through the selected gages
                with arcpy.da.SearchCursor(layer, 'OBJECTID') as cursor_selectedGages:
                    for row_gage in cursor_selectedGages:
                        print '  Downstream Gage ObjectID',row_gage[0]

        #delete the layer temporary group layer before starting the next dam
        arcpy.Delete_management(temporaryOutputGroupLayerLoc)

Re: identify the nearest gages downstream from dams

XianyingFan — Thu, 23 Jun 2016 23:03:38 GMT

Thank you so much Bill! We will try in our computer and get back to you if there is any further question!