Geoprocessing Flow Accumulation Tool Output Issues

[JCann]

I am currently running ArcGIS Pro 2.9.1 on my personal computer and ArcGIS Pro 2.9.1.179945 on my work computer. The geoprocessing Flow Accumulation tool is producing a very different output from the Arcpy based tool (and from the Flow Accumulation output that was generated for the project in 2.8.X, the project has spanned 2.8.X to 2.9.X). This is occurring on both computers, which have entirely separate installations of ArcGIS Pro and have had the DEMs downloaded to each, not transferred between the two.

The first screenshot is the output from the Arcpy based tool, (purple lines). This output matches the previous output generated in 2.8.X. The filled DEM, flow direction, and flow accumulation in the first screenshot were generated using Arcpy

The second screenshot is from the Flow Accumulation geoprocessing tool(red lines). The "missing" lines are not isolated to one section, the gaps occur across two mosaiced DEMs downloaded from USGS 3DEP (converted from .tif to geodatabase raster and projected to NAD 1983 2011 UTM Zone 10 N). The Flow Accumulation in the second screenshot used a Flow Direction Raster generated in the Geoprocessing toolbox from the same filled DEM that was used in the first screenshot and the same threshold (3000).

I have also replicated this with a clip of the mosaic, on both individual DEMs before mosaicing, and on a Topo To Raster generated from a single DEM (I have also re-downloaded the DEM's re-ran just to be sure). I also tested the Watershed Tools in Raster Functions and was able to get a "correct" output, but after I ran those tools, Pro crashed and is failing to initialize (work computer, whole other issue)

Has anyone else run across this recently? Is this an issue with the geoprocessing tool, or am I just missing something?

Output from Arcpy

Output from Geoprocessing Toolbox

[tlopezcantu]

Hello!

I am sorry that you are experiencing issues with Flow Accumulation.

I have used USGS 3DEP elevation data before in both arcpy and geoprocessing tool in the software version that you mentioned. So, the problem could be related to the specific data input to the workflow or any preprocessing steps.

To try to narrow down and identify the source of the problem, I would like to ask you a few questions:

- What is the output format, and did you specify a scratch workspace?  

- From the images that you shared, it looks like the output of the Geoprocessing tool is "incomplete", but not entirely wrong. By this, I mean that I see some similarities between the red lines and the purple lines. Are you using the Con tool to create the stream lines? Could you try using Reclassify or Greater than tools? or use a different threshold on the Geoprocessing Flow Accumulation output to see the resulting lines?

- What is the cell size of your original DEM data? and of your projected individual rasters and mosaic raster dataset?  Please make sure that the threshold that you are specifying is adequate for the raster cell size.

- Finally, how does the Flow Accumulation output compares between arcpy and geoprocessing tool? Is there any difference at all if you use the Minus tool? 

I will follow up with you requesting additional information, if the problem still persists.

Thank you,
Tania

[JCann]

Tina,

Output format is file geodatabase raster into the same file geodatabase that the DEM(s) and Flow Direction raster are stored. No scratch workspace.  The cell size for all rasters, from the initial downloaded .tif from the USGS,  to the final flow accumulation is "10" meters, with the actual cell size in properties being 9.03 x 9.03. I am not inputting a cell size into any tool, so the 9.03 x 9.03 is for everything. 

This project needs the stream lines to closely match the NHD flowlines for the area of interest due the the stream lines being used for watershed delineation and the NHD flowlines being used in the riparian buffer delineation.  For this project, a threshold value of 3000 was used. The stream lines in the screen shot are the "initial stream lines", direct from the flow direction raster, using Symbology, Classify, Natural Breaks, 2 classes, first class 0.001-3000, second  class 3000.001 - upper value. 
I use the Con tool from there to create the stream lines once I have settled on a threshold value. I usually use a process of both picking values, 100-5000, and seeing how they look and calculating a percentage of the maximum flow accumulation as another method.

I have not tried Greater Than, Reclassify, or Minus tools yet, in part due to needing a repeatable process that can be scaled up and down using the same tools, but varying threshold values depending on project size. As far as values between the flow accumulation from geoprocessing and Arcpy, if I remember correctly, there was no difference in values (I am at work and do not have the dataset with me). I found it odd that the flow accumulations values were the same, but the lines incomplete on the geoprocessing output.
The above process worked in 2.8.X for the initial stream lines and I found this issue when I needed to expand my project into an adjacent DEM and create more stream lines. I did test various thresholds to see if that made a difference, but it did not, and this case, a different threshold would mean redoing this portion of the project since the same value needs to be used across the whole project. I can provide more once I am not at work if needed. 

Thanks, 

John 

[tlopezcantu]

Hi John,

Thank you for your answers! I want to reiterate that when you use arcpy and Flow Accumulation tool from Geoprocessing Panel, it is the same operation. Based on your answers, I think there could be these two different things happening:

(1) The environment settings set for the arcpy execution are different than those set for the Flow Accumulation Geoprocessing tool.

(2) If the arcpy output and the Geoprocessing tool Flow Accumulation outputs have similar ranges, then I believe the issue is not in Flow Accumulation, but most likely from Con tool.

Could you please verify that you are running both arcpy and Flow Accumulation tool from Geoprocessing panel with the same environments, as well as try the Reclassify or Greater than tools? Reclassify or Greater than tool can also be part of your automated workflow and there will be no issue to make your work repeatable.  While in Con tool you specify the expression "Where VALUE > 3000", in Greater than tool, for the first input you will choose the Flow accumulation raster and for the second input you can directly write 3000. 

Please let me know if this worked for you.

Best regards,

Tania

[JCann]

A quick reply and I will check a few other things tomorrow. I should have clarified earlier; by Arcpy, I meant that I wrote a stand alone Python script, in Visual Studio Code, using the Python version of Flow Direction and Flow Accumulation. The VS Code script replicated the inputs and environments of the tools when ran in the geoprocessing pane, but with unique outputs into the same file geodatabase. Neither set of stream lines in screen shot was run through the Con tool, the stream lines were generated by the previously described classify process in symbology, my pre Con tool workflow. The missing lines screen shot is from the built-in geoprocessing tool, the "correct" stream lines are from the stand alone Python Script. I also ran Flow Accumulation from the raster functions pane, and was able to replicate the output from stand alone script. The only output that has not be accurate, is the one from the geoprocessing tool. I will run further test with Con, Greater, etc. tomorrow. 

Thanks, 

John