topic Re: Flow direction data error? in Water Resources Questions

Flow direction data error?

sarahoutekamer — Thu, 22 Mar 2018 08:04:19 GMT

for a school group project we were given a pre processed DEM built from lidar point cloud.

We have had tons of issues with the DEM (.tiff) due to the size of the file but have been able to run the tools by cutting out the required data and re-sampling call size to 5m.

After running fill sinks and flow direction the data shows a uniform grid that was undetectable in the DEM and Slope calculations.

I am curious if anyone has dealt with this before, or have any idea if this could come from some rendering issues with the large data, or if there is some kind of error in the data.

(The project is research based. I'm not just looking for easy answers for a lab assignment)

Thanks!

Re: Flow direction data error?

DanPatterson_Retired — Thu, 22 Mar 2018 09:31:46 GMT

What resampling method did you use? http://pro.arcgis.com/en/pro-app/tool-reference/data-management/resample.htm

Given the data are not categorical, did you examine the remaining options and their affect on the resultant dem prior to filling sinks.

What do you think would be the cause of the rectilinear pattern evident in the first two images?

Are they the result of any initial preprocessing to produce the dem or are they the result of resampling and other processing steps?

These are some of the questions I would ask if I had given the project out

Re: Flow direction data error?

XanderBakker — Thu, 22 Mar 2018 11:44:26 GMT

When you zoom in to the flow direction raster, does the grid remain visible? You may need to calculate your pyramids again: Build Pyramids—Help | ArcGIS Desktop

Re: Flow direction data error?

sarahoutekamer — Fri, 23 Mar 2018 04:08:40 GMT

We actually got the lines in the original interpolation and then re-sampled to try and eliminate them.

We don't actually know what other pre-processing was done... the DEM was given to us from an outside organization.

(I wish that this was a given assignment because then my prof. could help. we are doing a final research project on modeling changes in stream flow post fire to measure the effect on current infrastructure.)

Re: Flow direction data error?

sarahoutekamer — Fri, 23 Mar 2018 04:11:35 GMT

Yes they stay visible throughout. Which leads me to believe there must be a error in the data, because rendering issues would remedy itself at large scales (Maybe I dunno).

I will try and give the pyramids a shot!

thanks

Re: Flow direction data error?

DanPatterson_Retired — Fri, 23 Mar 2018 04:16:33 GMT

Sara, that is why I asked what kind of resampling you did. The bilinear and cubic should/might have smoothed out some of those artifacts.

Re: Flow direction data error?

sarahoutekamer — Fri, 23 Mar 2018 04:39:37 GMT

I was not sure what re-sampling my team member used. I'm sure he used nearest neighbor just because we where having trouble with the size.

I tried to run both cubic and bi linear and I got the same patterns but not in the same locations......

Re: Flow direction data error?

sarahoutekamer — Fri, 23 Mar 2018 04:44:43 GMT

oh actually the grid is the same

Re: Flow direction data error?

XanderBakker — Fri, 23 Mar 2018 11:22:30 GMT

Nearest neighbor should not be used for a continuous surface like a DEM (DTM). As Dan mentioned bilinear and cubic are the ones that are appropriate.

Re: Flow direction data error?

XanderBakker — Fri, 23 Mar 2018 11:25:31 GMT

Did you try to rebuild the pyramids? Although as you describe, if the line is there at all scales it will probably not be the solution.

If you zoom in to pixel level, are the lines a single pixel thick or are they wider?. You could set those pixels to NoData and then use a focal statistics operation to fill them using a mean of the known values around the NoData pixel. If the the lines and just a single pixel wide, and evenly spaced you could even create a fishnet and use the outlines in a SetNull or Con operation to set those pixels to NoData.

Re: Flow direction data error?

XanderBakker — Fri, 23 Mar 2018 11:28:46 GMT

In addition, you could load the data into a File Geodatabase. I have managed DTM's and DSM's derived from LiDAR data occupying more than 25GB in a FGDB and worked with them without a problem. You can also manage it as a mosaic dataset and apply raster functions which will speed up the process dramatically. Some more reading on this topic can be found here: https://community.esri.com/people/xander_bakker/blog/2016/07/18/using-mosaic-datasets-and-raster-chain-functions-to-analyze-interference-of-trees-on-an-overhead-power-line