topic Re: Problem with DEM and Path Distance in 3D Questions

Problem with DEM and Path Distance

MargaretBrown_Vega — Thu, 17 Feb 2011 13:58:07 GMT

Hi all,

I am trying to calculate path distance, using a vertical factor table (created by Nico Tripcevich based on Tobler's hiking function, discussed here http://forums.esri.com/Thread.asp?c=93&f=995&t=138683, among other places).

I created a DEM from SRTM version 4. When I calculate path distance using the vertical factor table, then generate contours, I get fairly evenly-spaced lines that look more like Euclidean distances. This is mountainous terrain. One should not be able to travel as far in the same amount of time when climbing a mountain. In fact, Euclidean distance contours seem to be further from the source point, even in areas where there are mountains that I know take two to three hours to climb!

In the past I've calculated path distance before using another DEM (an earlier version of SRTM). To try and diagnose the problem I snagged this earlier DEM, and calculated path distance from the same point feature on both DEM's for comparison. My problem is best illustrated in the two attached graphics. In the first graphic, the red contour lines represent what was generated with the earlier DEM, and is what I expect. The blue contours are my problem: that is what is produced doing the exact same calculation with this new DEM I've created. I am going mad trying to figure out what the problem is with this second DEM.

I've been able to calculate slope with this newer DEM (see attached figure 2), as well as some other compound indices. But there appears to be something wrong with the DEM. Other analyses have looked okay, but the path distance does not. Just to illustrate the point further the image shows slope from the second, problematic, DEM, and the evenly spaced path distance contours (which make no sense given the terrain).

I compared the two DEM's, which are of the same geographic area, and there seems to be a big difference in elevation (600 m).

My plan is to go back and use the older DEM. But, I would like to know what the problem is with the newer DEM (assuming that's what it is). Any suggestions or feedback would be much, much appreciated!

Sincerely,
Margaret

Re: Problem with DEM and Path Distance

Anonymous User — Thu, 17 Feb 2011 17:30:24 GMT

Original User: mybvega

I found the problem, and it does not appear to have been the DEM.

I was following a protocol for calculating path distance that didn't entail including a cost raster. Rather, I just used the DEM. For some reason, this does not now work. I took my slope raster calculated from my DEM and set that as my cost raster, and my DEM as the surface raster. This worked. Problem solved. An omission of a necessary input. Does anyone know why I would have been able to calculate it before without the slope raster input (simply using the elevation field from the DEM)?

Re: Problem with DEM and Path Distance

StevenWernke — Fri, 18 Feb 2011 02:31:53 GMT

I am having a similar problem with the exact same data types (I am also using Nico's Tobler-derived vertical factor tables, and conferred with him personally on it; I am sure the tables are not the issue).

To be clear, here is exactly how I ran it using the Path distance tool:

Spatial Analyst Tools > Distance > Path Distance...
a. Input: Sites (my source point data--one point selected)
b. Output: aniso_hrs1
c. Input cost raster (optional):<blank>
d. Input surface: colca_clip (my clipped 30m DEM)
e. Maximum distance (optional):<blank>
f. Output backlink raster (optional): aniso1_bklk
g. Vertical factor parameters (optional): Input Vertical Raster: colca_clip, Vertical factor (optional): Table, using Nico's "Tobler_away.txt"
h. "Environments..."> "Processing Extent..." > Set the Extent to "Same as Display"

The "good" news is that I can get it to run using the above, but ONLY if it is zoomed into a small area. Beyond a certain extent (not a very large one at that), it won't work anymore, even restricting the processing extent to Same as Display. If I use the extent of the DEM as the processing extent, it will just return a Euclidian distance grid, ignoring the DEM.

FYI, the DEM must be 32 bit floating point, with square pixels for it to work. That might be your problem. Mine, however, has to do with the extent.

Please ESRI people, throw us a bone.

Re: Problem with DEM and Path Distance

Anonymous User — Sat, 19 Mar 2011 15:05:33 GMT

Original User: rwhitlow

Steve,

I think it's a problem with the raster's cell count, not the extent. I ran into the same issue and was able to fix it by resampling the raster to a larger cell size. I don't know the ceiling for Path Distance, but I was able to roughly bracket the column x row measurements. A 388 x 484 (~188k cell count) raster works, a 542 x 677 (~367k) raster does not. I may keep playing with this, and let you know if I find the magic number.

- Ray Whitlow

Re: Problem with DEM and Path Distance

MathewSchmidtlein — Wed, 02 Nov 2011 13:48:46 GMT

Any news on this? I'm trying to figure out some issues with an analysis I'm running for a moderately sized study area, but with 1 meter pixels. It is an area of pretty low relief, but we are getting what seem to be dramatically low estimates for pedestrian travel times. It worked, however, before I upgraded to 10. Were there changes in the Path Distance tool at the 10 release that are causing this issue?

Re: Problem with DEM and Path Distance

MathewSchmidtlein — Fri, 04 Nov 2011 16:20:54 GMT

In case this is helpful to anyone: I have been using Path Distance to model pedestrian evacuations, using reclassified landcover as a cost surface, and elevation as the surface layer and vertical factor, all with 1 meter resolution data. When I was running it with ArcGIS Desktop 10, it would run for about 24 hours, and result in dramatic underestimates of travel times. I still had my copies of 9.3.1, so I removed 10 reinstalled that, and the whole process took about an hour, and is returning more reasonable results. So I am suspecting the issue here is something to do with changes made at the 10 release. I like 10 overall, but Path Distance doesn't seem to work well in it.

Re: Problem with DEM and Path Distance

Anonymous User — Wed, 08 Feb 2012 11:06:50 GMT

Original User: dcontre

I am having a similar problem (and did about a year ago as well). Last time, I was eventually able to solve the problem by converting my input point to a raster, and by making sure all raster inputs were in GRID format (which does not I think conform to Steve's suggestion of 32-bit floating point, but worked). I also had to eliminate hidden spaces in the file paths to which the Path Distance tool was writing (running VMWare on a Mac, the default file path includes a space).
It took addressing all of those issues to get an result that didn't ignore the input raster and just produce a Euclidean distance (the same pattern that Margaret illustrates above). Coupled with the various suggestions here, this makes me think that a variety of errors lead the Path Distance tool to produce Euclidean results. I did eventually get real results, though - and with a sizable underlying raster (23896 x 24107), 16 bit continuous signed integer.
At any rate, I am now stumped by the same problem again. I am trying to calculate Path Distance using a SRTM 90m pixel raster as the input. I've converted my origin point from feature to raster, double-checked file paths and raster formats, and tried with both UTM and Lambert Conformal Conic projections. I've even gone back to my old data (different region entirely), which I ran successfully last Feb., and tried to re-run those. The output is invariably a concentric spider-web. Does anyone have any suggestions? I notice the last post here was a few months ago - has anyone figured out the root problem and/or a solution?

Many thanks,

Dan

Re: Problem with DEM and Path Distance

DavidRedhouse — Thu, 15 Mar 2012 16:28:08 GMT

I am having a similar problem (and did about a year ago as well). Last time, I was eventually able to solve the problem by converting my input point to a raster, and by making sure all raster inputs were in GRID format (which does not I think conform to Steve's suggestion of 32-bit floating point, but worked). I also had to eliminate hidden spaces in the file paths to which the Path Distance tool was writing (running VMWare on a Mac, the default file path includes a space).
It took addressing all of those issues to get an result that didn't ignore the input raster and just produce a Euclidean distance (the same pattern that Margaret illustrates above). Coupled with the various suggestions here, this makes me think that a variety of errors lead the Path Distance tool to produce Euclidean results. I did eventually get real results, though - and with a sizable underlying raster (23896 x 24107), 16 bit continuous signed integer.
At any rate, I am now stumped by the same problem again. I am trying to calculate Path Distance using a SRTM 90m pixel raster as the input. I've converted my origin point from feature to raster, double-checked file paths and raster formats, and tried with both UTM and Lambert Conformal Conic projections. I've even gone back to my old data (different region entirely), which I ran successfully last Feb., and tried to re-run those. The output is invariably a concentric spider-web. Does anyone have any suggestions? I notice the last post here was a few months ago - has anyone figured out the root problem and/or a solution?

Many thanks,

Dan

I had this problem last week. It nearly reduced me to tears. It occurred in ArcGIS 9.2 (Desktop and Workstation) and ArcGIS 10 (Desktop and Workstation). In the end I got it working by converting my DEM to a 32-bit Float, then exporting that as a GRID outside of my File Geodatabase, and then using a Shapefile to hold the start points. I stored everything in C:\TEMP and my filenames were all shorter than 8 characters and used only letters in them. Also I waved a rubber chicken over the computer three times before pressing OK.

HTH

Re: Problem with DEM and Path Distance

Anonymous User — Fri, 15 Jun 2012 01:22:14 GMT

Original User: lkellett

Did anyone figure out the path distance problem in ArcGIS10, specifically in using a SRTM DEM? LK

Re: Problem with DEM and Path Distance

DanielBecker — Thu, 05 Jul 2012 08:55:07 GMT

Ohh.. same problem here. I work on my geography diploma thesis which is about, well.. prehistoric sites and distance/site catchment analysis around caves (and i have to work with arcgis).

Now i discovered the toblerAway-table and was very happy that this could be a nice solution, but it only works if i zoom in to a very small extent (while using processing-extent: same as display).

In other cases i also get the Euclidean Distance pattern in the resulting cost raster. 😞

Re: Problem with DEM and Path Distance

Anonymous User — Tue, 10 Jul 2012 16:40:30 GMT

Original User: ph0b0z

So, i tried different extents, these are the resulting rasters:

493x345 works
657x460 doesn't work (which means, the circular pattern is displayed).

Anyone else still interested in solving the problem? 🙂

Re: Problem with DEM and Path Distance

Anonymous User — Wed, 19 Sep 2012 16:09:38 GMT

Original User: CAUK

This is my problem as well, why hasn't there been any response from ESRI on this in over a year?

Re: Problem with DEM and Path Distance

MichalBirkenfeld — Wed, 24 Oct 2012 09:57:29 GMT

Hi everyone!

I'm not sure if seeing such a long thread about the same problem I'm having makes me feel better or worse...
I'm trying to calculate path distance travel time, using a 5m cell size DEM as the surface raster, and a ASCII table with the vertical factor. I keep getting this Eucledean "spider web" type raster...

Any suggestions?

Has anyone been able to solve this issue?

It is driving me INSANE...

Thanks
Michal

Re: Problem with DEM and Path Distance

Anonymous User — Wed, 07 Nov 2012 16:09:37 GMT

Original User: rwhitlow

Michal,

I'm about 90% positive that the problem is the path distance toolbar is unable to handle more than 250,000 cells in the raster analysis. The most effective workaround I've found so far is to either reduce your study area to a smaller size (a 500 x 500 cell square) or resample your raster to a larger cell size. Alternatively, if you are working in an area with relatively little elevation change you can omit the vertical raster from the analysis and use the cost surface and horizontal surface instead, or use the cost- tools rather than the path- tools.

Hope that helps.

Best,
Ray Whitlow

Re: Problem with DEM and Path Distance

MarkBoucher — Wed, 07 Nov 2012 23:09:56 GMT

There is a way to increase the size of a raster ArcMap will deal with. This post describes the "AdvancedArcMapSettings.exe" registry editor to use to do that. http://forums.arcgis.com/threads/48666-ArcHydro-doesn-t-output-to-shapefile?p=203314#poststop

Also, I found under the menu "Customize>ArcMap Options>Raster Tab>Raster Dataset subtab - Max Number of Unique Values to Render" has the same number I put into the "AdvancedArcMapSettings.exe" interface. As a matter of fact, it appears to be the same variable and either interface changes the same value.

Maybe increasing this value will allow you to process a larger raster.

Re: Problem with DEM and Path Distance

Anonymous User — Tue, 13 Nov 2012 19:38:32 GMT

Original User: rwhitlow

Mark,

Thank you! I just tried the ArcMap->Customize change with a larger raster and it calculated the path distance correctly.

Re: Problem with DEM and Path Distance

MargaretBrown_Vega — Mon, 07 Jan 2013 14:25:09 GMT

Hi All,

I returned to this problem after a fairly long hiatus. I think I successfully ran the path distance analysis after making two changes. I converted my DEM to a 32-bit floating point raster (this alone did not produce a good result). Then, I followed Mark's suggestion. I went into the Customize menu and changed the Max Number of Unique Values to Render. The value was set to 65536. I wasn't sure what to change it to, so I just opted for 200000. I've attached an image which shows travel time in hours away from major rivers (white) using Euclidean distance (red contour), and travel time in hours using a vertical factor table (yellow contour). No more concentric spider-web lines.
[ATTACH=CONFIG]20501[/ATTACH]
However, I do believe the result still underestimates travel time (perhaps another issue).

Just thought I'd chime in with Ray - Mark's solution appears to work.

Best,
Margaret

Still fighting the (good?) fight

Anonymous User — Tue, 09 Apr 2013 17:55:09 GMT

Original User: emarsh

I have been following this post since the beginning, with the same problem (and tears). I was enthused by the recent posts by Mark, Raymond, and Margaret. HOWEVER, once again, I have been let down (such a familiar feeling with Arc). I continue the plea for help. Even after adjusting the Max Number of Unique Values to Render, I was back to square one, with precisely problem (and settings) described by Margaret and Steve in their intitial posts. I get good results with a very small extent, making me think I have eliminated the myriad of other possible problems. My DEM is a 32 bit floating point GRID and my Max Number of Unique Values to Render is 10 million (just to be on the safe side).

One minor point that may be helpful. As described in Mark's post, the Max Number of Unique Values to Render can be adjusted two ways: In ArcMap (Customize>ArcMap Options>Raster Tab>Raster Dataset subtab) or in AdvancedArcMapSettings.exe (Raster tab). It took me a while to realize that my changes to this number within ArcMap were not taking effect. The new number I entered remained, even when I closed and re-opened the program.

However, when I opened AdvancedArcMapSettings.exe, the number was unchanged, set to the default (around 65000), and changing it had no effect. I had to open this program with Administrator Access (by right-clicking on the application, "Open as administrator", even though I was already logged in as the administrator), and only then did the change hold. Finally, the number was the same in both ArcMap and AdvancedArcMapSettings, as Mark points out it should be.

Much to my dismay, this did not solve the issue.

As I said, I am back to square one.
I am open to any ideas! Thanks for all the hints and input up to now.
Erik

Re: Problem with DEM and Path Distance

MarkBoucher — Tue, 09 Apr 2013 18:12:08 GMT

Sorry to hear you are having problems. One other thing I've done that seem to have greatly reduced the crashed (and tears) is to copy the project to my local PC and not have it on the network. My suspicion is that using Spatial Analysis and geoprocessing over a network takes a lot more time and introduces more chances of errors as the data goes back and forth through the routers. I can't verify this since my understanding of networks and routers is limited.

So, if you are trying to work on rasters and datasets over a network, copy them all to a folder on your PC and try to make it work then. I can guarantee things will go much faster and can almost guarantee you'll have fewer processing crashes.

An easy want to do this, if the files are all in one folder on the network, is to choose the option under the "File" menu for the mxd properties and make the references to layers "relative" (I think. I can't check now b/c my Arc Map is busy). Then after you copy it all to your PC you don't have to remap the layer sources.

Re: Problem with DEM and Path Distance

Anonymous User — Thu, 25 Apr 2013 16:18:33 GMT

Original User: sesary

Hi. I have been facing the same problem for some time, and in my case it happened to be an issue with the 10.1 version. Solved after installing Service Pack 1.
Hope this helps
Cheers

César Parcero-Oubiña