Suggestions on calculating Slope on a 5Tb Raster Dataset?

Question

I have a 5Tb Raster Dataset in SDE that had no problems calculating Hillshade, now I want to calculate Slope and it pops back in 5 seconds with a 99999 error Table not Registered.  ESRI Support suggesteed that the file is simply too big. Anyone out there playing with mega files that can offer suggestions.  I can dice it up into smaller rasters, but what about the edge effects?

VinceAngelo · Answer

There shouldn't be much difference between running this as a 64-bit Python script (64-bit Background Geoprocessing) and running it within Pro.

Given the size of the raster, and the fact that the ArcSDE API reads in tiles, not rows, constructing a cache for the tile height might have caused memory issues in 32-bit clients. Re-loading the raster with a W*2 x H/2 tile size might help for borderline cases, depending on the original tile Width & Height. Changing the tile size to be small enough to reside in-row might also result in significant performance improvement, as would using file-based storage (using rasters in databases was novel at ArcGIS 9.0, but is most of the way toward deprecated now).

In the future, please include image and tile dimensions in the question, since storage doesn't provide an indication of width, height, depth, or compression algorithm.

- V

DanPatterson_Retired · Answer

part of the bottleneck is undoubtedly SDE... but at least you are moving forward

deleted-user-xAqOOPuoS824 · Answer

Jayanta,

Thanks for the tips.  I tried running with background off and it still dies in 5 seconds.  Seems that ArcGIS isn’t ready for big rasters.  However, I did get a clue from Dan Patterson who said try it in ArcPro and it seems to be working.  It takes about 3 days to see results so I will find out Monday.

-John