Playa

Python Topo To Raster: Tiled Processing

Discussion created by Playa on Sep 1, 2016
Latest reply on Sep 1, 2016 by Dan_Patterson

I'm looking for a reasonable workflow to generate DEM's from contours for multiple study areas without having to clip my contours into tiles in order to generate a DEM. I'm looking for suggestions from the community in being able to process large datasets (contours) by processing the study area into usable tiled areas such as the Tiled processing that ESRI has developed for certain geoprocessing tools:

 

 

I'd like to generate a tiled index on the fly based on the extent of my study area and output cellsize, in order to process my study area in tiles and then stitch the tiles together to generate a single DEM. 

 

Topo to Raster is written in Fortran and doesn't take advantage of multiple processors or cores. Its limited in the amount of memory that is allocated to the single geoprocessing tool, so I'm trying to develop a Python workflow of splitting the study area into small enough areas that can be run simultanously and then stitched back together afterwards. 

 

My current study area is made up of nine local municipalities and from my calculations based on a 10m cellsize each local municipality will have the following amount of cells for each DEM:

 

  • Baviaans 226 791 600
  • Blue Crane Route 209 397 384
  • Camdeboo 250 726 124
  • Ikwezi 86 625 374
  • Kouga 50 539 474
  • KouKamma 73 776 259
  • Makana 40 357 026
  • Ndlambe 40 357 026
  • Nelson Mandela Bay 34 654 290
  • Sunday's River Valley 114 374 190

 

Nine Local Municipalities

Outcomes