Split Raster function produces partial or no output

[JohnGaiot]

The behaviour of the Split Raster function is rather unpredictable and produces partial or no output depending on the output raster data format chosen. Some tiles would complete successfully while others tiles would either contain NODATA or be skipped altogether. This would happen when producing stand-alone GRID's, rasters within file geodatabases, and also TIFF's (curiously, the pattern of which tiles were successful and which weren't was not consistent between formats). The only format that was successful in my trials was IMAGINE IMAGE (.IMG) which produced data for all the tiles successfully. Further testing would be required to determine if this is always the case.

I searched all the forums and found one entry by a user experiencing similar problems, but the solution to the issue was never pursued as the user found a manual 'work around' which doesn't work for me as I have to do this repeatedly for large areas in Ontario. Forum reference: http://gis.stackexchange.com/questions/211630/missing-output-from-arcgis-split-raster-tool

Illustration below was the result using the stand-alone GRID format but there were also instances where no tiles were created at all in other tests.

The following are the DEM characteristics and the splitting criteria I used:

DEM Projection: NAD_1983_CSRS_UTM_Zone_17N

Data Type: Float 32-Bit

Unsplit DEM Width Size (columns) = 11163
Unsplit DEM Height Size (rows) = 11519
Cell Width Size ( x ) = 2
Cell Height Size ( y ) = 2
Linear Unit Name = Meter
Split DEM Width Size (columns) = 4000
Split DEM Height Size (rows) = 4000

1 PIXEL overlap between tiles.

NEAREST resampling option

SIZE_OF_TILE option

I also encountered this problem using a DEM with a 30m cell size, NAD_1983_Ontario_MNR_Lambert projection, and on the following platforms.

arcgis-10-3‌ and arcgis-10-1‌

I am wondering whether later versions of ArcGIS have fixed this problem?

Thanks in advance.

John

[DanPatterson_Retired]

Lots of threads on support but nothing matching your case

In the ...Py_links... I maintain a list of ... issues addressed... by version.  These lists often cite enhancements/fixes that never make it to a published bug number, or one that you could find without a NIM number.  You could start with 10.5 then move backwards to see if split raster is mentioned anywhere.  Otherwise, this is a classic case of where you have a documented issue and the data.  You should forward it to esri Canada for a first look.

  Good luck

[JohnGaiot]

Thanks Dan,

I sifted through all the versions as you suggested but came up empty. I will forward to ESRI when I have some time to prepare. I have some rather lengthy custom Python scripts that are run as a series, and would like to isolate the parameters specifically related to the Split Raster task. But I'll get it done eventually.

Thanks again.

[DanPatterson_Retired]

If you are nimble with python, I find you can use numpy and then convert back to a raster

ie use rastertonumpy array to get an array... do the tiling, then use numpyarraytoraster to get the rasters back.  There is an np.array_split function that can be use.  But in short, you could write a fairly simple function to split your array/raster with the desired overlap and convert back to rasters.

Not the best situation in your case, but you won't have to sit and wait

>>> import numpy as np
>>> a = np.arange(5*7).reshape(5,7)
>>> a
array([[ 0,  1,  2,  3,  4,  5,  6],
       [ 7,  8,  9, 10, 11, 12, 13],
       [14, 15, 16, 17, 18, 19, 20],
       [21, 22, 23, 24, 25, 26, 27],
       [28, 29, 30, 31, 32, 33, 34]])
>>> a.shape
(5, 7)
>>> v, h = 3, 4
>>> a[:v, :h]
array([[ 0,  1,  2,  3],
       [ 7,  8,  9, 10],
       [14, 15, 16, 17]])
>>> a[v:, h:]
array([[25, 26, 27],
       [32, 33, 34]])
>>> a[:v, h:]
array([[ 4,  5,  6],
       [11, 12, 13],
       [18, 19, 20]])
>>> a[v:, :h]
array([[21, 22, 23, 24],
       [28, 29, 30, 31]])
>>> ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[JohnGaiot]

Thanks for the suggestions. I will look into this option.. As for the Split Raster function, I conducted further testing and it appears to be a memory issue. No matter the data type (it eventually happened with IMG files too), the output is random and should report a warning that not all tiles were processed but it continues as if successful. I eventually went to ArcCatalog and reran the script and it completed all the tiles successfully. I suppose 16 GB RAM is not good enough for ArcMap. I think the function should at least verify against the footprint of the original raster to see whether the output is complete, and if not, give a warning. Something to note for ESRI Canada I suppose

[DanPatterson_Retired]

ArcMap will only use 4Gig of ram at best.. Try the workflow in ArcGIS PRO

[JohnGaiot]

Even with 64-bit background geoprocessing enabled? I thought that was the limit for the 32-bit environment. The help says it can utilize "large amounts of RAM".

[DanPatterson_Retired]

Yeah... I have heard that too... but I continue to read solutions to problems saying '...turn off background geoprocessing...'... when using ArcMap... go figure

Try Pro, it won't cost you anything, and it is evolving nice ... unless of course you are running a 32bit machine.

[JohnGaiot]

Believe me, I would really like to jump over, but two big things are holding me back. Support for topology and geometric networks just aren't there yet. In the world of hydrology, this functionality is very important. And from what I'm seeing, the proposed network functionality is also going to be different from the desktop version which will mean some significant revamping of our tools.

[DanPatterson_Retired]

A common concern, however, waiting until the main dish is done shouldn't stop one from sampling while the cooking is going on.  You will be better prepared for the end result