topic Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters in ArcGIS Spatial Analyst Questions

errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

yvanaltchenko — Sat, 03 Feb 2018 15:02:22 GMT

Hi, I need to do some statistics (mean, std) in order to compare each cell of 27 rasters. I get error using "cell statistics" (error 99999) or "raster calculator" (000539). Any ideas where come from the error (my formula in raster calculator has been rechecked and it is correct).

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

DanPatterson_Retired — Sat, 03 Feb 2018 15:29:34 GMT

perhaps you could provide it anyway, but things to check when working with rasters would include

file paths with spaces
files in different coordinate systems
mismatched cell sizes, extents and/or alignment can cause problems when they are attempted to be accounted for on the fly
not saving a raster with a *.tif file extension produces an esri grid, which raises a whole different set of concerns
data sources sometimes on servers have caused issues which are resolved when working with locally stored data

the 99999xxxx errors are generic errors, suggesting there is no specific category of error but potentially many errors or one that doesn't fit nicely into a 'mistake'

Also, version of arcmap and/or Pro

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

DanPatterson_Retired — Sat, 03 Feb 2018 23:22:30 GMT

Error 000539... some from the ESRI support home

https://support.esri.com/en/bugs/nimbus/TklNMDg2MDQ1
http://resources.arcgis.com/en/help/main/10.1/index.html#//00vp00000007000539
https://support.esri.com/en/technical-article/000011928 raster calculator and python issues?

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

yvanaltchenko — Sun, 04 Feb 2018 07:18:24 GMT

Thank you very much for your advice. Unfortunetaly, all rasters seem to have appropriate configuration (the 27 rasters have been created in modelbuider using cell statistics and raster calculation tools from the same initial parameter and I did not get any issue....all file path have no space, I working on my internal hardrive only). The error comes from somewhere else.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

curtvprice — Mon, 05 Feb 2018 00:25:01 GMT

Please copy and paste your tool messages into the thread, this may shed some light as to what is going on. It really is hard to suggest a fix without more information.

That said, another thing to look for is to make sure your workspace and scratch workspace environments are set to a valid path.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

DanPatterson_Retired — Mon, 05 Feb 2018 04:27:36 GMT

Share your raster calculator expression or your modelbuilder script export if you can.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

yvanaltchenko — Mon, 05 Feb 2018 07:36:45 GMT

See the screen shot of both error. I never change the current workspace since I installed ARCGIS (and i did not get any issue with it) but I get sure that the scratch workspace doesnt have space. My raster caculator expression is the simple mean calculation as below (as raster statistics was not working I try raster calculator but I test with mean while I am more interested with std)

("GWDC_W1_A01.tif" + "GWDC_W1_A02.tif" + "GWDC_W1_A03.tif" + "GWDC_W1_A04.tif" + "GWDC_W1_A05.tif" + "GWDC_W1_A06.tif" + "GWDC_W1_A07.tif" + "GWDC_W1_A08.tif" + "GWDC_W1_A09.tif" + "GWDC_W1_A10.tif" + "GWDC_W1_A11.tif" + "GWDC_W1_A12.tif" + "GWDC_W1_A13.tif" + "GWDC_W1_A14.tif" + "GWDC_W1_A15.tif" + "GWDC_W1_A16.tif" + "GWDC_W1_A17.tif" + "GWDC_W1_A18.tif" + "GWDC_W1_A19.tif" + "GWDC_W1_A20.tif" + "GWDC_W1_A21.tif" + "GWDC_W1_A22.tif" + "GWDC_W1_A23.tif" + "GWDC_W1_A24.tif" + "GWDC_W1_A25.tif" + "GWDC_W1_A26.tif" + "GWDC_W1_A27.tif") / 27

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

XanderBakker — Fri, 10 Dec 2021 22:13:16 GMT

If you chop it up in three parts of 9 rasters does any of the expressions in the Raster calculator or using Cell Statistics run correctly? It might be related to 1 or more rasters?

Can you try to run these calculations and see if any one works:

("GWDC_W1_A01.tif" + "GWDC_W1_A02.tif" + "GWDC_W1_A03.tif" + "GWDC_W1_A04.tif" + "GWDC_W1_A05.tif" + "GWDC_W1_A06.tif" + "GWDC_W1_A07.tif" + "GWDC_W1_A08.tif" + "GWDC_W1_A09.tif") / 9
("GWDC_W1_A10.tif" + "GWDC_W1_A11.tif" + "GWDC_W1_A12.tif" + "GWDC_W1_A13.tif" + "GWDC_W1_A14.tif" + "GWDC_W1_A15.tif" + "GWDC_W1_A16.tif" + "GWDC_W1_A17.tif" + "GWDC_W1_A18.tif") / 9
("GWDC_W1_A19.tif" + "GWDC_W1_A20.tif" + "GWDC_W1_A21.tif" + "GWDC_W1_A22.tif" + "GWDC_W1_A23.tif" + "GWDC_W1_A24.tif" + "GWDC_W1_A25.tif" + "GWDC_W1_A26.tif" + "GWDC_W1_A27.tif") / 9‍‍‍‍‍‍

If they do, you can finish by calculating the mean of the 3 resulting rasters.

Some other questions:

What range of values do the rasters have?
What pixel type do they have?
Do the have NoData cells?

Try as Dan suggested earlier to use a proprietary data format (such as FGDB raster) to see if the problem continues. If possible share the rasters to this thread so we can investigate a little more to find out what the possible reason for the error might be.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

DanPatterson_Retired — Mon, 05 Feb 2018 12:22:05 GMT

For Cell Statistics rasters are entered separated by commas

CellStatistics([InRas1, InRas2, InRas3], "SUM", "NODATA")

You might want to try that, or see the examples in the link on how to use it in a script to get you over the immediate need.

The raster calculator isn't meant to be used in scripts in any event

How Raster Calculator works so it might be best just to use the required format for scripting for Cell Statistics in order to get your Standard Deviation

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

yvanaltchenko — Tue, 06 Feb 2018 07:47:09 GMT

Hi, thanks. The cell statitics is working if I am splitting into 3 group ou 2 group the 27 rasters (but I have to open a new map document for making it working) but it doesnt work with the 27 rasters.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

NawajishNoman — Tue, 06 Feb 2018 08:01:27 GMT

I think the long string you created with the raster has some issue. It didn't have any separator between rasters. There could be other issues.

Please create a list of your rasters and see if it works. Here is an example.

import arcpy
from arcpy import env
from arcpy.sa import *
env.workspace = "C:/sapyexamples/data"
outCellStats = CellStatistics(["degs", "negs", "cost"], "STD", "DATA")
outCellStats.save("C:/sapyexamples/output/outcellstats.img")

You can also specify the name of the raster in a variable and then construct a list as follows.

# Set variables
inRaster01 = "degs"
inRaster02 = "negs"
inRaster03 = "cost"

# Execute CellStatistics
outCellStatistics = CellStatistics([inRaster01, inRaster02, inRaster03], "RANGE", "NODATA")

Let us know if it doesn't work for you.

Thanks!

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

yvanaltchenko — Tue, 06 Feb 2018 08:10:44 GMT

Hi see below the script export of the modelbuilder. Thanks

# -*- coding: utf-8 -*-
# ---------------------------------------------------------------------------
# statistics.py
# Created on: 2018-02-06 10:09:49.00000
# (generated by ArcGIS/ModelBuilder)
# Description:
# ---------------------------------------------------------------------------

# Import arcpy module
import arcpy

# Local variables:
GWDC_W1_A01_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A01.tif"
GWDC_W1_A02_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A02.tif"
GWDC_W1_A03_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A03.tif"
GWDC_W1_A04_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A04.tif"
GWDC_W1_A05_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A05.tif"
GWDC_W1_A06_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A06.tif"
GWDC_W1_A07_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A07.tif"
GWDC_W1_A08_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A08.tif"
GWDC_W1_A09_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A09.tif"
GWDC_W1_A10_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A10.tif"
GWDC_W1_A11_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A11.tif"
GWDC_W1_A12_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A12.tif"
GWDC_W1_A13_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A13.tif"
GWDC_W1_A14_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A14.tif"
GWDC_W1_A15_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A15.tif"
GWDC_W1_A16_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A16.tif"
GWDC_W1_A17_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A17.tif"
GWDC_W1_A18_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A18.tif"
GWDC_W1_A19_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A19.tif"
GWDC_W1_A20_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A20.tif"
GWDC_W1_A21_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A21.tif"
GWDC_W1_A22_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A22.tif"
GWDC_W1_A23_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A23.tif"
GWDC_W1_A24_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A24.tif"
GWDC_W1_A25_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A25.tif"
GWDC_W1_A26_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A26.tif"
GWDC_W1_A27_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A27.tif"
GWDC_W1_A_std_tif = "D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A_std.tif"

# Process: Cell Statistics
arcpy.gp.CellStatistics_sa("D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A01.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A02.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A03.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A04.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A05.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A06.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A07.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A08.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A09.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A10.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A11.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A12.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A13.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A14.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A15.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A16.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A17.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A18.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A19.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A20.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A21.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A22.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A23.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A24.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A25.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A26.tif;D:\\PhD\\GIS\\janvier_2018\\GWIDC_calculation\\Uncertainty_on_factor_A\\GWDC_W1_A27.tif", GWDC_W1_A_std_tif, "STD", "DATA")

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

DanPatterson_Retired — Tue, 06 Feb 2018 10:58:46 GMT

This is beginning to sound like another thread where the raster dimensions are physically fixed and there was a need for obtaining a statistical parameter over a time period.

https://community.esri.com/message/742193-re-how-to-calculate-the-percentile-for-each-cell-from-timeseries-raster?commentID=742193#comment-742193

In that case and in several others some success was had by emulating cell statistics within arrays.

What is the size of your tiff files .. ie rows and columns If they aren't too onerously bit, then could you zip them into a file for testing?

The procedure that has been used in several occasions for esri grids, tiffs, and ascii files is to

convert each raster to a numpy array ... the array will have 2 dimensions (cols, rows)
when stacked they produce a 3 dimensional array (depth, cols, rows)
the 3d array is what is used to calculate Cell Statistics, obtaining a parameter on a cell-by-cell basis over the time (depth) axis
This procedure has been used to calculate means, running means, percentiles etc.

So if the files are relatively small, they may be able to be loaded into arrays and processed in python/numpy escaping the limits imposed on memory by arcmap.

Perhaps on that note, I can't remember of you have tried PRO to do the same work.

So if you can share the data or even 1 or 2 files, I can try to set up a workflow that might work.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

XanderBakker — Fri, 10 Dec 2021 22:13:19 GMT

You can write this code a lot shorter...

# Import arcpy module
import arcpy
import os

ws = r'D:\PhD\GIS\janvier_2018\GWIDC_calculation\Uncertainty_on_factor_A'
ras_name_template = 'GWDC_W1_A{0}.tif'
ras_list = [os.path.join(ws, ras_name_template.format("%02d" % (i,) )) for i in range(1, 28)]
ras_name_out = 'GWDC_W1_A_std.tif'
ras_path_out = os.path.join(ws, ras_name_out)

# Process: Cell Statistics
arcpy.gp.CellStatistics_sa(ras_list, ras_path_out, "STD", "DATA")
‍‍‍‍‍‍‍‍‍‍‍‍

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

yvanaltchenko — Tue, 06 Feb 2018 12:50:39 GMT

Thanks for all your advice. Very much appreciate. My raster are quite big covering Africa with 0.005 degre cell size (about 800 Mo each for a total of 21 Go). Value of the 27 rasters are between 1 an 4.8 and are not time serie. They have been built from 8 parameters which are been firstly ranked into 5 value (1, 2 , 3 , 4 , 5) then weighted (equal weight of the parameter). The difference between the 27 rasters is that I applied 3 different rankings on 3 parameters. Then I need to calculcate mean and std of the 27 rasters in order to check areas where results are quite similar (it is a quite of sensitivity analysis).

I am not too familar with python script (but I can deal with modelbuilder) and very unfamilar with nunpy array (i basicly arcgis self-taught). I just wonder if my computer did not reach its capacity as the files are big (or ARCGIS capacity). I dont have PRO.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

DanPatterson_Retired — Tue, 06 Feb 2018 13:17:47 GMT

You are probably running out of computing resources. Run your system monitor to see how it progresses as you run a batch.

If memory is peaking, you will need to run in a 64 bit environment (ArcMap with background geoprocessing... although I don't know if it is used by SA off hand. Or ArcGIS PRO)

Batching may be your only solution.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

DanPatterson_Retired — Tue, 06 Feb 2018 14:02:43 GMT

As an alternative, particularly in a memory starved environment, see

Cell Statistics made easy... raster data over time

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

curtvprice — Thu, 08 Feb 2018 04:24:34 GMT

> ArcMap with background geoprocessing... although I don't know if it is used by SA off hand.

ArcMap background processing is fully supported by Spatial Analyst. You do need to do the separate install, it isn't part of the main ArcMap install.

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

XanderBakker — Thu, 08 Feb 2018 12:03:11 GMT

In addition to the great post by Dan Patterson‌, since you have rasters that are large you should implement blocks, to read parts of the raster and not the entire raster as explained in code sample 2 on this help page: RasterToNumPyArray—ArcPy Functions | ArcGIS Desktop

Re: errors on performing "cell statistics" (error 99999) or "raster calculator" (000539) on 27 rasters

yvanaltchenko — Thu, 08 Feb 2018 12:41:18 GMT

Thank you for all your advices. I manage to calculate the standard deviation and the mean of my 27 rasters by cutting my raster into pieces. Cell statistics works when raster size is less than 80 Mo and when I process cell statistics one by one (if i create a model to go quicker it doesnt work). I guess it is link to my laptop capacity even if when processing the ram is not use fully.