10,000 inputs for Cell Statistics?

972
8
Jump to solution
10-25-2016 07:22 AM
Highlighted
New Contributor III

Hi all, 

I was wondering if it is possible to have 10,000 inputs for cell statistics? I keep getting an error, saying that I can't save the output as a TIF.  I get the following: ERROR 010240. 

Any ideas or suggestions on how to tackle this? 

Thanks a million, 

import arcpy
import os

arcpy.CheckOutExtension("Spatial")
arcpy.env.overwriteOutput = True
arcpy.env.mask = r'E:\Reference_ET\Verification\GCS_PRISM_4km.shp'

print "Loaded Pre-processing extents..."


TmaxWs = r'M:\PRISM\Daily\tmax'
TminWs = r'M:\PRISM\Daily\tmin'
outws = r'E:\Reference_ET\KT'
TmaxList = []
TminList = []

print "Completing Lists..."

for dirpath, dirnames, filenames in arcpy.da.Walk(TmaxWs, topdown=True, datatype="RasterDataset"):
   for filename in filenames:
      TmaxList.append(os.path.join(dirpath, filename))
      listfile = os.path.join(dirpath, filename)

print "\t- Successful \n"

print "Calculating Average Max Temp from 1984-2015..."
TmaxRas = arcpy.sa.CellStatistics(TmaxList, "MEAN")

print "\t- Completed Cell Statistics... \n"

TmaxRas.save(outws + r'\Tmax30yr.tif')
print "\t- Successful \n"

Reply
0 Kudos
1 Solution

Accepted Solutions
Highlighted
MVP Esteemed Contributor

Take the advice... shut everything else down except what you need and don't forget 32 bit ArcMap only uses 4 Gb or ram regardless how much you have.  Another reason to use Pro and python 3.x.  Memory is cheap when you compare it to lost or slowdowns in time (try 32 Gb or ram and get the best fully memory loaded video card you can afford)

View solution in original post

Reply
0 Kudos
8 Replies
Highlighted
MVP Esteemed Contributor

throw a print statement before

TmaxRas.save(outws + r'\Tmax30yr.tif')

just to confirm that the path is correct.

Options to try

  • a different location
  • a different file type
  • save to a geodatabase

One of those seems to work for the dozens of Dr Google suggestions.

Highlighted
New Contributor III

Hi Dan, 

I tried the following three actions and still received the same error. I'm wondering if this is due to the number of inputs? It's daily data from 1984 to 2015, almost over 11,000 rasters. I also tried just limiting the process to two years, 730 rasters and still received the same results. It is completing but fails at saving. Any ideas or other suggestions? Can it possibly be some settings I'm missing in the script? 

print "\t- Completed Cell Statistics... \n"

Thanks for all your help! 

Reply
0 Kudos
Highlighted
MVP Esteemed Contributor

Well I tend not to use cell statistics, but I use NumPy... fewer problems and more functionality.

/blogs/dan_patterson/2016/08/14/reclassify-raster-data-simply 

https://community.esri.com/thread/173806 

The latter is a similarly related problem which you might be interested in.  the discussion from which it is branched is quite long, has some other options and there are other samples.

Reply
0 Kudos
Highlighted
New Contributor III

Thanks for your reply Dan!  

I added a couple of more print messages and it seems to be something with the .save() function. It keeps failing at that line. Would there be a possible way to get a more specified error message?  Say for example, is this being caused by temp files, not enough space, etc. 

Regarding, NumPy and your method, I read the thread. Would the "aust_temperature_demo.py" be something that I could modify to get a mean tif of temperature? 

Thanks! 

Reply
0 Kudos
Highlighted
MVP Esteemed Contributor

Go to the Results window... copy the info from their and your error message

Using the Results window—Help | ArcGIS for Desktop 

And yes it could be an option, but I have several distractions on the go now so you would be on your own for a bit.

In the interim, you should be able to parse your works into parts, then assemble.

I would also still like to see your file names and those results window error messages and the actual error message... those I can deal with, even though they may seem cryptic 

Reply
0 Kudos
Highlighted
New Contributor III

Hi Dan, 

 

After running the processes on ArcMap python window. I received the following error: 

I clipped my images to my study area and just processed two years worth of data, the processed work. I think this is most likely a RAM issue, since I have 1 TB hard drive on my computer. 

 

Thanks a million! 

 

Francisco 

Reply
0 Kudos
Highlighted
MVP Esteemed Contributor

Take the advice... shut everything else down except what you need and don't forget 32 bit ArcMap only uses 4 Gb or ram regardless how much you have.  Another reason to use Pro and python 3.x.  Memory is cheap when you compare it to lost or slowdowns in time (try 32 Gb or ram and get the best fully memory loaded video card you can afford)

View solution in original post

Reply
0 Kudos
Highlighted
New Contributor III

Dan thanks a million! 

Switched to GIS pro and Python 3.4. Worked like a charm! 

 

Francisco