topic Re: ERROR 999998 when saving a raster after Con in Python Questions

ERROR 999998 when saving a raster after Con

AreendamChanda — Fri, 22 Jun 2018 16:32:50 GMT

I am trying to do a very simple calculation on a raster.

I have done this kind of work many times, but I can't figure out what has gone wrong this time around.

import arcpy, numpy
from arcpy import env
from arcpy.sa import *
import time
start_time = time.time()
arcpy.CheckOutExtension("Spatial")
env.overwriteOutput = True
env.workspace = "G:/GIS Data/DMSPOLSRC/Radiance_Calibrated"
arcpy.env.scratchWorkspace="G:/GIS Data/scratch"

#1996
inRaster1 = arcpy.Raster("F12_19960316-19970212/F12_19960316-19970212_rad_v4.avg_vis.tif")
outRaster="rescaled/f1296rsc"
tempRaster=0.915*inRaster1
outCon = Con((inRaster1 ==0), 0 , 4.336+tempRaster)
outCon.save(outRaster)
print time.time() - start_time, "seconds, f121996completed"

Python returns the following error message

Traceback (most recent call last):
File "G:\GIS Data\DMSPOLSRC\Radiance_Calibrated\rescaling.py", line 16, in <module>
outCon.save(outRaster)
RuntimeError: ERROR 999998: Unexpected Error.

In other words something is going wrong when it tries to save the raster. I cant figure out for the life of me whats happening.

I should note the following

a) If I comment out the problematic line, it moves on and prints the time taken. In other words, it seems like there i no problem with the raster calculation itself.

b) If I go into ArcMap and use the raster calculator there, there is no problem if I save the outRaster to an existing geodatabase folder. However, it will return the following error when I try to save it to the target folder above:

Messages
Executing: RasterCalculator Con("F12_19960316-19970212_rad_v4.avg_vis.tif"==0,0, 4.336+0.915 * "F12_19960316-19970212_rad_v4.avg_vis.tif") "G:\GIS Data\DMSPOLSRC\Radiance_Calibrated\rescaled\f121996rsc"
Start Time: Fri Jun 22 11:26:13 2018
Con(Raster(r"F12_19960316-19970212_rad_v4.avg_vis.tif")==0,0, 4.336+0.915 * Raster(r"F12_19960316-19970212_rad_v4.avg_vis.tif"))
ERROR 000539: Error running expression: rcexec()
Traceback (most recent call last):
File "<expression>", line 1, in <module>
File "<string>", line 8, in rcexec
RuntimeError: ERROR 999998: Unexpected Error.

Failed to create raster dataset
Failed to execute (RasterCalculator).
Failed at Fri Jun 22 11:26:14 2018 (Elapsed Time: 0.85 seconds)

I am running ArcGis Desktop 10.6 and the python version is 2.7.14 (which comes with ArcGis).

Thanks

Re: ERROR 999998 when saving a raster after Con

DanPatterson_Retired — Sat, 23 Jun 2018 00:55:50 GMT

spaces, dots, dashes and slashes I suspect. It is not clear from your workspace setting whether

"F12_19960316-19970212/F12_19960316-19970212_rad_v4.avg_vis.tif")

is a folder within a folder within the workspace, it has a slash, a dash and a double dot (the last is needed but 4.avg isn't

Try a simpler workspace on a local drive with simpler names

Re: ERROR 999998 when saving a raster after Con

AreendamChanda — Sat, 23 Jun 2018 03:27:19 GMT

Thanks. Whittled it down to

inRaster1 = Raster("F121996Copy/rad_v4avg_vis.tif")

but still got the same error.

Re: ERROR 999998 when saving a raster after Con

DanPatterson_Retired — Sat, 23 Jun 2018 04:02:06 GMT

I really meant simplify your paths and names. I know those are what you want but I was thinking

c:/test where the *tif resides

test.tif as the filename

in any event,

Con(Raster(test.tif) == 0, 0, 1)

that is simplify the Con statement to see if it is the Raster(test.tif) that is the problem OR the calculation

Re: ERROR 999998 when saving a raster after Con

curtvprice — Sat, 23 Jun 2018 20:56:50 GMT

a) If I comment out the problematic line, it moves on and prints the time taken. In other words, it seems like there i no problem with the raster calculation itself.

The calculation doesn't happen until you do the .save() - so if you have an issue with the output workspace it will only happen when you to the save. Are sure the folder path, which saves to Esri grid format in a folder:

G:\GIS Data\DMSPOLSRC\Radiance_Calibrated\rescaled\f121996rsc

is valid? Does the rescaled folder exists, and what is in it? highly recommend saving to the .tif format because avoids problems related to corrupted "info" folders, filename, and size limitations of Esri grids. You can save to .tif format by simply adding the extension to your path "rescaled/f121996rsc.tif"

Another issue is your use of a folder has a space in it, I recommend avoiding that especially for raster tool operations. https://community.esri.com/people/curtvprice/blog/2016/05/07/how-to-name-things-in-arcgis?sr=search&searchId=77846afb-5ef6-4c5a-bfb8-30d504c6f413&searchIndex=0‌

Third suggestion, though I don't think this is your issue, if you are using 10.5.x I suggest installing these two patches.

ArcGIS 10.5.1 (Desktop, Engine, Server) Spatial Analyst Int and Abs tools Patch

ArcGIS 10.5.1 (Desktop, Engine, Server) Spatial Analyst Zonal Statistics Tool Patch

Re: ERROR 999998 when saving a raster after Con

AreendamChanda — Sat, 23 Jun 2018 22:31:37 GMT

I tried out your suggestions. I don’t think the file name and/or folder path was the issue. It failed even with basic locations like "G:/f1996.tif". Toe weed out any confusion about file paths, I invoked the os package and worked with it. As Curtis suggested I also switched to tif. I also cleaned out the rescaled folder.

The outcon command was the problem. It was not doing the required calculations. However, it would do the simpler command that Dan suggested.

But, the behavior of the Outcon command is bizarre.

It turns out I can get my outcon command to work if it is preceded by the simpler outcon command that Don suggested. Otherwise, it does not work. So my program only works if I define testRaster below and first do the simply outCon routine before performing the one I actually need. Any ideas?

import arcpy, numpy, os

from arcpy import env

from arcpy.sa import *

import time

start_time = time.time()

arcpy.CheckOutExtension("Spatial")

env.overwriteOutput = True

env.workspace = "G:/GIS Data/DMSPOLSRC/Radiance_Calibrated"

arcpy.env.scratchWorkspace="G:/GIS Data/scratch"

inws="G:/GIS Data/DMSPOLSRC/Radiance_Calibrated"

outws="G:/GIS Data/DMSPOLSRC/Radiance_Calibrated/rescaled"

#1996

inRaster1 = Raster(os.path.join(inws,"F121996Copy/rad_v4avg_vis.tif"))

print inRaster1

testRaster=os.path.join(outws, "f121996test.tif")

outCon = Con((inRaster1==0), 0 , 1 )

outCon.save(testRaster)

print "Completed", testRaster

outRaster=os.path.join(outws, "f121996rsc.tif")

outCon = Con((inRaster1==0), 0 , 4.336+(0.915*inRaster1))

outCon.save(outRaster)

print "Completed", outRaster

Re: ERROR 999998 when saving a raster after Con

curtvprice — Sat, 23 Jun 2018 22:52:19 GMT

That really is weird. I'd report this to Esri support.

When doing map algebra I like to set the scratch and output workspace to the same folder location. If temporary rasters are created behind the scenes (they are saved as .tif starting at 10.5), then a .save() operation to .tif format only has to rename, not copy, the temp so-called 'in memory' raster. This makes a big difference when your datasets get big, as they do.

Also, the spaces in paths thing with raster data is real, Scout's honor, although it has become less fatal over time. Dan and I will both attest that spaces in paths seem to work fine, but our students find that their workflows always fail if there are spaces in their paths the week their projects are due! So, we both always cringe when see "C:\GIS Data". If your scratch workspace still had a space in it when you saved to your C drive root, that may have been contributing to your issue. (I suspect your complex map algebra expression is requiring the software to write an intermediate raster to the scratch workspace under the hood, and if the temp raster is created behind the scenes like that first to a path with a space... maybe that's a problem.)

This is strange though, definitely worth taking it to support.

For an illustration of how temp rasters work in map algebra under the hood see:

https://community.esri.com/people/curtvprice/blog/2017/03/03/temporary-rasters-in-arcpy?sr=search&searchId=9db247f4-562e-4bd0-b4d4-aeafd0735bed&searchIndex=0

Re: ERROR 999998 when saving a raster after Con

AreendamChanda — Sun, 24 Jun 2018 17:34:43 GMT

Thanks. This is really very useful. I have self-taught ArcGIS and Python over the past year but my knowledge on both the outer and inner workings of folders, directories, relationships between databases is very vague.

Re: ERROR 999998 when saving a raster after Con

DanPatterson_Retired — Sun, 24 Jun 2018 18:08:57 GMT

Adrian.... and Python 3.x is going to add an extra dimension as well.

For some examples of how things go wrong with paths...

/blogs/dan_patterson/2016/08/14/filenames-and-file-paths-in-python