ERROR 999998: Unexpected Error raster algebra

7547
42
02-08-2018 11:30 AM
PauloFlores
New Contributor II

Hi there,

 I am new at Pythin and during the last couple months I have working on writing some Python scripts to automatize some of my drone imagery analysis, such as the code bellow. I have written few scripts and used them on different datasets and they were working just fine. A couple days ago I realize that would be better to save my final raster (yellow) on the same folder that my original RGB raster (blue), so I can use that same path in other scripts for further analysis without having to copy rasters around. Then, I started to get a message ERROR 999998: Unexpected Error, and now does not matter which code I try to run, I get the same message. 

I have been looking around for a solution for my issue but I have been unsuccessful on that regard. By reading some material online, it seems that problem is related to memory issues, but I am not quite sure about that. Here are some details about my machine and software.

- desktop computer (Windows 10, version 1511, OS Build 10586.663, Processor Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz 3.41 GHz, RAM 32GB, System type 64-bit operating system, x64-based processor) . I have ArcGis 10.5  and I am using PythonWin (PythonWin 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (Intel)] on win32.) to write my scripts.

- I ran the code bellow on my Surface 3 laptop (Windows 10 Pro, version 10.0.16299 build 16299, system type x64-based PC, processor Intel Core i5-4300 CPU @ 1.90GHz, 2501 Mhz, 2 cores, 4 logical processors; RAM 8GB) this morning and it worked just fine.

-Last night I ran the same code from within ArcGIS and it worked just fine.

-I can not understand how everything was working just fine until a couple days ago, and suddenly the computer does not have enough memory to handle the same task. My RGB file is (around 1.8 GB in size, but I used files larger than that in the past with no problems.

 

Any help to fix this issue would be greatly appreciated.

Code:

import arcpy, string, os, errno

from arcpy import env

from arcpy.sa import*

from datetime import datetime

arcpy.CheckOutExtension("spatial")

arcpy.env.overwriteOutput = True

 

start_time = datetime.now()

 

imagery = r'E:\UAV2017\PythonField8ExGr\Field8_90ft_08012017.tif'

 

path = os.path.dirname(imagery)

base1 = os.path.basename(imagery)

base = os.path.splitext(base1)[0]

folder = "Results"

Results = os.path.join(path,folder)

if not os.path.exists(Results):

    os.makedirs(Results)

print "Results folder successfully created"

 

exgr_filename = base + "_ExGr.tif"

arcpy.env.workspace = imagery

bands = [Raster(os.path.join(imagery, b)) for b in arcpy.ListRasters()]

print bands

 

arcpy.env.workspace = Results

Ngreen = (Float(bands[1])) / (Float(bands[0]) + bands[1] + bands[2])

Nred = (Float(bands[0])) / (Float(bands[0]) + bands[1] + bands[2])

Nblue = (Float(bands[2])) / (Float(bands[0]) + bands[1] + bands[2])

exgr = (2 * Ngreen) - Nred - Nblue

exgr.save(exgr_filename)

print "Excess Green successfully calculated"

 

#Build pyramids and calculate statistics

arcpy.BuildPyramids_management(exgr_filename)

arcpy.BuildPyramids_management(exgr_filename)

print "Pyramids and Statistics successful"

 

end_time = datetime.now()

print('Duration: {}'.format(end_time - start_time))

 

Interactive Window messages:

E:\UAV2017\PythonField8ExGr

Field8_90ft_08012017

Results folder successfully created

Traceback (most recent call last):

  File "C:\Python27\ArcGIS10.5\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 325, in RunScript

    exec codeObject in __main__.__dict__

  File "C:\Users\Paulo.Flores\Desktop\Python Codes\New versions\ExGr_calc_02072018.py", line 45, in <module>

    exgr.save(exgr_filename)

RuntimeError: ERROR 999998: Unexpected Error.

0 Kudos
42 Replies
DanPatterson_Retired
MVP Emeritus

just to be sure... print is now 

print(.....enclose in round brackets.....)

unlike the python 2.7 legacy method

0 Kudos
PauloFlores
New Contributor II

I am not sure if understand your comment Dan. I still have Python2.7 with the new ArcGIS 10.6 installation.

0 Kudos
DanPatterson_Retired
MVP Emeritus

sorry... I thought Pro... no harm in planning ahead

0 Kudos
PauloFlores
New Contributor II

I have been playing around with the original code posted above tonight. I try to run it from PythonWin and from the Python window within ArcGIS. It worked once or twice using PythonWin and most of the times from within ArcGIS, but for the first time I got the same error message when running it within ArcGIS.

I was able to compile a different code to try to accomplish the same thing than the original one. You can find the code below. It works most of the time from PythonWin, but it does not yield the same results than the original code (see image below). I ran both versions of the code from within ArcGIS and compare the results (just min and max showing on the figure below) with one raster calculated step by step using the raster calculator in ArcGIS. Could someone help me to understand why the two codes yield different results?

Thanks.

import arcpy, string, os, errno
from arcpy import env
from arcpy.sa import*
from datetime import datetime
arcpy.CheckOutExtension("spatial")
arcpy.env.overwriteOutput = True

start_time = datetime.now()

imagery = r'E:\UAV2017\PythonField8ExGr\Field8_90ft_08012017.tif'
#Get path information to create Results folder, where NDVI and NDRE will be saved
path = os.path.dirname(imagery)
print path
base1 = os.path.basename(imagery)
base = os.path.splitext(base1)[0]
print base

#Saving name for Excess Green mosaic - using basename from the original imagery
exgr_filename = base + "_ExGr.tif"
Nred = "Nred.tif"
Nblue = "Nblue.tif"
Ngreen = "Ngreen.tif"

#Set workspace to multiband raster, then list rasters to get band names
arcpy.env.workspace = imagery
bands = [Raster(os.path.join(imagery, b)) for b in arcpy.ListRasters()]
print bands
Red = imagery + "\Band_1"
print Red
Green = imagery + "\Band_2"
print Green
Blue = imagery + "\Band_3"
print Blue

red_out = "Red.tif"
green_out = "Green.tif"
blue_out = "Blue.tif"

arcpy.CopyRaster_management(Red, red_out)
arcpy.CopyRaster_management(Green, green_out)
arcpy.CopyRaster_management(Blue, blue_out)
print "Rasters successfully copied"
#Set workspace to the location were the results will be saved
arcpy.env.workspace = path
nred_num = arcpy.sa.Float(Raster(red_out))
Denom = arcpy.sa.Float(Raster(red_out) + Raster(green_out)+ Raster(blue_out))
Nred_eq = arcpy.sa.Divide(nred_num, Denom)
Nred_eq.save(Nred)
print "Nred saved"
ngreen_num = arcpy.sa.Float(Raster(green_out))
Ngreen_eq = arcpy.sa.Divide(ngreen_num, Denom)
Ngreen_eq.save(Ngreen)
print "Ngreen saved"
nblue_num = arcpy.sa.Float(Raster(blue_out))
Nblue_eq = arcpy.sa.Divide(nblue_num, Denom)
Nblue_eq.save(Nblue)
print "NBlue saved"

Ngreen_x2 = arcpy.sa.Times(Ngreen, 2)
Nred_minus_Nblue = arcpy.sa.Minus(Nred, Nblue)
ExGr_eq = arcpy.sa.Minus(Ngreen_x2, Nred_minus_Nblue)
ExGr_eq.save(exgr_filename)
print "Excess Green calculation successful"

#Build pyramids and calculate statistics for each raster in env.worspace folder
arcpy.BuildPyramids_management(exgr_filename)
arcpy.BuildPyramids_management(exgr_filename)
print "Pyramids and Statistics successful"

end_time = datetime.now()
print('Duration: {}'.format(end_time - start_time))

0 Kudos
PauloFlores
New Contributor II

@ Dan Patterson, @ Xander Bakker

On my original code, posted above, does it matter the position of the item on line 25? Would the code work any different if it gets move to line 34? I am asking that because for the first time on my laptop I got the same error message that I am getting on my desktop. I was playing around with the code and for 3 times I got the same error message. Then, I moved the variable correspondent to exgr_filename (ndvi_filename on this case) to below the line that sets the workspace (for the code above that would be below line 33). Once I did that the code ran just fine. I am not sure it makes any difference code wise or it was a random luck moment.

Thanks.

0 Kudos
DanPatterson_Retired
MVP Emeritus

Paulo... could you edit the line numbers in your post, NOT the code, line 25

print bands

which I suspect is due to the code formatting changing line locations from your IDE position

0 Kudos
PauloFlores
New Contributor II

Dan,

I am not sure what code version you are looking at and founding line 25 as

print bands

 I am referring to the code posted on Feb 12, 2018 8:26 AM. 

On that version line 25 is as follow:

exgr_filename = os.path.join(Results, base + "_ExGr.tif")

and line 33 is this one

arcpy.env.workspace = Results

So, would make any difference on the code performance (sorry for the lack of a better term) if line 25 gets moved down to line 34 on that code?

Thanks again for your help.

0 Kudos
DanPatterson_Retired
MVP Emeritus

Sorry, the threads get out of order depending where you view them.

Performance-wise... no difference... as long as the objects are the same and the result is the same, you won't notice any

PauloFlores
New Contributor II

So, the code below has been working for me today on the same desktop that I am still having issues with the first code that I posted here. I ran the code below five times tonight, using three different rasters, ranging in size from around 0.5 to 1.8 GB, and it worked every time. I hope it will remain like that.

import arcpy, string, os, errno
from arcpy import env
from arcpy.sa import*
from datetime import datetime
arcpy.CheckOutExtension("spatial")
arcpy.env.overwriteOutput = True

start_time = datetime.now()

imagery = r'E:\UAV2017\MW70ftP10Python\MW70ft05122017.tif'
#Get path information to create Results folder, where NDVI and NDRE will be saved
folder = os.path.dirname(imagery)
print folder
base1 = os.path.basename(imagery)
base = os.path.splitext(base1)[0]
print base

#Saving name for Excess Green mosaic - using basename from the original imagery
exgr_filename = base + "_ExGr.tif"
nred_filename = base + "_Nred.tif"
ngreen_filename = base + "_Ngreen.tif"
nblue_filename = base + "_Nblue.tif"

#Getting the ratser's bands and saving them as individual rasters
desc = arcpy.Describe(imagery)  
for band in desc.children:
    bandName = band.name  
    band_path = os.path.join(imagery, bandName)  
    dest_path = os.path.join(folder, bandName + '.tif')  
    arcpy.CopyRaster_management(band_path, dest_path, "", "", "", "NONE", "NONE", "")
    print "Bands saved as rasters"
    
#Set workspace to the location were the results will be saved
arcpy.env.workspace = folder
band_1 = "Band_1.tif"
band_2 = "Band_2.tif"
band_3 = "Band_3.tif"
float_band1 = Float(band_1)
float_band2 = Float(band_2)
float_band3 = Float(band_3)

#Excess Green calculation - (2*Ngreen-Nred-Nblue)
nred = float_band1 / (float_band1 + float_band2 + float_band3)
print "Nred calculated"
ngreen = float_band2 / (float_band1 + float_band2 + float_band3)
print "Ngreen calculated"
nblue = float_band3 / (float_band1 + float_band2 + float_band3)
print "Nblue calculated"
exgr = (2 * ngreen) - nred - nblue
exgr.save(exgr_filename)
print "Excess Green successfully calculated"

#Build pyramids and calculate statistics for each raster in env.worspace folder
arcpy.BuildPyramids_management(exgr_filename)
arcpy.BuildPyramids_management(exgr_filename)
print "Pyramids and Statistics successful"

end_time = datetime.now()
print('Duration: {}'.format(end_time - start_time))
0 Kudos
CharlesFried
New Contributor III

Was there a resolution to this issue?

I have getting "RuntimeError: ERROR 999998: Unexpected Error." trying to save the results from arcpy.sa.EucAllocation running 64-bit python.  It appears to me that EucAllocation is creating a corrupted result, but the error message is most unhelpful it could be some environment setting but there are no clues.

0 Kudos