Select to view content in your preferred language

ERROR 999998: Unexpected Error raster algebra

8796
42
02-08-2018 11:30 AM
PauloFlores
Emerging Contributor

Hi there,

 I am new at Pythin and during the last couple months I have working on writing some Python scripts to automatize some of my drone imagery analysis, such as the code bellow. I have written few scripts and used them on different datasets and they were working just fine. A couple days ago I realize that would be better to save my final raster (yellow) on the same folder that my original RGB raster (blue), so I can use that same path in other scripts for further analysis without having to copy rasters around. Then, I started to get a message ERROR 999998: Unexpected Error, and now does not matter which code I try to run, I get the same message. 

I have been looking around for a solution for my issue but I have been unsuccessful on that regard. By reading some material online, it seems that problem is related to memory issues, but I am not quite sure about that. Here are some details about my machine and software.

- desktop computer (Windows 10, version 1511, OS Build 10586.663, Processor Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz 3.41 GHz, RAM 32GB, System type 64-bit operating system, x64-based processor) . I have ArcGis 10.5  and I am using PythonWin (PythonWin 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (Intel)] on win32.) to write my scripts.

- I ran the code bellow on my Surface 3 laptop (Windows 10 Pro, version 10.0.16299 build 16299, system type x64-based PC, processor Intel Core i5-4300 CPU @ 1.90GHz, 2501 Mhz, 2 cores, 4 logical processors; RAM 8GB) this morning and it worked just fine.

-Last night I ran the same code from within ArcGIS and it worked just fine.

-I can not understand how everything was working just fine until a couple days ago, and suddenly the computer does not have enough memory to handle the same task. My RGB file is (around 1.8 GB in size, but I used files larger than that in the past with no problems.

 

Any help to fix this issue would be greatly appreciated.

Code:

import arcpy, string, os, errno

from arcpy import env

from arcpy.sa import*

from datetime import datetime

arcpy.CheckOutExtension("spatial")

arcpy.env.overwriteOutput = True

 

start_time = datetime.now()

 

imagery = r'E:\UAV2017\PythonField8ExGr\Field8_90ft_08012017.tif'

 

path = os.path.dirname(imagery)

base1 = os.path.basename(imagery)

base = os.path.splitext(base1)[0]

folder = "Results"

Results = os.path.join(path,folder)

if not os.path.exists(Results):

    os.makedirs(Results)

print "Results folder successfully created"

 

exgr_filename = base + "_ExGr.tif"

arcpy.env.workspace = imagery

bands = [Raster(os.path.join(imagery, b)) for b in arcpy.ListRasters()]

print bands

 

arcpy.env.workspace = Results

Ngreen = (Float(bands[1])) / (Float(bands[0]) + bands[1] + bands[2])

Nred = (Float(bands[0])) / (Float(bands[0]) + bands[1] + bands[2])

Nblue = (Float(bands[2])) / (Float(bands[0]) + bands[1] + bands[2])

exgr = (2 * Ngreen) - Nred - Nblue

exgr.save(exgr_filename)

print "Excess Green successfully calculated"

 

#Build pyramids and calculate statistics

arcpy.BuildPyramids_management(exgr_filename)

arcpy.BuildPyramids_management(exgr_filename)

print "Pyramids and Statistics successful"

 

end_time = datetime.now()

print('Duration: {}'.format(end_time - start_time))

 

Interactive Window messages:

E:\UAV2017\PythonField8ExGr

Field8_90ft_08012017

Results folder successfully created

Traceback (most recent call last):

  File "C:\Python27\ArcGIS10.5\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 325, in RunScript

    exec codeObject in __main__.__dict__

  File "C:\Users\Paulo.Flores\Desktop\Python Codes\New versions\ExGr_calc_02072018.py", line 45, in <module>

    exgr.save(exgr_filename)

RuntimeError: ERROR 999998: Unexpected Error.

0 Kudos
42 Replies
DanPatterson_Retired
MVP Emeritus

Well I was going to try it on the Surface Book...

I suspect it is just the superiorness of the machines...

If it works on one machine and not another then it has something to do with the environments on that machine.

*.tif is the correct extension as well.

I will still have a look

PauloFlores
Emerging Contributor

Dan,

You mentioned that it has something to do with the environments on my desktop. How would explain the fact that it was working previously on that machine and suddenly it does not work anymore? I am the only user of this machine, and I have not made any changes to the environments on that desktop.

Thanks.

0 Kudos
XanderBakker
Esri Esteemed Contributor

I just ran the code with a small change and it ran without problems.

The only thing I changed was the setting of the workspace (your line 36) and I changed it to point at a fgdb. This was enough for it to finish correctly:

C:\GeoNet\UAV2017
Field8_90ft_08012017
Results folder successfully created
C:\GeoNet\UAV2017\Results\Field8_90ft_08012017_ExGr.tif

Excess Green successfully calculated
Pyramids and Statistics successful
Duration: 0:10:39.098000

Resulting image:

Before the change it created tmp results in the workspace for the 3 band calculations and those rasters may cause problems due to the length of the names and the less efficient data storage that TIF uses, hence the memory problems I guess.

PauloFlores
Emerging Contributor

Xander,

Would you mind sharing the changes that you made to the code with me? As I mentioned previously, I am new using Python and I have not used geodatabases before.

Thanks,

0 Kudos
XanderBakker
Esri Esteemed Contributor

Do you want the code to create the output FGDB too?

0 Kudos
PauloFlores
Emerging Contributor

Dear Xander and Dan,

I made a small change on the code an it is working now. Maybe you guys can help me understand why. I just basically comment out lines 31, 32, and 33. I ran the code using a shorter name version (without the base) and a long one (with the base as part of the name), and both of them worked. The only other change was line 36, where the env.workspace is set to path (line 13). You can see the results bellow and the raster image in ArcGIS.

import arcpy, string, os, errno

from arcpy import env
from arcpy.sa import*
from datetime import datetime
arcpy.CheckOutExtension("spatial")
arcpy.env.overwriteOutput = True

start_time = datetime.now()

imagery = r'E:\UAV2017\PythonField8ExGr\Field8_90ft_08012017.tif'
#Get path information to create Results folder, where NDVI and NDRE will be saved
path = os.path.dirname(imagery)
print path
base1 = os.path.basename(imagery)
base = os.path.splitext(base1)[0]
print base
folder = "Results2"
Results2 = os.path.join(path,folder)
if not os.path.exists(Results2):
    os.makedirs(Results2)
print "Results folder successfully created"

#Saving name for Excess Green mosaic - using basename from the original imagery
exgr_filename = os.path.join(Results, base + "_ExGr.tif")
ngreen_filename = os.path.join(Results, base + "Ngreen.tif")
nred_filename = os.path.join(Results, base + "Nred.tif")
nblue_filename = os.path.join(Results, base + "Nblue.tif")

#Set workspace to multiband raster, then list rasters to get band names
##arcpy.env.workspace = imagery
##bands = [Raster(os.path.join(imagery, b)) for b in arcpy.ListRasters()]
##print bands

#Set workspace to the location were the results will be saved
arcpy.env.workspace = path
#Excess Green calculation - (2*Ngreen-Nred-Nblue)
Ngreen = (Float(bands[1])) / (Float(bands[0]) + bands[1] + bands[2])
Ngreen.save(ngreen_filename)
print "Ngreen succesfully calculated"
Nred = (Float(bands[0])) / (Float(bands[0]) + bands[1] + bands[2])
Nred.save(nred_filename)
print "Nred succesfully calculated"
Nblue = (Float(bands[2])) / (Float(bands[0]) + bands[1] + bands[2])
Nblue.save(nblue_filename)
print "Nblue succesfully calculated"
exgr = ((2 * Ngreen) - Nred - Nblue)
exgr.save(exgr_filename)
print "Excess Green successfully calculated"

#Build pyramids and calculate statistics for each raster in env.worspace folder
arcpy.BuildPyramids_management(exgr_filename)
arcpy.CalculateStatistics_management(exgr_filename)
print "Pyramids and Statistics successful"

end_time = datetime.now()
print('Duration: {}'.format(end_time - start_time))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

E:\UAV2017\PythonField8ExGr
Field8_90ft_08012017
Results folder successfully created
Ngreen succesfully calculated
Nred succesfully calculated
Nblue succesfully calculated
Excess Green successfully calculated
Pyramids and Statistics successful
Duration: 0:02:50.354000‍‍‍‍‍‍‍‍‍

ExGr Image ArcGIS

0 Kudos
XanderBakker
Esri Esteemed Contributor

That would be a miracle, since you are referencing the bands variable that will never be set.  Can you post the code that actually did work?

0 Kudos
PauloFlores
Emerging Contributor

The code is posted above, with the results and the ArcGIS screenshot.

0 Kudos
XanderBakker
Esri Esteemed Contributor

Hi Paulo Flores , 

In my previous remark I was referring to the variable "bands" that is used on line 38, 41 and 44, but you commented out its creation on lines 31 and 32. That's why this code will break at line 38. If you were able to run the code like this, you probably did that using the Python window of ArcMap where variables used in a previous run still exist. This code when executed in a stand alone IDE will not run successfully...

0 Kudos
PauloFlores
Emerging Contributor

Hi Xander,

I ran the code twice previously using the PythonWin and it worked. I just try to run the code again, the same way, after restarting my computer and gives me a message that the “bands” is not defined, which is exactly what you are saying.

Thanks.