Area Solar Radiation from Python script - error

[toddsams]

Hello,

I am attempting to execute Area Solar Radiation in a stand-alone python script (shown below). The basis for the script was provided in the ESRI tool help. I made a few adjustments to this template by adjusting a few of the local variables and adding try/except etc.

The problem is that when I execute the script, it runs for about a minute then fails with an ERROR 999999 (also shown below in quoted script output).

Does anyone have an idea on why this may be occurring?

# Name: SolarRad_MayJuly.py # Description: Derives incoming solar radiation from a raster surface.  #              Outputs a global radiation raster and optional direct, diffuse and direct duration rasters #              for a specified time period. (May to July). #               # Requirements: Spatial Analyst Extension  # Import system modules import arcpy, time from arcpy import env from arcpy.sa import *  # Set environment settings env.workspace = "D:/Projects/Tasks/02_SpatialInputs/SolarRad/work/MayJulyOutput"  # Check out the ArcGIS Spatial Analyst extension license arcpy.CheckOutExtension("Spatial")  # Set local variables inRaster = "D:/Projects/Tasks/02_SpatialInputs/Geomorph/ElevationSlope/elev" latitude = 37.1567 skySize = 200 timeConfig = TimeMultipleDays(2012, 121, 212) dayInterval = 14 hourInterval = 1 zFactor = 1 calcDirections = 32 zenithDivisions = 8 azimuthDivisions = 8 diffuseProp = 0.3 transmittivity = 0.5 outDirectRad = "" outDiffuseRad = "" outDirectDur = ""  print time.asctime(time.localtime()) print "Running Area Solar Radiation..." try:     # Execute AreaSolarRadiation     outGlobalRad = AreaSolarRadiation(inRaster, latitude, skySize, timeConfig,        dayInterval, hourInterval, "NOINTERVAL", zFactor, "FROM_DEM",        calcDirections, zenithDivisions, azimuthDivisions, "UNIFORM_SKY",        diffuseProp, transmittivity, outDirectRad, outDiffuseRad, outDirectDur) except:     print "Error while processing solar radiation"     print (arcpy.GetMessages())  # Save the output  outGlobalRad.save("D:/Projects/Tasks/02_SpatialInputs/SolarRad/work/MayJulyOutput/globrad") print "Finished" print time.asctime(time.localtime())

Tue Jan 28 09:52:16 2014
Running Area Solar Radiation...
Error while processing solar radiation
Executing: AreaSolarRadiation D:/Projects/Tasks/02_SpatialInputs/Geomorph/ElevationSlope/elev D:/Projects/Tasks/02_SpatialInputs/SolarRad/work/MayJulyOutput\AreaSol_elev1 37.1567 200 "MultiDays   2012  121  212" 14 1 NOINTERVAL 1 FROM_DEM 32 8 8 UNIFORM_SKY 0.3 0.5 # # #
Start Time: Tue Jan 28 09:52:16 2014
ERROR 999999: Error executing function.
Failed to open raster dataset
Failed to execute (AreaSolarRadiation).
Failed at Tue Jan 28 09:53:33 2014 (Elapsed Time: 1 minutes 17 seconds)

[MathewCoyle]

It seems like something is wrong with your source data. What format is it in? Have you run this from ArcGIS with the same data source without issues?

[toddsams]

The source data are in GRID format. I have run Area Solar Radiation on GRID files successfully in ArcGIS in the past.

This particular data set does not successfully complete within ArcGIS. It just runs for about two days, then ArcCatalog quits with no error. I attributed this to the large size of the data set and the complexity of the Solar Radiation analysis. This is what led me to try in a standalone Python script, which aborts (with an error) much quicker than within ArcGIS.

[MathewCoyle]

I would first try moving it out of a GRID to a geodatabase raster or ascii. How big is your dataset? Have you tried breaking it up unto smaller sections? If performance is also something you want to gain you can look at multiprocessing. Esri has a blog entry on performing analysis on a large dataset using multithreading that you may find interesting.
http://blogs.esri.com/esri/arcgis/2011/08/29/multiprocessing/

[toddsams]

The data set is a 30m resolution with about 20,000 and 25,000 rows and columns (2GB integer GRID).

I have considered breaking into smaller sections, but would get edge effects if the sections do not overlap. I was just trying to give the full dataset a whirl before having to break up.

I will try other raster formats and look into multithreading. Right now, the process gets allocated to only a single core.

Thanks for your thoughts. Will check back if this does not work out.

[toddsams]

I tried using TIF, TXT, and a FDGDB raster. None of these worked.

Are there any other suggestions?

[toddsams]

I clipped the raster to a smaller area and it works ok. Looks like I will have to split the data up.

[MathewCoyle]

What are the specs of the machine you are running this on? You may be running in to memory issues, do you have the 64-bit background geoprocessing module installed?

[toddsams]

I have Windows 7 64-bit on a Core i7 with 12GB of RAM.

I had 64-bit geoprocessing installed with 10.0. I am not able to recall if I installed when I upgraded to 10.1. How would I check?

Since the process fails after only a minute, it seems that it does not like the size of the raster as it reads it into memory.

[MathewCoyle]

Easiest way is to open IDLE that installed with your version of ArcGIS. At the top it will say whether the default is 32 or 64 bit. As 10.0 and 10.1 use different versions of Python you almost definitely have the 32 bit version, so would have this tag.

...[MSC v.1500 32 bit (Intel)]...

64 bit would have this.

...[MSC v.1500 64 bit (AMD64)]...

Using 64 bit geoprocessing should solve this issue for you so I would look into getting that.