topic Re: python script exits in the middle of running with no error in Python Questions

python script exits in the middle of running with no error

Diana_Di_Leonardo — Fri, 21 Aug 2020 21:03:38 GMT

I am trying to create a Python script to calculate the volume of sediment required for a marsh creation project. I've implemented the steps in the attached .py file. The data needed to run the script is in the .zip folder.

The code is written in Python 3. I am using ArcPro 2.6.0 (though I updated it today during my troubleshooting). I have been loading the code into Arc by right clicking in an Arc Python window.

The code has run through with no issues several times, but then I started having problems getting it to complete. There were no major changes to the code between when it worked and when I started having issues.

The first problem was that Arc was executing a random number of steps before stopping mid script with no error. If I then copy and paste each successive line into the Python window, it runs. Most often Arc was exiting after executing line 52:

deep_water = Minus(mwl,dem), but not always.

ArcPro wanted me to update. So I tried that. And started having a different version of the same problem.

The script started exiting after line 48: fill_depth = Minus(mc_raster,dem), again no error and not always. In addition, The raster does not display and although it has symbology defined as
, the statistics are .

Once, the script ran to completion after the second error started occurring. Surprisingly the answer seemed correct.

Restarting Arc does not seem to have any effect on the problem. With no error messages, I'm lost as to where the problem could be happening. Any suggestions are appreciated.

Re: python script exits in the middle of running with no error

DavidPike — Sun, 23 Aug 2020 12:11:47 GMT

Only things I can think of are to check out the spatial analyst extension, and be explicit rather than from arcpy.sa import * (I assume since I don't really download people's zips - posting the code would get more traction) use arcpy.sa.Minus(..

Also I'd ensure that the rasters are properly cast into Raster objects to allow the raster algebra.

Re: python script exits in the middle of running with no error

Diana_Di_Leonardo — Sun, 23 Aug 2020 16:53:15 GMT

Here is the code. I have already been explicit with the import for spatial analyst.

I was under the impression from the documentation that if I used the arcpy.sa.Minus, etc. I didn't need to cast into raster first. Is that not correct? I also did see anything that would suggest which was better to use: arcpy.sa.Minus(ras1,ras2) or Raster(ras1)-Raster(ras2). Is one preferable?

Thanks,

Diana

#Marsh volume tool
#Generate estimates of marsh volume/area that can be built with a given sediment input

#Inputs
#1 - topobathy DEM
#2 - marsh creation polygon with design elevation as a field
#3 - maximum allowable depth for fill placement (master plan default is 2.5 ft) ft or meters
#4 - Mean water level (mwl) raster

import arcpy
from arcpy import env
from arcpy.sa import Times, Minus, Con, Raster, ZonalStatistics

#for using as a script
#env.workspace = 'C:\\Users\\ddileonardo\\Documents\\ArcGIS\\Projects\\POWC_new\\MyProject1.gdb' #path to geodatabase to save; use default save geodatase for testing
dem           = "ngom_port_clip" #either make sure that input data is on the current path or provide full path
mc_polygon    = "marsh_creation_test_polygon" #polygon of the marsh creation area; needs a field for the design elevation
mc_raster     = mc_polygon +"_raster" #name for the raster made from the marsh creation polygon
mc_elev_field = "design_elevation" #name of the field containing the design elevation for the marsh creation
max_depth     = 2 #maximum allowable depth in same units as the DEM
mwl           = 0.1 #mean water level in same units as the DEM
res           = 10 #DEM resolution
'''
Below are the calculations. You should not need to edit.

'''
#1 - Convert marsh creation (MC) polygon to a raster value set to the design elevation.
#Use the topobathy DEM as the snap raster so the two align.
arcpy.env.snapRaster = dem
arcpy.PolygonToRaster_conversion(mc_polygon,mc_elev_field,mc_raster)

#2 - Build a raster that is fill depth by subtracting  the DEM raster from the MC elevation
fill_depth = Minus(mc_raster,dem)

#3 - Build a deep water raster for areas that are deeper than your allowable depth. 
#Subtract DEM raster from mwl raster
deep_water = Minus(mwl,dem)

#4 - Reclassify the deep water to 0 and the shallow water to 1 (essentially making a the deep water raster a booleon)
#if value of deep water raster is greater than the maximum allowable depth, the value becomes zero, else the value becomes 1

depth_classified = Con(Raster("deep_water") > max_depth,0,1)

#5 - Multiply fill depth rater from step 2 by the deep water 0/1 raster in step 4. 
#This will zero out fill depths for deep water
fill_depth_classified = Times(fill_depth,depth_classified)
#6 - Reclassify any negative fill depths to zero. Any fill depths that are currently zero stay zero
#This will account for areas where the existing DEM is above the planned MC design elevation
fill_depth_final = Con(Raster("fill_depth_classified") <= 0,0,"fill_depth_classified")

#7 - Multiply the fill depth by your DEM resolution to get a fill volume
fill_volume = Times(fill_depth_final,res)

#8 - Do a zonal statistics sum using the original MC footprint as your zone to calculate the total fill volume required
fill_volume_total = ZonalStatistics(mc_polygon,"OBJECTID",fill_volume, "SUM")

Re: python script exits in the middle of running with no error

DavidPike — Sun, 23 Aug 2020 19:21:54 GMT

Hi yes, this was before seeing your script, so apologies you're correct on most counts.

I can't see the spatial analyst extension being checked out? Which may be the issue since you can use it in immediate mode?

There are string-only paths specified but the arcpy env workspace is commented out?

I'm not sure of the new python sorting this out, but you may want to explicitly put your values as floats in-case of integer division happening (max_depth = 2.0 etc.)

Re: python script exits in the middle of running with no error

JoshuaBixby — Sat, 11 Dec 2021 23:30:56 GMT

I downloaded your data and Python script. When I first ran it, I got the following error:

Traceback (most recent call last):
  File "<string>", line 45, in <module>
  File "r:\arcgis\pro\Resources\arcpy\arcpy\conversion.py", line 2568, in PolygonToRaster
    raise e
  File "r:\arcgis\pro\Resources\arcpy\arcpy\conversion.py", line 2565, in PolygonToRaster
    retval = convertArcObjectToPythonObject(gp.PolygonToRaster_conversion(*gp_fixargs((in_features, value_field, out_rasterdataset, cell_assignment, priority_field, cellsize), True)))
  File "r:\arcgis\pro\Resources\arcpy\arcpy\geoprocessing\_base.py", line 511, in <lambda>
    return lambda *args: val(*gp_fixargs(args, True))
arcgisscripting.ExecuteError: Failed to execute. Parameters are not valid.
 ERROR 000728: Field design_elevation does not exist within table
Failed to execute (PolygonToRaster).‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I believe since you uploaded a shape file, the "design_elevation" field was truncated to "design_ele".

After double checking the field names and layer names were correct, I loaded the Python code into the interactive Python window using "Load Code", and everything worked fine. I got three new raster outputs.

I then tried running the script as a tool, but I got the following error:

Traceback (most recent call last):
  File "R:\Temp\foo\Marsh_creation_fill_volume_tool.py", line 58, in <module>
    depth_classified = Con(Raster("deep_water") > max_depth,0,1)
  File "r:\arcgis\pro\Resources\arcpy\arcpy\sa\Raster.py", line 79, in __new__
    return super().__new__(cls, in_raster, is_multidimensional)
RuntimeError:  ERROR 000732: Input Raster: Dataset deep_water does not exist or is not supported
 Failed to execute (Script)‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This error made me realize you are passing strings of variable names instead of the the variables themselves. For example, the following

depth_classified = Con(Raster("deep_water") > max_depth,0,1)‍‍

should be

depth_classified = Con(Raster(deep_water) > max_depth,0,1)‍‍

Once I cleaned up the strings, the script ran as a script tool just fine. The reason the original code ran in the Interactive Python window with the strings is that layers were being created and added to the TOC when the tools were run from the Python window. The layers that got added had the names of your strings, and so when you pass a string the geoprocessing tool was picking it up and a layer and referencing back to the data set.

Re: python script exits in the middle of running with no error

Diana_Di_Leonardo — Mon, 24 Aug 2020 14:31:06 GMT

It seems like passing the variable names as strings was the issue. I've just run it through successfully twice in a row. Thanks to both of you for your help!