Python Crashing Unexpectedly

[KarlZimmer]

Hello,

I have created a bunch of python scripts that seemed to be working fine. However, I attempted to run one over-night and it failed about halfway through the process, so only half the rasters were processed. Now for some unknown reason all my scripts are causing python to crash unexpectedly.  Its very strange as they were working fine yesterday.  In this script it appears to crash at the reclassify stage. I'm trying to run them in PythonWin but Python command line also seems to crash. Any ideas on why this would start happening all the sudden?  I ran some simpler code and PythonWin worked fine.

[KarlZimmer]

Hello,

I wanted to give an update on my crashing issue. I re-installed Python and Python win and still am having crashing issues; however, my script did run for a couple hours this time before giving me the following message:

Runtime Error!
Program:C:\Python25\Lib\sitepackages\Pythonwin\Pythonwin.exe

This application has requested the runtime to terminate in an unusual way. Please contact the application support for more assistance.

I also tried running it in Python Command Line and the code terminated early and gave me an output thats said:
FATAL ERROR
DIRECTORY FULL

My hard drive containing the data has over 40 GB of free space so I don't think its a hard drive issue. Is it perhaps some sort of RAM  memmory getting full as the code reiterates?

Any help would be appreciated. Thanks in advance.

[ChrisSnyder]

How many rasters are you processing?

Does it always crash on the same raster? And if so, what sets that raster apart from the others that successfully finish? What is that raster's order number (example: is it the 255th in the list?).

Some comments on your code:

1. Just check out the SA license once at the top, you code could? be checking out a license on every loop.
2. I see that you are twice resetting the gp.workspace in the loop, however your input raster and output raster BOTH use gp.workspace variable. you really only need to set the gp.workspace when using the gp.list*() commands. Otherwise I would recommend doing something like this:

outputDir = r"C:\temp\test"
outputRaster = outputDir + "\\rcl_" + rs

Another question: Your input files seem to be .tif files with rather long names. Grid format (the naive output of the Spatial Analyst tools) has a name limit of 13 characters. Could it be that you are exceeding the limit using your "rs" variable names?

[KarlZimmer]

Hi Chris,

Thanks for the reply and programming advice. I'm new to python so I appreciate all tips!

As for the problem, I'm processing about 400 rasters. The problem doesn't seem to occur with any one particular raster.  I updated the code with the recommendations you provided including checking out the extension at the start and set the outputs to output directories rather than the workspace. I didn't rename the rasters as I don't think this is the problem as occasionally the code works successfully.  I've been doing some reading on the message board and think there could be a memory leak issue and the code is terminating when the memory is full. Unfortunately, I know very little about what this actually means or how to go about fixing it. Its happening to me on multiple python codes now (see attached).When I ran the the Cloud fill one on ArcGIS 9.2 it ran without problems.  However, when I try to run it on 9.3.1 it is getting the same run-time error. I think there might be a connection between the scripts as well as the runtime error on the reclass code seems to mostly occur after I've ran the larger cloud fill code. I also noticed intermediate layers in the scratch workspace are not being deleted and .aux files are continually accumulating in the scratch workspace as the scripts run. Correct me if I'm wrong but shouldn't these files be deleted upon completion of geoprocessing?

Any more insight, comments or insight would be greatly appreciated.

Thanks for all your help.

[KarlZimmer]

I think I partially solved the problem. It looks like as my code is running the intermeidate files are not deleting as the script runs so the scratch workspace folder continues to accumulate files, which in this case it appears as if the scratch workspace folder becomes full at 50,000 files and therefore causes the program to crash.  As a result as soon as I manually delete some of these files form the scratch folder all scripts using this same scratch workspace folder function fine until the 50,000 file threshold is reached.  As such I fixed the problem by simply coding in a create and remove command that removes and recreates the Scratch folder every time the script iterates. This temporarily solves the problem but still doesn't solve the problem of why the intermediate files are not automatically deleted from the scratch workspace as the script runs.

Thanks again for your help Chris.

Anyone have any idea how to set the geoprocesser so that the intermediate files get deleted as the script runs?