topic Re: Python keeps crashing in Geoprocessing Questions

Python keeps crashing

MarcGraham — Sun, 19 Sep 2010 09:39:17 GMT

Hi all,

I have written a script that finds all the mxds in a directory tree and creates a new shapefile with their extents as features. It works pretty well until python keeps crashing. This is driving me crazy, as it isn't an error as such, just a straight "python has stopped working".

If anybody could try this out, or let me know if they can find any issues with it that would be great.

One other problem is that sometimes the code fails as python cannot add the fields to the index featureclass, it says it doesn't exist, even though it has literally just been created.

Please excuse my beginners coding...

## This script is written to recursively search through a directory, find all the MXD files and create a polygon
## shapefile with a feature showing the extent of each dataframe.  The features have several attributes related
## to the dataframe extents.


## Import Modules
import os, fnmatch, arcpy, shutil, datetime, time

## Define Variables
print datetime.datetime.now(), "Defining Variables"

search_path = "C:\\Temp\\Python\\"
index_shape_path = search_path + "mxd_index\\"
mxd_index_name = "mxd_index.shp"
full_mxd_name = index_shape_path + mxd_index_name

geometry_type = "POLYGON"
template = ""
has_m = "DISABLED"
has_z = "DISABLED"
coordinate_system = "C:\Program Files\ArcGIS\Desktop10.0\Coordinate Systems\Projected Coordinate Systems\UTM\WGS 1984\Northern Hemisphere\WGS 1984 UTM Zone 60S.prj"

file_filter = "*.mxd"
file_type = file_filter[-3:]

arcpy.env.overwriteOutput = True
arcpy.env.workspace = index_shape_path

## Define Functions
print datetime.datetime.now(), "Defining Functions"

def locate(pattern,root=os.curdir):
    matches = []
    for path, dirs, files in os.walk(os.path.abspath(root)):
        for filename in fnmatch.filter(files,pattern):
            matches.append(os.path.join(path,filename))
    return matches

## 1a. Create a folder to hold the index featureclass
print datetime.datetime.now(), "Creating Folder for Index Featureclass"

if os.path.isdir(index_shape_path):
    shutil.rmtree(index_shape_path)
os.mkdir(index_shape_path)

## 1b. Create the empty featureclass with the required attributes
print datetime.datetime.now(), "Creating Empty Index Featureclass"

arcpy.CreateFeatureclass_management(index_shape_path, mxd_index_name, geometry_type, template, has_m, has_z, coordinate_system)

time.sleep(1)

arcpy.AddField_management(mxd_index_name, "MXD_PATH", "TEXT", "", "", 255)
arcpy.AddField_management(mxd_index_name, "COORD_SYS", "TEXT", "", "", 50)
arcpy.AddField_management(mxd_index_name, "SCALE", "DOUBLE")
arcpy.AddField_management(mxd_index_name, "X_MIN", "DOUBLE")
arcpy.AddField_management(mxd_index_name, "Y_MIN", "DOUBLE")
arcpy.AddField_management(mxd_index_name, "X_MAX", "DOUBLE")
arcpy.AddField_management(mxd_index_name, "Y_MAX", "DOUBLE")

## 1c.  Create a textfile to log errors

log_file = open(index_shape_path + "mxd_index_error_log.txt","w")

## 2. Create the insert cursor
print datetime.datetime.now(), "Creating Insert Cursor and Polygon Array"

insert_rows = arcpy.InsertCursor(full_mxd_name, coordinate_system)
polygon_array = arcpy.Array()
point = arcpy.Point()

## 3. Count and report the number of mxd files
print datetime.datetime.now(), "Counting MXD Files"

mxd_file_count = 0
for found_file in locate(file_filter, search_path):
    mxd_file_count += 1

print "There are", mxd_file_count, file_type, "files in", search_path

## 4. Insert a feature for each dataframe in each mxd in the list
print datetime.datetime.now(), "Inserting Dataframe Features"

for found_file in locate(file_filter, search_path):
    try:
        mxd = arcpy.mapping.MapDocument(found_file)
        
        for dataframe in arcpy.mapping.ListDataFrames(mxd,"*"):
            
            dataframe_extent = dataframe.extent
            dataframe_spatialreference = dataframe.spatialReference
            dataframe_spatialreference_name = dataframe.spatialReference.name
            
            print "Found", dataframe_spatialreference_name, "dataframe"
            
            insert_rows = arcpy.InsertCursor(index_shape_path+mxd_index_name, dataframe_spatialreference)
            polygon_feature = insert_rows.newRow()
            
            for vertex in [0,1,2,3,4]:
                point.ID = vertex
                if vertex in [0,3,4]:
                    point.X = dataframe_extent.XMax
                else:
                    point.X = dataframe_extent.XMin
                if vertex in [0,1,5]:
                    point.Y = dataframe_extent.YMax
                else:
                    point.Y = dataframe_extent.YMin
                polygon_array.add(point)
            
            polygon_feature.shape = polygon_array
            polygon_feature.MXD_PATH = found_file
            polygon_feature.COORD_SYS = dataframe_spatialreference_name
            polygon_feature.SCALE = dataframe.scale
            polygon_feature.X_MIN = dataframe_extent.XMin
            polygon_feature.Y_MIN = dataframe_extent.YMin
            polygon_feature.X_MAX = dataframe_extent.XMax
            polygon_feature.Y_MAX = dataframe_extent.YMax

            print "Inserting new dataframe extent from", found_file
            
            insert_rows.insertRow(polygon_feature)
            polygon_array.removeAll()

            log_file.write("Successfully added " + str(found_file) + "\n")
    except:
            log_file.write("Failed to add " + str(found_file))

print "Killing search cursor"

del insert_rows

Cheers,

Marc

Re: Python keeps crashing

DanPatterson_Retired — Sun, 19 Sep 2010 10:11:30 GMT

before I look at the script too much, this line
coordinate_system = "C\....
is going to cause problems since Python expects double backslashes \\ or a single forward slash /
or put a r in front of path (raw string, check Python help), so you line could be
coordinate_system = r"C\.....

Re: Python keeps crashing

MarcGraham — Sun, 19 Sep 2010 16:30:37 GMT

Oh yep, cheers for that heads up. Sloppy copy and paste... Later in the code it resets the cursor with the coordinate system from the dataframe that it is reading, so I am not sure if this matters... However I will change it and see if this helps at all.

Re: Python keeps crashing

MarcGraham — Tue, 21 Sep 2010 04:56:27 GMT

Fixed the path that was incorrect, and it still keeps crashing. I only have one machine available to test it on with ArcGIS 10 installed, so if anyone could try this out on their ArcGIS machine I would really appreciate it. About 30 secs before Python crashes a white square appears on the top left corner of my PC screen, about 3x3cm, and then Python crashes. I am not sure if it is specific to my computer, if it is bad coding or if there is a bug, so any help would be great.

Re: Python keeps crashing

runegullstrom — Wed, 22 Sep 2010 01:04:24 GMT

Marc,

I've debugged your script up until line 84.
The projection file path should be \Southern Hemisphere\ not Northern.
Just use an editor like PyScripter and debug your script for more errors,

cheers

Rune

Re: Python keeps crashing

MarcGraham — Thu, 23 Sep 2010 04:38:39 GMT

Thanks Rune.

That was a copy/paste error when I was pasting the code into the forum post. The code runs fine when the coordinate system is good, until some arbitrary point when python crashes, rather than getting an error...

Re: Python keeps crashing

runegullstrom — Fri, 24 Sep 2010 01:11:46 GMT

Marc,

I don't know what you're trying to do in this script, but why don't you step through your code rather than running it until it collapses?

I ran it again, this time till line 96 where it bails out 'cause I don't have the shape you're referring to.

If you don't know how to debug with your editor include a try / except with something like this:
except:
    tb = sys.exc_info()[2]
    tbinfo = traceback.format_tb(tb)[0]
    pymsg = "PYTHON ERRORS:\nTraceback Info:\n" + tbinfo + "\nError Info:\n    " + \
                  str(sys.exc_type)+ ": " + str(sys.exc_value) + "\n"
    print pymsg

be sure to include module traceback,

cheers

Re: Python keeps crashing

JPGlutting — Tue, 12 Oct 2010 15:03:22 GMT

Marc,

I don't know what you're trying to do in this script, but why don't you step through your code rather than running it until it collapses?

I ran it again, this time till line 96 where it bails out 'cause I don't have the shape you're referring to.

If you don't know how to debug with your editor include a try / except with something like this:
except:
    tb = sys.exc_info()[2]
    tbinfo = traceback.format_tb(tb)[0]
    pymsg = "PYTHON ERRORS:\nTraceback Info:\n" + tbinfo + "\nError Info:\n    " + \
                  str(sys.exc_type)+ ": " + str(sys.exc_value) + "\n"
    print pymsg

be sure to include module traceback,

cheers

I have been fiddling around with ArcMap error messages, and all you really need is:

except:
gp.AddError(traceback.format_exc())

AddError also prints to the terminal, at least for me, so there is no need to do an additional print. I'm not sure why tracebacks aren't automatic. [that didn't format right the first time, but it should be indented, of course]