Error 99999: create csv of pathways, filenames, and fields...works in beta, not final

ArielleSimmons · ‎09-24-2012

Hi all,

I have been working on a data inventory script. Idea is to use various os and arcpy functions to build a simple csv (i. .txt file) catalog of my data. In my beta testing stages, this worked perfectly. However, when running it on my server, I am getting an ???999999: Error executing function??? when I am not even a 1/4th of the way through recording my server files. Also, I noticed the .txt file I am outputting to is locked (which means my f.close() isn???t working right???but I don???t see how that could be???.looks right to me, and like I said, it works fine on my subset of data).

Anyone know what could be going on here?

Attached is the code. Copy and paste as is to keep the indents.

###############################################
## Arielle Simmons                           ##
## Planner/GIS Specialist                    ##
## Pioneer Valley Planning Commission        ##
## Date created: September 22, 2012          ##
## Date modified:                            ##
###############################################

## This script records the complete pathways of
## specified GIS data for documentation and inventory
## purposes. Also prints the date of the shapefile
## (Warning: this date could be EITHER the date
## modified or the creation date of the file...it
## is only pointing to the date of the file with
## that has been identified by the search pattern).

## Results of this script are all printed to
## an excel readable file via csv.

## This script can be modified to search
## for any ESRI feature class type (i.e. shapefile, geodatabase, etc).
## Due to the volume of data present, it is recommended that this
## script only be run for one feature class at a time. To do this, 
## modify the following:
## 1) the search pattern which is based
##     on file extensions (i.e. pattern = r"(file extension)")
## 2) the 'shp' variable name (rename for better identification of the
##     data type)
## 3) the directory where the feature classes you wish to search for
##     are contained (i.e. folder = r"(folder)"
## 4) BEFORE RUNNING THIS CODE,
##     In explorer, create and name a .txt file according to the
##     file extension you are configuring the code to search
##     on. MAKE SURE to set the f = open("filename.txt") path to it!!!


## Import modules and create geoprocessor object
## the csv module contains classes for working with csv files

import arcpy
import os
import fnmatch
import sys
import string
import os.path, time
import datetime
import csv
import traceback

from arcpy import env


# Set the workspace. List all of the shapefiles
# input workspace

arcpy.env.workspace = r"P:\GIS\Python_test\Ari_Python_script"

try:
# returns a list of shapefiles in the specified folder
# that match the specified wildcard search pattern

       folder =r"P:\GIS\Raid"
       pattern = "*.shp"

       shpList = []

       for path, dirs, files in os.walk(folder):
              for filename in fnmatch.filter(files, pattern):
                     shpList.append(os.path.join(path, filename))


# Warning: arcpy.AddMessage is only for importing this script into the
# ArcGIS ModelBuilder. Uncomment out to have it run properly.

# arcpy.AddMessage("Shapefiles returned:")

# open an output csv file

       f = open("P:\GIS\Python_test\Data_inventory\shapefiles_test.txt", 'w')

# for loop for printing outputs

       for shp in shpList:

# Warning: arcpy.AddMessage is only for importing this script into the
# ArcGIS ModelBuilder. Uncomment out to have it run properly.

# arcpy.AddMessage(shp)

# first create a datetime object (of when the file was modified OR created
# could be EITHER) which captures the dates of the files that the
# wildcard search is looking for...convert it to a string

              date_object = datetime.datetime.fromtimestamp(os.path.getmtime(shp))
#
              date = date_object.strftime('%m/%d/%Y')

# this is the 'for' loop that generates...
# ...a list of Fields, then it
# prints the list of fields for each featureclass

              allFields = ""
              fieldList = arcpy.ListFields(shp)
              for field in fieldList:
                     allFields = allFields + field.name + ", "


# concatenate the path, filename, date of when file was modified OR created
# (COULD BE EITHER), and field names into the identified csv file

       
              f.write(shp + ", " + os.path.basename(shp) + ", " + date + ", " + allFields + "\n")
# print path

              print shp
              
# print file name
              print os.path.basename(shp) + ","

# print date of when file was modified and or created

              print datetime.datetime.fromtimestamp(os.path.getmtime(shp)) 


       f.close()

except arcpy.ExecuteError:

# get the geoprocessing error messages

       print arcpy.GetMessages(2)


except Exception as e:

# print the error message

       print e.message

AndrewChapkowski · ‎09-24-2012

For your text file, you can try using a with statement:

  with open(r"P:\GIS\Python_test\Data_inventory\shapefiles_test.txt", 'w') as f:
     #do stuff

This will ensure it closes properly.

You can also try using try/except statements in your code and return the exact location where everything is failing.

import sys
import traceback
import inspect
def trace():
    '''
    trace finds the line, the filename and error message and returns it
    to the user
    '''
    tb = sys.exc_info()[2]
    tbinfo = traceback.format_tb(tb)[0]
    # script name + line number
    line = tbinfo.split(", ")[1]
    filename = inspect.getfile( inspect.currentframe() )
    # Get Python syntax error
    #
    synerror = traceback.format_exc().splitlines()[-1]
    return line, filename, synerror

This function should be called in the except part of your code.

try:
   # use your code
except:
   print trace() # or write to log file

This should help you catch your bug/error.