topic Re: Overwrite of output not working in Python Questions

Overwrite of output not working

JennyAxelrod — Mon, 10 Sep 2012 21:55:33 GMT

Hi,
I have a script that writes .dbf files to a folder. The first time the script runs it does what it is supposed to. The second time I run it however, it says the first .dbf to be created already exists and stops. It's not overwriting the files.
I have arcpy.env.overwriteOutput = True and had it successfully working to overwrite at one point in the development of the script. I'm not sure what made it stop working. I'm pretty sure I added the field mapping and the listing of the files in the log file after it was running smoothly but, have commented them out and it still gives me the error message. Any suggestions would be appreciated....I have no idea what is causing the problem.

# Import arcpy module
import arcpy, sys, traceback, time, datetime, os

# Set overwrite option
arcpy.env.overwriteOutput = True

# INPUT NEEDED: Set Read me file workspace
CURDATE = datetime.date.today()
logpath = "X:\\Script\Out\ReadMe"
logfile1 = logpath + str (" ") + str(CURDATE) + ".txt"

if arcpy.Exists(logfile1):
    arcpy.Delete_management(logfile1)

log1 = open(logfile1, "a")

print "Generate Readme file " + str(CURDATE)
print >> log1, time.strftime("%c")
print >> log1, "End Processing "

#Set the current workspace for input file geodatabase
arcpy.env.workspace = "X:\\Script\Input.gdb"

#Set Local variables specifyting the output table folder:
outTable = "X:\\Script\Out"

print "Start Processing"

#Set a Search Cursor on the Polygons
srows = arcpy.SearchCursor("Polygons", "","", "WATERSHED", "" )
srow = srows.next()

print "Get Polygon"

mapFields = ["ID", "STUDY", "NAME"]
fieldMappings = arcpy.FieldMappings()
for mapField in mapFields:
    fieldMap = arcpy.FieldMap()
    fieldMap.addInputField("Points", mapField)
    fieldMappings.addFieldMap(fieldMap)

while srow:

    pid = srow.ObjectID
    poly_lyr = arcpy.MakeFeatureLayer_management("Polygons",'pgon_select',""" "ObjectID" = """+ str(pid))

    print "Watershed is " + str(srow.WATERSHED)

    arcpy.MakeFeatureLayer_management("Points", "Points_lyr", "", "", )
    print "Make point feature layer"


    arcpy.SelectLayerByLocation_management("Points_lyr", "INTERSECT", "pgon_select", "", "NEW_SELECTION")
    print "Select points by " + str(srow.WATERSHED) + " watershed"

    arcpy.TableToTable_conversion("Points_lyr", outTable , str(srow.WATERSHED)+ ".dbf","")
    print "Make table " + str(srow.WATERSHED)

    srow = srows.next()
    print "Next Polygon"

print "Write .dbf tables to log file"
print >> log1, "Files generated = "
file_list = os.listdir(outTable)
for file in file_list:
    if file.endswith (".dbf"):
        log1.write(file + "\n")

print "End Processing"

log1.close()

Re: Overwrite of output not working

markdenil — Tue, 11 Sep 2012 10:53:48 GMT

Honor the Zen of Python

The second line of the Zen of Python doxology is:
Explicit is better than implicit.

It is better to test for and remove files you want to replace (Explicit )
than to rely on an Arc environment setting (implicit).

Although python can import arcpy, python is not itself arcpy.
It does not necessarily honor an Arc environment.

You can get the Zen of Python by typing
import this
on the Python command line.

Re: Overwrite of output not working

curtvprice — Tue, 11 Sep 2012 15:55:46 GMT

# Import arcpy module import arcpy, sys, traceback, time, datetime, os  # Set overwrite option arcpy.env.overwriteOutput = True  # INPUT NEEDED: Set Read me file workspace CURDATE = datetime.date.today() logpath = "X:\\Script\Out\ReadMe" logfile1 = logpath + str (" ") + str(CURDATE) + ".txt"   if arcpy.Exists(logfile1):     arcpy.Delete_management(logfile1)  log1 = open(logfile1, "a")  print "Generate Readme file " + str(CURDATE) print >> log1, time.strftime("%c") print >> log1, "End Processing "    #Set the current workspace for input file geodatabase arcpy.env.workspace = "X:\\Script\Input.gdb"  #Set Local variables specifyting the output table folder: outTable = "X:\\Script\Out"  print "Start Processing"  #Set a Search Cursor on the Polygons srows = arcpy.SearchCursor("Polygons", "","", "WATERSHED", "" ) srow = srows.next()  print "Get Polygon"  mapFields = ["ID", "STUDY", "NAME"] fieldMappings = arcpy.FieldMappings() for mapField in mapFields:     fieldMap = arcpy.FieldMap()     fieldMap.addInputField("Points", mapField)     fieldMappings.addFieldMap(fieldMap)   while srow:      pid = srow.ObjectID     poly_lyr = arcpy.MakeFeatureLayer_management("Polygons",'pgon_select',""" "ObjectID" = """+ str(pid))          print "Watershed is " + str(srow.WATERSHED)         arcpy.MakeFeatureLayer_management("Points", "Points_lyr", "", "",  )     print "Make point feature layer"          arcpy.SelectLayerByLocation_management("Points_lyr", "INTERSECT", "pgon_select", "", "NEW_SELECTION")     print "Select points by " + str(srow.WATERSHED) + " watershed"      arcpy.TableToTable_conversion("Points_lyr", outTable , str(srow.WATERSHED)+ ".dbf","")     print "Make table " + str(srow.WATERSHED)          srow = srows.next()     print "Next Polygon"    print "Write .dbf tables to log file" print >> log1, "Files generated = " file_list = os.listdir(outTable) for file in file_list:     if file.endswith (".dbf"):         log1.write(file + "\n")   print "End Processing"   log1.close()

You have neglected to close the cursor when you are done. You must do this, otherwise the cursor stays open -- and you may run into file locking on the next run.

del srows

Also, using a layer object ("Polygons") with a tool that is attached to an open cursor (outside of directly accessing the geometry or actions like that) is generally bad practice as you are accessing the dataset from two objects at the same time.

I think it is much safer to create a list of unique watershed ids first, closing the cursor and then looping on that list.

Note the "for" loop construct below. This is the best way to loop through cursors in arcpy (until 10.1 when you should use arcpy.da).

#Set a Search Cursor on the Polygons srows = arcpy.SearchCursor("Polygons", "","", "WATERSHED", "" ) sheds= list() for srow in srows:   sheds.append(srow.WATERSHED) del row,srows # close/delete cursor variables  for shed in sheds:   ...    arcpy.MakeFeaturelayer_management('"WATERSHED" = \'' + shed + "'"...)   ...

Hope this helps.

Re: Overwrite of output not working

JennyAxelrod — Tue, 11 Sep 2012 18:12:28 GMT

mdenil,
Although the Zen is enlightening, the overwrite function still won't work as needed. Is there a specific piece of code you could suggest? I'm a python novice.

Re: Overwrite of output not working

curtvprice — Fri, 10 Dec 2021 21:03:10 GMT

mdenil,
Although the Zen is enlightening, the overwrite function still won't work as needed. Is there a specific piece of code you could suggest? I'm a python novice.

I think what Mark is getting at is inserting something like this before the TableToTable call to explicitly delete the table if it's already there:

tabName = srow.WATERSHED + ".dbf")
tabPath = os.path.join(outTable,tabName) 
if arcpy.exists(tabPath): arcpy.Delete_management(tabPath)
arcpy.TableToTable_conversion("Points_lyr", outTable , tabName)

However, my experience is that arcpy.env.overwriteOutput seems to work fine as long as there are no outstanding file locks (ie active layers pointing to it, etc).

Re: Overwrite of output not working

DanaDiotte1 — Fri, 10 Dec 2021 21:03:13 GMT

Hi. I ended up here because i was having the same problem. The suggestions here make sense as well. The thing is i don't want to put an

if arcpy.Exists()

for every layer.

I was having the same problem until i put my section of code between the

try: 
   <code> 
except: 
   <code>

and it worked. Why that makes all the difference is beyond me but it might be worth trying because it worked for me.

Re: Overwrite of output not working

LaurelKish — Wed, 24 Oct 2012 14:03:30 GMT

I don't know if it's a python issue or an ESRI/arcpy issue and I don't know why, but this last tidbit of info (try:except) solved a problem I've been having off and on for months. Halle-freaking-lujah ! Thank you.

Re: Overwrite of output not working

curtvprice — Fri, 10 Dec 2021 21:03:15 GMT

I don't know if it's a python issue or an ESRI/arcpy issue and I don't know why, but this last tidbit of info (try:except) solved a problem I've been having off and on for months. Halle-freaking-lujah ! Thank you.

Even better is a try - except - finally, to make sure things are cleaned up no matter what happens. (Layers, table views, open files, cursors, etc.)

For example:

try:
     lyr1 = arcpy.MakeFeatureLayer("myfile.shp","myLyr")
     # ... do stuff with lyr 1
except:
    # ... do stuff you want to do if it fails
finally:
    # do stuff you want to do no matter what, for example, try to delete the layer:
    try:
        arcpy.Delete_management(lyr1)
    except:
        pass