Cleaning up temporary layers created by python toolbox script

[JohnJackson]

ArcMap 10.2.1 / Python 2.7
I have several tools in a python toolbox. When I run through the tools in succession, temporary layers created in previously run tools show up as parameters in later tools.  These temporary layers are created with arcpy.MakeFeatureLayer_management and the corresponding feature classes are in the scratch workspace.  The layers show up in other tools for parameters where parameterType = GPFeatureLayer.

For example: Tool_A creates temporary layers temp1 and temp2 and then adds "Final Layer" to the TOC.  If I run Tool_B which takes a layer parameter, I would expect "Final Layer" to show up in the list but NOT layers temp1 & temp2.  The behavior I am seeing however is that all three layers show up as choices.

I am setting arcpy.env.addOutputsToMap = False at the beginning of each tool script and the temporary layers do not show up in the TOC as expected.  I also delete all layers from scratch at the end of each tool script.  Any help would be greatly appreciated.

[AdamCox1]

Maybe try arcpy.RefreshTOC() and arcpy.RefreshActiveView() after you delete the temporary layers?

[JohnJackson]

Thanks Adam.  I am actually calling both functions when I add the final layer to the map.

[AdamCox1]

Hmm, it may help to add those operations to the initializeParameters() method in the ToolValidator class for each tool, so that the parameters are definitely referencing an up-to-date mxd. Otherwise storing those temp feature classes in "in_memory\" instead of a scratch gdb may help too.

[FilipKrál · Accepted solution]

Hi,
Strange. I thought in-memory objects like layers and in_memory feature classes are deleted when a script finishes, but anyway...

Make sure that by the end of each script you delete the temporary feature classes (the actual data) as well as the temporary layers (in-memory objects pointing to the data).

I would love to hear how you guys do this and if there is good practice that really works. So far in my scripts I have been doing something like this (pseudo-code):

import arcpy # start of script  try:     todlt = [] # list to store everything I want to delete in the end.          # ... body of the script, whatever ...      my_buffer = arcpy.Buffer_analysis(...).getOutput(0)     todlt.append(my_buffer) # if my_buffer is a temporary feature class          my_buffer_lyr = arcpy.MakeFeatureLayer_management(my_buffer, 'lyrname', ...).getOutput(0)     todlt.append(my_buffer_lyr) # my_buffer_lyr is a temporary feature layer      # ... rest of the body of script ...      except arcpy.ExecuteError as e:     # handle ... except Exception as e:     # handle ... finally:     # cleanup     if 'todlt' in dir():         for dl in todlt:             try:                  arcpy.Delete_management(dl)             except:                 arcpy.AddWarning('Could not delete temporary data ' + str(dl))

By the way, I often use functions from my arcapi repository you may want to check out: https://github.com/NERC-CEH/arcapi
Specifically, function dlt() comes handy for the cleanup:

if 'todlt' in dir():     for dl in todlt:         ap.dlt(dl)

and function tstamp is useful to generate temporary layer names:

my_buffer_lyr = arcpy.MakeFeatureLayer_management(my_buffer, "lr" + ap.tstamp(tf = "%H%M%S") , ...).getOutput(0)

Filip.

[JohnJackson]

Thank you Filip!  After looking at your code and re-reading the documentation on Internal Layers http://resources.arcgis.com/en/help/main/10.2/index.html#//00210000000m000000, specifically:

When you create a layer or table view using geoprocessing tools, the new layer or table view is stored in an internal layer list, which is a different list from the ArcMap table of contents. This means that geoprocessing actually keeps two lists of layers and table views:

• The list of layers in the ArcMap table of contents


• The internal list of layers created by geoprocessing tools

I can see what I was doing wrong.  I thought I was deleting the temporary layers by getting a list of layers for the data frame and deleting from that list:

def clearLayers():
    mxd = arcpy.mapping.MapDocument('CURRENT')
    for df in arcpy.mapping.ListDataFrames(mxd):
        for lyr in arcpy.mapping.ListLayers(mxd, "", df):
            arcpy.mapping.RemoveLayer(df, lyr) 
    del mxd

This was not clearing the internal layer list (layers created by other geoprocessing tools) that the python tools use in part to generate parameter lists.  Having arcpy.env.addOutputsToMap = False had no effect on the internal layer list either (which it shouldn't).

[curtvprice]

This is worth discussing further. The help is not explicit about this and it's too bad -- if you don't clean up your layers, you leave joins and file locks polluting your session -- especially with script tools launched in-process from ArcGIS Desktop.

Here's the approach that I have been using.

from datetime import datetime
dtag = datetime.now().strftime("%Y%m%d%H%M%S") # '20140915110231'
lyrName = "lyr" + dtag
# set up temp variables
tmp_layer, tmp_table = [None] * 2
try:
    # ... create tmp_table name
    tmp_table = arcpy.CreateScratchName("", dtag, "table")
    arcpy.CreateTable_management(os.path.dirname(tmp_table),
                                os.path.basename(tmp_table))
    tmp_layer = arcpy.MakeFeatureLayer(tmp_table, lyrName)
    # ...
except:
    pass # normal error handling here
finally:
    for f in [tmp_layer, tmp_table]:
        if f:
            try:
                arcpy.Delete_management(f)
            except:
                pass

Note - we don't need to clean up lyrName because it's just a simple string object that will be deleted when the python script ends. The variable tmp_layer is assigned a result object returned from MakeFeatureLayer, but when you use it as input to a tool (like Delete_management) the tool is only passed the result object's string representation, which is just the string (above, "lyr201409015110231", or, below, "lyrname") so we don't need to unpack it with result.getOutput(0).

Here are some details, shown from the Python prompt (in ArcMap). Gotta love it.

>>> lyr = arcpy.MakeFeatureLayer_management(r"temp.shp", "lyrname")
>>> type(lyr)
<class 'arcpy.arcobjects.arcobjects.Result'>
>>> lyr
<Result 'lyrname'>
>>> str(lyr)
'lyrname'
>>> lyr.getOutput(0)
<map layer u'lyrname'>
>>> arcpy.Delete_management(lyr)
<Result 'true'>
>>> print(arcpy.GetMessages())
Executing: Delete lyrname #
Start Time: Mon Sep 15 11:32:05 2014
Succeeded at Mon Sep 15 11:32:05 2014 (Elapsed Time: 0.00 seconds)

[JamesCrandall]

I just call this def() before/after any in_memory processing.  Seems legit.

def clearINMEM():


  arcpy.env.workspace = r"IN_MEMORY"

  fcs = arcpy.ListFeatureClasses()
  tabs = arcpy.ListTables()

  ### for each FeatClass in the list of fcs's, delete it.
  for f in fcs:
    arcpy.Delete_management(f)
    arcpy.AddMessage("deleted: " + f)
  ### for each TableClass in the list of tab's, delete it.
  for t in tabs:
    arcpy.Delete_management(t)
    arcpy.AddMessage("deleted: " + t)

[curtvprice]

James, I believe we are discussing layers, not datasets (in_memory, on disk, or in an RDBMS). I realize I confused the thread a bit by including a table in my code sample. If you delete a feature class that is referenced by a open layer in memory, the Delete_management will not throw an error but the dataset will persist until the python session (not the script) is over. At least that's my experience.

For this reason, you'll see in my finally block that I listed layer first, then the dataset in my delete list.

[JamesCrandall]

Ah.  Your Delete_management of FeatureLayers works for me too.