Frustrating .lock files

[jllorens]

I have created a Python package for use with python toolboxes (.pyt) which does a lot of the file management  and parameter management grunt work that I need my tools to do.  The point of this is to greatly simplify file naming and working with file directories as well as allowing me to access parameter values by name rather than index, all of which works well.  The user selects an output workspace, and depending on various other parameters the user sets and the nature of the output workspace, a file GDB is created.  An external script is called and this GDB is passed to it, all "intermediate" files which are outputs of other ESRI tools called in the script but not intended to be the final output of the script are stored in this file GDB.  The script completes and the final outputs are written to disk, and then the script tries to clean up by deleting all of the files out of the GDB and remove the GDB.  The point is to give the user the option of keeping these files or deleting them if they are not needed, since many can be used to review the output or used in further geoprocessing.

The problem is that sometimes the file GDB is NOT completely deleted.  All of the actual data in the GDB is removed and all of the files required for the GDB to function as a GDB are removed, but a few .lock files pertaining to a single raster dataset produced in the workflow remain along with the .GDB folder.  It seems to depend on the actual arcpy functions that I use to create these intermediate files, as some files created by some functions just really do not appreciate being deleted.

I have attached images of the problem below. (is there a way to post code?)

In the image below, you can see no errors are produced by the script and it completes successfully.  Here is an example of me running a simple script from this API which performs the standard workflow to produce watersheds from a DEM raster and a pour points feature class.  The outputs (a watersheds polygon feature class and a pour points feature class) look fine and are created properly.  

Tool window.

In the image below, you can see the file GDB is not entirely removed, a nonfunctional "remnant" .gdb folder is left with a few .lock files inside.  All of these .lock files pertain to a Pour_Point_Raster dataset which was saved to this file GDB during the script.  The dataset itself was removed successfully, along with all other "intermediate" datasets, yet the .lock files for this particular dataset remain.  Closing ArcMap causes these .lock files to dissapear and the .gdb folder can then be removed. 

The "remnant" file gdb folder.Below is the main body of the script.  I have pointed out the lines of code which pertain to the Pour_Points_Raster.  There are no references to this raster dataset after arcpy.RasterToPoint_conversion().

The main body of the script.The image below shows the bit of the package that runs after the above script has been completed (somewhere around when the output window says "Finalizing..."  Note that the issue is not related to the file name/path not being listed in self._intm_file_list, all "intermediate" files were purged from the file GDB and only the .lock files pertaining to the Pour_Points_Raster remained.  Also note that Compact_management() does not work (and removing this line doesn't fix the problem either). 

The portion of the script which is responsible for deleting the file gdb.

[curtvprice]

This all sounds good to me. I am not that versed in the details of how python does things as much as you are! You may have run into a bug in the snap pour point tool that should be fixed, please report it!

I wonder if your deferred-save scheme is similar to what the raster tools do when they only execute the tool if you call .save() or use it in executing a calculation later ("deferred execution" is what Esri raster heads call it).

Thanks for this thread, it was illuminating to me.

[jllorens]

No problem.  Keep in mind, I am theorizing here, and have extremely limited programming experience.  I have been using python for about 2 months now, and have only done some minor programming in C++ years ago on an Arduino.  I really have no idea what I am talking about, and am just going off of what I have read in the past few weeks about how all this works.  It just "looks" to me like one of the arcpy tool methods is "forgetting" to remove .lock files when it is done with the dataset.  

[DanPatterson]

The order that you delete things is important too.  You could still have a lock on something that is being used by another