Add Context Manager Methods to ArcPy Cursor Classes

[GIS_Spellblade]

TLDR: ArcPy Cursors do not universally respect Context Managers; Add Appropriate Methods to the Cursor Class to enable graceful exits (discard, disconnect, close) as part of the __exit__() call.

Recently an Esri blog was published detailing a relatively common "gotcha" pertaining to cursors applied against a file geodatabase: https://www.esri.com/arcgis-blog/products/arcgis-pro/data-management/locked-by-another-application-u...

The given solution was to embed a del call within a context manager (a with statement). This idea runs contrary to the syntactic sugar abstraction that is a context manager, or put otherwise, we use a context manager to not have to worry about some of the nitty-gritty of programming details like closing files and cursors. The suggested solution is problematic as it requires a user to have in-depth knowledge of what is likely an oversight due to the long-lived nature of the ArcPy library; further compounded by the lack of documentation outside of the blog's reference. https://realpython.com/python-with-statement/

This idea is to add in certain methods within the ArcPy Cursor Class to have it conform to the Python DBA Specification (https://peps.python.org/pep-0249/); while this sounds like a lot, in reality the additions would be fairly minimal and would be similar to the solution discussed in this blog post: https://dev.to/c_v_ya/sql-cursor-via-context-manager-2gc7

Having an implementation would make the ArcPy library more robust and allow the ArcPy Cursor to behave the same way regardless of the underlying data source-- --which would be the expectation for any other library, a function to behave the same way for all the inputs it is programmed to accept. This would benefit the users as they would not need to have an arcane understanding of a library that is possibly older than some of its users; they would be able to trust that ArcPy cursors comply with context managers which is a common usage pattern since Python 2.5.

[philnagel]

I believe we run into this issue quite a bit as well. I honestly had no idea the context manager for arcpy.da cursors was not a true context manager. This is a very frustrating issue and it seems crazy to me this is even a thing, considering that even ESRI staff are surprised by this as documented by the blog post you linked. ESRI, please fix this!

[HannesZiegler]

We're working on this!  This status does not guarantee that the functionality will be in the next release, but development work has begun. Release cycles vary by product so make sure to check the product life cycle information to get an idea of when to expect the next release.  

[HannesZiegler]

Congratulations! The idea made its way into the software (ArcGIS Pro 3.7) - arcpy.da Cursors now properly dispose of resources upon exiting the context scope. Manually deleting the cursor object to release locks is no longer necessary. You can see the difference if you run the sample code from the blog post referenced by the idea: Locked by another application using ArcPy and a File geodatabase.

Some workflows may unintentionally have leveraged this bug. For example, consider this code:

with arcpy.da.UpdateCursor(...) as cursor:
    for row in cursor:
        cursor.updateRow([...])

cursor.reset()            # Now fails with "I/O operation on closed file" (but did not pre 3.7)
for row in cursor: ...    # Now fails with "I/O operation on closed file" (but did not pre 3.7)

(Note the use of the context manager after it should have been disposed)

Going forward, context manager methods will no longer function outside of the scope of the context manager (as expected), such that cases like the above will no longer work. Keep everything within the context of the existing context manager or start a new context manager.

with arcpy.da.UpdateCursor(...) as cursor:
    for row in cursor:
        cursor.updateRow([...])

    cursor.reset()            # OK
    for row in cursor: ...    # OK