Proper Exception Handling in Cursors

[RogerDunnGIS]

I am experienced programmer, and no stranger to exceptions, try blocks, and exception classes in programming languages.  But I continue to be proven wrong when it comes to exceptions being thrown during arcpy cursor operations.

I am working with a file geodatabase and whenever an exception is thrown during a cursor operation (whether Search, Insert, or Update), when I go back to my map, my layers don't draw; when I go back to my attribute table, it is empty and shows an error message sometimes.  Here is pseudo-code for the pattern I use.  Any feedback and explanations would be helpful.  Note that this pattern works just fine when no exceptions are thrown.

def sampleFunc(fc, flds, query, data):
    with arcpy.da.UpdateCursor(fc, flds, query) as uCursor:
        try:
            for row in uCursor:
                # some business logic here which works for most rows but
                # may throw an unhandled exception on a particular row.
                row[4] = aFunction(row[3])
                uCursor.updateRow(row)
        finally:
            del uCursor

In order to see my map and attribute tables again, I simply close down the project (with or without saving) and open it up again.

[BlakeTerhune]

When using a with statement, there's no need to explicitly clean up the cursor yourself. Maybe that's causing some weirdness. Just let the cusor's context manager take care of it. Since you don't have an except block, try removing the try/finally.

When an exception occurs in your business logic, do you want to continue updating rows?

[RogerDunnGIS]

When I used to write code without the try...finally and deleting the cursor, it wasn't much better.  I would get an error when looking at the attribute which mentioned something like ArcGIS Pro was unable to see the fields.  And although the map functions would still work (like pan and zoom), any layers in the whole workspace wouldn't draw in new areas I zoomed to.

To try to prove my old aches and pains, I wrote the following code just now, which should throw an exception at OBJECTID 5:

with arcpy.da.UpdateCursor("AnchorPoints", "*") as uCursor:
    for row in uCursor:
        row[2] = 4.0
        if row[0] == 5:
            row[2] = 1/0
        uCursor.updateRow(row)

And a strange thing happened.  The first four records got a 4.0 in the 3rd field and the row with OBJECTID 5 didn't get one.  A ZeroDivisionError was thrown (as expected), but my map and attribute table are working fine!

Am I suffering from mechanic syndrome?  ("I swear it was making the noise before I drove it here today, but it's not making it now")

[RogerDunnGIS]

Note that, in the next cell, I did this:

print(type(uCursor))

And I got this:

<class 'da.UpdateCursor'>

I thought arcpy would "take care of it."  Doesn't the existence of the UpdateCursor keep an unnecessary lock on my feature class and/or workspace?

[BlakeTerhune]

---

@RogerDunnGIS wrote:

I thought arcpy would "take care of it."  Doesn't the existence of the UpdateCursor keep an unnecessary lock on my feature class and/or workspace?

---

I have not done a deep dive, but the documentation does say

Update cursors also support with statements to reset iteration and aid in removal of locks. However, using a del statement to delete the object or wrapping the cursor in a function to have the cursor object go out of scope should be considered to guard against all locking cases.

However, I have never experienced an issue with locks using a with statement. Even with an Editor session or list comprehension like

a_field_values = [i[0] for i in arcpy.da.SearchCursor("the_fc", ["a_field"])]

[RogerDunnGIS]

It's true that when my code behaves, all locks are removed and I can access both the layer and the attribute table.  It's just when an exception is thrown from the cursor that the layer stops drawing and the attribute table appears empty, sometimes with an error message.

Would you mind writing code on your end that throws an exception in the updateRow method?  For example, provide a value that is the wrong type for the field, or a string that is too long to fit.  And could you write code which creates an UpdateCursor but provide an incorrect field name and see how that should best be handled?  I'd just like to see some professional best practices which remove the locks and allow the map to draw again.  Thank you.

[BlakeTerhune · Accepted solution]

In Python 3 (ArcGIS Pro) this is what I typically do, mostly to track where errors occur so I can more easily troubleshoot to data. Especially if this update cursor work is in a separate function than your main code.

with arcpy.da.UpdateCursor("myFeatureClass", ["someUniqueID", "Field1", "Field2"]) as uCursor:
    for uid, field1, field2 in uCursor:
        try:
            # Do some business logic.
            field2 = field1 * 100
            # Commit new values to table.
            # To throw an error with the cursor:
            # use a value that does not conform to your field type
            # or change the number of values in the row.
            uCursor.updateRow([uid, field1, field2, "thisWillError"])
        except Exception as e:
            raise Exception(f"Error updating row at someUniqueID {uid}") from e

If you want the most coverage possible, following the "suggestions" in the documentation, I suppose you could do something like this (though I never have).

with arcpy.da.UpdateCursor("AnchorPoints", ["someUniqueID", "Field1", "Field2"]) as uCursor:
    try:
        for uid, field1, field2 in uCursor:
            try:
                # Do some business logic.
                field2 = field1 * 100
                # Commit new values to table.
                uCursor.updateRow([uid, field1, field2])
            except Exception as e:
                raise Exception(f"Error updating row at someUniqueID {uid}") from e
    finally:
        del uCursor

As for having an incorrect field name, the cursor will fail to be created so there's no cursor object to delete. In that case, you would have to follow the suggestion in the documentation and have it in a separate function so the everything gets cleaned up with garbage collection when it goes out of scope on exit (regardless of an exception). That's pretty tedious though.

You might need to contact Esri support about the behavior your seeing when the cursor fails.

[RogerDunnGIS]

Those are both great solutions in your last post.  I will mark it as a solution.  Although I frequently handle an exception in the try block, I didn't think of raising one, though I was deleting the cursor in the finally.  I bet you're right that since the exception was rethrown, the cursor will free the lock on the table, as designed.

One thing I discovered with my experiments is that if the map stops drawing due to an exception being thrown in cursor operation, I can close just the map tab, reopen it from the Catalog tab > Maps folder, and everything will start drawing again.  Thank you, Blake for your interest in this thread.  I hope it will benefit other scripters.