SystemError: error return without exception set ; arcpy.da.Editor

PaulPanehal · ‎10-09-2019

I have had this never-ending question regarding how to run a Python Script to make updates to Versioned data.

How can I enter an Edit Session, on a pre-existing version, complete the edit session, review the changes, then reconcile and post WITHOUT causing a schema lock.

For some background, the below script is not in it's entirety:

I have created a database connection that is pointing to the intended, pre-existing version. The update cursor reads and updates a feature class using the above database connection. The update cursor completes successfully, updating correctly.

It is intended that the user will then manually review the changes, if the changes are good, reconcile and post. If bad, delete the version (I need a better process for this as well).

The Problem:

If the user enters an ArcMap Editor Toolbar Edit Session post update, checks everything, reconciles and posts. It works, no locks. Then, if you run the tool again, SystemError: error return without exception set. If I go to the Version Manager to view current locks, there is a lock on that version even after successful rec/post.

How does one prevent this lock from occurring. The python editing process is not documented in depth enough, and does not touch on how the delta tables are affected (I have tried to research the ArcObjects docs, which are mostly copied and pasted into arcpy). Why would I have schema locks if I am editing in one toolset vs. the other, but they're taking place after operations? I have tried many workarounds, numerous times, yet I am clearly missing something.

## Editor Class
"""
https://gis.stackexchange.com/a/158627
"""
def find_ws(path, ws_type=''):
    """finds a valid workspace path for an arcpy.da.Editor() Session

    Required:
        path -- path to features or workspace

    Optional:
        ws_type -- option to find specific workspace type (FileSystem|LocalDatabase|RemoteDatabase)

    """
    # try original path first
    if os.sep not in path:
        path = arcpy.Describe(path).catalogPath
    desc = arcpy.Describe(path)
    if hasattr(desc, 'workspaceType'):
        if ws_type and ws_type == desc.workspaceType:
            return path
        elif not ws_type:
            return path

    # search until finding a valid workspace
    SPLIT = filter(None, path.split(os.sep))
    if path.startswith('\\\\'):
        SPLIT[0] = r'\\{0}'.format(SPLIT[0])

    # find valid workspace
    for i in xrange(1, len(SPLIT)):
        sub_dir = os.sep.join(SPLIT[:-i])
        desc = arcpy.Describe(sub_dir)
        if hasattr(desc, 'workspaceType'):
            if ws_type and ws_type == desc.workspaceType:
                return sub_dir
            elif not ws_type:
                return sub_dir


class EditorUpdateCursor(object):
    """Wrapper class for arcpy.da.UpdateCursor to automatically
    implement editing (required for versioned data, and data with
    geometric networks, topologies, network datasets, and relationship
    classes"""
    def __init__(self, *args, **kwargs):
        """init wrapper class for update cursor Supported args:
            in_table, field_names, where_clause=None, spatial_reference=None
            explode_to_points=False, sql_clause=(None, None)"""
        self.args = args
        self.kwargs = kwargs
        self.edit = None

    def __enter__(self):
        ws = None
        if self.args:
            ws = find_ws(self.args[0])
        elif 'in_table' in self.kwargs:
            ws = find_ws(self.kwargs['in_table'])
        self.edit = arcpy.da.Editor(ws)
        self.edit.startEditing()
        self.edit.startOperation()
        return arcpy.da.UpdateCursor(*self.args, **self.kwargs)

    def __exit__(self, type, value, traceback):
        if traceback is None:
            self.edit.stopOperation()
            self.edit.stopEditing(True)
            self.edit = None
        else:
            self.edit.abortOperation()
            self.edit.stopEditing(False)
            self.edit = None

## Update Logic
if updatedStructuresCount > 0:
            arcpy.AddMessage('\nPole Numbers are being updated...')

            for structure in structureDict:
                where = """GLOBALID = '{}'""".format(structure)
                with editUpdCursor.EditorUpdateCursor(strPath, ['POLENUMBER'], where) as cursor:
                    for row in cursor:
                        newPoleNum = structureDict.get(structure)
                        row[0] = newPoleNum
                        cursor.updateRow(row)
            arcpy.AddMessage('Pole Numbers update complete...')
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

JoshuaBixby · ‎10-09-2019

If you run the tool once, rec/post and then exit out of ArcMap, does running the tool a second time work in a clean ArcMap session? I ask because this seems to be the ArcMap GUI session holding the lock, not ArcPy, so it might be a GUI workflow change that is needed and not changes to ArcPy code.

PaulPanehal · ‎10-09-2019

EDIT: It seems this did not resolve my problem entirely as it still seems to randomly lock the version. Here is the interesting part, if I use Reconcile Versions using the Database Connection point to the Version that did the edits and will be Reconciled and Posted, it may lock the version, and block entering an edit session with that version. If I do not use a Database Connection that points to that Version, it may still lock the version, but I will be able to enter an Edit Session using the Editor Toolbar, and then use the Versioning toolbar to Reconcile and Post, and it will remove the lock. Like I mentioned originally, it is absolutely bizarre to me.

You made a good point, so I decided to test out a number of cases to determine what would cause the error and resulting version lock.

There is some difference on how the arcpy.da.Editor editing session works / ends and how ArcMap's Editor Toolbar > Versioning Toolbar (GUI Session) works in the end.

The way I have avoided this error was to only use the Data Management Reconcile Versions geoprocessing tool. By reconciling and posting with that, I could continue using my tool without the error arising. The error seemed to occur when I would run my tool successfully, enter an edit session using the Editor Toolbar and Reconciling and Posting, then running my tool again wherein it would fail before the arcpy.da.Editor Session could begin.

The testing process was as follows:

All cases of running my tool were outside of an ArcMap Editor Toolbar editing session.
Case 1 (FAIL):
Run tool many times and reconcile and post using Editor Toolbar and Versioning Toolbar.
I ran my tool using a database connection that was pointed to a specific version numerous times without reconciling and posting. I then entered an edit session using the Editing Toolbar and using the Versioning Toolbar to reconcile and post. I ran my tool again, and it failed with the SystemError: error return without exception set.
I closed ArcMap and killed it's process. I ran the tool again numerous times, and followed the above bullet, failing again.
Case 2 (SUCCESS):
Restart ArcMap, run tool, use Reconcile Versions geoprocessing tool to Reconcile and Post on my version.
Ran tool again, and it successfully runs.
Case 3 (FAIL):
Consider possibility of there being a hidden lock after ArcMap GUI Reconcile and Post
Ran my tool, use ArcMap GUI, and wait some arbitrary time (~5 Minutes)
Ran my tool again, SystemError: error return without exception set
Case 4 (SUCCESS):
Test to see if ArcMap GUI works after Reconcile Versions geoprocessing tool
Ran my tool successfully, run Reconcile Versions geoprocessing tool
Enter ArcMap GUI editing session and reconcile and post
Run my tool again after successfully.

This might resolve my immediate issue, but I still hesitate to consider this solved without there being further discussion on the differences between the ArcObjects/Arcpy editing versus the toolbars in ArcMap.