SystemError: error return without exception set & arcpy.da.Editor(workspace)

Procyon · ‎10-01-2019

I would like to create a script tool to loop through feature classes in a versioned Oracle database and replace a specified value within two specific fields (if they do exist). I am quite new to ArcPy so I am sure some of the tasks could have been implemented much more elegant. Below is the code I have written so far. Unfortunately, I stumble upon the following problems:

In a first approach, I used an UpdateCursor without starting an edit session and everything seemed to work fine. The problem was that changes did not seem to be permanent. I ran the script in a script tool in ArcMap and as soon as I closed the program, all changes were reverted
After that, I switched to the da.UpdateCursor as it is much faster. When I run the script in PyScripter, it stops at line 20 ("for row in cursor") with the following error message:

RuntimeError: Objects in this class cannot be updated outside an edit session [owner.example_edit]

Next, I tried to implement starting an edit session and I get the generic error message (line 14, "edit.startEditing(False, True)"):

SystemError: error return without exception set

I have consulted many forum posts related to this error and have tried many different things. At one point, it even run through but stopped at "edit.stopOperation()" with the same error. Unfortunately, I cannot figure out the exact problem and I am not able to solve it.

Further, but not so important, ListFeatureClasses() is very slow when I run the script in PyScripter. It takes several minutes to list all feature classes. On the other hand, if I run the same snippet directly in the python console in ArcMap, it takes only a few seconds.

Any kind of help is much appreciated!

Cheers

import arcpy
from arcpy import env
import os

env.workspace = r"G:\aaa\bbb\ccc.sde"
user_old = arcpy.GetParameterAsText(0)
user_new = arcpy.GetParameterAsText(1)
fields = "CREATE_BY", "CHANGE_BY"

fcList = arcpy.ListFeatureClasses("owner.*")
fcList.sort()

edit = arcpy.da.Editor(env.workspace)
edit.startEditing(False, True)
edit.startOperation()

for fc in fcList:
    if len(arcpy.ListFields(fc,"CREATE_BY"))>0:
        with arcpy.da.UpdateCursor(fc, fields) as cursor:
            for row in cursor:
                if row[0] == "domain" + "\\" + user_old:
                    row[0] = "domain" + "\\" + user_new
                    if row[1]  == "domain" + "\\" + user_old:
                        row[1] = "domain" + "\\" + user_new
                        cursor.updateRow(row)
                        arcpy.AddWarning("field in " + fc + " updated")

edit.stopOperation()
edit.stopEditing(True)

arcpy.AddWarning(user_old + " --> " + user_new + " successfully changed in all feature classes!")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

NeilAyres · ‎10-02-2019

Are you sure that you have these parameters correct?

NeilAyres · ‎10-02-2019

And does your user id have the correct permissions on that database?

Procyon · ‎10-02-2019

Thanks for the fast response!

I am not sure about the parameters, though. I read several times that for versioned data in a DB one should use the setting True, False. I also tried the default True, True and the opposite, False, True. Unfortunately, both without success.

About the permission: I am using a connection file for a technical user (TU) to connect to the default version (owner SDE). I only want to manipulate feature classes belonging to TU, but with the specified connection, I can also see feature classes belonging to other users (without privileges to manipulate them). E.g.:

- AB.parkinglots

- TU.agriculture

- TU.forestareas

- TU.waterbodies

- XY.airports

and so on ...

To avoid listing feature classes belonging to AB and XY, I used a wildcard in ListFeatureClasses (line 10). In the example above, it would be

fcList = arcpy.ListFeatureClasses("TU".*)

So, even if I don’t try to manipulate the feature classes from AB and XY, is it a problem to start an edit session and don’t have sufficient privilieges? Could that be the problem?

NeilAyres · ‎10-02-2019

I think you have to be the owner (sde) if you want to edit.

NeilAyres · ‎10-02-2019

Can you edit the data manually inside ArcMap or Pro?

Procyon · ‎10-02-2019

I can edit the data manually in ArcMap and Pro. We also use the data in map services and edit them via the web browser.

I'm not a DB specialist, but in my understanding, sde "only" owns the system objects. Maybe I'm wrong. TU is the owner of the user data and can therefore manipulate them.

I don't know if it is a problem that TU has also "Select" but no "Insert, Update and Delete" privileges on data of e.g. AB and XY when I try to start an edit session with ArcPy.

PaulPanehal · ‎10-09-2019

You should have a syntax error 'SyntaxError: EOL while scanning string literal' due to your fields variable, which should be a list.

You have a logical error that prevents the row from being updated within the Update Cursor unless the conditional is satisfied; disregard if that is intentional.



# Original
fields = "CREATE_BY", "CHANGE_BY" # Not a list as would be expected

for fc in fcList:
    if len(arcpy.ListFields(fc,"CREATE_BY"))>0:
        with arcpy.da.UpdateCursor(fc, fields) as cursor:
            for row in cursor:
                if row[0] == "domain" + "\\" + user_old:
                    row[0] = "domain" + "\\" + user_new
                    if row[1]  == "domain" + "\\" + user_old: #<--- Conditional
                        row[1] = "domain" + "\\" + user_new
                        cursor.updateRow(row)             # <------ Here
                        arcpy.AddWarning("field in " + fc + " updated")

# Updated
fields = ["CREATE_BY", "CHANGE_BY"] # Is a list, and is expected

for fc in fcList:
    if len(arcpy.ListFields(fc,"CREATE_BY"))>0:
        with arcpy.da.UpdateCursor(fc, fields) as cursor:
            for row in cursor:
                if row[0] == "domain" + "\\" + user_old:
                    row[0] = "domain" + "\\" + user_new
                    if row[1]  == "domain" + "\\" + user_old:
                        row[1] = "domain" + "\\" + user_new
                        arcpy.AddWarning("field in " + fc + " updated")
                    cursor.updateRow(row)         # <------ Will update if first cond. True

"""The original would only update your row when the conditional below is True:
         if row[1]  == "domain" + "\\" + user_old:
       The cursor.updateRow(row) was placed inside of the first conditional
       since it is assumed that you'd like it to only update when an update
       occurs."""‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

As for the 'SystemError: error return without exception set' error, this error is fleeting, and I have not figured out the exact reasoning for it either, but what you will likely find is that the version you were accessing has a schema lock on it. The only way to remove this lock is to have your SDE admin disconnect it, or close out of ArcMap, and possibly kill the ArcMap.exe process in the Task Manager.

I have found that this schema lock will happen when you are in an edit session through ArcMap's Editor Toolbar, and you try to run the arcpy.da.Editor tools on that versioned data, or the opposite, you run arcpy.da.Editor, then try to use the Editor Toolbar. I have yet to figure out the best way to be able to view the changes, and completely prevent this lock from occurring.

On another note, after a successful run of your script, if your Table of Contents in ArcMap is set to the versioned data source of your update, you can go up to the Versioning Toolbar, and click on Refresh, and it will pull your changes to be viewed, in or out of a Editor Toolbar Edit Session.

Procyon · ‎10-10-2019

Thank you for the interesting and detailed answer. I think there are many good points you mentioned I will have to take a closer look at.

Unfortunately, I am out of the office right now and will be back again in a week and a half.

I am sure that follow-up questions will arise and I would be glad if I could come back to you again.

Cheers

PaulPanehal · ‎10-10-2019

You're welcome, I also made a thread about the error SystemError: error return without exception set where I tested a number of cases where the error arose, and I found that it seems to happen in one case where you try to enter an edit session from arcpy.da.Editor, then later (after successful completion) with the ArcMap Editor Toolbar Editing tools. I linked it below.

SystemError: error return without exception set ; arcpy.da.Editor