I've written a number of python scripts, involving UpdateCursor and InsertCursor, that are meant to be used during an edit session in ArcMap. The workspace is a multi-versioned SDE geodatabase. They scripts work fine as long as I perform at least one manual edit prior to running the script, which runs in ArcToolbox. However, if I begin an edit session and run the script prior to performing a manual edit, I receive an error: "The requested operation is invalid on a closed state."
My understanding of the reference is that unless an edit (either spatial or attribute) is first performed on a feature in the SDE database, the workspace remains in a closed state. I mentioned the workaround above; however, this is necessarily convenient and it's easy to forget. Is there something that can be written into the python script, a function perhaps, that can change the workspace to an open state? This would alleviate the need to perform a manual edit before running the script.
I appreciate any help or suggestions.
Chris
Chris,
Thanks for the manual edit workaround. You're right...it isn't convenient to remember to have to do manual edits, and it's hard to explain to others who might use the scripts.
I added arcpy.da.edit code to my scripts that edit SDE data, even if the scripts might be executed from within ArcMap during an edit session. The "edit.startOperation()" function prevents "The requested operation is invalid on a closed state" error from being thrown. It's an ugly workaround, but I had to use it with a similar issue within my Python Add-ins to bypass an apparent hardwired "protection" against making edits to SDE data without being in an edit session.
However...there are some weird consequences of starting the edit session within the script when it already exists in ArcMap. Although there is a "fatal" error in the script when it encounters edit.stopEditing(True), the edits actually persist. The edits made by the script can also be "undone" if there was a manual edit before the script was run, but the previous manual edit will also be undone. I remain somewhat concerned about the potential issues due to improper handling of the edit session and edit operations with SDE data, but it might be less of an issue for us than others, since we register our data as "versioned moving edits to base" and rarely have to deal conflicting edits.
Kerry
Kerry - I appreciate your reply. I hadn't thought about using the edit functions as a workaround. Typical ESRI - we have to use their tools to compensate for one of their glitches. Instead of using True, what happens if you replace with edit.stopEditing(False)? Would that avoid the fatal error?
I'll give your idea a shot. Thanks for the suggestion.
Chris
Hi Chris,
I haven't tried using edit.stopEditing(False), but I did try simply commenting out edit.stopEditing(True), which removes the error.
I'm assuming that "edit" was never really a valid Python edit object because the editor arcobject (at the heart of the Python editor object) is a singleton object, so only one instance can exist at a time. An "edit" session can't be closed if it doesn't really exist, hense the error. No clue why the start/stopOperations don't cause errors. Unfortunately the add-in framework requires edit sessions to be started within the add-in for SDE edits, so I have to put in the edit.startEditing(True,True) code just to avoid the errors, but the "editor" within the addin otherwise unfunctional.
I'm pretty sure that my add-ins worked with all of the edit code included, as long as the user did not have a pre-existing edit session running in ArcMap, but that doesn't seem to be the case now (10.3 upgrade, maybe?), so now I don't have much of a reason to keep the edit.stopEditing(True) line in my add-ins. However, I just discovered another post by someone else who just removes that line, and they noticed some issues if they try to make additional edits on newly created features that haven't been properly saved.
Bottom line, I think, is that there is no "correct" way to handle this arcpy/SDE edit issue... just different workarounds with different disadvantages.
Kerry
You are correct that it requires a start/stop editing process in arcpy. I found this to be annoying, so I created some wrapper classes to automatically stop and start the edit sessions for cursors. I made one for Update and Insert Cursors that make use of the __enter__ and __exit__ methods to handle the edit session initializing and closing (must be used with a "with" statement). These make it much more convenient:
import arcpy
import os
class WSMixin(object):
@staticmethod
def find_ws(path, ws_type='', return_type=False):
"""finds a valid workspace path for an arcpy.da.Editor() Session
Required:
path -- path to features or workspace
Optional:
ws_type -- option to find specific workspace type (FileSystem|LocalDatabase|RemoteDatabase)
return_type -- option to return workspace type as well. If this option is selected, a tuple
of the full workspace path and type are returned
"""
def find_existing(path):
if arcpy.Exists(path):
return path
else:
if not arcpy.Exists(path):
return find_existing(os.path.dirname(path))
# try original path first
if isinstance(path, (arcpy.mapping.Layer, arcpy.mapping.TableView)):
path = path.dataSource
if os.sep not in str(path):
if hasattr(path, 'dataSource'):
path = path.dataSource
else:
path = arcpy.Describe(path).catalogPath
path = find_existing(path)
desc = arcpy.Describe(path)
if hasattr(desc, 'workspaceType'):
if ws_type == desc.workspaceType:
if return_type:
return (path, desc.workspaceType)
else:
return path
else:
if return_type:
return (path, desc.workspaceType)
else:
return path
# search until finding a valid workspace
path = str(path)
SPLIT = filter(None, str(path).split(os.sep))
if path.startswith('\\\\'):
SPLIT[0] = r'\\{0}'.format(SPLIT[0])
# find valid workspace
for i in xrange(1, len(SPLIT)):
sub_dir = os.sep.join(SPLIT[:-i])
desc = arcpy.Describe(sub_dir)
if hasattr(desc, 'workspaceType'):
if ws_type == desc.workspaceType:
if return_type:
return (sub_dir, desc.workspaceType)
else:
return sub_dir
else:
if return_type:
return (sub_dir, desc.workspaceType)
else:
return sub_dir
class UpdateCursor(WSMixin):
"""wrapper clas for arcpy.da.UpdateCursor, to automatically
implement editing (required for versioned data, and data with
geometric networks, topologies, network datasets, and relationship
classes"""
def __init__(self, *args, **kwargs):
"""initiate wrapper class for update cursor. Supported args:
in_table, field_names, where_clause=None, spatial_reference=None,
explode_to_points=False, sql_clause=(None, None)
"""
self.args = args
self.kwargs = kwargs
self.edit = None
self.alreadyInEditSession = False
def __enter__(self):
ws = None
if self.args:
ws = self.find_ws(self.args[0])
elif 'in_table' in self.kwargs:
ws = self.find_ws(self.kwargs['in_table'])
try:
self.edit = arcpy.da.Editor(ws)
self.edit.startEditing()
self.edit.startOperation()
self.cursor = arcpy.da.UpdateCursor(*self.args, **self.kwargs)
return self.cursor
except Exception as e:
# explicit check for active edit session, do not attempt starting new session
if isinstance(e, RuntimeError) and e.message == 'start edit session':
self.cursor = arcpy.da.UpdateCursor(*self.args, **self.kwargs)
self.alreadyInEditSession = True
return self.cursor
else:
raise e
def __exit__(self, type, value, traceback):
if not self.alreadyInEditSession:
try:
self.edit.stopOperation()
self.edit.stopEditing(True)
except:
pass
self.edit = None
try:
del self.cursor
except:
pass
class InsertCursor(WSMixin):
"""wrapper clas for arcpy.da.InsertCursor, to automatically
implement editing (required for versioned data, and data with
geometric networks, topologies, network datasets, and relationship
classes"""
def __init__(self, *args, **kwargs):
"""initiate wrapper class for update cursor. Supported args:
in_table, field_names
"""
self.args = args
self.kwargs = kwargs
self.edit = None
self.alreadyInEditSession = False
def __enter__(self):
ws = None
if self.args:
ws = self.find_ws(self.args[0])
elif 'in_table' in self.kwargs:
ws = self.find_ws(self.kwargs['in_table'])
try:
self.edit = arcpy.da.Editor(ws)
self.edit.startEditing()
self.edit.startOperation()
self.cursor = arcpy.da.InsertCursor(*self.args, **self.kwargs)
return self.cursor
except Exception as e:
# explicit check for active edit session, do not attempt starting new session
if isinstance(e, RuntimeError) and e.message == 'start edit session':
self.cursor = arcpy.da.InsertCursor(*self.args, **self.kwargs)
self.alreadyInEditSession = True
return self.cursor
else:
raise e
def __exit__(self, type, value, traceback):
if not self.alreadyInEditSession:
try:
self.edit.stopOperation()
self.edit.stopEditing(True)
except:
pass
self.edit = None
try:
del self.cursor
except:
pass
I have named the above module as "cursors.py". With this, you can simply do the following:
import cursors
fc = r'<some sde feature class>'
fields = ['Field1', 'Field2', ...]
# use update cursor wrapper with "with" statement
with cursors.UpdateCursor(fc, fields) as rows: # initializing edit session with __enter__
for r in rows:
# do stuff
rows.updateRow(r)
# the __exit__ method will handle closing the edit session, so at this point we are out of the edit session!
This saves me a bunch of redundant code, so I hope others find it useful as well. This code will also catch the error thrown if an edit session is already started, if so, you could revert to normal cursors if already in an edit session.
Following for the ArcPy Pro rewrite
Coming soon...we have to migrate a lot of code to be compatible with 3.x
Here is your 3.x compatible version
import os
import arcpy
import warnings
class WSMixin(object):
@staticmethod
def find_ws(path, ws_type='', return_type=False):
"""finds a valid workspace path for an arcpy.da.Editor() Session
Required:
path -- path to features or workspace
Optional:
ws_type -- option to find specific workspace type (FileSystem|LocalDatabase|RemoteDatabase)
return_type -- option to return workspace type as well. If this option is selected, a tuple
of the full workspace path and type are returned
"""
def find_existing(path):
if arcpy.Exists(path):
return arcpy.Describe(path).catalogPath
else:
if not arcpy.Exists(path):
return find_existing(os.path.dirname(path))
# try original path first
if hasattr(arcpy, 'mapping'):
if isinstance(path, (arcpy.mapping.Layer, arcpy.mapping.TableView)):
path = path.dataSource
else:
if isinstance(path, (arcpy._mp.Layer, arcpy._mp.Table)):
path = path.dataSource
if os.sep not in str(path):
if hasattr(path, 'dataSource'):
path = path.dataSource
else:
path = arcpy.Describe(path).catalogPath
path = find_existing(path)
# first make sure it's not an "in_memory" workspace
if (path or '').startswith('in_memory'):
return ('in_memory', 'LocalDatabase') if return_type else 'in_memory'
desc = arcpy.Describe(path)
if hasattr(desc, 'workspaceType'):
if ws_type == desc.workspaceType:
if return_type:
return (path, desc.workspaceType)
else:
return path
else:
if return_type:
return (path, desc.workspaceType)
else:
return path
# search until finding a valid workspace
path = str(path)
split = list(filter(None, str(path).split(os.sep)))
if path.startswith('\\\\'):
split[0] = r'\\{0}'.format(split[0])
# find valid workspace
for i in range(1, len(split)):
sub_dir = os.sep.join(split[:-i])
desc = arcpy.Describe(sub_dir)
if hasattr(desc, 'workspaceType'):
if ws_type == desc.workspaceType:
if return_type:
return (sub_dir, desc.workspaceType)
else:
return sub_dir
else:
if return_type:
return (sub_dir, desc.workspaceType)
else:
return sub_dir
class InsertCursor(WSMixin):
"""wrapper clas for arcpy.da.InsertCursor, to automatically
implement editing (required for versioned data, and data with
geometric networks, topologies, network datasets, and relationship
classes"""
def __init__(self, *args, **kwargs):
"""initiate wrapper class for update cursor. Supported args:
in_table, field_names
"""
self.args = args
self.kwargs = kwargs
self.edit = None
self.alreadyInEditSession = False
def __enter__(self):
ws = None
if self.args:
ws = self.find_ws(self.args[0])
elif 'in_table' in self.kwargs:
ws = self.find_ws(self.kwargs['in_table'])
try:
self.edit = arcpy.da.Editor(ws)
self.edit.startEditing()
self.edit.startOperation()
self.cursor = arcpy.da.InsertCursor(*self.args, **self.kwargs)
return self.cursor
except Exception as e:
# explicit check for active edit session, do not attempt starting new session
if isinstance(e, RuntimeError) and e.message == 'start edit session':
self.cursor = arcpy.da.InsertCursor(*self.args, **self.kwargs)
self.alreadyInEditSession = True
return self.cursor
else:
raise e
def __exit__(self, type, value, traceback):
if not self.alreadyInEditSession:
try:
self.edit.stopOperation()
self.edit.stopEditing(True)
except Exception as e:
warnings.warn("Exception On Insert Cursor! Records May Not Have Inserted: {}".format(e))
self.edit = None
try:
del self.cursor
except:
pass
class UpdateCursor(WSMixin):
"""wrapper clas for arcpy.da.UpdateCursor, to automatically
implement editing (required for versioned data, and data with
geometric networks, topologies, network datasets, and relationship
classes"""
def __init__(self, *args, **kwargs):
"""initiate wrapper class for update cursor. Supported args:
in_table, field_names, where_clause=None, spatial_reference=None,
explode_to_points=False, sql_clause=(None, None)
"""
self.args = args
self.kwargs = kwargs
self.edit = None
self.alreadyInEditSession = False
def __enter__(self):
ws = None
if self.args:
ws = self.find_ws(self.args[0])
elif 'in_table' in self.kwargs:
ws = self.find_ws(self.kwargs['in_table'])
try:
self.edit = arcpy.da.Editor(ws)
self.edit.startEditing()
self.edit.startOperation()
self.cursor = arcpy.da.UpdateCursor(*self.args, **self.kwargs)
return self.cursor
except Exception as e:
# explicit check for active edit session, do not attempt starting new session
if isinstance(e, RuntimeError) and e.message == 'start edit session':
self.cursor = arcpy.da.UpdateCursor(*self.args, **self.kwargs)
self.alreadyInEditSession = True
return self.cursor
else:
raise e
def __exit__(self, type, value, traceback):
if not self.alreadyInEditSession:
try:
self.edit.stopOperation()
self.edit.stopEditing(True)
except Exception as e:
warnings.warn("Exception On Update Cursor! Records May Not Have Updated: {}".format(e))
self.edit = None
try:
del self.cursor
except:
pass
Caleb Mackey Amazing! I thought I will die in this creepy ArcGIS Python API, but your cursors immediately fixed last issues.
How many years will it take for esri to adopt these workarounds...
I just ran into this problem too on with ArcMap 10.2.2. I have an .mxd that uses Task Assistant Manager. Some of the steps in Task Assistant point to a custom python toolbox to insert/update/delete records or related records. All of these tools run while ArcMap is already within an edit session. My tools worked when I had my sde database versioned with edits saved to base, but when I reconfigured the versioning without going to base, the python tools would throw a "closed state" error.
My fix, thanks to the comments above was, before cursors I start an edit session, then startOperation() and stopOperation() after the cursor has completed. I do not use edit.stopEditing because that throws an error. The user can then save their edits from the editing toolbar.
It sure would be nice to have a way for python to identify if the Arcmap is in an editing session.
Christina