Layers Disappear after running Arcpy Notebook with Edit Session, and Nested Update Cursors

RWATT — Fri, 14 Oct 2022 17:03:00 GMT

Hello,

I'm working on an ArcGIS Pro (v3.0.2) notebook that copies attributes between a point (1) and a line (2) and then copies the line feature (2) to an identical line feature (3).

Details on each of these features:

1. A Feature Service Feature Class of Survey Entries from Survey123. They have important attributes from the survey, but no meaningful geographic component.

2. An Enterprise Geodatabase in PostgreSQL (v10.6.1) Feature Class of lines. This feature class is receiving the attributes from 1.

3. A Feature Service Feature Class of lines. This feature class recieves the entire feature from 2 via Copy>Paste Special, or arcpy.management.Append.

This process works consistently and without error messages.

However, these code cells will occasionally cause some or all layers in my map and corresponding layout to "disappear".

These layers remain in the Contents pane, I can still access their attribute tables, and sometimes I can still select them by clicking in the map.

Their visibility is restored by any of the following:

1. Closing and reopening the map and layout.

2. Removing the layer from the Contents and re-adding it.

3. Clearing the cache via Project>Options>Display>"Clear Cache Now"

4. Closing and reopening the project.

Their visibility is NOT restored by:

1. Refreshing the map in the lower-right corner.

2. Clearing the layer cache in the contents pane.

3. Refreshing the database connection

4. Clearing the workspace environment. arcpy.ClearEnvironment('workspace')

5. Deleting the workspace. del arcpy.env.workspace, workspace

6. Clearing the workspace cache. arcpy.management.ClearWorkspaceCache()

I think this is likely an issue with the database connection, but I'm out of ideas on how to avoid it. Any suggestions are appreciated. Thanks!

Three Code Cells:

#This ipynb is built for use in ArcGIS Pro to produce new digital service cards from Survey123 and service points collected in the field. #This cell sets up the context for future cells #imports. getpass is to pull the windows username for the PDF details at the end. import arcpy, getpass #define the project, map, mapview, layout, map frame in the layout, survey entry layer, and workspace. aprx = arcpy.mp.ArcGISProject('CURRENT') m = aprx.listMaps("Editing_Map")[0] mv = aprx.activeView layout = aprx.listLayouts("CombinedLayout")[0] map_frame = layout.listElements("MAPFRAME_ELEMENT", "Map Frame")[0] lyr = m.listLayers("Survey_Entries")[0] #arcpy.env.workspace = r"C:\Users\USER\Documents\ArcGIS\Projects\Service Cards\arcgis(2).sde"

#This cell will: #validate that only a single gisdb service is selected, and: #Pull all of the survey content into the gisdb service feature based on matching Work Order # #Pull the Service ID from the gisdb service to the Survey point (We use this to track which surveys have been processed) #Copy the entire gisdb service feature into the AGOL layer. workspace = r"C:/Users/USER/Documents/ArcGIS/Projects/Service Cards/arcgis(2).sde" #Check that only one record is selected in the gisdb services layer if int(arcpy.management.GetCount("gisdb.sde.Service").getOutput (0)) == 1: #start an edit session edit = arcpy.da.Editor(workspace) #Start editing. I don't need to undo in this process and the workspace is versioned. edit.startEditing(with_undo=False, multiuser_mode=True) edit.startOperation() #Sync services to survey. ## field names in the service feature class. Many have been removed for brevity service_fields = ['w_o_','city','installati', 'premises', 'house_no', 'street', 'lot_no', ] ## field names in the survey feature class. Many have been removed for brevity. survey_fields = ['USER_WO_Number', 'USER_Town', 'USER_Date_Completed', 'premise_ids', 'house_number', 'street_name', 'apt_unit'] ##select the service and survey feature classes service_lines = m.listLayers("gisdb.sde.Service")[0] survey = lyr #Create update cursor "service_cursor" for services with arcpy.da.UpdateCursor(service_lines, service_fields) as service_cursor: #iterate through lines/features/services in service_lines for service_row in service_cursor: #Create update curor "service_cursor" for survey points with arcpy.da.UpdateCursor(survey, survey_fields) as survey_cursor: for survey_row in survey_cursor: ## find the entry with the same WO ID if service_row[0] == survey_row[0]: ## WO ## set each field in the service layer equal to the value in the ## corresponding field from the survey layer ## Towns formatted to fit in server domain if "" in survey_row[1]: service_row[1] = "" elif "" in survey_row[1]: service_row[1] = "" elif "" in survey_row[1]: service_row[1] = "" elif "" in survey_row[1]: service_row[1] = "" elif "" in survey_row[1]: service_row[1] = "" elif "" in survey_row[1]: service_row[1] = "" elif "" in survey_row[1]: service_row[1] = "" elif "" in survey_row[1]: service_row[1] = "" else: service_row[1] = survey_row[1].replace("_", " ") service_row[2] = survey_row[2].replace(hour=12) ## completion date service_row[3] = survey_row[3] ## premises service_row[4] = survey_row[4] ## house number service_row[5] = survey_row[5].title() ## street service_row[6] = survey_row[6] ## lot/unit service_row[7] = survey_row[7] ## size service_row[8] = survey_row[7] ## size service_row[9] = survey_row[8] ## material - text if survey_row[8] == 'Steel': ## if the survey has material as steel service_row[10] = 3 ## then set service material to 3, which is coded as steel else: service_row[10] = 5 ## otherwise set it to 5 which is plastic #Many attributes have been removed for brevity and privacy here. survey_row[37] = service_row[46] ## service ID #save the service ID to a new variable for use in the next cell. service_input = service_row[46] service_cursor.updateRow(service_row) survey_cursor.updateRow(survey_row) # Stop the edit operation. edit.stopOperation() # Stop the edit session and save the changes edit.stopEditing(save_changes=True) #Tidying up that I thought might help with this error. It has had no apparant impact. arcpy.management.ClearWorkspaceCache() del edit del survey_fields, service_fields, service_lines, survey, service_cursor, survey_cursor ################################################################ print("Survey2Services Complete!") else: #print an error to remind the user to select only one service print('Select only 1 Service in the svc db layer and rerun this cell!')

#Check that only one service is selected if int(arcpy.management.GetCount("gisdb.sde.Service").getOutput (0)) == 1: #Copy selected svc database layer to AGOL layer. arcpy.management.Append("gisdb.sde.Service", "Services_AGOL", "TEST")

Re: Layers Disappear after running Arcpy Notebook with Edit Session, and Nested Update Cursors

RWATT — Mon, 17 Oct 2022 16:21:30 GMT

I think I might have found this bug in the issues addressed for v10.7 at https://downloads.esri.com/support/downloads/other_/107-IssuesAddressedList-03182019.pdf.

"BUG-000117721
High rate of edit operations and reconciles generates lock contention on the sde.states table in the version_util procedure new_edit_state".

I'm hoping an update to 10.7 or later will resolve this issue.

Re: Layers Disappear after running Arcpy Notebook with Edit Session, and Nested Update Cursors

RWATT — Mon, 24 Oct 2022 19:00:19 GMT

I wrote a temporary workaround that minimizes edit operations on the database by doing all the editing locally and then just making a single edit to append the completed feature on the gdb.

I will wait until I can run the original code on an upgraded version to accept this as the solution, in case there's something else at play here.

topic Re: Layers Disappear after running Arcpy Notebook with Edit Session, and Nested Update Cursors in ArcGIS Pro Questions

Layers Disappear after running Arcpy Notebook with Edit Session, and Nested Update Cursors

Re: Layers Disappear after running Arcpy Notebook with Edit Session, and Nested Update Cursors

Re: Layers Disappear after running Arcpy Notebook with Edit Session, and Nested Update Cursors