- Home
- :
- All Communities
- :
- Products
- :
- ArcGIS Pro
- :
- ArcGIS Pro Questions
- :
- Re: Layers Disappear after running Arcpy Notebook ...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Layers Disappear after running Arcpy Notebook with Edit Session, and Nested Update Cursors
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I'm working on an ArcGIS Pro (v3.0.2) notebook that copies attributes between a point (1) and a line (2) and then copies the line feature (2) to an identical line feature (3).
Details on each of these features:
1. A Feature Service Feature Class of Survey Entries from Survey123. They have important attributes from the survey, but no meaningful geographic component.
2. An Enterprise Geodatabase in PostgreSQL (v10.6.1) Feature Class of lines. This feature class is receiving the attributes from 1.
3. A Feature Service Feature Class of lines. This feature class recieves the entire feature from 2 via Copy>Paste Special, or arcpy.management.Append.
This process works consistently and without error messages.
However, these code cells will occasionally cause some or all layers in my map and corresponding layout to "disappear".
These layers remain in the Contents pane, I can still access their attribute tables, and sometimes I can still select them by clicking in the map.
Their visibility is restored by any of the following:
1. Closing and reopening the map and layout.
2. Removing the layer from the Contents and re-adding it.
3. Clearing the cache via Project>Options>Display>"Clear Cache Now"
4. Closing and reopening the project.
Their visibility is NOT restored by:
1. Refreshing the map in the lower-right corner.
2. Clearing the layer cache in the contents pane.
3. Refreshing the database connection
4. Clearing the workspace environment. arcpy.ClearEnvironment('workspace')
5. Deleting the workspace. del arcpy.env.workspace, workspace
6. Clearing the workspace cache. arcpy.management.ClearWorkspaceCache()
I think this is likely an issue with the database connection, but I'm out of ideas on how to avoid it. Any suggestions are appreciated. Thanks!
Three Code Cells:
#This ipynb is built for use in ArcGIS Pro to produce new digital service cards from Survey123 and service points collected in the field.
#This cell sets up the context for future cells
#imports. getpass is to pull the windows username for the PDF details at the end.
import arcpy, getpass
#define the project, map, mapview, layout, map frame in the layout, survey entry layer, and workspace.
aprx = arcpy.mp.ArcGISProject('CURRENT')
m = aprx.listMaps("Editing_Map")[0]
mv = aprx.activeView
layout = aprx.listLayouts("CombinedLayout")[0]
map_frame = layout.listElements("MAPFRAME_ELEMENT", "Map Frame")[0]
lyr = m.listLayers("Survey_Entries")[0]
#arcpy.env.workspace = r"C:\Users\USER\Documents\ArcGIS\Projects\Service Cards\arcgis(2).sde"
#This cell will:
#validate that only a single gisdb service is selected, and:
#Pull all of the survey content into the gisdb service feature based on matching Work Order #
#Pull the Service ID from the gisdb service to the Survey point (We use this to track which surveys have been processed)
#Copy the entire gisdb service feature into the AGOL layer.
workspace = r"C:/Users/USER/Documents/ArcGIS/Projects/Service Cards/arcgis(2).sde"
#Check that only one record is selected in the gisdb services layer
if int(arcpy.management.GetCount("gisdb.sde.Service").getOutput (0)) == 1:
#start an edit session
edit = arcpy.da.Editor(workspace)
#Start editing. I don't need to undo in this process and the workspace is versioned.
edit.startEditing(with_undo=False, multiuser_mode=True)
edit.startOperation()
#Sync services to survey.
## field names in the service feature class. Many have been removed for brevity
service_fields = ['w_o_','city','installati', 'premises', 'house_no', 'street',
'lot_no', ]
## field names in the survey feature class. Many have been removed for brevity.
survey_fields = ['USER_WO_Number', 'USER_Town', 'USER_Date_Completed', 'premise_ids', 'house_number',
'street_name', 'apt_unit']
##select the service and survey feature classes
service_lines = m.listLayers("gisdb.sde.Service")[0]
survey = lyr
#Create update cursor "service_cursor" for services
with arcpy.da.UpdateCursor(service_lines, service_fields) as service_cursor:
#iterate through lines/features/services in service_lines
for service_row in service_cursor:
#Create update curor "service_cursor" for survey points
with arcpy.da.UpdateCursor(survey, survey_fields) as survey_cursor:
for survey_row in survey_cursor:
## find the entry with the same WO ID
if service_row[0] == survey_row[0]: ## WO
## set each field in the service layer equal to the value in the
## corresponding field from the survey layer
## Towns formatted to fit in server domain
if "" in survey_row[1]:
service_row[1] = ""
elif "" in survey_row[1]:
service_row[1] = ""
elif "" in survey_row[1]:
service_row[1] = ""
elif "" in survey_row[1]:
service_row[1] = ""
elif "" in survey_row[1]:
service_row[1] = ""
elif "" in survey_row[1]:
service_row[1] = ""
elif "" in survey_row[1]:
service_row[1] = ""
elif "" in survey_row[1]:
service_row[1] = ""
else:
service_row[1] = survey_row[1].replace("_", " ")
service_row[2] = survey_row[2].replace(hour=12) ## completion date
service_row[3] = survey_row[3] ## premises
service_row[4] = survey_row[4] ## house number
service_row[5] = survey_row[5].title() ## street
service_row[6] = survey_row[6] ## lot/unit
service_row[7] = survey_row[7] ## size
service_row[8] = survey_row[7] ## size
service_row[9] = survey_row[8] ## material - text
if survey_row[8] == 'Steel': ## if the survey has material as steel
service_row[10] = 3 ## then set service material to 3, which is coded as steel
else:
service_row[10] = 5 ## otherwise set it to 5 which is plastic
#Many attributes have been removed for brevity and privacy here.
survey_row[37] = service_row[46] ## service ID
#save the service ID to a new variable for use in the next cell.
service_input = service_row[46]
service_cursor.updateRow(service_row)
survey_cursor.updateRow(survey_row)
# Stop the edit operation.
edit.stopOperation()
# Stop the edit session and save the changes
edit.stopEditing(save_changes=True)
#Tidying up that I thought might help with this error. It has had no apparant impact.
arcpy.management.ClearWorkspaceCache()
del edit
del survey_fields, service_fields, service_lines, survey, service_cursor, survey_cursor
################################################################
print("Survey2Services Complete!")
else:
#print an error to remind the user to select only one service
print('Select only 1 Service in the svc db layer and rerun this cell!')#Check that only one service is selected
if int(arcpy.management.GetCount("gisdb.sde.Service").getOutput (0)) == 1:
#Copy selected svc database layer to AGOL layer.
arcpy.management.Append("gisdb.sde.Service", "Services_AGOL", "TEST")
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think I might have found this bug in the issues addressed for v10.7 at https://downloads.esri.com/support/downloads/other_/107-IssuesAddressedList-03182019.pdf.
"BUG-000117721
High rate of edit operations and reconciles generates lock contention on the sde.states table in the version_util procedure new_edit_state".
I'm hoping an update to 10.7 or later will resolve this issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I wrote a temporary workaround that minimizes edit operations on the database by doing all the editing locally and then just making a single edit to append the completed feature on the gdb.
I will wait until I can run the original code on an upgraded version to accept this as the solution, in case there's something else at play here.
