Arcpy script fails because "table is being edited"?

AlexSmith8 · ‎10-18-2017

Hi,
I'm using ArcGIS Pro 2.0.1. I have migrated some arcpy scripts from ArcMap that we use for buffering and clipping right-of-ways generated from line features. The script works successfully except for when I attempt to delete layers at the end. It gives me "error 496: table is being edited". I thought ArcGIS Pro doesn't have edit sessions so I don't know how to deal with this? This script never gave me problems in ArcMap even when an edit session was open. Any thoughts on how to deal with this? I assume ArcGIS Pro is creating a lock somewhere where ArcMap would not.
alex

DanPatterson_Retired · ‎10-18-2017

Don't blame Pro... that message is at least 6 years old under similar circumstances

https://community.esri.com/message/35989 for example

perhaps your script might illuminate the issue

AlexSmith8 · ‎10-20-2017

Hi,

This is a pretty long script so no worries if you don't want to dig through it. But if anyone wants to do some skimming the data manipulation starts at line 218. Ignore the messy stuff with field manipulation and the dissolve so the final output matches our schema.

Basically, the script takes selected line features and buffers them by a width input by the user. It then intersects those buffers with US township-range-section polygons. Each buffer segment is then attributed with the township-range-section information and some other information preset by the user. Then, all of the buffer segments in each township-range are copied to separate feature classes named by the township-range. The script generates a lot of intermediate feature classes in in_memory but cleans them up at the end.

I'm mainly wondering if there are contexts in ArcGIS Pro where locks are generated differently from in ArcMap. I was surprised that the program complained about tables not being editable when there aren't even edit sessions in ArcGIS Pro. Moreover, the same operations in the Python 2 version of the script never presented any problems in ArcMap even when an editor session was open.

# -*- coding: utf-8 -*-

import os
import sys
import datetime
import arcpy
from arcpy import env
from natsort import natsorted

# ROW Single Buffer Python 3 version for ArcGIS Pro
# Version October 17, 2017

# This script requires a third-party library called natsort.
# A folder called natsort should have been included with the script
# and should be placed in the same folder as this .py file.


###############    DEFINE FUNCTIONS    ###############


def ReorderFields(table, out_table, field_order, add_missing = True):
    # Reorders fields in input featureclass/table
    # :table:         input table (fc, table, layer, etc)
    # :out_table:     output table (fc, table, layer, etc)
    # :field_order:   order of fields (objectid, shape not necessary)
    # :add_missing:   add missing fields to end if True (leave out if False)
    # -> path to output table

    existing_fields = arcpy.ListFields(table)
    existing_field_names = [field.name for field in existing_fields]

    existing_mapping = arcpy.FieldMappings()
    existing_mapping.addTable(table)

    new_mapping = arcpy.FieldMappings()

    def add_mapping(field_name):
        mapping_index = existing_mapping.findFieldMapIndex(field_name)

        # required fields (OBJECTID, etc) will not be in existing mappings
        # they are added automatically
        if mapping_index != -1:
            field_map = existing_mapping.fieldMappings[mapping_index]
            new_mapping.addFieldMap(field_map)

    # add user fields from field_order
    for field_name in field_order:
        if field_name not in existing_field_names:
            raise Exception('Field: {} not in {}'.format(field_name, table))

        add_mapping(field_name)

    # add missing fields at end
    if add_missing:
        missing_fields = [f for f in existing_field_names if f not in field_order]
        for field_name in missing_fields:
            add_mapping(field_name)

    # use merge with single input just to use new field_mappings
    arcpy.Merge_management(table, out_table, new_mapping)
    return out_table
        
def GetWorkspace(featureClass):
    # Set workspace to geodatabase containing the input feature class
    dirname = os.path.dirname(arcpy.Describe(featureClass).catalogPath)
    desc = arcpy.Describe(dirname)

    # Checks to see if the path of the layer is a feature dataset and if so changes the output path
    # to the geodatabase
    if hasattr(desc, 'datasetType') and desc.datasetType == 'FeatureDataset':
        dirname = os.path.dirname(dirname)
    return dirname

def WhereClauseFromList(table, field, valueList):
    # Takes a list of values and constructs a SQL WHERE
    # clause to select those values within a given field and table.

    # Add DBMS-specific field delimiters
    fieldDelimited = arcpy.AddFieldDelimiters(arcpy.Describe(table).path, field)

    # Determine field type
    fieldType = arcpy.ListFields(table, field)[0].type

    # Add single-quotes for string field values
    if fieldType == 'String':
        valueList = ['\'{}\''.format(value) for value in valueList]

    # Format WHERE clause in the form of an IN statement
    whereClause = '{} IN({})'.format(fieldDelimited, ', '.join(list(map(str, valueList))))
    return whereClause

def GetFloat(i):
    return float(''.join(x for x in i if x.isdigit() or x == '.'))
    
def GetInt(i):
    return int(''.join(x for x in i if x.isdigit()))

    
###############    INITIAL CLEANUP    ###############
 
 
arcpy.Delete_management('in_memory')
   
lyrList = ['lyr_rowLines', 'lyr_trsPolygons', 'lyr_buffer', 'lyr_buffer_intersect',
           'lyr_intersect_dissolve', 'lyr_buffer_dissolve', 'lyr_rowLines_intersect']

for i in lyrList:
    if arcpy.Exists(i):
        arcpy.Delete_management(i)


###############    SCRIPT TOOL INPUT    ###############
  
  
# Setting this to FALSE to ensure script is smart
env.overwriteOutput = False

rowLines = arcpy.GetParameterAsText(0)
trsPolygons = arcpy.GetParameterAsText(1)
env.workspace = arcpy.GetParameterAsText(2)
bufferTxt = arcpy.GetParameterAsText(3)             # Buffer width (in feet)
keNumber = arcpy.GetParameterAsText(4)              # The script now requires the KE
generalComments = arcpy.GetParameterAsText(5)       # Comments for all features
county = arcpy.GetParameterAsText(6)                # Select county from list of counties
updateDate = arcpy.GetParameterAsText(7)           # Updates date


###############    ERROR CHECKING    ###############


# Checks if a value exists in keNumber and if the length is less than 50
if len(keNumber) > 50:
    arcpy.AddError('ERROR: KE Number must be 50 characters or less.')
    sys.exit('Script failed.')
    
# Uses in_memory workspace to create intermediate feature classes
# This way you don't have lots of temporary files floating around
# Deletes in_memory first to ensure it's fresh

# Test to make sure buffer is a number
try:
    bufferNum = float(bufferTxt)
except:
    arcpy.AddError('Buffer width must be a numeric value.')
    sys.exit('Script failed.')

# Generate buffer width text for buffer tool
if bufferNum.is_integer():
    bufferNum = int(bufferNum)
    
bufferTag = bufferNum / 2

bufferWidth = '{} feet'.format(str(bufferTag))
    
# Test to make sure lines are selected
if not arcpy.Describe(rowLines).FIDSet:
    arcpy.AddError('This script requires you to select line features.')
    sys.exit()

# If comments, test to make sure they are less than 255 characters
if generalComments and len(generalComments) > 254:
    arcpy.AddError('Comments field has a maximum of 254 characters.')
    sys.exit()

    
###############    INITIAL SETUP    ###############


# County lookup dictionary
counties = {'Apache (1)': [1, 'APACHE'],
            'Cochise (2)': [2, 'COCHISE'],
            'Coconino (3)': [3, 'COCONINO'],
            'Gila (4)': [4, 'GILA'],
            'Graham (5)': [5, 'GRAHAM'],
            'Greenlee (6)': [6, 'GREENLEE'],
            'Maricopa (7)': [7, 'MARICOPA'],
            'Mohave (8)': [8, 'MOHAVE'],
            'Navajo (9)': [9, 'NAVAJO'],
            'Pima (10)': [10, 'PIMA'],
            'Pinal (11)': [11, 'PINAL'],
            'Santa Cruz (12)': [12, 'SANTA CRUZ'],
            'Yavapai (13)': [13, 'YAVAPAI'],
            'Yuma (14)': [14, 'YUMA'],
            'La Paz (15)': [15, 'LA PAZ'],
            'Misc (16)': [16, 'MISC'],
            'Misc (17)': [17, 'MISC'],
            'Misc (18)': [18, 'MISC']}

countyNum = counties[county][0]
countyName = counties[county][1]

aprx = arcpy.mp.ArcGISProject('CURRENT')
projectMap = aprx.listMaps()[0]

# Make feature layers for lines and polygons    
arcpy.MakeFeatureLayer_management(rowLines, 'lyr_rowLines')
arcpy.MakeFeatureLayer_management(trsPolygons, 'lyr_trsPolygons')

# Adds data to lines (which will be propagated to buffers)
with arcpy.da.UpdateCursor('lyr_rowLines', ['OID@', 'KE_NUMBER', 'COUNTY', 'COUNTYNM', 'EDIT_DATE', 'COMMENT']) as cursor:
    for row in cursor:
        
        row[1] = keNumber
        row[2] = countyNum
        row[3] = countyName
        if updateDate == 'true' and updateDate != '#':
            row[4] = datetime.date.today()
        if generalComments and generalComments != '#':
            row[5] = generalComments
        
        cursor.updateRow(row)


###############    PROCESSING BUFFERS    ###############
   
   
# Buffer lines  
arcpy.Buffer_analysis('lyr_rowLines', 'in_memory/inMem_lyr_buffer', bufferWidth, method = 'PLANAR')

arcpy.MakeFeatureLayer_management('in_memory/inMem_lyr_buffer', 'lyr_buffer')

arcpy.AddMessage('\nGenerated {} foot buffer.'.format(str(bufferNum)))

# Intersect buffers with sections
arcpy.Intersect_analysis(['lyr_buffer', 'lyr_trsPolygons'], 'in_memory/inMem_lyr_buffer_intersect')

arcpy.MakeFeatureLayer_management('in_memory/inMem_lyr_buffer_intersect', 'lyr_buffer_intersect')

arcpy.AddMessage('Generated intersect of buffer with TRS layer.')

# Set to contain each township-range (for splitting into separate feature classes later)
buffer_trSet = set()

# Cursor gets the township-range-section from each intersected feature and puts it into the set
# The script also writes the township-range-section info to different fields so the final output matches our schema
with arcpy.da.UpdateCursor('lyr_buffer_intersect', ['OID@', 'TOWNSHIP', 'RANGE', 'SECTION', 'TWP', 'RNG', 'SEC']) as cursor:
    for row in cursor:
        twp = GetFloat(row[1])
        rng = GetFloat(row[2])
        sec = GetInt(row[3])
                
        buffer_trSet.update([(row[1], row[2])])
        
        row[4] = twp
        row[5] = rng
        row[6] = sec
        
        cursor.updateRow(row)
 
# Create ordered list of township-ranges for output 
buffer_orderedTR = natsorted([i for i in buffer_trSet])

# Dissolve buffer segments
arcpy.Dissolve_management('lyr_buffer_intersect', 'in_memory/inMem_lyr_buffer_dissolve', ['TOWNSHIP', 'RANGE', 'SECTION', 'TWP', 'RNG', 'SEC', 'KE_NUMBER', 'COUNTY', 'COUNTYNM', 'EDIT_DATE', 'COMMENT'])

arcpy.AddField_management('in_memory/inMem_lyr_buffer_dissolve', 'TSSW_UNQID', 'LONG')
arcpy.AddField_management('in_memory/inMem_lyr_buffer_dissolve', 'TWNSHPLAB', 'TEXT', field_length = 20)
arcpy.AddField_management('in_memory/inMem_lyr_buffer_dissolve', 'LANDNUM', 'TEXT', field_length = 40)
arcpy.AddField_management('in_memory/inMem_lyr_buffer_dissolve', 'QUAD', 'LONG')
arcpy.AddField_management('in_memory/inMem_lyr_buffer_dissolve', 'FUND', 'LONG')
arcpy.AddField_management('in_memory/inMem_lyr_buffer_dissolve', 'PARCEL', 'LONG')
arcpy.AddField_management('in_memory/inMem_lyr_buffer_dissolve', 'TSSW_ACRES', 'DOUBLE')
arcpy.AddField_management('in_memory/inMem_lyr_buffer_dissolve', 'TSSW_DISTANCE', 'DOUBLE')

# Field order for reorder fields function
fieldOrder = ['TSSW_UNQID', 'TWNSHPLAB', 'LANDNUM', 'QUAD', 'TWP', 'RNG', 'SEC', 'COUNTY', 'COUNTYNM', 'FUND', 'PARCEL', 
                'EDIT_DATE', 'KE_NUMBER', 'COMMENT', 'TSSW_ACRES', 'TSSW_DISTANCE']

arcpy.MakeFeatureLayer_management('in_memory/inMem_lyr_buffer_dissolve', 'lyr_buffer_dissolve')               

# For each township-range, output a feature class containing the buffer segments in that township             
for tr in buffer_orderedTR:
    township = tr[0]
    range = tr[1]
    
    bufferNum = 0
    bufferTag = 'T_{}_R_{}_KE_{}'.format(township, range, keNumber).replace('.', '_').replace('-', '_').replace(' ', '_')
    bufferName = bufferTag
    
    while arcpy.Exists('{}'.format(bufferName)):
        bufferNum += 1
        bufferName = '{}_{}'.format(bufferTag, str(bufferNum))
    
    inMem_bufferName = 'in_memory/inMem_{}'.format(bufferName)

    trWhere = 'TOWNSHIP = \'{}\' AND RANGE = \'{}\''.format(township, range)
    
    arcpy.SelectLayerByAttribute_management('lyr_buffer_dissolve', selection_type = 'NEW_SELECTION', where_clause = trWhere)
    arcpy.CopyFeatures_management('lyr_buffer_dissolve', inMem_bufferName)
    
    ReorderFields(inMem_bufferName, bufferName, field_order = fieldOrder, add_missing = False)

    arcpy.CalculateField_management(bufferName, 'TSSW_ACRES', '!shape.area@acres!', 'PYTHON_9.3')
    
    projectMap.addDataFromPath('{}\\{}'.format(env.workspace, bufferName))
            
arcpy.AddMessage('\nCalculated acreage of each buffer segment in field TSSW_ACRES.')

arcpy.AddMessage('\nGenerated Buffer Feature Classes.')

    
###############    FINAL CLEANUP    ###############


arcpy.Delete_management('in_memory')

for i in lyrList:
    if arcpy.Exists(i):
        arcpy.Delete_management(i)

arcpy.AddMessage('\nScript complete.')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

DanPatterson_Retired · ‎10-20-2017

Try to substitute those in_memory saves because it doesn't look like they are the best use of workflow... in fact, try saving them to disk, then use 'delete' from arcpy if you don't want them around....

See the help topic here on Considerations when Using in_memory...

especially towards the middle beginning with

Running single geoprocessing tools one at a time from either the Geoprocessing pane or the Python window does not make good use of the in_memory workspace; in fact, doing so might incur a performance cost. Because ArcGIS Pro is a multithreaded application, it can levera..... etc