Iterate selection and calculate field not working

[ThomasColson]

With the following, I am trying to select each polygon by its name, select points from another feature class that are within the selected polygon, then calculate a field using the same name that was used to select the polygon (by attributes). A spatial intersect, or any other method that results in creating a new FC as output and having to do joins won't work here, as this is (hopefully) to be an unattended nightly update script. It iterates through the polygons fine, but the result is that ALL points get the name of the last polygon. 

import os
import errno
import arcpy
import sys  
import traceback  
from arcpy import env
update_feature_class = r'C:\Users\tcolson\Documents\ArcGIS\Projects\MyProject\MyProject.gdb\GRSM_RESEARCH_LOCATIONS'
county_feature_class = r'C:\Users\tcolson\Documents\ArcGIS\Projects\MyProject\MyProject.gdb\CountyBoundaries'

with arcpy.da.SearchCursor(county_feature_class,['SHAPE@','NAME']) as cursor:
    for row in cursor:
        print('Selecting '+row[1])
        expression = "NAME = '"+row[1]+"'"
        arcpy.SelectLayerByAttribute_management (county_feature_class, "NEW_SELECTION", expression)
        arcpy.SelectLayerByLocation_management(update_feature_class, "INTERSECT", county_feature_class)
        arcpy.CalculateField_management(update_feature_class, "COUNTY","'"+row[1]+"'", "PYTHON3")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[DanPatterson_Retired]

Simple.... delete the old field first (using code of course) and 'extend' the table with the new data.

It is a limitation of esri's implementation that they don't allow direct replacement of the old field...

But it doesn't matter anyway.

because... I never trust 'updating' a column in any event, just in case things go wrong... I would rather add a new field, compare to the old field, then delete the old field.

BUT … on the upside, imagine having an incremental record of your updates!  Poly_id01, Poly_id02, …. Poly_id07... and at the end of the week you can review, summarize and consolidate

This was just a demo though, so I am sure that a temporary field could be added using extend table, then a CalculateField to update and existing field, then a Delete of the extended field could be quite readily added and be transparent to the user since there is no 'join'

[MichaelVolz]

Thomas:

I am curious about the statement "Ok I got it now, sort of. Turns out there WAS bad geometry, the OGC check in Pro found it."

I have some SDE polygon data that I initially thought only had broken links when added to Pro 2.2.x as it works fine in ArcMap 10.5.1 (Just like you the OGC check in Pro detected the issue but not the ESRI check).  I have started to investigate upgrading to ArcMap 10.6.1 as Pro does not support several important workflows at this time.  The same data also does not load in ArcMap 10.6.1.  So would you know if your data that needed to be fixed could be used in 10.6.x before it was fixed (Although I'm not sure if your org is even using 10.6.1 at this time)?

It seems that Pro 2.2.x as well as ArcMap 10.6.1 has higher standards for data that worked in earlier versions of ArcMap.

[JoshuaBixby]

A spatial intersect, or any other method that results in creating a new FC as output and having to do joins won't work here, as this is (hopefully) to be an unattended nightly update script.

As much as I like ArcPy, there are times using geoprocessing tools are both the fastest and best way to get a task done.  What it is about this being an unattended nightly update script that you think disallows the use of geoprocessing tools that results in new feature classes?

[RichardFairhurst]

You original code fails because you have not created layers for the two feature classes and are using the feature classes directly.  I am surprised the select actions did not cause an error, because they only work with layers, not feature classes.  The Calculate Field will always calculate every record if the input is a feature class and not a layer.

Your code would work if you create layers from the feature classes and use the layers:

import arcpy
update_feature_class = r'C:\Users\tcolson\Documents\ArcGIS\Projects\MyProject\MyProject.gdb\GRSM_RESEARCH_LOCATIONS'
county_feature_class = r'C:\Users\tcolson\Documents\ArcGIS\Projects\MyProject\MyProject.gdb\CountyBoundaries'

arcpy.MakeFeatureLayer_management(update_feature_class,"update_lyr")
arcpy.MakeFeatureLayer_management(county_feature_class,"county_lyr")
with arcpy.da.SearchCursor(county_feature_class,['NAME']) as cursor:
    for row in cursor:
        county_name = row[0]
        print('Selecting '+county_name)
        expression = "NAME = '"+county_name+"'"
        arcpy.SelectLayerByAttribute_management ("county_lyr", "NEW_SELECTION", expression)
        arcpy.SelectLayerByLocation_management("update_lyr", "INTERSECT", "county_lyr")
        arcpy.CalculateField_management("update_lyr", "COUNTY","'"+county_name+"'", "PYTHON3")‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[ThomasColson]

Hi all, thanks for all the detailed replies. I got sideways with some other more priority mapping requirements last few weeks, I'm skipping cutting firewood today so I can wade through the responses here!

This one worked: 

https://community.esri.com/thread/221620-iterate-selection-and-calculate-field-not-working#comment-8... 

[DanPatterson_Retired]

Thomas... still ruled out ExtendTable with a Delete field, if needed, as an option? It certainly gets around that cursor stuff and could  be added to to check for bad geometries if needed

[ThomasColson]

Dan, it's been a few days and things are going sideways at work but: I got your code to run, but the problem(s) I'm running into are twofold: I need a string (county name) to be added to an existing string field in the points layer. The numpy solution seems to be limited to only adding an integer to a new field. I was reading the numpy documentation and fiddling around with string arrays, and getting nothing done other than throw errors. Another limitation here is temporarily adding fields, doing joins, etc...as these feature classes are pushing feature services, which put a schema lock on the feature classes, so adding fields is not possible. 

[DanPatterson_Retired]

I can fix that, extendtable is like a join, but permanent, you have a common id field, then you can join anything, string, integer float, a single field or multiple fields

I use it all the time, with or without field deletion.

But it sounds like you aren't doing this stuff when the data can be worked with or it wouldn't have locks

I will post a blog, scripts and a toolbox so you can read it at your leisure

[ThomasColson]

I think an example like that would have pretty broad application beyond my use. Lots of folks needing to make spatial type updates to feature services that are driving feature services, e.g a field survey of stormwater drains using collector and the python updates what zoning district or whatever, without requiring alterations to the schema or service disruption. 

[RichardFairhurst]

Try the code below.  It does not alter the schema at all and updates values in an existing field.  It is also more efficient than your original approach and will minimize the number of polygon to point comparisons required to match all of the county names to the points.

import arcpy

update_feature_class = r'C:\Users\tcolson\Documents\ArcGIS\Projects\MyProject\MyProject.gdb\GRSM_RESEARCH_LOCATIONS'
county_feature_class = r'C:\Users\tcolson\Documents\ArcGIS\Projects\MyProject\MyProject.gdb\CountyBoundaries'

county_count = 0
relateDict = {}
# Set up search cursor for county fc
with arcpy.da.SearchCursor(county_feature_class,['NAME','SHAPE@']) as search_cursor:

    # Iterate through counties
    for polygon in search_cursor:
        if not polygon[0] in relateDict:
            relateDict[polygon[0]] = polygon[1]
        else:
            relateDict[polygon[0]] = relateDict[polygon[0]].union(polygon[1])

for name in relateDict:
    county_count += 1

point_count = 0
comparison_count = 0
# Set up update cursor for point fc
with arcpy.da.UpdateCursor(update_feature_class, ['SHAPE@', 'COUNTY']) as update_cursor:

    # Iterate through points
    for point in update_cursor:
        point_count += 1
        for name in relateDict:
            comparison_count += 1
            county = relateDict[name]
            # Check if the current point is within the current polygon
            if point[0].within(county):  # Returns True or False
                # Set the value of the field as the county name
                point[1] = name
                # Update the cursor with the new value
                update_cursor.updateRow(point)
                break
print("Processed {0} County/Point comparisons for {1} Counties and {2} Points".format(comparison_count, county_count, point_count))
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍