Calculate Field using a layer on Portal - Fails every time

KBKnight

Hi all,

I am trying to do a (relatively) simple thing. I have a polygon dataset on AGOL. I need to update a field with new values. It's not possible to overwrite from Pro because it's used in Field Maps and synced (replicas issues). So, I have tried the following:

1. load layer from Portal in Arc Pro (works great!)

2. Join to table with updated field (no problem)

3. Calculate the field in the polygon dataset to be the field from the joined table (fails every time after a LOOOONG run time (maybe hours) of the progress bar slowly moving.

I've tried using a smaller, sample dataset and this still doesn't work. I CAN manually change an entry for the field in question so changing one row's data works fine. The field is text with a domain applied, but otherwise nothing special at all. 

Next steps? Things I could try? Workarounds?

jcarlson

 

from arcgis.gis import GIS
from arcgis.features import FeatureCollection
import pandas as pd

# Log in. Will prompt for password using getpass
gis = GIS('your-portal-url', 'user')

# Get polygon layer
lyr = gis.content.get('polygon-layer-itemID').layers[0] # or whatever the layer index is

# Create dataframe from layer; using a query limits output fields and geometry for performance
l_df = lyr.query(as_df=True, out_fields=['the-field', 'identifier'])

# Get the table
tbl = pd.read_csv('path-to-csv')

# Merge dataframes; if field names are identical, use on='identifier'
df = l_df.merge(tbl, left_on='identifier', right_on='matching-identifier')

# Replace field values
df['the-field'] = df['the-other-field']

# Submit updates in batches
n = 0
while n < len(df):
    fs = df.iloc[n:n+200].spatial.to_featureset()
    lyr.edit_features(updates=fs)
    n += 200

 

 

Notes:

• If your table is a feature class somewhere, you can try using arcgis.features.GeoAccessor.from_featureclass('path-to-fc')
• The edit_features function will automatically ignore any fields not present in the published layer, so if your dataframe has ancillary fields from the merge, don't worry about it. As long as the field being updated and the ObjectID / GlobalID match, the edit will apply
• The API docs suggest using the append function for larger edits. I admittedly wrote a version of this script prior to 10.8.1, when that function was unavailable to Enterprise users, hence the 200-row batch edits. I've not gone back to test it on the new function. But it's probably worth it to look into.

Good luck! I haven't edited a layer quite as large as yours, but updating the attributes on a layer w/ about 60k rows only took a few minutes. Since you're not adjusting the geometry of the layer, it should be pretty fast.