python to update a field in a feature layer based on records in csv file

[Nahomdinka]

Hello:

I am a little bit new to coding. I would like to update an existing feature layer in AGOL using an open-source python library. Only one field (let us say field1) needs to be updated in the feature layer based on source data in csv. The feature layer and the .csv file have a common field to join them. Field_ID in the feature layer and CSV_ID in CSV file are used to join or map the source and the target. I do not think it matters but Field_ID is not unique. I am looking for a python code to automate the process of updating features in the feature layer using data from csv. I can not overwrite the feature layer because there are records or fields in the feature layer that are not in the csv. 

Below is what I have...

import pandas as pd

import os

from arcgis.gis import GIS

from arcgis import features

from arcgis.features import SpatialDataFrame

from arcgis.mapping import WebMap

import arcpy

from arcgis.features import FeatureLayerCollection

csvFile = pd.read_csv("csvfile.csv")

portal ="htttps://xyz.maps.arcgis.com/"

username  = "abc"

pw = "pword"

gis = GIS(portal, username, pw)

Feature_layerID  = gis.content.get("123456789")

#replace the records in the "field1" of the feature layer based on the data in csv file, using the two common IDs in the feature layer and the csv file (Field_ID in the feature layer and CSV_ID in CSV file are used to join or map the source and the target). 

[HuubZwart]

Not tested and this will overwrite the data in your feature layer. More could be done like mapping different names, only apply those edits that are actually changed etc. But this should get you on the right track.

# Feature_layerID is a bit confusing, gis.get("some id") returns a feature layer collection, so I changed that
fl_id = "abcde"
flc = gis.content.get(fl_id)
layer_id = 0 # index of the layer in the flc
fs = flc.layers[layer_id].query(where="1=1", returnGeometry=False).to_dict()
fields = [field["name"] for field in fs["fields"]] # gets the fields from your fl 
features = fs["features"] # the existing features
csv_file_path = # your absolute file path
df = pd.read_csv(csv_file_path) #read the csv,  add sep= to specify separator if neccessary
update_dict = df.set_index("CSV_ID")[fields].to_dict("index") # get a dictionary with the ID field as index 
edits = [] # empty list to fill with features to update
for f in features:
    updated_attributes = update_dict.get(f["attributes"]["Field_ID"]) # get the new values from the dictionary
    if(updated_attributes is not None):
        f["attributes"].update(updated_attributes) # update the feature, all values from the csv will overwrite the old ones 
        edits.append(f)

# apply the edits to the feature layer
flc.layers[layer_id].edit_features(updates=edits)

[Nahomdinka]

Thanks a lot, @HuubZwart . I am almost there. I believe that on line #8, you meant to say [layer_id].

fs = flc.layers[layer_id).query(where="1=1", returnGeometry=False).to_dict()

Then, I got an error after #10. 

update_dict = df.set_index("CSV_ID")[fields].to_dict("index")

The error says...

KeyError: "None of [Index([;ogc_fld;, 'name', 'field3', 'field4'))] are in the [columns]"

[HuubZwart]

You're right - I edited my original reply. 

KeyError: "None of [Index([;ogc_fld;, 'name', 'field3', 'field4'))] are in the [columns]"

This means these fields are in your feature layer, but not in the csv. To circumvent this we can select only the the fields from the feature layer that are also in the csv. Make sure to move this below the declaration of df. 

fields = [field["name"] for field in fs["fields"] if field["name"] in df.columns]

[Nahomdinka]

Thanks again. 

I got an error on the last step...

# apply the edits to the feature layer
flc.layers[layer_id].applyEdits(updates=edits)

AttributeError: 'FeatureLayer' object has no attribute 'applyEdits'

[HuubZwart]

My bad had the REST function in mind, that should be

flc.layers[layer_id].edit_features(updates=edits) 

[Nahomdinka]

NP. 

Once I run

flc.layers[layer_id].edit_features(updates=edits)

, I got this message "Parameters not valid for edit_features"

[HuubZwart]

Can you share what the structure of "edits" is now?

[Nahomdinka]

The aim is to replace the records in the field called "field1" of the feature layer (fl) based on a record from a field called "FLAT"  in the csv file. The two common IDs in the feature layer and the csv file are Field_ID and CSV_ID, respectively. 

[Nahomdinka]

@HuubZwart  Here is what it looks like