topic Re: python to update a field in a feature layer based on records in csv file in ArcGIS API for Python Questions

python to update a field in a feature layer based on records in csv file

Nahomdinka — Mon, 28 Feb 2022 19:07:45 GMT

Hello:

I am a little bit new to coding. I would like to update an existing feature layer in AGOL using an open-source python library. Only one field (let us say field1) needs to be updated in the feature layer based on source data in csv. The feature layer and the .csv file have a common field to join them. Field_ID in the feature layer and CSV_ID in CSV file are used to join or map the source and the target. I do not think it matters but Field_ID is not unique. I am looking for a python code to automate the process of updating features in the feature layer using data from csv. I can not overwrite the feature layer because there are records or fields in the feature layer that are not in the csv.

Below is what I have...

import pandas as pd

import os

from arcgis.gis import GIS

from arcgis import features

from arcgis.features import SpatialDataFrame

from arcgis.mapping import WebMap

import arcpy

from arcgis.features import FeatureLayerCollection

csvFile = pd.read_csv("csvfile.csv")

portal ="htttps://xyz.maps.arcgis.com/"

username = "abc"

pw = "pword"

gis = GIS(portal, username, pw)

Feature_layerID = gis.content.get("123456789")

#replace the records in the "field1" of the feature layer based on the data in csv file, using the two common IDs in the feature layer and the csv file (Field_ID in the feature layer and CSV_ID in CSV file are used to join or map the source and the target).

Re: python to update a field in a feature layer based on records in csv file

HuubZwart — Tue, 01 Mar 2022 16:09:31 GMT

Not tested and this will overwrite the data in your feature layer. More could be done like mapping different names, only apply those edits that are actually changed etc. But this should get you on the right track.

# Feature_layerID is a bit confusing, gis.get("some id") returns a feature layer collection, so I changed that fl_id = "abcde" flc = gis.content.get(fl_id) layer_id = 0 # index of the layer in the flc fs = flc.layers[layer_id].query(where="1=1", returnGeometry=False).to_dict() fields = [field["name"] for field in fs["fields"]] # gets the fields from your fl features = fs["features"] # the existing features csv_file_path = # your absolute file path df = pd.read_csv(csv_file_path) #read the csv, add sep= to specify separator if neccessary update_dict = df.set_index("CSV_ID")[fields].to_dict("index") # get a dictionary with the ID field as index edits = [] # empty list to fill with features to update for f in features: updated_attributes = update_dict.get(f["attributes"]["Field_ID"]) # get the new values from the dictionary if(updated_attributes is not None): f["attributes"].update(updated_attributes) # update the feature, all values from the csv will overwrite the old ones edits.append(f) # apply the edits to the feature layer flc.layers[layer_id].edit_features(updates=edits)

Re: python to update a field in a feature layer based on records in csv file

Nahomdinka — Tue, 01 Mar 2022 15:20:34 GMT

Thanks a lot, @HuubZwart . I am almost there. I believe that on line #8, you meant to say [layer_id].

fs = flc.layers[layer_id).query(where="1=1", returnGeometry=False).to_dict()

Then, I got an error after #10.

update_dict = df.set_index("CSV_ID")[fields].to_dict("index")

The error says...

KeyError: "None of [Index([;ogc_fld;, 'name', 'field3', 'field4'))] are in the [columns]"

Re: python to update a field in a feature layer based on records in csv file

HuubZwart — Tue, 01 Mar 2022 15:48:58 GMT

You're right - I edited my original reply.

KeyError: "None of [Index([;ogc_fld;, 'name', 'field3', 'field4'))] are in the [columns]"

This means these fields are in your feature layer, but not in the csv. To circumvent this we can select only the the fields from the feature layer that are also in the csv. Make sure to move this below the declaration of df.

fields = [field["name"] for field in fs["fields"] if field["name"] in df.columns]

Re: python to update a field in a feature layer based on records in csv file

Nahomdinka — Tue, 01 Mar 2022 16:01:53 GMT

Thanks again.

I got an error on the last step...

# apply the edits to the feature layer
flc.layers[layer_id].applyEdits(updates=edits)

AttributeError: 'FeatureLayer' object has no attribute 'applyEdits'

Re: python to update a field in a feature layer based on records in csv file

HuubZwart — Tue, 01 Mar 2022 16:03:44 GMT

My bad had the REST function in mind, that should be

flc.layers[layer_id].edit_features(updates=edits)

Re: python to update a field in a feature layer based on records in csv file

Nahomdinka — Tue, 01 Mar 2022 16:06:00 GMT

NP.

Once I run

flc.layers[layer_id].edit_features(updates=edits)

, I got this message "Parameters not valid for edit_features"

Re: python to update a field in a feature layer based on records in csv file

HuubZwart — Tue, 01 Mar 2022 16:10:52 GMT

Can you share what the structure of "edits" is now?

Re: python to update a field in a feature layer based on records in csv file

Nahomdinka — Tue, 01 Mar 2022 16:21:45 GMT

The aim is to replace the records in the field called "field1" of the feature layer (fl) based on a record from a field called "FLAT" in the csv file. The two common IDs in the feature layer and the csv file are Field_ID and CSV_ID, respectively.

Re: python to update a field in a feature layer based on records in csv file

Nahomdinka — Tue, 01 Mar 2022 17:14:07 GMT

@HuubZwart Here is what it looks like

Re: python to update a field in a feature layer based on records in csv file

Robert_LeClair — Tue, 01 Mar 2022 22:03:11 GMT

@JeffBigos - can you assist the Python question here? The customer - Nohomdinka - requested my assistance but I do not program in Python so am out of my wheelhouse here. Thx!