topic Re: How to update Feature Layer contents from a Spatially Enabled Dataframe? in ArcGIS API for Python Questions

How to update Feature Layer contents from a Spatially Enabled Dataframe?

MatthewThompson — Sun, 12 Dec 2021 16:49:05 GMT

Hi all. Sorry, I'm relatively new to all of this and wanted to ask what the easiest way to accomplish this would be. Below is an example of the code I am working with:

import pandas as pd
from arcgis.gis import GIS
gis = GIS(ServerURL, username, password, verify_cert=False)
df1 = pd.DataFrame([["foo", -90.0, 30.0], ["bar", -91.0, 31.0]], columns = ["ID", "Longitude", "Latitude"])
df1SEDF = df1.spatial.from_xy(df1, "Longitude", "Latitude")
dataFrameLayer = df1SEDF.spatial.to_featurelayer("DataFrame to a layer", gis)
# df1SEDF is published to the server

df2 = pd.DataFrame([["foo", -90.0, 30.0], ["fighter", -93.0, 33.0]], columns = ["ID", "Longitude", "Latitude"])
df2SEDF = df1.spatial.from_xy(df2, "Longitude", "Latitude")

The first six lines of code runs fine, takes the two points in the dataframe and adds them to the server (as a Shapefile and Feature Layer (hosted)).

What would be the easiest way to update the layer with the contents of the second dataframe? The ID column is unique if that matters. It could be an overwrite, or update only the changes (remove "bar" and add "fighter").

Any help is much appreciated. Thanks!

Re: How to update Feature Layer contents from a Spatially Enabled Dataframe?

JakeSkinner — Sun, 12 Dec 2021 05:22:04 GMT

Hi Matthew,

You could truncate the service and then add the features if editing is enabled on the feature service:

import pandas as pd
from arcgis.gis import GIS

# Variables
portal = 'https://portal.esri.com/portal'
username = 'jskinner@ESRI'
password = '*******'

# Create connection to AGOL/Portal
gis = GIS(portal, username, password, verify_cert=False)

# Publish feature services from DF
df1 = pd.DataFrame([["foo", -90.0, 30.0], ["bar", -91.0, 31.0]], columns = ["ID", "Longitude", "Latitude"])
df1SEDF = df1.spatial.from_xy(df1, "Longitude", "Latitude")
dataFrameLayer = df1SEDF.spatial.to_featurelayer("DataFrame to a layer", gis)

# Get Item ID of previous published data frame
searchResults = gis.content.search('title:DataFrame to a layer AND owner:{0}'.format(username), item_type='Feature Layer')
sourceId = searchResults[0]['id']

# Reference layer in feature service
fLayer = gis.content.get(sourceId)
editTable = fLayer.layers[0]

# Truncate feature service
editTable.manager.truncate()

# List of edits as dictionaries
addFeatures1 =  {
    "attributes" : {
        "id" : "foo",
        "longitude" : -90.0,
        "latitude" : 30.0
    },
    "geometry":
           {"x": -90.0, "y": 30.0}
    }

addFeatures2 =  {
    "attributes" : {
        "id" : "fighter",
        "longitude" : -91.0,
        "latitude" : 31.0
    },
    "geometry":
           {"x": -91.0, "y": 31.0}
    }

# Add features
editTable.edit_features(adds=[addFeatures1, addFeatures2])‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: How to update Feature Layer contents from a Spatially Enabled Dataframe?

MatthewThompson — Thu, 22 Oct 2020 09:50:17 GMT

Thanks for the response Jake. The data is about 10,000 point objects, so I was hoping to do this with SEDF or file, but I will certainly give this a try and report back.

Re: How to update Feature Layer contents from a Spatially Enabled Dataframe?

JakeSkinner — Thu, 22 Oct 2020 11:16:33 GMT

Are you working with ArcGIS Online or Portal hosted feature services?

Re: How to update Feature Layer contents from a Spatially Enabled Dataframe?

MatthewThompson — Sun, 12 Dec 2021 05:22:07 GMT

Hi Jake Skinner, thanks for continuing to engage with this. I will be adding the data to our Enterprise Server.

Your approach seems to be a fine workflow with the exception of the time between the table truncation and the data being added. I can likely work around this using a mix of add/edit/delete once I get going. Question for you: is it possible to add the features in your example as a featureset after converting the second SEDF? Here is what I am trying:

import pandas as pd
from arcgis.gis import GIS
from arcgis.features import FeatureSet

#GIS Server credentials
gis = GIS(GISURL, username, password, verify_cert=False)

df1 = pd.DataFrame([["foo", -90.0, 30.0], ["bar", -91.0, 31.0]], columns = ["ID", "Longitude", "Latitude"])

df1SEDF = df1.spatial.from_xy(df1, "Longitude", "Latitude")

dataFrameLayer = df1SEDF.spatial.to_featurelayer("DataFrame to a layer", gis)‍‍‍‍‍‍‍‍‍‍‍‍

This continues to work excellently and places the objects on the server as expected.

However, this portion seems to be giving me an issue:

df2 = pd.DataFrame([["foo", -90.0, 30.0], ["fighter", -93.0, 33.0]], columns = ["ID", "Longitude", "Latitude"])

df2SEDF = df1.spatial.from_xy(df2, "Longitude", "Latitude")

# Get Item ID of previous published data frame
searchResults = gis.content.search('title:DataFrame to a layer AND owner:{0}'.format(username), item_type='Feature Layer')
sourceId = searchResults[0]['id']

# Reference layer in feature service
fLayer = gis.content.get(sourceId)
editTable = fLayer.layers[0]

# Truncate feature service
editTable.manager.truncate()

df2FeatureSet = FeatureSet.from_dataframe(df2SEDF)

editTable.edit_features(adds = df2FeatureSet)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The code returns what appears to be success (apologize for the non congruent objectids - I ran it twice):

{'addResults': [{'objectId': 5, 'success': True}, {'objectId': 6, 'success': True}], 'updateResults': [], 'deleteResults': [], 'attachments': {'addResults': [], 'updateResults': [], 'deleteResults': []}}

However, the data on server is blank:

Any ideas why this could be the case? Am I incorrectly using the FeatureSet?

Thanks a bunch!

Re: How to update Feature Layer contents from a Spatially Enabled Dataframe?

MatthewThompson — Mon, 26 Oct 2020 17:23:34 GMT

Turns out it was an issue with our server implementation. Going from SEDF > FeatureSet and using Feature set to update the table works well.