How to update Feature Layer contents from a Spatially Enabled Dataframe?

[MatthewThompson]

Hi all. Sorry, I'm relatively new to all of this and wanted to ask what the easiest way to accomplish this would be. Below is an example of the code I am working with:

import pandas as pd
from arcgis.gis import GIS
gis = GIS(ServerURL, username, password, verify_cert=False)
df1 = pd.DataFrame([["foo", -90.0, 30.0], ["bar", -91.0, 31.0]], columns = ["ID", "Longitude", "Latitude"])
df1SEDF = df1.spatial.from_xy(df1, "Longitude", "Latitude")
dataFrameLayer = df1SEDF.spatial.to_featurelayer("DataFrame to a layer", gis)
# df1SEDF is published to the server

df2 = pd.DataFrame([["foo", -90.0, 30.0], ["fighter", -93.0, 33.0]], columns = ["ID", "Longitude", "Latitude"])
df2SEDF = df1.spatial.from_xy(df2, "Longitude", "Latitude")

The first six lines of code runs fine, takes the two points in the dataframe and adds them to the server (as a Shapefile and Feature Layer (hosted)).

What would be the easiest way to update the layer with the contents of the second dataframe? The ID column is unique if that matters. It could be an overwrite, or update only the changes (remove "bar" and add "fighter").

Any help is much appreciated. Thanks!

[JakeSkinner · Accepted solution]

Hi Matthew,

You could truncate the service and then add the features if editing is enabled on the feature service:

import pandas as pd
from arcgis.gis import GIS

# Variables
portal = 'https://portal.esri.com/portal'
username = 'jskinner@ESRI'
password = '*******'

# Create connection to AGOL/Portal
gis = GIS(portal, username, password, verify_cert=False)

# Publish feature services from DF
df1 = pd.DataFrame([["foo", -90.0, 30.0], ["bar", -91.0, 31.0]], columns = ["ID", "Longitude", "Latitude"])
df1SEDF = df1.spatial.from_xy(df1, "Longitude", "Latitude")
dataFrameLayer = df1SEDF.spatial.to_featurelayer("DataFrame to a layer", gis)

# Get Item ID of previous published data frame
searchResults = gis.content.search('title:DataFrame to a layer AND owner:{0}'.format(username), item_type='Feature Layer')
sourceId = searchResults[0]['id']

# Reference layer in feature service
fLayer = gis.content.get(sourceId)
editTable = fLayer.layers[0]

# Truncate feature service
editTable.manager.truncate()

# List of edits as dictionaries
addFeatures1 =  {
    "attributes" : {
        "id" : "foo",
        "longitude" : -90.0,
        "latitude" : 30.0
    },
    "geometry":
           {"x": -90.0, "y": 30.0}
    }

addFeatures2 =  {
    "attributes" : {
        "id" : "fighter",
        "longitude" : -91.0,
        "latitude" : 31.0
    },
    "geometry":
           {"x": -91.0, "y": 31.0}
    }

# Add features
editTable.edit_features(adds=[addFeatures1, addFeatures2])‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[MatthewThompson]

Thanks for the response Jake. The data is about 10,000 point objects, so I was hoping to do this with SEDF or file, but I will certainly give this a try and report back.

[JakeSkinner]

Are you working with ArcGIS Online or Portal hosted feature services?

[MatthewThompson]

Hi Jake Skinner, thanks for continuing to engage with this. I will be adding the data to our Enterprise Server.

Your approach seems to be a fine workflow with the exception of the time between the table truncation and the data being added. I can likely work around this using a mix of add/edit/delete once I get going. Question for you: is it possible to add the features in your example as a featureset after converting the second SEDF? Here is what I am trying:

import pandas as pd
from arcgis.gis import GIS
from arcgis.features import FeatureSet

#GIS Server credentials
gis = GIS(GISURL, username, password, verify_cert=False)

df1 = pd.DataFrame([["foo", -90.0, 30.0], ["bar", -91.0, 31.0]], columns = ["ID", "Longitude", "Latitude"])

df1SEDF = df1.spatial.from_xy(df1, "Longitude", "Latitude")

dataFrameLayer = df1SEDF.spatial.to_featurelayer("DataFrame to a layer", gis)‍‍‍‍‍‍‍‍‍‍‍‍

This continues to work excellently and places the objects on the server as expected.

However, this portion seems to be giving me an issue:

df2 = pd.DataFrame([["foo", -90.0, 30.0], ["fighter", -93.0, 33.0]], columns = ["ID", "Longitude", "Latitude"])

df2SEDF = df1.spatial.from_xy(df2, "Longitude", "Latitude")

# Get Item ID of previous published data frame
searchResults = gis.content.search('title:DataFrame to a layer AND owner:{0}'.format(username), item_type='Feature Layer')
sourceId = searchResults[0]['id']

# Reference layer in feature service
fLayer = gis.content.get(sourceId)
editTable = fLayer.layers[0]

# Truncate feature service
editTable.manager.truncate()

df2FeatureSet = FeatureSet.from_dataframe(df2SEDF)

editTable.edit_features(adds = df2FeatureSet)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The code returns what appears to be success (apologize for the non congruent objectids - I ran it twice):

{'addResults': [{'objectId': 5, 'success': True}, {'objectId': 6, 'success': True}], 'updateResults': [], 'deleteResults': [], 'attachments': {'addResults': [], 'updateResults': [], 'deleteResults': []}}

However, the data on server is blank:

Any ideas why this could be the case? Am I incorrectly using the FeatureSet?

Thanks a bunch!

[MatthewThompson]

Turns out it was an issue with our server implementation. Going from SEDF > FeatureSet and using Feature set to update the table works well.