Update existing hosted feature layer with a new shapefile

[TommyTaylorDev]

I am attempting to use ArcGIS API for Python to overwrite the underlying data of a hosted feature layer with an shapefile. As this layer is used within a Web App and has pre-configured filters, it cannot simply be replaced with a new layer.

Workflow

1) Published the service

# publish shapefile as feature service
gis = GIS("https://anonymous/arcgis/")
zip_path = '\\\\<anonymise>\\GDW_Catalogue_Items_OS.zip'
published_item = gis.content.add(
    item_properties = {
        "type": "Shapefile", 
        "title": parent,
        "description": "printed maps and charts",
        "tags": "GDW, Catalogue, Footprints, Printed Maps, Printed Charts",
        },
    data = zip_path,
    overwrite=True
)
published_layer = published_item.publish()

# share published feature layer with groups
published_layer.share(groups=['3214a47ac81e4c839a044edf7c42e5ab', '3b3f17e9f2b84aeda789e337da033a7f', 'b4856c293ab84eb389ab323979bd253d', '404b73b5e78b49a0863f8a8cb0f1b6ce'])

2) Sometime later, I wanted to update the data using a new Shapefile:

# Access hosted feature layer
gis = GIS("https://anonymous/arcgis/")
search_results = gis.content.search('title:GDW_Catalogue_Items_OS AND type:Feature Service')   
feature_layer_item = search_results[0]

# feature collection
from arcgis.features import FeatureLayerCollection
feature_layer_collection = FeatureLayerCollection.fromitem(feature_layer_item)

# call the overwrite() method to update data
zip_path = '\\\\<anonymise>\\GDW_Catalogue_Items_OS.zip'
feature_layer_collection.manager.overwrite(zip_path)

I am given the following error:

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
In  [96]:
Line 14:    feature_layer_collection.manager.overwrite(zip_path)

File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\managers.py, in overwrite:
Line 2183:  layer_info = self._gis._con.get(lyr_url_info, {"f": "json"})

File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py, in get:
Line 506:   ignore_error_key=ignore_error_key,

File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py, in _handle_response:
Line 625:   self._handle_json_error(data["error"], errorcode)

File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py, in _handle_json_error:
Line 648:   raise Exception(errormessage)

Exception: Token Required
(Error Code: 499)
---------------------------------------------------------------------------

Troubleshooting:

1. Manually updated hosted feature layer using portal (Update Data > Overwrite Entire Layer) - ERROR: 'There was an Error". No more information given.
2. Manually created a new feature layer on Portal and manually updated using my new shapefile - successful. But does this not work for Step #1?
3. Ensured my role has the necessary permissions
4. Ensured 'enable editing' in Settings was ticked
5. Ensured 'enable sync' was unticked
6. Ensured both shapefile names were identical
7. Ensured both shapefile schemas were identical

Please would someone help 🙂

[EarlMedina]

Hi @TommyTaylorDev ,

Since you said the data is being used in an app with filters, I'm assuming the schema doesn't change all the often if at all? If that's the case, you could do what a lot of people do and just truncate/append new records. This is a good option if the new data coming in is just an updated version of what's already published. It also causes less disruption to your end users than an overwrite, I'd say.

There are a few different examples of how to do this if you just search "truncate append". Here's a script you can start with: Overwrite ArcGIS Online Feature Service using Trun... - Esri Community

[TommyTaylorDev]

Hi @EarlMedina,

Appreciate your quick response!

Yes you're right that my schema does not change at all.

I had attempted the truncate>append method as part of my troubleshooting. Unfortunately I am still given the same "Exception: Token Required (Error Code: 499)" at the point where I append the new data to the existing layer.

I don't suppose you know why it is requesting a token when I am already logged in and have the correct permissions?

I thought there was something unusual happening when I couldn't even overwrite the data manually in Portal.

Any suggestions would be amazing 🙂  

[EarlMedina]

That doesn't sound expected. Does the same thing happen if you try to add record manually via the UI/Pro? Maybe you're missing editing privileges? I doubt the process is taking so long it's exceeding the token expiration.