Hosted Feature Layer View Does Not Update When Overwriting Hosted Feature Layer

[AmandaBeck]

I just used the Overwrite Web Layer in ArcGIS Pro to update a Hosted Feature Layer in ArcGIS Online. That Hosted Feature Layer updated successfully, however, the Hosted Feature Layer View that we have in a lot of ArcGIS Online maps/apps did not update. An error message pops up saying that the layer is not available anymore.

I did this successfully back in September 2022, but it is not working right now. Does anybody know what may be happening here? Did ESRI change this capability? Any comments are appreciated.

Thanks! 

[jcarlson]

There are a lot of reasons why this might have happened. In general, overwriting a service to update its contents is asking for trouble. You should only do it if something has changed about the layer structure, or if there were significant changes to the schema; most everything else can be modified in-place, or by using a truncate/append process.

When you overwrote the layer, were there any changes to the order of your layers or the schema?

[AmandaBeck]

Hi Josh, 

The data in which I needed to overwrite was address point data - which I send to the state twice a year. My GIS department came up with this procedure of overwriting the layer from ArcGIS Pro to update the hosted feature layer which would automatically update the view layer. It worked perfectly fine when I did the last update in August. I understand the positives of appending data to a layer in ArcGIS Online, however, when we are talking about such a large layer with points that could have been moved/deleted/and added, it seems better to just overwrite the entire layer completely. 

To answer your question, there were no changes made to the layer or schema before overwriting it. 

Thanks for your response! 

[jcarlson]

Well, that eliminates any easy explanations. We used to update some of our layers this way, and would occasionally run into this very problem, especially with view layers. We'd overwrite a layer 8 times, and on the 9th, it would break.

Sometimes we could identify an error in the data that got through the publishing tool's analyze step, but more often, there was no discernible difference.

For the record, truncating and appending basically is overwriting, but only the data itself, while leaving all the deeper settings in place. It's what most people actually want to be doing when they use the built in "overwrite web layer" tool. You "zero out" the layer first, then append into it, so every record, whether new, modified, or otherwise, is reflected in the end product, and there's no risk of out-of-date data.

[AmandaBeck]

Well it's nice to know that we're not the only ones that have run into this issue! I just want to verify that I am correct in saying that when updating a hosted feature layer (either overwriting or appending), the view layer should update itself automatically? From what you are explaining that has happened to you in the past, this seems to be the case as to why we would even try doing this in the first place. 

Do you now only use the append data in ArcGIS Online to update hosted feature layers to show in view layers? I just want to know what a solution would be for this to not run into this problem again. Aside from recreating the view layer, there doesn't seem to be a way to fix it at this point. 

To add to the confusion, when I am looking at the details page in ArcGIS Online of the View Layer, under layers where the data is, when I click on it, it shows the date I updated the data in the hosted feature layer. It's almost as if it worked partially, just not all the way if you know what I mean. 

Thanks again, 

Amanda Beck - Warren County GIS 

[jcarlson]

You're correct, updating the main layer should update the associated views as well. The trouble is that overwriting replaces the entire service definition of the layer, but the view layer retains its original definition, so there can be underlying differences between main layer and view layer that render the view incompatible.

We use the ArcGIS Python API to update our layers, whether they have associated views or not, if the original source of the data is elsewhere.

Here's a very simple version of the process:

from arcgis import GIS
from arcgis.features import GeoAccessor

# log in to portal
gis = GIS('portal-url', 'user', 'password')

# get feature layer by id and index
fl = gis.content.get('itemid of service').layers[0]

# load source data; this will vary depending on the source
df = GeoAccessor.from_featureclass('path to file')

# truncate existing data
fl.manager.truncate()

# load latest data in
fl.edit_features(adds=df.spatial.to_featureset())

For larger updates, you may wish to do this in smaller batches, rather than submitting a single, massive update.

I know that there is an actual append functionality in AGOL and the Python API, but for various reasons specific to our source data, I've not been able to get that process working for our updates. But the truncate/append script runs perfectly well, so I don't see any particular reason to alter it.

We use this for lots of things, including some address data. For about 50k address points, the entire process runs in only a few minutes.

[MichaelVolz]

Thanks for providing the code sample Josh, as this looks very powerful.

[Laura]

Would that code work for a group layer? Do I need to separate the group layer I have in AGOL to 11 different hosted feature layers and update individually?

[DanielWebb]

Thanks for this code, Josh.

When I run it, it throws an error: 'Exception: The page was not displayed because the request entity is too large.'

You mentioned above about dividing it up into smaller chunks. Do you know the best way to do that?

I'm trying to update an AGOL layer from a file geodatabase.

[FabioLuiz2]

Hello Amanda,

I don't know how to explain this problem that also happened to me, but I managed to get around it by activating the "Assign layer IDs" option in ArcGIS Pro 3.
In previous versions I remember not needing this.

https://pro.arcgis.com/en/pro-app/latest/help/sharing/overview/assign-layer-ids.htm 

(this message was translated from PT-BR)