ArcGIS Online, Python, Overwrite Feature Services

Question

I have a strange problem when using python code to overwrite a feature service on our ArcGIS Online account.  I created two point feature layers (fire hydrants data), one is for the fire department and another version with some fields removed (not visible) for the public.  I can manually upload and/or share each feature layer to ArcGIS Online and even overwrite the feature services.  The data looks fine and I can create webmaps with no problems.  I then used Python code that we found on ESRI’s website to upload the feature layers.  Below is a link to the discussion of the code that we use with no problems in other project for a few years now.

https://community.esri.com/t5/python-questions/automate-overwrite-web-layer-feature-class/td-p/20067

The problem I’m having is after the automated Python code runs overnight, the attribute table for the “fire department” version is missing data.  See the attached screen shot.  The “public” version feature service attribute table’s data is visible.  When I select Visualization option for the “fire department” feature layer (hosted), I do see points on a map but no data when I select a point.  If I re-run the manual overwrite option within the ArcGIS Pro project, it corrects the problem and I can see the data in the attribute table.  Any suggestions on where to the look?  I'm planning to add a field that was removed for the "public" table one at time and run the python code until I see a failure.

Thanks,

Screen Shot 20240417.JPG

Williams_Gregory · Answer

Josh,

Thanks for the reply, if figure there was some sort append coding options.  But after looking over your code and searching the internet for other append/truncate python code it appears to be way beyond my Python coding capabilities.  I'll keep working on it but for right now I'll just manually overwrite the feature service via ArcGIS Pro.  I don't why they (ESRI) doesn't provide an append option in the menu, maybe in the future.  Thanks for your help.

jcarlson · Answer

This kind of thing comes up pretty regularly in posts here. I'm assuming the overwrite is to update the data, yes?

This doesn't help you identify the specific problem you're seeing, but avoid overwrites unless absolutely necessary. Lots of things can break that way.

At the very least, consider a truncate/append, where the AGOL feature layers are emptied out, and new data loaded. This leaves all the layer schema and settings untouched.

A step further is to compare the source and destination data and identify which specific rows changed, then selectively apply edits where needed. It's more work to set up, but if the source data sees relatively low edit volume, it can definitely be worth it.

I presented on it a couple years ago, you might find some python in here that could help: https://github.com/jdcarls2/ilgisa-2022/blob/main/hosted-copy/hosted-copy.ipynb