FeatureLayerCollection manager overwrite() not working as expected

[Trippetoe]

Hi There.

I am trying to use the ArcGIS python API to overwrite an AGOL hosted feature table that i own. The process seems to work, i.e. the FeatureLayerCollection.manager.overwrite() method returns {'success': True}, but the data in the AGOL hosted table are not updated. The items detail page of the hosted feature table indicates that it was updated at the same time as my log file indicates that the overwrite() operation completed. So all indications are that the overwrite worked, except for the data.

(from Item Details page)

(from my log file)

If i update the hosted table via the AGOL UI, the process works as expected - the new data are present in the table.

The table was originally created by publishing a csv file to AGOL. The csv file i am using to overwrite the existing hosted table has the same name and schema as the original file. 

my conda environment is using arcgis v2.1.0.2.

Any ideas what's going on?

files_to_update = json.loads((settings.SURVEY_LOOKUP_TABLES_TO_UPDATE).replace('\n',''))
for file in files_to_update:
    lookup_file = update_survey_lookup_file(settings.SURVEY_LOOKUP_FILES_LOCAL_SOURCE, file)
    lookup_table = gis.content.get(file['agolItemId'])     
    lookup_collection = FeatureLayerCollection.fromitem(lookup_table)
    LOGGER.debug("Start to overwrite AGOL table with " + lookup_file)
    response = lookup_collection.manager.overwrite(lookup_file)
    LOGGER.debug("the response from the overwrite operation: " + str(response))
    LOGGER.debug("Done with overwrite of AGOL table with " + lookup_file)
    lookup_table.update(item_properties = {"title" : file['agolTitle']})

[EarlMedina]

So the schema is not changing? f not I would just simplify this to be a truncate/append operation. I suspect the problem is the JSON, but without an example it's hard to say.

There are tons of truncate/append examples if you search the board.

[Trippetoe]

Hello @EarlMedina .  Thanks for replying.

Correct, the schema is not changing. What JSON could be the problem - the service's JSON?

I'll poke around on community for some truncate/append examples and give those a whirl.

[EarlMedina]

Oh, I'm sorry I think I misread what you were saying. I see you are reading csv files. To convert the process, you could load the csv files into spatial dataframe (assuming there is coordinate information of some sort). and then use sedf.to_featureset() - this would be the thing you use for your adds.

Here's a quick example:

import pandas as pd
from arcgis.gis import GIS
from arcgis.features import GeoAccessor, FeatureLayer

url = "https://services3.arcgis.com/rtycvyukgbyuklbnhjkb/arcgis/rest/services/test/FeatureServer/0"
csv_path = r"C:\Users\you\Documents\xy.csv"

gis = GIS("https://www.arcgis.com", "user", "pass"")
fl = FeatureLayer(url, gis)

df = pd.read_csv(csv_path)
sdf = GeoAccessor.from_xy(df=df, x_column="LONGITUDE", y_column="LATITUDE", sr=4326)
fs = sdf.spatial.to_featureset()
          
fl.manager.truncate()
fl.edit_features(adds=fs)  

[Trippetoe · Accepted solution]

@EarlMedina Thanks again for replying. I really appreciate you sharing your time and expertise.

I may have stumbled over the problem and solution myself. I have two candidates:

1. I noticed that in the config file i'm using to set various parameters like AGOL itemid, file name, etc, the file name did not include the ".csv" extension. So, i included that info. And,
2. I noticed that i had deleted the original source csv from AGOL. I think i read somewhere in the docs that the source csv file needed to be present for the overwrite to work (or something like that). So i deleted the hosted feature service, reuploaded the original csv file and published that.

Since doing those two things, the code is working as expected with the service data being updated.

Regardless of what the source of the problem was, I wonder why the overwrite() method returned success = True when the update didn't work.  I guess that's a question for another thread.

Thanks again for your help.