topic Re: Update a CSV/hosted feature service in AGOL in ArcGIS Online Questions

Update a CSV/hosted feature service in AGOL

gismoe — Mon, 20 Apr 2020 11:55:13 GMT

Hi there,

i want to upload a CSV to AGOL and then later on i have to update this file.
Publishing the file works. A hosted feature service is created.
But i have to update the file every day. Either a fully upload or an append will do the job. The id mustn't change, because the data is used in a dashboard.

My script doesn't work. The update returns a 'TRUE' but the data isn't changed.

Any suggestions how i get the job done.

Thanks.

from arcgis.gis import GIS
username = "username"
password = "pwd"
gis = GIS("https://www.arcgis.com", username, password)
csv_file = 'd:\\file.csv'
csv_item = gis.content.add({}, csv_file)
csv_item.move('some_folder')
csv_lyr = csv_item.publish()
# -------- works up to here ---------
ret=csv_lyr.update(data='d:\\file_update.csv')
print (ret)

Re: Update a CSV/hosted feature service in AGOL

IvanBrown — Mon, 20 Apr 2020 14:48:16 GMT

I have a similar situation. I have a CSV-file-based 1-table feature-service; I want to script the process of refreshing that service via new CSV files (schema is same).

A truncate() and overwrite() approach seems to work. However, I can't get a truncate() and append() approach to work.

I also need the option of just-appending; this means there is no truncate(), just appends to add new records. My code:

#Get Feature Service by Getting List of Item Objects via Query
print("Searching for feature service...")
the_services = gis.content.search(query = 'title:' + item_title + " AND " + 'id:' + item_id)
if len(the_services) != 1:
print("Couldn't find feature service " + item_title + " in ArcGIS Online.")
sys.exit()

#Get record count of feature service before update.
#Do this by getting "tables" property of the feature-service item;
#that property is a list of Table objects.
the_tables = the_services[0].tables
if len(the_tables) != 1:
print("Didn't find a 1-table feature-service as expected.")
sys.exit()
#(Capture table object.)
t = the_tables[0]
print("Before refresh, feature-service " + t.properties.name + " has " + str(t.query(return_count_only=True)) + " records.")

#Truncate.
#Do this by getting the "manager" property of the Table object, "manager" being a FeatureLayerManager object
#from the arcgis.features.managers module; FeatureLayerManager has the truncate() method.
print("Truncating...")
the_result = t.manager.truncate()
if the_result["success"] == False:
print("Something went wrong with the truncate.")
sys.exit()

#Get and Update CSV Table in AGO
print("Searching for CSV table in AGO...")
the_CSVs = gis.content.search(query = 'title:' + csv_item_title + " AND " + 'id:' + csv_item_id)
if len(the_CSVs) != 1:
print("Couldn't find CSV table " + item_title + " in ArcGIS Online.")
sys.exit()
c = the_CSVs[0]

#Update CSV Table in AGO from local CSV file
print("Updating CSV table in AGO...")
the_result = c.update(data = csv_path)
if the_result == False:
print("Something went wrong with updating CSV table.")
sys.exit()

#Analyze CSV Table, to get a publishParameters object (from dictionary from analyze())
print("Analyzing CSV table...")
the_analysis = gis.content.analyze(item = csv_item_id, file_type = "csv")
the_publishParameters = the_analysis["publishParameters"]

#Append to Feature Service from CSV Table
#---> HERE'S THE PROBLEM, CAN'T GET append() TO WORK! SERVICE HAS 0 RECORDS AFTER THIS!
print("Appending to feature service from CSV table...")
the_result = t.append(item_id = csv_item_id, source_info = the_publishParameters, upload_format="csv")
if the_result == False:
print("Something went wrong with the append.")
sys.exit()

#HOWEVER, THIS SEEMS TO WORK INSTEAD OF append()
t.container.manager.overwrite(csv_path)

Re: Update a CSV/hosted feature service in AGOL

CynthiaKozma — Thu, 17 Sep 2020 17:56:52 GMT

Did you ever come up with a solution to this? I have the same problem. I can get the csv to update if I delete it and upload it again, but this changes the id and then breaks the dashboard that is pointing to it. I need to have the CSV update with current data that changes weekly and keep that dashboard up to date.

Re: Update a CSV/hosted feature service in AGOL

KarstenRank — Thu, 17 Sep 2020 18:23:20 GMT

Hi,

what about updateing the feature service created from the csv file?

Overwriting feature layers | ArcGIS for Developers

Re: Update a CSV/hosted feature service in AGOL

CynthiaKozma — Sat, 11 Dec 2021 18:16:54 GMT

I think that might do it. I think using the FeatureLayerCollection was the key.

from arcgis.features import FeatureLayerCollectioncities_flayer_collection = FeatureLayerCollection.fromitem(cities_item)
cities_flayer_collection.manager.overwrite(os.path.join('data', 'updating_gis_content',                               'updated_capitals_csv', 'capitals_1.csv'))

Thank you!