topic Append data from csv to hosted table using Python API in ArcGIS Online Questions

Append data from csv to hosted table using Python API

Jean-SebastienLauzon — Mon, 30 Mar 2020 14:14:04 GMT

I would like to append data to a hosted Table from a csv file using the Python API.

This can be done in AGOL, by choosing Update Data -> Append data to layer, how would that be done in Python.

I've tried to follow this: Append Features | ArcGIS for Developers

When I try to append the table using:

dataset_tbl.append(item_id=new_csv_item_id, upload_format='csv')

I get an error message "Unable to append data. Missing appendSourceInfo parameters.

I see that I should enter something here, but I'm not sure what.

source_info

optional dictionary. This is only needed when appending data from excel or csv. The appendSourceInfo can be the publishing parameter returned from analyze the csv or excel file.

Note that this table has no geometry, it is a data table only. Also this table is joined, so it cannot be overwritten.

Re: Append data from csv to hosted table using Python API

JakeSkinner — Mon, 30 Mar 2020 15:21:48 GMT

Hi Jean-Sebastien,

You can use ArcGIS Pro's Append tool to do this. If you initially run into issues with the CSV as the input table, you may want to try converting it to a geodatabase table, and then using this as the input for the Append.

Re: Append data from csv to hosted table using Python API

Jean-SebastienLauzon — Mon, 30 Mar 2020 15:55:16 GMT

I just got it to work by feeding in the result of gis.content.analyze as the source_info argument

param1 = gis.content.analyze(item=csvUpdate_item.id)

dataset_tbl.append(item_id=csvUpdate_item.id,upload_format='csv',source_info=param1)

Re: Append data from csv to hosted table using Python API

Anonymous User — Sat, 05 Sep 2020 17:18:32 GMT

What is the content or correct form of the content of the variable dataset_tbl.

I tried the following three:

oldcsv = gis.content.get(tbl) where the tbl is the item id?

response "item object has no attribute 'append'"

oldcsvFLC = FeatureLayerCollection.fromitem(oldcsv)?

response 'FeatureLayerCollection' object has no attribute 'append'

tbl = oldcsvFLC.tables

response "append() takes no keyword arguments"

My source data is a csv file loaded into AGOL.

My target table (oldcsv) is a hosted table on AGOL.

Thanks

Re: Append data from csv to hosted table using Python API

Jean-SebastienLauzon — Tue, 08 Sep 2020 11:25:49 GMT

Assuming you have a hosted table on AGOL that you want to update using a csv file already uploaded on AGOL. This is what I do. Replace {} by your file/id.

para1=gis.content.analyze(item='{your new csv ID}')

ExistingData= gis.content.get({ID of your hosted table on AGOL)

dataset_tbl = ExistingData.tables[0]

dataset_tbl.append(item_id = {your new csv ID}, upload_format = 'csv', source_info = para1, upsert=False)

I hope that helps

Re: Append data from csv to hosted table using Python API

Anonymous User — Thu, 10 Sep 2020 21:10:15 GMT

Thank you very much. It worked like a charm.

Re: Append data from csv to hosted table using Python API

RichardHowe — Mon, 05 Aug 2024 11:13:48 GMT

@Jean-SebastienLauzon I arrived here looking for this solution and it works like a charm...so 4 years later (as it's still not in the ESRI documentation) your help is very much appreciated!

@JakeSkinner I was also pointed towards using Pro's append tool by ESRI staff when I asked a similar question, but when you want to include it as part of a scheduled notebook in AGOL that isn't an option! And when an append command exists in the Python API it should be usable!

Re: Append data from csv to hosted table using Python API

JakeSkinner — Mon, 05 Aug 2024 11:39:20 GMT

@RichardHowe just an FYI, the Advanced Runtime for AGOL Notebooks contains arcpy.

https://doc.arcgis.com/en/arcgis-online/create-maps/specify-the-runtime-of-a-notebook.htm#:~:text=the%20Advanced%20runtime%20contains%20all%20of%20the%20same%20libraries%20as%20well%20as%20ArcPy%20and%20some%20related%20libraries

Re: Append data from csv to hosted table using Python API

RichardHowe — Mon, 05 Aug 2024 12:26:10 GMT

Thanks Jake, noted...albeit at a cost 🙂