topic Re: Overwrite or update hosted table with new fields in Python Questions

Overwrite or update hosted table with new fields

IonSola_Torralba — Mon, 12 Apr 2021 07:48:59 GMT

I am trying to update a hosted table in AGOL (published from a CSV). The updated CSV has the same name and structure except a new field. When I try to update or overwrite the table, both manually from AGOL and using script with Python API the new field is not added to the hosted table.

I also tried to add the field (both manually from AGOL and using 'Add field' in Python) prior to the overwritting but when I overwrite the new field dissapears.

# Overwrite CSV
oldcsv = gis.content.get('itemid')
oldcsvFLC = FeatureLayerCollection.fromitem(oldcsv)
oldcsvFLC.manager.overwrite('path')

I attach a subset of old and new CSV as an example

Re: Overwrite or update hosted table with new fields

jcarlson — Mon, 12 Apr 2021 13:14:57 GMT

Changing the schema during an overwrite can lead to problems. It is technically possible, but the docs themselves advise against this.

4. The schema (column names, column data types) of the data_file should be the same as original. You can have additional or fewer rows (features).

Try adding the field to your hosted layer first, then overwriting.

Re: Overwrite or update hosted table with new fields

IonSola_Torralba — Fri, 07 May 2021 11:36:49 GMT

Thank you, but I already tried that, and it didn't work. I add the field to my hosted layer with no problem, but after the overwrite this new field dissapears again.

The overwriting works fine from ArcGIS PRO if the original table was shared from ArcGIS PRO too, but when I try to overwrite a table in AGOL which was published from a CSV the overwriting doesn't create new fields.

Re: Overwrite or update hosted table with new fields

LARAPAdmin — Sat, 20 Nov 2021 01:41:03 GMT

Hey, I have also experienced this exact issue. My only solution has been to recreate everything to add the new field(s). This method is not ideal and is super time consuming. Have you been able to find a workaround or an alternative solution?

Thanks!

Re: Overwrite or update hosted table with new fields

jayt70 — Thu, 07 Jul 2022 16:27:16 GMT

I'm having this same problem. Has a solution been identified yet?

Re: Overwrite or update hosted table with new fields

IonSola_Torralba — Tue, 26 Jul 2022 10:04:13 GMT

As far as I know, there is no solution for a key issue like this. Managing hosted tables in AGOL is clearly improvable. The only workaround I know is deleting the old table, publishing the new table and replace the itemid of the table everywhere. I share the script. I hope it helps:

# Importe las bibliotecas necesarias.
from arcgis.gis import GIS, Item

# Cree una conexión al usuario de ArcGIS Online o Portal for ArcGIS deseado.

user = "XXXXXXXX" # introduzca el usuario deseado
password = "XXXXXXXX" # introduzca la contraseña

url = "https://www.arcgis.com" # cambie la URL de Portal si es necesario
gis = GIS(url, user, password)
me = gis.users.me
my_items = me.items(max_items=500000)

# ItemIDs
old = "olditemid"
new = "newitemid"

carpetas = ['Name of AGOL folder where data is stored']

itemTypes = ['Dashboard']

# Help: https://developers.arcgis.com/python/sample-notebooks/clone-portal-users-groups-and-content/

folders = me.folders
for folder in folders:
carpeta = folder['title']
if carpeta in carpetas:
print('__________________________________')
print(' ')
print('Carpeta: ' + folder['title'])
print('__________________________________')
print(' ')
folder_items = me.items(folder=folder['title'],max_items=5000)
if len(folder_items)>0:
for item in folder_items:
# Si encuentra algún itemid viejo printa el item, el número de veces que se accede a ese itemid y los sustituye
if item.type in itemTypes:
#print(item.title + '. Tipo: ' + item.type)
itemid = item.id
# Recupere los datos de la aplicación.
app = Item(gis, itemid)
appdata = app.get_data(False)

# Saber el número de veces que se utiliza item viejo
pattern = old
count =0
flag=True
start=0

if appdata != None: # Si no hay texto no entra al loop

while flag:
a = appdata.find(pattern,start) # find() returns -1 if the word is not found,
#start i the starting index from the search starts(default value is 0)
if a==-1: #if pattern not found set flag to False
flag=False
else: # if word is found increase count and set starting index to a+1
count+=1
start=a+1

if count > 0:
print(item.title + '. Tipo: ' + item.type)
print('Se han encontrado ' + str(count) + ' items viejos')

# Ejecute el script para sustituir itemid viejo por nuevo en los datos de la aplicación.
new_appdata = appdata.replace(old, new)

# Comprobar si queda algún itemid viejo a sustituir
count =0
flag=True
start=0
while flag:
a = new_appdata.find(pattern,start) # find() returns -1 if the word is not found,
#start i the starting index from the search starts(default value is 0)
if a==-1: #if pattern not found set flag to False
flag=False
else: # if word is found increase count and set starting index to a+1
count+=1
start=a+1
print('Tras la sustitución, quedan ' + str(count) + ' items viejos')

# Actualice el elemento con la URL especificada en el paso 5 e imprima una sentencia de cierre.
app.update({"text": new_appdata})

print("El item con nombre " + item.title + " ha sido actualizado con éxito")
print(' ')

Re: Overwrite or update hosted table with new fields

JonJones1 — Thu, 10 Aug 2023 17:33:30 GMT

Hey! I'm curious about how you resolved your problem. Currently, the only solution I can follow is what you mentioned earlier. I make sure the layer is originally published in ArcGIS Pro, and then, when I overwrite the web layer from within ArcGIS Pro, the new columns do appear. How did you handle this on your end?