Error code 500 on appending data with REST API to ArcGIS Online table

[mikaël]

I have this process that runs daily, where I get some values from an external source, I transform & store them in a local DB. I then match existing values on AGOL to this local table to make sure the objectids are the same (just to make sure everything is in sync) and add new objectids sequentially locally from the last existing one on AGOL. Then I append (with python) this data to my AGOL table (upload_format = 'geojson', upsert = True, edits = the results from a query on my local DB).

This was running fine, every day, until last week, when I started getting an Exception: Unknown Error (Error Code: 500). The sync and changeTracking parameters are deactivated and I've tried deleting and recreating the table (multiple times)...

What to do?

Please help!

EDIT march 11th

Working with ESRI staff on this, will update with developments. Also noticed that I only have this problem with tables, appending to layers is working correctly.

[JeffSilberberg]

If it was running fine until last week, I would open a support ticket since AGOL was updated last week and this may be a behavior change or a new bug. 

[mikaël]

Well that was exactly what I was thinking, since the problems started occuring on the 26th and the "what's new" blog is dated February 28th, but I did not see anything recent for the REST or Python API.

But yeah good idea, I'll send them an email.

Thanks!

[twhammond]

Any word on this one yet? I have a similar situation, except I am appending from an existing gdb.

fLyr.append(item_id=fgd_item.id, upload_format="filegdb", upsert=False, field_mappings=[])

I set up the process to run in task scheduler every few minutes for testing purposes to see what kind of errors I would get.  The script worked for weeks beforehand, and after setting it up in task scheduler, it worked for a couple days. Now when I run it, I get the same result as you: 

Exception: Unknown Error

(Error Code: 500)

[mikaël]

Last update was yesterday, an ESRI analyst is still looking into this issue.

I changed my workflow to use edit_features() which is more complicated because I have to check beforehand which features I have to update and which I have to add, which I did not have to do with append(). But at least it's working now.

[twhammond]

Thanks, I'll start looking at moving away from append() in the meantime.  Hopefully they get to the bottom of it soon.

[mikaël]

Here is the latest update:

The outcome of the escalation was Esri Inc. logging the following defect:

BUG-000166918: Error 500 "Unknown error" encountered when overwriting hosted tables with any file type data using ArcGIS API for Python. 

A workaround for this issue is to use a File Geodatabase to append data to the table instead of GeoJSON (or CSV).

[twhammond]

That's interesting.  I got the issue while appending data using a file geodatabase.  However, the bug mysteriously disappeared after a couple days and has been working normally ever since.  I honestly have no idea what caused it to break, or why I stopped encountering the error.

So I can't say the workaround doesn't work, but I can say as of last Wednesday I was encountering the same error when appending to table from a geodatabase.

[gmaz]

Glad ESRI could help a bit. I'm also appending a file gdb into a truncated dataset and getting the Error Code: 500. It doesn't seem to be a work around that actually works around. I've looked into edit_features(), but given the size of this dataset getting overwritten daily (~15,000 records) it seems like append() would/should be the tool for the job. Curious if you've heard anything else from ESRI. 

[mikaël]

Nope nothing yet. I've started using AGOL regularly 2 years ago and ESRI opened a couple of bugs from my emails.

No changes yet, so you have to be patient or try to workaround a problem or use something else completely 🙂