Error code 500 on appending data with REST API to ArcGIS Online table

[mikaël]

I have this process that runs daily, where I get some values from an external source, I transform & store them in a local DB. I then match existing values on AGOL to this local table to make sure the objectids are the same (just to make sure everything is in sync) and add new objectids sequentially locally from the last existing one on AGOL. Then I append (with python) this data to my AGOL table (upload_format = 'geojson', upsert = True, edits = the results from a query on my local DB).

This was running fine, every day, until last week, when I started getting an Exception: Unknown Error (Error Code: 500). The sync and changeTracking parameters are deactivated and I've tried deleting and recreating the table (multiple times)...

What to do?

Please help!

EDIT march 11th

Working with ESRI staff on this, will update with developments. Also noticed that I only have this problem with tables, appending to layers is working correctly.

[gmaz]

Ahh...thank you. I also opened a case with ESRI about this. I'm looking into some workarounds and will let you know if anything comes of the case. 

[AndyHess]

Has there been an update to this? I am trying to truncate and append using a modified version of the script here: https://community.esri.com/t5/arcgis-online-documents/overwrite-arcgis-online-feature-service-using/... and I am getting the "Error appending features: Error reading features.
(Error Code: 500)" issue. I changed the script to use a shapefile because apparently using a file geodatabase also doesn't work with the arcgis for python api  for some unknown reason. (https://community.esri.com/t5/arcgis-api-for-python-questions/problem-update-hosted-feature-in-porta...). Thanks for any updates.

[mikaël]

No update, although you're not getting the same error I was ("Error reading features"). Maybe in your case the problem is not append() ?

[Justin_Greco]

I just started noticing this issue with the 500 error code with one of my scripts that uses append().  But after downloading the ZIP file of the FGDB and using the Update Features on the item page, it worked with no issues.  So I tried again with the developer tools open and noticed it was sending the append_fields and field_mappings, which I was omitting in Python since they are optional.  The append in Python was successful after setting the append_fields and field_mappings.  Though it was the append_fields that fixed it, since I tried without field_mappings.

[mikaël]

wow thanks, will try it out!

edit: @Justin_Greco finally managed to test your solution. Unfortunately it doesn't work for me.

I created a new empty feature service, added a couple of records manually on AGOL then tried to append features with a python dict (to FeatureSet to FeactureCollection), either to add or update existing ones, attributes and/or geometry. Tried adding/removing/changing parameters field_mappings and append_fields and also played with upsert and update_geometry. Still getting error 500.

Plus, weirder thing is that when I had troubleshooted this problem in march, I was only getting the problem when appending to tables, not layers. But now I'm getting it with this item which is a layer.

Oh well, at least edit_features() works for me and I changed my current flow to use that function instead so it's not too much of a bother anymore, it's just weird (and I would prefer to use append() which is much simpler).

Leaving the question opened in case there's another answer in a not too distant future 🙂

Thanks!

[mikaël · Accepted solution]

Well, I think I got it.

My usual workflow was getting data from a local (PostgreSQL/PostGIS) DB and transforming that into a geojson and appending to a layer/table with the upload_format='geojson' parameter. For some reason, it stopped working and I created this post.

When I tried again last time (my last post), I was creating a manual dictionary, transforming that into a featureCollection and using the upload_format='featureCollection' and edits=fc parameters, where fc was my feature collection (from_featureset(from_dict(data))).

Upon further testing today for another project, I was looking yet again at the reference, and noticed that under the edits parameters description, it's written: "Only feature collection json is supported". But I was not sending a feature collection json, I was sending the feature collection object. So I modified the parameter to edits=fc.properties and it was successful!!

The edits parameters description also states: "Append supports all format through the upload_id or item_id". But when I was sending my geojson, I was never using those parameters. So I don't know if it changed at some point. So anyway, I'll try again with a geojson with the upload_id or item_id, and if it still fails, I'll just change my DB query to return a feature collection json instead.

There you go! 🎉