Feature size increasing after truncate and append

Question

Hi Everybody,

I've encountered some inexplicable result when I'm running daily/weekly python scripts to updated hosted feature layers in ArcGIS Online where the feature storage size seems to balloon in size after each truncate and append option. The hosted feature layer started a few weeks ago at like 60mb of feature storage, but it's now 1.5gbs and I'm at a loss at what's happening.

The layer is sync and edit enabled.

Basically we have two methods of updating feature layers.

1. Is where we call rest endpoint directly and perform arcpy delate rows and append. And it seems to work and the end user doesn't seem to notice any issues.

HFL URL = "[URL]"arcpy.DeleteRows_management(HFL URL)arcpy.Append_management(Local FC Source, HFL URL,"NO_TEST")

2. The other method, we zip up a file geodatabase, upload it to AGO and perform a truncate and append (from the fgdb) to the feature layer. Below are the steps after the zipped fgdb gets uploaded to AGO. flc.manager.update_definition({"capabilities": "Create,Delete,Query,Update,Editing,Sync", "syncEnabled": False})featLyr.manager.truncate()flc.manager.update_definition({"capabilities": "Create,Delete,Query,Update,Editing,Sync", "syncEnabled": True})featLyr.append(item_id=fgd_item.id,                    upload_format="filegdb",                    upsert=False,                    field_mappings=[])

We also have scripts that just item.publish(overwrite=True), but it's a little more finicky I've found to use than the truncate and append, until now. But I've not noticed feature layers exploding in storage space.

I almost feel like there should be like there's an adds and deletes table that needs to be compressed or something in AGO.

Has anyone encountered this issue or have any insights?

Thank you,

Thomas

RPGIS · Answer

My suggestion, if your goal is to reduce for size, is instead of truncate and append to use the cursor functions to add, update, or delete records. This will skip over rows that don't need updating or add/ delete records.

AaronKoelker · Answer

Ah that makes sense, I didn't catch that in the original code you shared. Moving the reenable sync step to after the append was almost certainly it. Glad you were able to figure it out.

twangtx · Answer

Hi Aaron,

Thank you for responding.

I think we've got the issue "resolved" in that we kind of just went with the nuclear option. We ended up just overwriting the hosted feature layer from Pro with a feature class of the same schema. And the method that has been working for us so far is:

* Zip up a file geodatabase 
* Upload to AGO
* Disable sync
* Truncate
* Append from file geodatabase
* Reenable sync
* Delete the file geodatabase in AGO.

I think maybe the critical thing we did was append before reenabling the sync.

It's been running daily for a few weeks, and the feature storage size has been staying consistent.

Thank you,

Thomas