Updating SDE Using AGOL Layers

[JoshBillings]

Hey all,

When I set up a new geodatabase for field collection a couple years ago I may have did it a bit backwards. Instead of uploading the layers to ArcGIS Server and then using that layer to collect data using Field Maps, I uploaded it straight to AGOL and began collecting data using the AGOL layer. This feature layer has 19 sublayers associated with it and all have attachments.

After more than 5,000 features collected, I need to get these data into our SDE and have it continuously updated as new features are added. The data also has domains and subtypes.

I have tried to export the layers as a FGDB, but AGOL will never finish the download (it just spins like it's working but never finishes). When I import the AGOL layer into ArcGIS Pro, the domains don't come over with it.

What would be the best way to be able pull data from AGOL and update our SDE using that data?

Thanks!

[jcarlson]

I'd suggest using the ArcGIS Python API. With it, you could easily query the AGOL service and push edits to your SDE. Easiest if the SDE layer is also a published service, and if you have editor tracking enabled, all the better.

import arcgis

gis = arcgis.GIS('your portal url', 'user', 'password')

hosted = gis.content.get('itemid of your field layer').layers[0] # assuming it's layer index 0, but it could be something else

sde = gis.content.get('itemid of published sde layer').layers[0]

# get recent edits
date_range = '07'
recent_edits = hosted.query(where=f"last_edited_date > NOW() - INTERVAL '{date_range}' DAY AND created_date < NOW() - INTERVAL '{date_range}' DAY")

# get recent adds
recent_adds = hosted.query(where=f"created_date > NOW() - INTERVAL '{date_range}' DAY")

# possible reshaping of schema here

sde.edit_features(adds=recent_adds, updates=recent_edits)

You'll probably need some way of identifying deletes, if features are deleted on the AGOL side. And to successfully apply the updates portion, you need some way of reliably linking objects in one table with those in another. I'd suggest adding a GUID field to one or both tables for this purpose.

Alternatively, you could just truncate the SDE table and use the AGOL data to load in a fresh set of data. Simpler, but more time-consuming.