Survey123 - migrate from hosted feature layer to enterprise GDB

966
6
10-08-2020 05:38 PM
AdrianDoss
New Contributor III

I use Survey123 Connect / XLSForms to create surveys hosted on our own portal. Survey data is stored in a hosted feature layer on the portal. This is great for prototyping/developing surveys. But now I want to store and sync the data in a feature class in our enterprise GDB (MS SQL Server). What I want to achieve is the same end result as creating a feature class in EGDB, publishing that as a portal service, and creating a survey from that service, but sort of in reverse, even if I have to publish the reverse-engineered feature layer as a service with sync capabilities and re-connect the survey to the published feature service. To add another complication, we allow photo attachments in our survey.

I understand that there is a Synchronize Survey Tool, but from what I have read it may be a little error-prone.

Has anyone been able to do this successfully?

0 Kudos
6 Replies
ZacharySutherby
Esri Regular Contributor

Hello Adrian Doss‌,

Like you had mentioned James has a script that will perform the workflow of syncing Hosted Feature Services to an Enterprise Geodatabase. 

That script is a little older and uses the Python 2 libraries. We are in the process of updating the script to use the ArcGIS API for Python and Python 3 libraries, but the script is still early in it's life. 

I have attached a zip file that contains the updated python API/python 3 script here. 

The workflow of the script is: 

- Connects to your ArcGIS Online or Enterprise organization

- Grabs the Hosted Feature Layer based on the REST endpoint. 

- Verifies if Sync is enabled on the service and if not temporarily enables it. (Sync will then be disabled later on in the script if it was disabled before running the script)

- Creates and downloads a File Geodatabase replica from the service.

- The script then looks at the content of the File Geodatabase and checks to see if a Feature Class in the Enterprise Geodatabase exists. 

- If it does not the script will then copy over all layers, tables, and attachments to the Enterprise Geodatabase, creates an index on the GlobalID field, and adds a field for the last time the script was run.

- If the data does exist in the Enterprise Geodatabase then the script will join the Enterprise Geodatabase Feature class to the File Geodatabase Feature Class to determine what records are new and don't exist in the Enterprise Geodatabase.

- Those records that already exist will be removed from the File Geodatabase only leaving the new records that haven't been added just yet.

- The script then copies over the filtered File Geodatabase Feature Class to the Enterprise Geodatabase as a temporary Feature Class and appends the new records in the Feature Classes and Tables to their respective counterparts. 

- Finally the script deletes the temporary Feature Class that was copied over and recalculates the field containing the last time the script was run. 

A few things we are looking to fix in the script is automating the process of grabbing the Feature Service layer indices instead of manually adding them to a variable. As well as deleting the temporary File Geodatabase that is downloaded from ArcGIS Online/Enterprise.  

The script has been tested with a handful of Feature Services, but definitely has not been tested with all scenarios. I would suggest creating a copy of your survey and testing on the copy prior to running the script on the production data. Please feel free to reach out with any issues or errors encountered any feedback is greatly appreciated! 

Thank you, 

Zach

Thank you,
Zach
AdrianDoss
New Contributor III

Hi Zach

Sorry for my late reply - I have been away for a while. In your message you mentioned a zip file containing an updated script for Python 3, but I cannot find it. Would you mind re-sending?

Regards

Adrian

0 Kudos
TylerGrosshuesch3
New Contributor II

Hi Zach,

I successfully used the script yesterday, thank you. My use case was pretty simple, with relatively few records and no repeats or attachments. Is the updated version you mentioned available? I didn't come across it but may have missed it.

Tyler

EBaker
by
New Contributor

Hi Zach,

Is the updated version available yet? 

Thanks!

Elliott

0 Kudos
MARIEL-U
New Contributor

 Where can we find the updated version?     

i am using ArcGIS Pro 2.8.0 

Thank you.

0 Kudos
FredMitchell
New Contributor III

Hi @ZacharySutherby,

Just came across this post and wanted to see if there's an updated script available? Thanks!

Fred