Clone surveys from one organization to another
Introduction:
A common question the Survey123 team has received from organization administrators is, "What's the best way to clone my surveys from one organization to another?"
There are two common use cases for cloning surveys:
- Create a copy of a survey in another ArcGIS organization. For example, a city’s transportation and water departments have different ArcGIS Online organizations and the water department would benefit from having a copy of one of the transportation department’s surveys as well as its associated web map and dashboard.
- Clone a survey from a development organization in ArcGIS Enterprise to staging and production organizations.
This blog and sample Python notebook demonstrate how to clone surveys and associated content from one organization to another. This workflow can be used to clone surveys from ArcGIS Online to ArcGIS Online, ArcGIS Online to ArcGIS Enterprise, or ArcGIS Enterprise to ArcGIS Enterprise. The direction of cloning does not matter.
The foundation of the workflow is the clone_items() method in the ArcGIS API for Python. This is the infrastructure that allows us to clone surveys from a source organization to a target organization. Given the different content and item types, possible ArcGIS Enterprise configurations, potential ArcGIS Online configurations, security considerations, and item dependencies, the clone_items() method aims to produce an exact duplicate of an item that retains all of its functionality.
Please note that cloning relies on the sharing model to determine the items a user can clone. If a user can access an item, that user can clone it. However, a user can’t create any items in the target organization if they don’t have the appropriate privileges to create.
For more in-depth content on how the clone_items() method works please review the ArcGIS API for Python Guide on cloning content, as well as the documentation.
A copy of the full notebook is available for download in our Survey123-tool GitHub repo.
Prepare to clone:
To start, we are going to need two GIS connections; one to our “source” organization, which is the organization in which the survey and content currently resides that we would like to clone; and another to a “target” organization, which is the organization that we would like to clone the survey and content to.
In our first example we are going to highlight a workflow where we have a few surveys shared to a group that we would like to clone to a different organization. In order to work with our surveys a Survey Manager is defined. A survey in the Survey Manager is a single instance of a survey project that contains the item information, properties, and provides access to the underlying survey dataset. Please use this link for more information on working with the Survey Manager. In this example, four surveys are shared to a group. Using the group ID, a connection is made to the group and a list is created containing all form items within the group.
Now that we have our forms as a list, we are ready to clone the content from our source organization to the target organization. As previously noted, a use case for using the clone_items() method is to clone surveys between development, staging, and production organizations. This first example clones the surveys from our existing group (as defined above) located in the source organization to the target organization and shares the cloned surveys to a group with the same name in the target organization.
Clone related items:
The code below starts by creating a new group in the target organization using the same title and tags as the existing group in the source organization. Once the surveys are cloned, they will be shared to this newly created group.
Each form item in the source group is looped through, obtaining the feature service associated with the survey through the “Survey2Service” relationship, as well as any additional items related to the survey using the “Survey2Data” relationship. This would include any linked content or report templates associated with the survey. All related items are merged into a list to be cloned. Please review the relationship types documentation for more information on related items in ArcGIS.
Next, a new folder in the target organization is created based on the survey name, and the items in the list are cloned. Since the `copy_data` parameter is set to “False” in the clone_items() method, the resulting services created in the target organization will not contain any data. This is ideal when cloning from a development environment to a staging or production environment, as you might not wish to retain any test data.
Finally, the form items are shared to the group in the target organization that was defined previously, including the feature service, and tags are added to each item.
As a confidence check, let’s query the newly created group and confirm our four surveys have been shared.
The example above is a useful workflow for cloning surveys between development, staging, and production environments, and will only clone items that are linked to the form item including: report templates, linked web maps, CSVs, and map packages. But what if we would like to clone not only the survey and its related items, but also the survey data that’s already been collected? Additionally, what if we have web maps that aren’t linked to the survey, web apps, or dashboards that use the survey data?
If those items are stored in the survey’s folder, a slightly different method can be used to clone all the content from this folder.
Clone full survey folder:
The code below demonstrates connecting to one specific survey within our organization. Using the properties of the survey, the folder ID where the survey resides is assigned to a variable. Next, all the folders in the source organization for the source username are listed. Using list comprehension, the folder in the `full_folder` variable is matched with the folder ID obtained from the survey properties.
Once the correct folder has been identified the contents of the folder are listed.
Now that all items to be cloned are in a list, a new folder is created in the target organization to store the content. After the folder is created the content is cloned to the target environment. Since the `copy_data` parameter is not defined the default value for the parameter is “True”, meaning all the underlying data will also be cloned. This means the resulting content in the target organization will be an identical clone of the original data. If you do not wish to retain the source data, setting the `copy_data` parameter to “False” will only clone the data schema and architecture to the target organization. Meaning the survey, web maps, web apps, dashboards etc. will configured as per their original items; the only difference is the feature layer will be empty.
In this blog we’ve covered two use cases of using the clone_items() method. This sample notebook is intended to be used as a guide; you can take what’s here and incorporate it into your own workflows.
What workflows or use cases do you have that we missed? Please let us know your use cases and workflows and we can work on incorporating them into the notebook.
Notes on limitations:
- Clone Fails with Non-Ascii Characters in Service Name
- Cloning is limited to 1000 records