It's always a good practice to have a DEV/STAGING environment along with a PRODUCTION environment for ArcGIS Enterprise. I frequently get asked how to migrate content from one to the other. Hosted services can be straight forward, and can easily be scripted (expect an update later to this toolset for this functionality), but referenced services can be rather difficult. For example, there are numerous prerequisites that are required for a service to be published from an Enterprise Geodatabase, such as:
These tools pick up once your services have been published to the other environment and you would like to begin migrating web maps, apps, and dashboards. Take a look at the video below on how to execute these tools.
These tools should only be used to migrate web maps/apps where the environments are the same version (i.e. 11.1). Migrating content between two ArcGIS Online organizations is also supported.
Currently, these tools have only been tested in ArcGIS Enterprise 11.1 and ArcGIS Online. If there are any issues, please report in the comment section below.
1/16/24: Added tools to copy hosted feature services and file based items.
1/29/24: Added option to run when portal has Windows Authentication enabled. Also, updated the Copy Dashboard tool to specify stand-alone layers.
2/12/24: Web Map item IDs now update
3/28/24: Added tool to copy Story Maps
6/30/24: Group layers and tables are now supported when copying web maps
7/10/24: Experience Builder Templates are now supported
Hello Jake,
Thank you so much for this solution! You are a life saver!
I was just going to tackle the task of deploying dashboards from dev to prod and your post popped up. It saved me so much effort and time. Thank you!
Here are my 2 cents:
I have tried the Copy Web Map and Copy Dashboard tools.
We use Integrated Windows Authentication and an error popped up while filling out the parameters that connection to the portal cannot be made.
I changed the validation code to not provide user and password as arguments, as they are assumed from the user currently logged in and it worked:
source = GIS(self.params[0].value)
target = GIS(self.params[6].value)
I made the same changes in all .py scripts:
source = GIS(sourcePortal)
target = GIS(targetPortal)
And I was able to copy a web map and a dashboard that has a map widget.
A lot of our dashboards don't have a map widget, and the tool cannot be used, but I was able to use your script to get the JSON of the original dashboard, change the item IDs of the source to the target ones and then use the second part of the script to copy the dashboard to the target portal. And it worked.
Thank you again for your priceless contribution!
Darina
@DarinaTchountcheva thank you for this valuable feedback. I've updated the tools to include options for when Windows Authentication is enabled for Portal, and also updated the Copy Dashboard tool to include updating the item IDs of stand-alone layers.
@Jake you are truly amazing!
Thank you so much for taking the time to make the modifications. I and a lot of other users will benefit greatly from it and will be more efficient.
I will let you know how it worked as soon as I get a chance to test it.
Best regards,
Darina
I'm trying to use these tools with an IWA integrated portal and an AGOL. I'm getting the following error when I use my portal in the Source URL and check 'Source Portal Windows Authentication Enabled'. I've tried with my Pro signed into my portal and out of my portal, as well as with different Python environments. My user does have admin rights to the Portal. Do you have any thoughts? Thanks!
Error 032659 updateParameters Error: Traceback (most recent call last):
File "Q:\admin\Template\expressions\ESRI_ScriptsToolboxs\Migrate Content\Migrate Content.atbx\CopyWebMap.tool\tool.script.validate.py", line 203, in <module>
File "Q:\admin\Template\expressions\ESRI_ScriptsToolboxs\Migrate Content\Migrate Content.atbx\CopyWebMap.tool\tool.script.validate.py", line 109, in updateParameters
File "Q:\admin\Template\expressions\ESRI_ScriptsToolboxs\Migrate Content\Migrate Content.atbx\CopyWebMap.tool\tool.script.validate.py", line 40, in usersDropDown
File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\__init__.py", line 4943, in search
users = self._portal.get_org_users(
File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 1244, in get_org_users
resp = self._org_users_page(
File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 2922, in _org_users_page
return self.con.post("portals/self/users", postdata)
File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1524, in post
return self._handle_response(
File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1000, in _handle_response
self._handle_json_error(data["error"], errorcode)
File "C:\Users\EReid\AppData\Local\ESRI\conda\envs\arcgispro-py3-ereid\Lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1023, in _handle_json_error
raise Exception(errormessage)
Exception: You do not have permissions to access this resource or perform this operation.
(Error Code: 403)
Open the python console, or new Notebook in Pro by going to Analysis tab > Python dropdown:
Enter the following code, replacing https://gis.esri.com/portal with your URL to portal:
from arcgis.gis import GIS
gis = GIS('https://gis.esri.com/portal')
source_users = gis.users.search(max_users=10000)
source_users
Ex:
Does this execute successfully?
It does not, same error.
I just double checked and our Portal security actually says SAML is set up, not IWA. I assumed they were the same thing. I'm not sure if they work the same.
@Eliot_Reid I had a feeling this may be the case. SAML is not supported. If you can, create a built-in Administrator account and use that instead.
@JakeSkinner
I am getting same error as @Eliot_Reid could you help me how can I fix this error? How can I create a built-in Administrator account? I am admin in our portal org
Best,
@anonymous55 go to Organization > Members > Add Members:
There you will have an option to create a Built-in account:
@MichaelWestbrook right-click on the tool you are trying to execute and click on the Execution tab on the left. It will list the path to the python script. Make sure it exists in the path referenced here. Ex:
Thank you. I had to create all of the folders in the the and copy the data there. It works perfectly now and does exactly what I need it to do. Are you going to make it so you can copy EXB templates as well?
@MichaelWestbrook EXB templates are now supported. You will need to re-download the tool.
@JakeSkinner Thank you
Hey @JakeSkinner
Doing a staging to production migration now and just found your tool, seemingly I just found the blog sections of Community! Great tools you have here and great work you've done, thank you for providing!
Cody
Thank you @CodyPatterson, I appreciate the feedback.
Hi Jake
Great work. I have sendt you a suggestion for a updated version of the "hosted feature service" script on mail.
Best regards
David
@JakeSkinner Good morning, I am trying to copy web maps and as soon as I select the web, PortalUrl gets an error
@MichaelWestbrook I found the issue, the validation code checks the web map for tables, if none exists, it produces this error. Redownload the tools, and it should work successfully now.
@JakeSkinner Works perfect now! Thank you!
@JakeSkinner I copied the web maps over successfully. When I copied the Experience Builder it ran successfully but out of the 7 web maps in it, it only mapped one successfully.
@MichaelWestbrook I've seen sometimes if you don't tab out of each Web Map ID box, the GP tool won't pick up the IDs correctly. Try tabbing out of the New Web Map ID and re-run the tool again to see if it works.
Hi @JakeSkinner have you considered adding this to Github? Potentially as a Python Toolbox to enable easier contribution to the validation rules?
Keen to provide some contributions to this useful resource.
The transfer worked but I was wondering if this error meant something that I need to be wary of:
Hi @JakeSkinner , I'm trying to migrate a dashboard from my ArcGIS Online account to an ArcGIS Portal 11.1 account, apparently the tool works fine but when I check the item it is empty. I even checked the .json through assitant and it has the dashboard structure copied. Do you know how I could solve this problem?
PD: I am the administrator of both accounts.
@jdcayetano no, this should not be a concern. The script did not delete a temporary zip file mentioned in the path above.
@JakeHanson-eaglenz once I learn GitHub, I'll post it there.
@CommunityMapsEsriColombia migrating from ArcGIS Online to Enterprise will not be supported. ArcGIS Online is always at a later release than Enterprise, so applications will fail to open. What should work is migrating Web Maps and Hosted Feature services from AGOL to Enterprise, but I have not extensively tested this. Ideally, you should only be migrating content between the same environments (i.e. AGOL --> AGOL, Enterprise 11.1 --> Enterprise 11.1).
@JakeSkinner Thanks for your great work.
I need to copy non hosted Feature services but it does not seem to be possible.
I tried many things without success like "content.add" and "content.clone_items".
There must be a way to copy a non hosted Feature service. Do you have any clue?
@RejeanLabbe copying referenced services is very difficult, especially if the data source will change. Due to the many possible parameters when publishing referenced services, this is not easily scripted.
@JakeSkinner Thanks for your answer. If anything changes about this in the future, I would be glad to ear about that.