How to copy content from one ArcGIS Online account to another using Python

11795
13
08-20-2020 08:54 AM

How to copy content from one ArcGIS Online account to another using Python

Most content (feature services, maps) can be copied using ArcGIS Online Assistant. The Python method described below is required for more complex content. You can use it for dashboards, web apps, surveys, and configurable apps. It will not work for ArcGIS StoryMaps or ArcGIS Experience Builder.

First, you'll identify the items you want to copy.

1. Go to ArcGIS Online and sign in with your source ArcGIS account—the one that contains your content.

2. Click the Content tab and locate the item you want to copy. Click the item name to open the item details page.

3. Copy the id from the address bar in your browser. This is the long string of numbers and lower-case letters that appear at the very end of the URL. Do not include id=.

4. Paste the id into a note or other temporary document.

5. Find any other items you want to move. Copy the id for each.

Some items contain other items as dependencies. For example, a dashboard probably references a map, and that map probably references a feature layer. The dashboard, map, and feature layer are three different items, but you only need to copy the dashboard. The dependent items will move with it automatically.

 

Note: This document will first demonstrate how to copy a few items. At the end, it will explain how to adapt the process to copy many items at once.

 

Next, you will create a new Jupyter Notebook in ArcGIS Pro.

Jupyter Notebooks are an environment that you will use to write the Python code to transfer your content.

6. Open ArcGIS Pro. It does not matter which account you sign in with.

7. Click Start without a template.

8. On the ribbon, click the Insert tab and click the New Notebook

The first thing you'll do in your new notebook is to connect to your source account—the one where the content is currently stored.

9. In the first cell, paste this text:

username = input("Enter username of origin organization: ")
from arcgis.gis import GIS
gis1 = GIS("https://arcgis.com", username)

10. Click Run.

11. In the textbox that appears, type your username and press Enter.

12. In the Enter password textbox, type your password and press Enter.

Next, you'll connect to your target ArcGIS account—the one you want to move your content into.

13. In the next cell, paste the following text:

username_2 = input("Enter username of target organization: ")
gis2 = GIS("https://arcgis.com", username_2)

14. Click Run.

15. Enter the username and password of your target account.

 

Finally, you'll identify the item(s) that you want to transfer and copy them to your target account.

16. In the next cell, type the following text, but do not run the code yet.

items = gis1.content.search(query="id:")
print(str(len(items)) + " item will be cloned:")
items

17. Position your curser after the text query="id: and paste the id string that you copied earlier.

18. If you have multiple items, you can reference them like this:

items = gis1.content.search(query= "id:(5f3b5c9325314fc3a11c1d649d37461e OR fc1b539234f44ee89845ea926e5e393f)")

19. Click Run.

20. The name of the item(s) to be cloned is printed. Confirm that it is correct.

Finally, you will clone your content into your target account.

21. In the next cell, paste the following text:

def deep_copy_content(input_list):
    for item in input_list:
        try:
            print("Cloning " + item.title)
            copy_list = []
            copy_list.append(item)
            gis2.content.clone_items(copy_list, copy_data=True, search_existing_items=True)
            print("Successfully cloned " + item.title)
        except Exception as e:
            print(e)
    print("The function has completed")
deep_copy_content(items)‍‍‍‍‍‍‍‍‍‍‍‍

22. Click Run. Wait until the text The function has completed has printed.

24. Go to ArcGIS Online and sign in with your target account.

25. Click the Content tab and verify that the item has been copied. If you were already on the Content page, refresh your browser to see changes.

If you are copying an item that is dependent on other items, you should see all items copied over. In the example below, I only moved TestDashboard. Test Map and Test_Data were also copied.

If you don't want this to happen, set copy_data=False in the code above.

You can read more about the clone_items function here: https://developers.arcgis.com/python/api-reference/arcgis.gis.toc.html#arcgis.gis.ContentManager.clo...

 

How to copy items in bulk:

1. If you want to copy more than a few items at a time, return to the cell you wrote in step 16. Replace the text there with this:

items = gis1.content.search(query="owner: {}".format(username), max_items=num_items, sort_field='id', sort_order='desc')
items

2. Re-run the cell. It will ask you how many items you want to copy.

3. Re-run the last cell to copy the content.

You can find more ways to search for items in the Search reference document.

 

How to copy a Classic story map:

This method will not work with ArcGIS StoryMaps. Only Classic Story Maps.

1. Use the instructions above to copy both the story and any web maps that are used in the story.

2. Open the new story in edit mode (Configure App). You'll see that the maps and images are blank.

3. Click the Edit button and Change Media to reconnect maps and image.

Alternatively, you can use the item_mapping parameter in the python script. See an example and instructions here: The Cloning Process.

Tags (1)
Comments

Hello @HeatherSmith,

great and well explained article. You mentioned "It will not work for ArcGIS StoryMaps or ArcGIS Experience Builder". Where in the Python API or REST API documentation are these limitations listed and updated ?

Regards, Bernard

Thank you @BernardRIXEN 

The documentation for cloning content is here: https://developers.arcgis.com/python/guide/cloning-content/ which lists the items that are possible to clone. Future changes should be reflected there.
There is also now an article which presents a workaround method for copying new StoryMap stories: https://developers.arcgis.com/python/sample-notebooks/clone-storymap-version2/ 

Has someone made this into a notebook that can just be shared directly already as opposed to copying from here?

Also, I got this error trying to clone a hosted feature service with related tables... any insight?

Cloning XXXXX
('Failed to create Feature Service XXXXX: list index out of range', <Item title:"XXXXX" type:Feature Layer Collection owner:XXXXX>)
The function has completed

NikolasHawley_1-1618694929005.png

 

 

@NikolasHawley that issue is normally related to data not being able to be copied to/from a destination.  If you have a `field` with HTML in it or generic text, make sure it's sanitized before the clone operation.

Here is a list of supported HTML tags:

https://doc.arcgis.com/en/arcgis-online/reference/supported-html.htm  

 

 

Has anyone had the issue "failed to create feature service name of item : 'available', None

edward33_0-1619147905316.png

I am trying to transfer a web map

 

Update: I realized the issue was because I was trying to transfer a web map with a feature layer that was hosted on my original account. The account I was trying to transfer to is a public account and they can't host feature layers.

HI guys,

 

 im runnig the script in jupyter and im getting this error in several items, any idea??

 

('Failed to create Feature Service Departamentos: Your request has timed out.\n(Error Code: 504)', <Item title:"Departamentos" type:Feature Layer Collection owner:NicolasM>)

 thanks!!!

Cloning Plan Compare (latest)
('Failed to create Feature Service Thu Duc City Proposed Land Use Plan: list index out of range', <Item title:"Thu Duc City Proposed Land Use Plan" type:Feature Layer Collection owner:gisencityadmin>)
The function has completed

Hi, I am also facing the same problem. I don't use any HTML tags on this application, What could be the problem? 

@HeatherSmith and @cmlosaria_gis 

To those of you who were getting the "list out of range" error, how did you resolve it? I have no issues cloning service definitions, file geodatabases and hosted feature layers with only one layer. I get the error anytime I try to clone a hosted feature layer that contains multiple layers and/or tables. These layers and tables are related to one another, if that makes a difference.

Thanks,

Teri

Is it possible to use this workflow to copy from ArcGIS Online to Enterprise (Portal)? @HeatherSmith 

 

Thanks,

@Juan_Toro-Killion I am told that yes, you can use clone_items() to move content between any two organizational deployments.

@HeatherSmith Thank you!

Hi - I used the code exactly as above to copy a dashboard from one AGOL to another, but it only copied the dashboard and not the associated web map and feature layers.  The username owns all the items, any thoughts on way it only copied the dashboard?

def deep_copy_content(input_list):
for item in input_list:
try:
print("Cloning " + item.title)
copy_list = []
copy_list.append(item)
gis2.content.clone_items(copy_list, copy_data=True, search_existing_items=True)
print("Successfully cloned " + item.title)
except Exception as e:
print(e)
print("The function has completed")
deep_copy_content(items)

 

Cheers

Chris

P.S if I just copy the web map then the associated data does get copied.  But when using the dashboard ID just the dashboard copies over. 

Thanks for this @HeatherSmith

I successfully cloned a couple of feature layers across from AGOL to our Enterprise system which definitely is a win for us. However, when I try to copy a complete Survey it gives me the following error:

Failed to update Form Death Animal Survey - Jordi: 'NoneType' object is not iterable"

It seems to be pointing at the form but I am not sure how to fix this? I have also tried to only clone the form part of the survey to see if that would work but unfortunately not. Thank you very much!

 

Version history
Last update:
‎12-12-2021 03:42 AM
Updated by: