The attached toolbox contains 3 tools for copying content between Portals.
File Geodatabase,CSV, Image, KML, Locator Package, Map Document, Shapefile, Microsoft Word, PDF, Microsoft Powerpoint, Microsoft Excel, Layer Package, Mobile Map Package, Geoprocessing Package, Service Definition, Scene Package, Tile Package, Vector Tile Package
For the tool to successfully update the web map, web application, and dashboard IDs the client machine executing this tool must be able to reach the Target Portal's server where Portal for ArcGIS (not the Web Adaptor) is installed (i.e. https://portal.esri.com:7443/arcgis). It uses this URL to make the post requests if Windows Authentication is enabled on Portal's web adaptor.
Each tool will share the content with Everyone, and the ArcGIS Enterprise Organization (if it is shared with these in the Source Portal). It will also share to each Group in the Target Portal if it has the same name as the Source Portal Group.
Note: This tool is in it's early development and has not been extensively tested. So far I've tested transferring content where both Portals are version 10.7.1. Looking to have any issues or recommendations reported in the comments below.
I can confirm that these tools work to copy Vector Tile Packages between Portal versions 10.6 to 10.7.1
noting this for reference:
Clone Portal users, groups and content | ArcGIS for Developers
Andres Castillo just a note on the above web link, those scripts will not copy any of the services. See the Conclusion section:
Note, this notebook did not copy over the services that power the service based items. Such items continue to point to the same URL as the ones in source portal did
For the Copy Map and Applications tool, when I attempt to specify mulitiple contents, the script tool loads endlessly.
Any suggestions?
I haven’t seen that issue before, sounds like something not right with the script. I suggest reaching out to Esri Support.
E.J. McNaughton
Andres Castillo What version of ArcGIS Pro are you running?
2.5.0
When I try to run outside of pro in vs code, I notice the token is not being stored:
Processing item: u
WARNING: ------------------
Error occurred processing item u
Error: Token Required
(Error Code: 499)
Deleting Connections
some modifications/observations I made to the script:
Andres Castillo if Windows Authentication is not enabled on Portal's web adaptor, you will not have to update the :7443/arcgis to /webadaptor. I just updated a note about this in the tool's description above. Also, thank you for catching the misspelling of Exception. I updated the code and re-uploaded.
I haven't tested going from 10.7.1 to 10.5.1. Are you able to go from 10.5.1 to 10.7.1? It could possibly be a version issue.
For the copy services script,
I can successfully copy over hosted feature layers from 10.7.1 to 10.5.1.
Windows Authentication is enabled on the wa.
Are your comment and the document description above in alignment?
The token error I experienced above was due to the fact that I didn't explicitly store my credentials in the GIS() object.
I sometimes get away with not putting my credentials because I've noticed that in other cases, portal auto-reads them using Windows Authentication and my AD credentials.
Testing transfer on Pro 2.5.1 with both AGS Enterprise systems at 10.8. So far, keep getting 'error occurred processing items' for all items. I love the concept. In the future, it might be good to have an 'all' where we can transfer all content from source regardless of user into one folder or the choice to copy content into default folder schema from source. This will help in staging and transferring AWS containers as well as solution transfers from Dev, Test and Prod.
Jake Skinner This script seems like exactly what we need. I just tried a single web map test with Pro 2.5.1 with the origin portal at 10.6.1 and the destination portal at 10.8.
I also got errors similar to the guy above: "Error occurred processing item" <class 'keyerror'> 402
Any ideas on how to get this running correctly would be greatly appreciated!
verify you are an admin on the portals
verify you are an admin on the portals
Yes Admin and verified.
Evan Marshall do you receive this error for all web maps, or just one in particular? Are you able to copy feature services successfully?
Luke Savage are you attempting to copy feature services, or web maps/apps?
Any content.
Hey Jake. I had not tried it with any services, as we are just looking to move some apps and maps over and will likely republish some services manually since we will be moving some of them to hosted layers in the portal. The map I tried was a simple web map with only our parcel layer in it.
Evan Marshall how did you add the parcel layer to the web map? If you add the layer with the URL (i.e. Add > Layer From Web), you will receive an error when trying to copy the web map. Instead, add the parcel as an Item in Portal (i.e. Content tab > Add Item). Then, add the parcel layer to the map by going to Add > Search for Layers.
By any chance, can this be used with copying over services from AGO to portal? I put in our AGO URL and it did not accept it.
I tend to use AGO Assist, but copying over feature services is unreliable. Please share if you know of other tools that can accomplish this.
Reza Tehranifar I've had sporadic results going from AGOL to Portal. I've written the below code to do this and it is much more reliable. It was originally written to copy all services from all users. You can specify an individual user to copy services from on line 117:
Removing this line will copy all services from all users.
import arcpy, requests, json, sys, os, time
from arcgis.gis import GIS
from arcgis.gis import User
arcpy.env.overwriteOutput = 1
startTime = time.clock()
# Variables
sourcePortal = 'https://www.arcgis.com' # AGOL URL
source_username = 'jskinner_CountySandbox' # AGOL Admin Username
source_password = '********' # AGOL Admin Password
targetPortal = 'https://jake2.esri.com/portal' # Portal URL
target_username = 'jskinner@AVWORLD' # Portal Admin Username
target_password = '*************' # Portal Admin Password
targetOwner = 'jskinner@AVWORLD' # Portal User to copy services to
targetFolder = 'Copy' # Portal User's folder to copy services to
# Create GIS objects for source and target portals
source = GIS(sourcePortal, source_username, source_password)
target = GIS(targetPortal, target_username, target_password)
export_folder = arcpy.env.scratchFolder
# Function to log errors
def errorLog(msg, e):
print(msg)
print("Error: {0}".format(e))
# Function to Copy Hosted Feature Services
def copyContent(item):
# Export hosted feature service to FGD, and downlaod to scracth folder
print("Processing {0}".format(item.title))
export_name = "{0}".format(item.title)
try:
print("Exporting {0}".format(item.title))
result_item = item.export(export_name, 'File Geodatabase', wait=True)
except Exception as e:
errorLog("Error exporting {0} File Geodatabase".format(item.title), e)
return
try:
print("Saving File Geodatabase to: {}".format(export_folder))
download_result = result_item.download(export_folder)
except Exception as e:
errorLog("Error saving {0} File Geodatabase to {1}".format(item.title, export_folder), e)
result_item.delete()
return
try:
print("Deleted {0} File Geodatabase from source portal".format(item.title))
result_item.delete()
except Exception as e:
errorLog("Error deleting {0} File Geodatabase from source portal".format(item.title), e)
return
# Set Item Properties
item_properties = {
'title':item.title,
'type':'File Geodatabase',
'description':item.description,
'snippet': item.snippet,
'tags':item.tags,
'extent': item.extent,
'accessInformation': item.accessInformation,
'licenseInfo': item.licenseInfo
}
# Get thumbnail and metadata
try:
thumbnail_file = item.download_thumbnail(arcpy.env.scratchFolder)
except Exception as e:
errorLog("Error getting thumbnail", e)
return
try:
metadata_file = item.download_metadata(arcpy.env.scratchFolder)
except Exception as e:
errorLog("Error getting metadata", e)
return
# Add File Geodatabase to portal
try:
print("Adding {0} File Geodatabase to target portal".format(item.title))
if targetFolder == 'ROOT':
fgd = target.content.add(item_properties=item_properties, owner=targetOwner, data=download_result, thumbnail=thumbnail_file, metadata=metadata_file)
else:
fgd = target.content.add(item_properties=item_properties, owner=targetOwner, folder=targetFolder, data=download_result, thumbnail=thumbnail_file, metadata=metadata_file)
except Exception as e:
errorLog("Error adding {0}".format(item.title), e)
return
# Publish File Geodatabase
try:
print("Publishing {0} File Geodatabase".format(item.title))
published_service = fgd.publish()
except Exception as e:
errorLog("Error publishing {0} File Geodatabase".format(item.title), e)
fgd.delete()
return
# Delete File Geodatabase from target portal and scratch folder
try:
print("Deleting {0} File Geodatabase in target portal".format(item.title))
fgd.delete()
except Exception as e:
errorLog("Error deleting {0} File Geodatabase in target portal".format(item.title), e)
return
# Main
source_users = source.users.search('!esri_ & !system_publisher', max_users=10000)
for user in source_users:
if user.user_types()['name'] not in ('Viewer', 'Editor', 'Field Worker'):
if user.username == 'jskinner_CountySandbox':
print("----------------------------------")
print("User: {0}".format(user.username))
user_content = user.items()
folders = user.folders
for folder in folders:
for item in user.items(folder=folder['title']):
user_content.append(item)
for item in user_content:
if 'Hosted Service' in item.typeKeywords:
if 'Map Service' not in item.typeKeywords and 'View Service' not in item.typeKeywords and 'Scene Service' not in item.typeKeywords:
copyContent(item)
# Clean up downloaded File Geodatabases
print("Cleaning up Scrath Workspace")
for (path, dirs, files) in os.walk(arcpy.env.scratchFolder):
for file in files:
os.remove(os.path.join(path, file))
endTime = time.clock()
elapsedTime = round((endTime - startTime) / 60, 2)
print("Script finished in {0} minutes".format(elapsedTime))
Thanks Jake Skinner. I will test this out!
Ideally instead of user, I would prefer to pass item ID's. But there is enough there for me to just change username to item ID.
Jake Skinner
Hi, the code looks very promising for some of my current work. I've had to hardcode the parameters again as I can't see any script tools in the zipped toolbox, so i'm struggling with some of the parameters and my trial-runs don't do anything... Has anyone got a sample of the parameters that have worked for them please?
Many thanks for creating this.
David Pike what version of ArcGIS Pro are you using? In the zip file there is a Copy Content.tbx file that will show up as a toolbox in ArcGIS Pro.
I'm using ArcMap, that may be the issue!
I tried to get it to load in my ArcGIS Pro 2.6.2 but I am not seeing it under tool boxes?
What about transferring 'Site Application' and 'Site Page' types?