ArcGIS Portal - How to create portalitem with always the same ID?

[LorenzMeyer1]

We're creating portal items through the ArcGIS API for python. This works quite very well. But there's one thing I don't get:

We provide on the portal several services (WMTS, Map Image Layer) as a base for portal users. So a portal users can create their own WebMaps and use those base portal items in their WebMaps. Let's assume now, that we have a use case where we want to redeploy all those base services. So we run again our Jupyter notebook to create all those base services through the ArcGIS API for pyhton. 

The thing is now, that all those base services get a new itemID when deploying through the jupyter notebook, meaning that all WebMaps from portalusers that used our base portal items won't work anymore. It's not quite right. The layer in the WebMap is still shown, but an error message pops up and says that the layer could be loaded. 

So how can we PRESERVE this itemID?

This issue is one little puzzle in a bigger question like: How do we migrate portal from one version to another without loosing any user content.

[MatthiasSchenker]

Hi Lorenz

Example: Update the URL of a service in a web map—Portal for ArcGIS (10.6) | ArcGIS Enterprise contains a script which allows you to update all the service references in a webmap.

You could run this as part of your notebook updating the service, I do not think that it is possible to preserve the URL and ID of hosted raster layers.

I know that you only mentioned raster layers, but just FYI, Manage hosted tile layers—Portal for ArcGIS | ArcGIS Enterprise describes how you can use "Replace Layer" for Vector Tile Layers to only update the content but not the URL or ID.

Added later: Also, have a look at http://cloudygis.maps.arcgis.com/home/item.html?id=d829e15ca7d840f29b1a1d8d2e8b7ac7 . The description states "This script uploads an updated tile package and extracts the tiles to a new tile service in ArcGIS Online." but I am not sure that it does indeed create a new tile service, it seems to take the URL of an existing service as input. I have not tested it but it could be that the ID stays the same.

Best,

Matthias

[LorenzMeyer1]

Hi Matthias 

Thanks for your feedback.

It's not exactly what I'm looking for. I mean, updating the URL of an existing item is just one small piece of the answer to the big questions that keeps me busy acutally.

How can we copy / clone content with preserved portalitem-IDs from one stage (one portal) to another, even through different versions? Or how can we migrate portalcontent to a new version without doing it manually (updating arcgis server, updating portal, updating datastore, updating webadaptor ....)? We have infrastructure as code, means everything is scripted and we stage from scratch. What we would like to do is the following: Save complete content of the portal, reinstall the complete portal (same architecture, but "probably" different version), restore saved content to this new portal.

In my opinion there's no way to do that. Let me explain what I found out so far:

- There's no tool to just backup or save the complete content (and only content) of the portal

- Backup / Restore of portal (ArcGIS Server Site, Datastore and Portalitems) does not work since it also stores site related configurations for the portal and the underlying site of the arcgis server. Hostnames, IP addresses and so on, they're all stored in this backup, means I cannot restore that on another portal.

- I found an example script (clone_portal_users_groups_and_content | ArcGIS for Developers) to copy portal item from one stage to another. It seems that it also preserves the IDs. But unfortunately it does not copy all items. For example Hosted Feature Layers are not working!

- Then I found the clone_items() function from the arcgis api for pyhton, which should handle hosted feature layers. Unfortunately it does not work either. Some hosted feature layers can be cloned, some not. I found this blog entry here: Error when cloning items from one portal to another with this sentence: "ESRI recommends republishing everything regardless and they admit clone item and copy content doesn't always work." What??? Do you really think it's a good idea to tell hundreds of portal users they have to redeploy all their hosted feature layers? You must be kidding...

So I currently have no idea on how to solve that with the tools available. Do you?

Other things I don't get with portal:

Using ArcGIS API for python (jupyter notebokk) I can create scripts to create content like WMTS layers like this:

But before I can create content like this I need the content inside the "text" json. For this I need to add the WMTS in portal manually, open the ArcGIS Online Assistant and view the items json data. Then copy paste this in my python script. Like this:

I mean, seriously?? I would just hand over the URL of the WMTS to the python function and the function does the rest itself. Why do I need the AGO Assistant? I'm pretty sure the JSON is generated inside the portal and not in the AGO Assistant.

We have infrastructure as code and deploy basic portal items (items for everyone) on each stage through a script I don't want the id's of the items to be changed everytime I deploy them. Existing webmaps which used one of this basic portal items, will no longer work, because one of the id's of the layer inside the webmap changed. This would mean, every user that created a webmap with one of this basic portal items has to fix his webmap.

If you have an idea on how to solve all this points, I would appreciate that. For sure I will take all those questions with me to the ESRI DevSummit in Berlin this year.

Best regards

Lorenz

[MatthiasSchenker]

Hi Lorenz

I am certainly not claiming that we are where we would like to be when it comes to backups. There is still a long way to go, but please bear in mind that there is also a fair amount of complexity coming from the fact that there is no backup strategy which is right for every organization, even at a broad level.

Dev Summit Europe in Berlin is definitely a good place to discuss this with of our developers and product engineers, I am happy to support you in finding the right contacts. Our engineers are however also monitoring Geonet and the ideas discussed here, such as ArcGIS Online Back-ups - hosted feature layers and web maps/apps. I know that this idea is asking for a backup option in the UI, but that does not mean that you could also add your thoughts from a scripting point of view. FYI, the above idea has actually already been reviewed and is currently under consideration. 

If it is OK with you, I will also share your thoughts with some colleagues in the US, who are most likely not monitoring the "Geodev Switzerland" space.

Regarding the WMTS item: I am well aware of this and have already discussed it with the product engineers earlier this year. I can therefore confirm that we are aware of this and are looking into possible ways to improve this experience.

Best,
Matthias

[LorenzMeyer1]

Hi Matthias

Thanks for the reply.

I'm really looking forward to the DevSummit for this specific topic. I mean, there must be a way to do what we would do, or how does ESRI migration their AGOL from one version to another. I don't think they're migrating the software manually.

Yes, please share my thoughts with some other colleagues from US. I would appreciate that.

Thanks and best regards

[CarlosNantes]

I'm wondering if this is a problem in 2020. I have a Portal and I want to migrate the items to another Portal, without changing the itemIDs. Is this possible now, without using ArcGIS Online Assistant tool?