Add Individual Layers within ArcGIS Server Services as Items to ArcGIS Online/Portal

[#1]

As many of you know, it's easy to share your ArcGIS Server service to ArcGIS Online/Portal for ArcGIS when you are publishing the service.  One thing that I don't find so easy is adding individual layers from this service as items to AGOL/Portal.  For example, say I am publishing the following MXD to ArcGIS Server:

You can see the MXD contains 9 different layers.  When sharing this service with AGOL/Portal, it will create a single Map Image Layer:

When adding this layer to a web map, it will bring in all 9 layers.  Say I am only interested in adding one of these layers, i.e. Rivers.  I would have to remove all the other layers in the web map, or I can add the individual layer as an item to AGOL/Portal using the REST URL and the corresponding index number, ex:

https://ags.esri.com/arcgis/rest/services/Philadelphia/MapServer/5

The layer will then be added as a Feature Layer to AGOL/Portal:

The rivers layer is now discoverable within my AGOL/Portal Organization and can be easily added as an individual layer to a web map.  However, you can see that this can be time consuming to add all 9 layers as items to your Organization.  The attached GP tool will do this automatically for you.  Ex:

After the tool is executed, the individual layers are all added as Feature Layers within AGOL/Portal:

Some may question why not just publish all 9 layers as individual ArcGIS Server service.  This is a feasible workflow, but one thing to keep in mind is that this would consume more resources on your ArcGIS Server instance.  For example, below is the CPU and Memory usage when only the single service is published:

Compare this to the CPU and Memory when all 9 layers are published as services.  There is a slight increase in both:

[deleted-user-AYsXjhkrwuAA]

Thanks Jake Skinner‌ for the script. I had trouble authenticating with our portal and server as these are only setup with Active Directory users and integrated windows authentication.  In the end I adapted your tool to use the windows API which can use built-in authentication to connect as the logged on user, the ArcPy Toolbox may be handy for anyone else in the same boat.

[Anonymous User]

Thanks Jake to sharing you work and experience with us

[FrankGoehner1]

I was trying your tool but I keep getting the error:

Failed script AddItemAGOL...

Traceback (most recent call last):
File "C:\Users\fgoehner\Documents\ArcGIS\AddIns\AGOL.py", line 37, in <module>
for layer in data['layers']:
KeyError: 'layers'

Failed to execute (AddItemAGOL).

Any ideas on how to fix this?

Frank

[JakeSkinner]

Hi 1_fgoehner‌,

Can you send a screen shot of how you have the tool setup?

[FrankGoehner1]

Hi Jake,

I tried to run the toolbox in ArcCatalog and ArcMap.

Here is a screenshot of the setup:

Here is a screenshot of the error

Thanks

Frank

[SimonJackson]

Really handy tool!  Nice work.

Would be great to have capabilities like this in the free version of GeoJOBE Admin Tools and the ArcGIS Online Assistant.
Has anyone seen any alternative versions to this tool, perhaps accessible from a browser?

 Admin Tools for ArcGIS Online &amp; Portal

[SimonJackson]

I am getting the exact same error as Frank, trying to register a public facing dynamic map service (10.3.1) with my ArcGIS Online account (logged in as a publisher).

Any ideas how to resolve?

Have you also seen any scripts that scrape through the entire ArcGIS REST Services directory to create items of every layer within every service?

[JakeSkinner]

sjacksonesriaustralia-com-au-esridist‌, can you send me the URL you are using?

[VivekDey1]

Hi Jake,

I am able to add individual layers via Add Item. However, I am unable to add them to my map on ArcGIS Online. I am getting the following error: Do you know what can be the reason. I am adding a public facing URL. I can add the map service but not the individual layers.

Thanks,

Vivek

[JakeSkinner]

Hi Vivek,

Can you provide the URL of the layer you are trying to add to a web map?

[VivekDey1]

Thanks Jake!, I have figured it out. One need to have a SHAPE feature class to be added as a layer.

[MehmetCuhadaroglu]

Hi Jake,

I am trying to use this tool but I keep getting this error message:

And my URL:   

MarinMap2/Open_Data_Download (MapServer) 

Thanks,

Mehmet Cuhadaroglu

County of Marin

[MehmetCuhadaroglu]

Hi Jake,

Never mind , we figured it out.  Perhaps I downloaded older version of the script which was missing a line at the end that we added:

name = layer['name']

it works great, very useful script.

Thanks,

Mehmet Cuhadaroglu

County of Marin

[TimMinter]

Nice work, Jake Skinner!  I like it...  I'm trying to use it with our Portal deployment, which uses ADFS for security, and am getting this message:

• URLError: <urlopen error [Errno 10061] No connection could be made because the target machine actively refused it>

My starting point for troubleshooting the code is to assume that it does not support ADFS.  I'm going to start hacking as soon as I sign off here, and I thought you might be able to point me in a good direction...

Ooh!  just had a thought.  be right back.  

Nope, dang.  I thought "ArcGIS Administrator > Connect to Portal > ArcMap > Sign In to Portal > then try to run tool"  

So, anyway, I'm hitting the documentation and messing with the code.  Let me know if you have ideas about how to proceed.

[JakeSkinner]

Tim Minter‌ the key would be to generate the token using ADFS.  I'm not quite sure how to do this, yet.

[TimMinter]

Thanks Jake Skinner‌, that's pretty much where I ended up with my wild guesses after poking at the code with my favorite stick.  There seems to be some discussion around the subject here - https://community.esri.com/thread/191593-how-to-connect-to-my-portal-when-our-security-is-adfs, but I'm not yet clear about it applying to my environment.  No biggie, I ended up rolling some tunes and getting manual with it just to get it done.

[NasirAhmad2]

I faced these problems while trying to add layers individually; adding individual layers in a web map causes the Collector for ArcGIS (only windows version) to crash when you start the edit session. SQL Views as a service added individually will not display the changes until you restart the Collector, change the base map or something to force the Collector to re-establish the connection with ArcGIS for Server.
But everything will work fine if you add them as a group.

Let me know if anyone has a solution for this. I really like to add the layers individually.

Thanks,

Nasir