topic Possible to find out where feature layers are being used in web applications? in ArcGIS Online Questions

Possible to find out where feature layers are being used in web applications?

ZachRobinson_SantaClaraCounty — Wed, 09 Feb 2022 16:12:29 GMT

I am wondering if it is possible to easily find out which web applications in our ArcGIS Online organization are using a specific feature layer.

For example, if I delete a layer from our organization, I want to make sure I know which web maps and applications the deletion will affect. This can be difficult and tedious to search manually when an organization has many web maps and applications.

Re: Possible to find out where feature layers are being used in web applications?

Katie_Clark — Wed, 09 Feb 2022 16:32:53 GMT

I would love to know if this is possible as well! As far as I know there isn't an "easy" solution, but maybe something can be done with the Python API?

Re: Possible to find out where feature layers are being used in web applications?

jcarlson — Wed, 09 Feb 2022 18:05:56 GMT

@Katie_Clark, you guess right: you can do this with the Python API!

The text version:

For a given layer's URL
Find all maps that reference that layer
Fina all apps that reference those maps or the layers directly

The tricky part is # 3. An app can reference a layer directly in one of its widgets, like the Search widget. Other apps, like Experience Builder and Dashboards, can also reference layers directly without going through a map. Further complicating this is that certain apps will make direct references to a layer using their ID, rather than their URL.

To make sure our search is comprehensive we can't just search for the apps that reference the maps; we'll have to search for the layers directly as well. It gets a bit convoluted, but it works!

The code:

from arcgis.gis import GIS import pandas as pd # Log in to portal; prompts for PW automatically gis = GIS('your-portal-url', 'username') # Layer ID to search for and its URL find_id = 'a405b06ae8e24f94a2768a4581b79e73' find_url = gis.content.get(find_id).url # Pull list of all web maps in portal webmaps = gis.content.search('', item_type='Web Map', max_items=-1) # Return subset of map IDs which contain the service URL we're looking for matches = [m.id for m in webmaps if str(m.get_data()).find(find_url) > -1] # Pull list of all web apps in portal webapps = gis.content.search('', item_type='Application', max_items=-1) # Create empty list to populate with results app_list = [] # Check each web app for matches for w in webapps: try: # Get the JSON as a string wdata = str(w.get_data()) criteria = [ wdata.find(find_url) > -1, # Check if URL is directly referenced any([wdata.find(i) > -1 for i in matches]) # Check if any matching maps are in app ] # If layer is referenced directly or indirectly, append app to list if any(criteria): app_list.append(w) # Some apps don't have data, so we'll just skip them if they throw a TypeError except: continue pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type} for a in app_list])

The output DataFrame will be any apps that reference your layer:

You can easily amend the final line of the expression to include additional details about the items.

Re: Possible to find out where feature layers are being used in web applications?

MichaelVolz — Wed, 09 Feb 2022 20:31:54 GMT

Have you considered looking at 3rd party vendor solutions designed specifically for this issue? I would look at GeoJobe especially if this is a time consuming task for GIS personnel at your org as AGOL and Portal mature and there is a good deal of content.

Re: Possible to find out where feature layers are being used in web applications?

Katie_Clark — Wed, 09 Feb 2022 21:21:09 GMT

Thank you, Josh! I'll definitely look into this.

Re: Possible to find out where feature layers are being used in web applications?

Katie_Clark — Wed, 09 Feb 2022 21:23:35 GMT

I think I've looked into GeoJobe before....correct me if I'm wrong, but it is not a free service, correct? It would be difficult to justify the expense to higher management (non-GIS) who would have to approve something like that, unfortunately.

Re: Possible to find out where feature layers are being used in web applications?

ZachRobinson_SantaClaraCounty — Wed, 09 Feb 2022 21:27:06 GMT

Fantastic, thank you so much for your thorough response!

Re: Possible to find out where feature layers are being used in web applications?

NavaSheer — Wed, 16 Mar 2022 09:55:29 GMT

Is there any other way to do this without Python or other coding? I am not a developer but really need this tool. Geo Jobe offers it in Pro for a fee, but would appreciate if ESRI would offer for free!

Re: Possible to find out where feature layers are being used in web applications?

Katie_Clark — Wed, 16 Mar 2022 13:37:53 GMT

The short answer for right now is "no", but you can create a custom tool in a Toolbox by right clicking on the Toolbox > New > Script.

Since @jcarlson was so generous with sharing his code, I'd like to share what I did to take it a step further and set up this tool so that others in my organization who didn't have programming experience could search for items. (setup for ArcGIS Pro)

When setting up the tool parameters, I entered "Web Map" and "Web Application" in the Value List I created for that parameter. This creates a drop down in the tool and ensures there won't be any typos in the user input that would break the code.

Also, if anyone reads this who is completely new to Python, I wanted to point out that when setting up tools to use within Pro or ArcMap, you use arcpy.AddMessage() to print messages to the console instead of a traditional print() statement.

And this is what the output looks like (nothing fancy) - this particular layer is only used in one web map. (side note - the time it takes to run depends on the number of items in your Organization's contents. It may run much quicker if you have fewer items to search through).

This community has helped me so much throughout the years, I like to give back when I can. I hope this helps.....

Best,

Katherine

from arcgis.gis import GIS import pandas as pd # Log in to portal; 'home' uses the credentials used to login within Pro gis = GIS('home') # Set up input parameters to use in the GUI find_id = arcpy.GetParameterAsText(0) search_type = arcpy.GetParameterAsText(1) find_url = gis.content.get(find_id).url if search_type == 'Web Map': arcpy.AddMessage("Searching for Web Maps. This could take a few minutes...") # Pull list of all web maps in portal webmaps = gis.content.search('', item_type='Web Map', max_items=-1) # Return subset of map IDs which contain the service URL we're looking for matches = [m.id for m in webmaps if str(m.get_data()).find(find_url) > -1] # Create empty list to populate with results map_list = [] # Check each web map for matches for w in webmaps: try: # Get the JSON as a string wdata2 = str(w.get_data()) criteria = [ wdata2.find(find_url) > -1, # Check if URL is directly referenced any([wdata2.find(i) > -1 for i in matches]) # Check if any matching maps are in app ] # If layer is referenced directly or indirectly, append map to list if any(criteria): map_list.append(w) # Some apps don't have data, so we'll just skip them if they throw a TypeError except: continue output = pd.DataFrame([{'title': m.title, 'id': m.id, 'type': m.type} for m in map_list]) arcpy.AddMessage(f"OUTPUT TABLE: \n \n {output}") if search_type == 'Web Application': arcpy.AddMessage("Searching for Web Applications. This could take a few minutes...") # Pull list of all web apps in portal webapps = gis.content.search('', item_type='Application', max_items=-1) # Create empty list to populate with results app_list = [] # Return subset of map IDs which contain the service URL we're looking for matches = [a.id for a in webapps if str(a.get_data()).find(find_url) > -1] # Check each web app for matches for w in webapps: try: # Get the JSON as a string wdata = str(w.get_data()) criteria = [ wdata.find(find_url) > -1, # Check if URL is directly referenced any([wdata.find(i) > -1 for i in matches]) # Check if any matching maps are in app ] # If layer is referenced directly or indirectly, append app to list if any(criteria): app_list.append(w) # Some apps don't have data, so we'll just skip them if they throw a TypeError except: continue output = pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type} for a in app_list]) arcpy.AddMessage(f"OUTPUT TABLE: \n \n {output}")

Re: Possible to find out where feature layers are being used in web applications?

SSMIC3038 — Thu, 25 Aug 2022 16:47:17 GMT

Thank you all for sharing this information, it's greatly appreciated and will be much used!!

Re: Possible to find out where feature layers are being used in web applications?

NickShannon2 — Wed, 28 Sep 2022 05:41:26 GMT

This is great! Thank you.

Can the script be modified to include 'web maps' in the output DataFrame?

Re: Possible to find out where feature layers are being used in web applications?

jcarlson — Wed, 28 Sep 2022 13:10:06 GMT

Of course! Since our list of maps is already defined earlier, that last line could just be amended like so:

pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type} for a in matches])

If you want the maps together with the apps in the output, you can just use the pandas method concat:

dependencies = pd.concat( [ pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type, 'url':f'{gis.url}/home/item.html?id={a.id}'} for a in app_list]), pd.DataFrame([{'title':m.title, 'id':m.id, 'type':m.type, 'url':f'{gis.url}/home/item.html?id={m.id}'} for m in matches]) ] )

But if the properties you want from the maps / apps are the same, you could also just merge the two lists together:

all_items = app_list + matches pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type, 'url':f'{gis.url}/home/item.html?id={a.id}'} for a in all_items])

Re: Possible to find out where feature layers are being used in web applications?

Big-Chill — Tue, 25 Oct 2022 21:52:14 GMT

Thanks so much for this. However, it does not catch items being utilized by a Web Experience. You will need to add this (apply code where necessary). Again, this is great stuff

# Pull list of all web experience apps webexp = gis.content.search('', item_type='Web Experience', max_items=-1) exp_list = [] # Check each web experience app for matches for wx in webexp: try: # Get the JSON as a string wxdata = str(wx.get_data()) criteria = [ wxdata.find(find_url) > -1, # Check if URL is directly referenced any([wxdata.find(i) > -1 for i in matches]) # Check if any matching maps are in app ] # If layer is referenced directly or indirectly, append app to list if any(criteria): exp_list.append(wx) # Some apps don't have data, so we'll just skip them if they throw a TypeError except: continue s1 = pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type} for a in app_list]) s2 = pd.DataFrame([{'title':b.title, 'id':b.id, 'type':b.type} for b in exp_list]) # Concatenate with original DataFrame pd.concat([s1, s2])

Re: Possible to find out where feature layers are being used in web applications?

HaThach1 — Tue, 22 Nov 2022 13:48:30 GMT

Hi, I implement this script above to include the maps together with the apps but got an error for m in matches section.

pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type, 'url':f'{gis.url}/home/item.html?id={a.id}'} for a in app_list]),
pd.DataFrame([{'title':m.title, 'id':m.id, 'type':m.type, 'url':f'{gis.url}/home/item.html?id={m.id}'} for m in matches])

Have any idea? thx

Re: Possible to find out where feature layers are being used in web applications?

OnnoKeller — Thu, 13 Apr 2023 12:36:21 GMT

This looks really great! I am new to python scripts though and when trying to run the tool I get an error message:

Any help on how to resolve this error would be much appreciated!

It should be noted that my maps and feature layers are in ArcGIS Online and not portal...

Re: Possible to find out where feature layers are being used in web applications?

jcarlson — Thu, 13 Apr 2023 12:57:17 GMT

The API should work the same regardless of where you're searching.

Try replacing line 1 with import arcgis

Then replace line 5 with gis = arcgis.gis.GIS('home')

Re: Possible to find out where feature layers are being used in web applications?

OnnoKeller — Mon, 17 Apr 2023 12:40:52 GMT

Thanks for the quick reply and help.

Your proposed changes made the script work for me!

Re: Possible to find out where feature layers are being used in web applications?

MDB_GIS — Fri, 05 May 2023 11:57:42 GMT

Hey,

Just curious, but how long should it take this tool to run? We have a fairly small AGOL org with less than 500 items but the tool has been running for about 15 minutes so far. This is my second attempt at running it.

I'm running it in the Python window and it appears to be running, but it is just bogged down somewhere.

Re: Possible to find out where feature layers are being used in web applications?

MDB_GIS — Fri, 05 May 2023 11:20:16 GMT

Katherine,

Thank you for sharing this! It is very helpful! How long on average does it take for it to search for Web Applications for you? We have a relatively small AGOL org (<500 items) and yet it still took the tool 3.5 hours to run. The Web maps part finishes in less than a minute. I'm not sure if I've misconfigured something or if there is sosmething weird about our organization that I'm not aware of but this is a real head scratcher.

Re: Possible to find out where feature layers are being used in web applications?

EmmaCurran18 — Tue, 18 Jul 2023 15:55:43 GMT

@jcarlson I am getting the same error as @HaThach1 . Have we found a solution to this? I haven't been able to get something to work.