Possible to find out where feature layers are being used in web applications?

12749
37
Jump to solution
02-09-2022 08:12 AM
ZachRobinson_SantaClaraCounty
New Contributor II

I am wondering if it is possible to easily find out which web applications in our ArcGIS Online organization are using a specific feature layer.

For example, if I delete a layer from our organization, I want to make sure I know which web maps and applications the deletion will affect. This can be difficult and tedious to search manually when an organization has many web maps and applications. 

1 Solution

Accepted Solutions
jcarlson
MVP Esteemed Contributor

@Katie_Clark, you guess right: you can do this with the Python API!

The text version:

  1. For a given layer's URL
  2. Find all maps that reference that layer
  3. Fina all apps that reference those maps or the layers directly

The tricky part is # 3. An app can reference a layer directly in one of its widgets, like the Search widget. Other apps, like Experience Builder and Dashboards, can also reference layers directly without going through a map. Further complicating this is that certain apps will make direct references to a layer using their ID, rather than their URL.

To make sure our search is comprehensive we can't just search for the apps that reference the maps; we'll have to search for the layers directly as well. It gets a bit convoluted, but it works!

The code:

 

from arcgis.gis import GIS
import pandas as pd

# Log in to portal; prompts for PW automatically
gis = GIS('your-portal-url', 'username')

# Layer ID to search for and its URL
find_id = 'a405b06ae8e24f94a2768a4581b79e73'
find_url = gis.content.get(find_id).url

# Pull list of all web maps in portal
webmaps = gis.content.search('', item_type='Web Map', max_items=-1)

# Return subset of map IDs which contain the service URL we're looking for
matches = [m.id for m in webmaps if str(m.get_data()).find(find_url) > -1]

# Pull list of all web apps in portal
webapps = gis.content.search('', item_type='Application', max_items=-1)

# Create empty list to populate with results
app_list = []

# Check each web app for matches
for w in webapps:
    
    try:
        # Get the JSON as a string
        wdata = str(w.get_data())

        criteria = [
            wdata.find(find_url) > -1, # Check if URL is directly referenced
            any([wdata.find(i) > -1 for i in matches]) # Check if any matching maps are in app
        ]

        # If layer is referenced directly or indirectly, append app to list
        if any(criteria):
            app_list.append(w)
    
    # Some apps don't have data, so we'll just skip them if they throw a TypeError
    except:
        continue

pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type} for a in app_list])

 

 

The output DataFrame will be any apps that reference your layer:

jcarlson_0-1644429780295.png

You can easily amend the final line of the expression to include additional details about the items.

- Josh Carlson
Kendall County GIS

View solution in original post

37 Replies
Katie_Clark
MVP Regular Contributor

I would love to know if this is possible as well! As far as I know there isn't an "easy" solution, but maybe something can be done with the Python API?

Best,
Katie


“The goal is not simply to ‘work hard, play hard.’ The goal is to make our work and our play indistinguishable.”
- Simon Sinek
0 Kudos
MichaelVolz
Esteemed Contributor

Have you considered looking at 3rd party vendor solutions designed specifically for this issue?  I would look at GeoJobe especially if this is a time consuming task for GIS personnel at your org as AGOL and Portal mature and there is a good deal of content.

0 Kudos
Katie_Clark
MVP Regular Contributor

I think I've looked into GeoJobe before....correct me if I'm wrong, but it is not a free service, correct? It would be difficult to justify the expense to higher management (non-GIS) who would have to approve something like that, unfortunately. 

Best,
Katie


“The goal is not simply to ‘work hard, play hard.’ The goal is to make our work and our play indistinguishable.”
- Simon Sinek
0 Kudos
jcarlson
MVP Esteemed Contributor

@Katie_Clark, you guess right: you can do this with the Python API!

The text version:

  1. For a given layer's URL
  2. Find all maps that reference that layer
  3. Fina all apps that reference those maps or the layers directly

The tricky part is # 3. An app can reference a layer directly in one of its widgets, like the Search widget. Other apps, like Experience Builder and Dashboards, can also reference layers directly without going through a map. Further complicating this is that certain apps will make direct references to a layer using their ID, rather than their URL.

To make sure our search is comprehensive we can't just search for the apps that reference the maps; we'll have to search for the layers directly as well. It gets a bit convoluted, but it works!

The code:

 

from arcgis.gis import GIS
import pandas as pd

# Log in to portal; prompts for PW automatically
gis = GIS('your-portal-url', 'username')

# Layer ID to search for and its URL
find_id = 'a405b06ae8e24f94a2768a4581b79e73'
find_url = gis.content.get(find_id).url

# Pull list of all web maps in portal
webmaps = gis.content.search('', item_type='Web Map', max_items=-1)

# Return subset of map IDs which contain the service URL we're looking for
matches = [m.id for m in webmaps if str(m.get_data()).find(find_url) > -1]

# Pull list of all web apps in portal
webapps = gis.content.search('', item_type='Application', max_items=-1)

# Create empty list to populate with results
app_list = []

# Check each web app for matches
for w in webapps:
    
    try:
        # Get the JSON as a string
        wdata = str(w.get_data())

        criteria = [
            wdata.find(find_url) > -1, # Check if URL is directly referenced
            any([wdata.find(i) > -1 for i in matches]) # Check if any matching maps are in app
        ]

        # If layer is referenced directly or indirectly, append app to list
        if any(criteria):
            app_list.append(w)
    
    # Some apps don't have data, so we'll just skip them if they throw a TypeError
    except:
        continue

pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type} for a in app_list])

 

 

The output DataFrame will be any apps that reference your layer:

jcarlson_0-1644429780295.png

You can easily amend the final line of the expression to include additional details about the items.

- Josh Carlson
Kendall County GIS
Katie_Clark
MVP Regular Contributor

Thank you, Josh! I'll definitely look into this. 

Best,
Katie


“The goal is not simply to ‘work hard, play hard.’ The goal is to make our work and our play indistinguishable.”
- Simon Sinek
0 Kudos
ZachRobinson_SantaClaraCounty
New Contributor II

Fantastic, thank you so much for your thorough response!

0 Kudos
NickShannon2
New Contributor III

This is great! Thank you. 

Can the script be modified to include 'web maps' in the output DataFrame? 

 

0 Kudos
jcarlson
MVP Esteemed Contributor

Of course! Since our list of maps is already defined earlier, that last line could just be amended like so:

pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type} for a in matches])

If you want the maps together with the apps in the output, you can just use the pandas method concat:

dependencies = pd.concat(
    [
        pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type, 'url':f'{gis.url}/home/item.html?id={a.id}'} for a in app_list]),
        pd.DataFrame([{'title':m.title, 'id':m.id, 'type':m.type, 'url':f'{gis.url}/home/item.html?id={m.id}'} for m in matches])
    ]
)

But if the properties you want from the maps / apps are the same, you could also just merge the two lists together:

all_items = app_list + matches

pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type, 'url':f'{gis.url}/home/item.html?id={a.id}'} for a in all_items])
- Josh Carlson
Kendall County GIS
0 Kudos
HaThach1
New Contributor II

Hi, I implement this script above to include the maps together with the apps but got an error for m in matches section.

pd.DataFrame([{'title':a.title, 'id':a.id, 'type':a.type, 'url':f'{gis.url}/home/item.html?id={a.id}'} for a in app_list]),
pd.DataFrame([{'title':m.title, 'id':m.id, 'type':m.type, 'url':f'{gis.url}/home/item.html?id={m.id}'} for m in matches])

HaThach1_0-1669124670796.png

Have any idea? thx

 

 

0 Kudos