Possible to find out where feature layers are being used in web applications?

[ZachRobinson_SantaClaraCounty]

I am wondering if it is possible to easily find out which web applications in our ArcGIS Online organization are using a specific feature layer.

For example, if I delete a layer from our organization, I want to make sure I know which web maps and applications the deletion will affect. This can be difficult and tedious to search manually when an organization has many web maps and applications. 

[Anonymous User]

Thank you to the contributors on this thread.  I've got the script running in ArcGIS Pro toolbox but after 5-6 minutes I get a message;

"TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond"

I am logged into ArcGIS Pro with my AGOL user account and I am testing the script using an item I "own" so it shouldn't be a security issue.

I've done some online research and a lot of people are suggesting it could be a firewall issue and I will need to alter my network settings.  I'd hate to get messing around with my firewall or network settings as our IT department would likely frown on that.

Anyone else encounter this issue? 

[lzk222]

I've not encountered this problem. I'd try using a VPN and running this script. If you're already using one, try running it with the VPN turned off.

[lzk222]

I've used this script to great effect over the years. Thanks to all who have contributed to it. I just wanted to post my slightly improved version that includes Web Experiences (kudos @Big-Chill), Web Maps, better exception handling, and hyperlinks the titles (in Jupyter Notebooks). 

Output should look something like this:

from arcgis.gis import GIS
import pandas as pd
from IPython.display import display, HTML

# Log in to portal; prompts for PW automatically
gis = GIS('your-portal-url', 'username')

# Layer ID to search for
find_id = 'a405b06ae8e24f94a2768a4581b79e73'

# Get the content based on ID
content_item = gis.content.get(find_id)

# Check if content is found
if content_item:
    find_url = content_item.url

    # URL snippet for web maps e.g. https://osit.maps.arcgis.com/apps/mapviewer/index.html?webmap=
    web_map_url_snippet = 'your-webmap-domain-path'

    # Pull list of all web maps in portal
    webmaps = gis.content.search('', item_type='Web Map', max_items=-1)

    # Return subset of map IDs which contain the service URL we're looking for
    matches = [m.id for m in webmaps if m.url and find_url in m.url]

    # Create empty list to populate with results
    app_list = []
    exp_list = []
    web_map_list = []  # New list for web maps

    # Pull list of all web apps in portal
    webapps = gis.content.search('', item_type='Application', max_items=-1)

    # Check each web app for matches
    for w in webapps:
        try:
            # Get the JSON as a string
            wdata = str(w.get_data())

            criteria = [
                wdata.find(find_url) > -1,  # Check if URL is directly referenced
                any([wdata.find(i) > -1 for i in matches])  # Check if any matching maps are in app
            ]

            # If layer is referenced directly, append app to list
            if any(criteria):
                app_list.append(w)

        # Some apps don't have data, so we'll just skip them if they throw a TypeError
        except:
            continue

    # Pull list of all web experience apps
    webexp = gis.content.search('', item_type='Web Experience', max_items=-1)

    # Check each web experience app for matches
    for wx in webexp:
        try:
            # Get the JSON as a string
            wxdata = str(wx.get_data())

            criteria = [
                wxdata.find(find_url) > -1,  # Check if URL is directly referenced
                any([wxdata.find(i) > -1 for i in matches])  # Check if any matching maps are in app
            ]

            # If layer is referenced directly, append app to list
            if any(criteria):
                exp_list.append(wx)

        # Some apps don't have data, so we'll just skip them if they throw a TypeError
        except:
            continue

    # Check each web map for matches
    for wm in webmaps:
        try:
            # Get the JSON as a string
            wmdata = str(wm.get_data())

            criteria = [
                wmdata.find(find_url) > -1,  # Check if URL is directly referenced
                any([wmdata.find(i) > -1 for i in matches])  # Check if any matching maps are in app
            ]

            # If layer is referenced directly, append web map to list
            if any(criteria):
                # Construct the complete web map URL
                wm_url = f'{web_map_url_snippet}{wm.id}'
                web_map_list.append({'Title': f'<a href="{wm_url}" target="_blank">{wm.title}</a>', 'ID': wm.id, 'Type': 'Web Map'})

        # Some web maps don't have data, so we'll just skip them if they throw a TypeError
        except:
            continue

    # Create DataFrames
    app_df = pd.DataFrame([
        {'Title': f'<a href="{a.url}" target="_blank">{a.title}</a>', 'ID': a.id, 'Type': a.type}
        for a in app_list
    ])

    exp_df = pd.DataFrame([
        {'Title': f'<a href="{b.url}" target="_blank">{b.title}</a>', 'ID': b.id, 'Type': b.type}
        for b in exp_list
    ])

    web_map_df = pd.DataFrame(web_map_list)

    # Display the HTML content
    display(HTML(web_map_df.to_html(escape=False)))
    display(HTML(app_df.to_html(escape=False)))
    display(HTML(exp_df.to_html(escape=False)))

else:
    print(f"Content with ID '{find_id}' not found.")

[Katie_Clark]

Wow, thank you for your contributions, @lzk222 !! 🙂

[JoshuaClanton]

Wow thank you for this! Great script

[TobyStewart2]

That looks amazing @lzk222 

I am trying to run it as a script in an ArcPro toolbox similar to how @Katie_Clark had set it up in her previous post. It runs without errors but there is no output at the end. Im not very good with scripting so am unsure what Im missing.

Thanks again

[lzk222]

Hi @TobyStewart2 , I am not that familiar with running scripts from the Toolbox. Because the script will return hyperlinks, it is probably best to run it in Jupyter Notebooks since this is browser based. Here's a video on how to get that started: Installing Jupyter Notebooks.

[MichaelRobb2]

Hi,

This sounds like an ideal solution for us as we have a tidy up of our AGOL resources.  Having forgotten about so much work we've done in the past, it is so difficult to ascertain what web maps, applications etc reference what shapefiles, feature layers and so on.  

Can I ask where the best place to actually run this Python script is i.e Python Notebook within ArcPro?  What form should the 'your-portal-url' be in, should it be "https://<organisation>.maps.arcgis/com/" ?

I've tried running it in a Notebook and after a fair while just get a connection error below.  It is possibly a firewall issue, it has been known!

Many thanks,

Mike

[lzk222]

Hi @MichaelRobb2 , 

The best place to run the script is in Jupyter Notebooks. 

The portal-url should look something like this:

https://osit.maps.arcgis.com/apps/mapviewer/index.html?webmap=

You can find it from any webmap in your organization.

Please try running it from here. If you're still getting errors let me know, though what you've posted appears to be firewall related.

[MichaelRobb2]

Many thanks.  I ran the Python script in Jupyter notebooks from home and it worked, so must be an organisational firewall thing indeed.  It's not the first time I've had problems!