Possible to find out where feature layers are being used in web applications?

32486
51
Jump to solution
02-09-2022 08:12 AM
ZachRobinson_SantaClaraCounty
Emerging Contributor

I am wondering if it is possible to easily find out which web applications in our ArcGIS Online organization are using a specific feature layer.

For example, if I delete a layer from our organization, I want to make sure I know which web maps and applications the deletion will affect. This can be difficult and tedious to search manually when an organization has many web maps and applications. 

51 Replies
MichaelRobb2
Occasional Contributor

Many thanks.  I ran the Python script in Jupyter notebooks from home and it worked, so must be an organisational firewall thing indeed.  It's not the first time I've had problems!  

0 Kudos
Jesse_Albers
Emerging Contributor

@lzk222Thank you for your post. I have noticed a discrepancy when searching for a layer that should appear in a web application. Despite the layer being present in the web map when using this Python tool, it does not appear in the web application that uses the same web map. However, the same layer appears in another web map and web application that uses that web map. Any clue why this would happen? I can post a screenshot if that would help.

0 Kudos
lzk222
by
Regular Contributor

Hi @Jesse_Albers , sorry for the slow response.  A screenshot would be helpful. 

Jesse_Albers
Emerging Contributor

This was a user error. Sorry about that, @lzk222.

0 Kudos
JustinMaysHMIS
Emerging Contributor

Hello All,

I want to thank everyone for the code on this forum as this is how I started my journey. I have taken the original code and made some changes that fix some of the issues people were having. The original script was working but it was not returning some of the web apps in the list. Below is a list of changes to the code. Double check indents if not working. 

  1. Ensures get_data() is not None.
  2. Increases max_items to 1000.
  3. Normalizes find_url for better matching (removes trailing slashes, converts to lowercase).
  4. Checks for URLs in web apps instead of just IDs.
  5. Improves exception handling (specific errors and warnings).
  6. Deeply searches JSON structures (better that just .find()).
  7. Finds both direct references and indirect ones through web maps.
  8. Handles errors gracefully without skipping valid results.
  9. Uses json.dumps() for more flexible searching.
  10. Improves search range by increasing max_items.

 

Below is the code:

import arcgis
import pandas as pd
import arcpy
import json


# Log in to portal; 'home' uses the credentials used to login within Pro
gis = arcgis.gis.GIS('home')


# Set up input parameters
find_id = arcpy.GetParameterAsText(0)
search_type = arcpy.GetParameterAsText(1)


# Get the URL of the service and normalize it
find_url = gis.content.get(find_id).url.rstrip('/').lower()
def deep_search(data, search_term):
       """Recursively search for a term in JSON data (dictionaries/lists)."""
       if isinstance(data, dict):
            return any(deep_search(value, search_term) for value in data.values())
       elif isinstance(data, list):
            return any(deep_search(item, search_term) for item in data)
       elif isinstance(data, str):
            return search_term in data.lower()
       return False
if search_type == 'Web Map':
     arcpy.AddMessage("Searching for Web Maps... This could take a few minutes.")

     # Retrieve all web maps (increase max_items)
     webmaps = gis.content.search('', item_type='Web Map', max_items=1000)

     # Find web maps that contain the service URL
     map_list = []
     for w in webmaps:
           try:
                wdata = w.get_data()
                if wdata and deep_search(wdata, find_url):
                       map_list.append(w)
           except Exception as e:
                    arcpy.AddWarning(f"Error processing Web Map {w.id}: {str(e)}")
     output = pd.DataFrame([{'title': m.title, 'id': m.id, 'type': m.type} for m in map_list])
     arcpy.AddMessage(f"OUTPUT TABLE: \n\n {output}")
elif search_type == 'Web Application':
     arcpy.AddMessage("Searching for Web Applications... This could take a few minutes.")

     # Retrieve all web apps (use advanced search for larger result sets)
     webapps = gis.content.search('', item_type='Application', max_items=1000)

     # Find web maps that reference the service URL
     webmaps = gis.content.search('', item_type='Web Map', max_items=1000)
     matching_webmaps = [w.id for w in webmaps if w.get_data() and deep_search(w.get_data(), find_url)]

     # Find web apps that contain either the service URL OR a matching web map
     app_list = []
     for w in webapps:
           try:
                wdata = w.get_data()
                if wdata:
                    wdata_str = json.dumps(wdata).lower() # Convert JSON to string for searching
                    criteria = [
                          find_url in wdata_str, # Direct service URL reference
                          any(m in wdata_str for m in matching_webmaps) # Indirect reference through web maps
                     ]
                     if any(criteria):
                         app_list.append(w)
             except Exception as e:
                      arcpy.AddWarning(f"Error processing Web Application {w.id}: {str(e)}")
    output = pd.DataFrame([{'title': a.title, 'id': a.id, 'type': a.type} for a in app_list])
    arcpy.AddMessage(f"OUTPUT TABLE: \n\n {output}")

SarahWigley
Occasional Contributor

Hi, I am really keen to get this code working. I am a complete novice with python, but I have tried the various methods mentioned in this thread. I.e. I've run the original script (jcarlson 02/09/22) from a notebook within ArcGIS Pro; the modified script that runs as an Arc tool (Katie_Clark 03/16/22); the modified script run from Jupyter Notebooks (lzk222 11/09/23); another modified script run from Notebooks (WesleyMills 04/10/24); and your code which I presumed was designed to be run from a tool in ArcGIS Pro (JustinMaysHMIS 3wks ago). However, I'm having issues with all. For your method, I'm having issues with the indents. I know you said check these, but I'm not sure how to identify or correct them?

I've ran the original method in stages and it seems an issue occurs at stage 4 - returning a subset of map IDs which contain the service URL we're looking for. This comes out empty, when I know there are maps present that contain the specified layer.

I can see it has pulled the list of web maps in the portal correctly (by printing 'webmaps'). And I can see that it has found the correct url for the layer (by printing 'find_url'). I noticed that clicking the layer url hyperlink in Notebooks opened the browser but gave an error; however copying and pasting the url into the browser opened the layer page successfully. Not sure if that's normal?

The format of the organisational portal url I used was 'https://xyz.maps.arcgis.com'. However, for your code I did not edit anything - presumed it would pick up organisational url from being logged in to ArcGIS Pro.

I am using ArcGIS Pro 3.3.1. I am running Windows 11.

Any advice hugely appreciated! @JustinMaysHMIS @jcarlson 

0 Kudos
JustinMaysHMIS
Emerging Contributor

Hello Sarah,

One of the issues I get from time to time when the script returns no results is that ArcGIS Pro is set to the wrong portal. Double check to make sure Pro is set to the correct portal you are looking at. Let me know if this helps. 

Justin

0 Kudos
SarahWigley
Occasional Contributor

Many thanks for your quick reply. The format I used for the portal (in the original code) was this: 'https://xyz.maps.arcgis.com'. Is that correct? I tried with a / at the end, but that seemed to make no difference.  However, I shouldn't need to enter the portal url anywhere with your code, should I? I used your code within a new tool, and presumed it took the portal automatically from my ArcGIS Pro log-in?

0 Kudos
JustinMaysHMIS
Emerging Contributor

Hello Sarah,

Sorry for the late response. We are behind a firewall, and we connect to Enterprise server portals. When I run this script, I need to make sure my active portal (within ArcGIS Pro up at the top of the Pro window) is set to where I want to search. For our servers, the portal address looks something like this https://<server name>/arcgis/. Hope that helps.

If that does not help, I have been working on another tool that will search based on the service and looks at all web map/apps/experience builder locations where that service is being used. I do want to point out that sometimes when running scripts over and over again in Pro, it gets a little wonkey and you might need to restart Pro to clear everything out.

Good luck and let me know if you get it working.

Justin

0 Kudos
ErikAIversen
Occasional Contributor

Hi!

Made my own script that can search for "Web Maps", "Web Application", "Experience Builder", "Dashboard" or "All". 

1. Open notepad, pase the code below, choose "All files" save as your desired filename and location. Remember to add .py at the end of the filename

filename.png

2. Create a new toolbox in ArcGIS Pro and remember to add the parameters.

parameters.png

3. Link the toolbox to the code you saved in step 1.
script.png

import arcpy
from arcgis.gis import GIS
import time

# Connect to ArcGIS Pro portal session
gis = GIS("home")

# Parameters
find_id = arcpy.GetParameterAsText(0)
search_type = arcpy.GetParameterAsText(1)  # Web Map, Web Application, Dashboard, Experience Builder, All

start_time = time.time()

# Get feature service and its URL
content_item = gis.content.get(find_id)
if not content_item or not hasattr(content_item, "url") or not content_item.url:
    arcpy.AddError(" Could not retrieve a valid URL from the provided ID.")
    raise SystemExit

find_url = content_item.url
arcpy.AddMessage(f"🔍 Searching for usage of: {find_url}")

# 1. Pull all Web Maps and collect those referencing the feature service
all_webmaps = gis.content.search('', item_type='Web Map', max_items=1000)
map_ids = []

for wm in all_webmaps:
    try:
        if find_url in str(wm.get_data()):
            map_ids.append(wm.id)
            arcpy.AddMessage(f"✔️ Found referencing Web Map: {wm.title} ({wm.id})")
    except:
        continue

# 2. Load other content types conditionally
webmaps = all_webmaps if search_type in ('Web Map', 'All') else []
webapps = gis.content.search('', item_type='Application', max_items=1000) if search_type in ('Web Application', 'All') else []
dashboards = gis.content.search('', item_type='Dashboard', max_items=1000) if search_type in ('Dashboard', 'All') else []
webexp = gis.content.search('', item_type='Web Experience', max_items=1000) if search_type in ('Experience Builder', 'All') else []

# 3. Recursive matching function for nested itemId references
def references_map_id(item_data, map_ids):
    def search_dict(d):
        for key, value in d.items():
            if key == "itemId" and value in map_ids:
                return True
            elif isinstance(value, dict):
                if search_dict(value):
                    return True
            elif isinstance(value, list):
                for item in value:
                    if isinstance(item, dict) and search_dict(item):
                        return True
        return False
    return search_dict(item_data)

# 4. Results
web_map_list, app_list, dash_list, exp_list = [], [], [], []

# --- Web Maps ---
if search_type in ('Web Map', 'All'):
    for i, wm in enumerate(webmaps):
        arcpy.AddMessage(f":hourglass_not_done: Checking Web Map {i+1}/{len(webmaps)}: {wm.title}")
        try:
            data = str(wm.get_data())
            if find_url in data:
                web_map_list.append(wm)
        except:
            continue
    arcpy.AddMessage(f":package: Checked {len(webmaps)} web maps.")
    if web_map_list:
        arcpy.AddMessage(f" Found {len(web_map_list)} Web Map(s):")
        for item in web_map_list:
            arcpy.AddMessage(f"🔗 {item.title} ({item.id}) [{item.type}]")
    else:
        arcpy.AddMessage(" No Web Maps found.")

# --- Web Applications ---
if search_type in ('Web Application', 'All'):
    for i, wa in enumerate(webapps):
        arcpy.AddMessage(f":hourglass_not_done: Checking Web App {i+1}/{len(webapps)}: {wa.title}")
        try:
            data = str(wa.get_data())
            if find_url in data or any(mid in data for mid in map_ids):
                app_list.append(wa)
        except:
            continue
    arcpy.AddMessage(f":package: Checked {len(webapps)} web apps.")
    if app_list:
        arcpy.AddMessage(f" Found {len(app_list)} Web App(s):")
        for item in app_list:
            arcpy.AddMessage(f"🔗 {item.title} ({item.id}) [{item.type}]")
    else:
        arcpy.AddMessage(" No Web Applications found.")

# --- Dashboards ---
if search_type in ('Dashboard', 'All'):
    if dashboards:
        for i, db in enumerate(dashboards):
            arcpy.AddMessage(f":hourglass_not_done: Checking Dashboard {i+1}/{len(dashboards)}: {db.title}")
            try:
                raw = db.get_data()
                if references_map_id(raw, map_ids):
                    dash_list.append(db)
                    arcpy.AddMessage(f":collision: MATCH FOUND in Dashboard: {db.title}")
            except:
                arcpy.AddMessage(f":warning: Failed to parse Dashboard: {db.title}")
                continue
        arcpy.AddMessage(f":package: Checked {len(dashboards)} dashboards.")
        if dash_list:
            arcpy.AddMessage(f" Found {len(dash_list)} Dashboard(s):")
            for item in dash_list:
                arcpy.AddMessage(f"🔗 {item.title} ({item.id}) [{item.type}]")
        else:
            arcpy.AddMessage(" No Dashboards found.")

# --- Experience Builders ---
if search_type in ('Experience Builder', 'All'):
    if webexp:
        for i, we in enumerate(webexp):
            arcpy.AddMessage(f":hourglass_not_done: Checking Experience Builder {i+1}/{len(webexp)}: {we.title}")
            try:
                raw = we.get_data()
                if references_map_id(raw, map_ids):
                    exp_list.append(we)
                    arcpy.AddMessage(f":collision: MATCH FOUND in Experience Builder: {we.title}")
            except:
                arcpy.AddMessage(f":warning: Failed to parse Experience Builder: {we.title}")
                continue
        arcpy.AddMessage(f":package: Checked {len(webexp)} experience builders.")
        if exp_list:
            arcpy.AddMessage(f" Found {len(exp_list)} Experience Builder(s):")
            for item in exp_list:
                arcpy.AddMessage(f"🔗 {item.title} ({item.id}) [{item.type}]")
        else:
            arcpy.AddMessage(" No Experience Builders found.")

# --- Final Summary
arcpy.AddMessage("\n:bar_chart: ==== FINAL SUMMARY ====")
if search_type in ('Web Map', 'All'):
    arcpy.AddMessage(f"Web Maps         : {len(web_map_list)}")
if search_type in ('Web Application', 'All'):
    arcpy.AddMessage(f"Web Applications : {len(app_list)}")
if search_type in ('Dashboard', 'All'):
    arcpy.AddMessage(f"Dashboards       : {len(dash_list)}")
if search_type in ('Experience Builder', 'All'):
    arcpy.AddMessage(f"Experience Builds: {len(exp_list)}")

total = len(web_map_list) + len(app_list) + len(dash_list) + len(exp_list)
arcpy.AddMessage(f"\n🧮 TOTAL MATCHES : {total}")
arcpy.AddMessage(f":stopwatch:️ Runtime        : {round(time.time() - start_time, 2)} seconds")

 

0 Kudos