Uncovering related content in your stories and beyond

[Warren_Davison]

As authors, we find ourselves crafting stories from a growing collection of content within ArcGIS Online. As we weave together layers, maps, and apps into stories, it can become a challenge to decipher which layer appeared in that map, which map was used in that story, and which apps were embedded in that story.

Our team has been working on a Python script (available here) to tackle this problem. This script helps reveal the relationships between items within your organization, enabling you to better understand how layers, maps, apps, and stories are connected.

Revealing connections

This tool helps you understand how items, such as stories, dashboards, maps, layers, and surveys, are connected across your ArcGIS Online content, allowing you to:

• Simplify content management: Quickly inspect the owner, sharing level, and item type of content across your organization.
• Declutter efficiently: Be confident in knowing which items are referenced by others and which items can be safely archived or removed without breaking anything.
• Visualize relationships: Visually inspect how your content is connected and identify dependencies.

How it works

This notebook uses the ArcGIS API for Python to query the items in your organization. As it explores the results, the script sifts through each item's JSON and identifies the item ids referenced within. It then builds a list of these relationships between items and constructs a graph that can be used to visualize your content as a network of connected items.

This simplified example demonstrates how the connection between items can be visualized in a graph network.

Running the tool

To get started and run this script to discover content in your organization, you need to configure just a few parameters.

1. Download and setup the script: Download the Python script from the link below.

Due to the nature of the recursive search, it could time out when run from an ArcGIS Notebook. Therefore, it is recommended that the script be downloaded and run locally.

2. If running on a local machine without an ArcGIS Pro installation, install the ArcGIS API for Python

• This script uses the ArcGIS API for Python, see these install and setup instructions.

3. Update the script arguments

• The descriptions below provide a more in-depth breakdown of each argument.
• As it may take some time to run, we recommend testing it on a small subset of your content. Either filter the QUERY_START_DATE to a date in the past 30 days or set the TEST_MAX_PROCESSED_ITEMS to a smaller value, such as 50.

# PARAMETERS
# Portal to investigate
PORTAL = "https://www.arcgis.com"
# Username to log in to the portal
USERNAME = "<USERNAME>"
# Password associated with the portal
PASSWORD = "<PASSWORD>"
# Output CSV file location
OUTPUT_FILE = "related_items.csv"
# Location for CSV where missed items and error messages will be added
OUTPUT_MISS_FILE = "missed_items.csv"
# Location of graphing html file. Will only output when CREATE_GRAPH_HTML is True
GRAPH_FILE = "graph.html"
# Should data be graphed at the end of the process
CREATE_GRAPH_HTML = True
# Unix timestamp (in milliseconds) marking the starting point for the search algorithm. Default is 00:00:00, January 1st, 2016
QUERY_START_DATE = 1451624400000
# The name of the account to be analyzed. If left blank the script will use the logged in user.
ACCOUNT = "<USERNAME>"
# Max number of items to analyze. FOR dev/testing purposes. Modify this value to an integer if you want to test this script with a shortened run
TEST_MAX_PROCESSED_ITEMS = None
# Max number of items to find.. FOR dev/testing purposes.  Modify this value to an integer if you want to test this script with a shortened run
TEST_MAX_FOUND_ITEMS = None

• Portal: This argument determines the organization that the tool will access when searching for content. Modify this argument to suit your organization. 
  • Enterprise: use `https://portal.domain.com`
  • ArcGIS Online: use `https://arcgis.com`
• Username and password: Provide a username and password that will be used to access the organization.
• Output files: Define the name for the two CSV report files.
  • OUTPUT_FILE will contain a list of items found by the tool and their related items.
  • OUTPUT_MISS_FILE will contain a list of instances where an item could not be identified and a potential reason.
• Create graph file: Configure whether the tool will also create a graph file with the results (TRUE or FALSE)
  • If not running in a Notebook, change line `675` in the script to 'notebook=False`.

net = Network(notebook=False, select_menu=True, filter_menu=True)

• Graph file: Define a name for the output graph file that can be created alongside the CSV reports.
• Query start date: Provide a for the script. Content created before this date will be excluded from the query.
• Maximums:
  • TEST_MAX_PROCESSED_ITEMS: This argument defines the maximum number of items that will be processed in the script’s main loop.
  • TEST_MAX_FOUND_ITEMS: This argument defines the maximum number of items fetched per query.

4. Run the script

• Once you've reviewed the results from a test run, you can expand the scope of the tool to a greater time frame and maximum number of items and the script will query the content within your organization that satisfies the arguments supplied in step 3.

5. Exploring the results: The tool creates several outputs that can be further explored.

• The script will create a ‘related_items.csv’ listing each item in the query and various attributes, including item id, owner, title, sharing, type, and related items. This can be turned into a pivot table (see the example below) to help take stock of items and confirm their owner or sharing level.

 This pivot table demonstrates how the output 'related_items.csv' can be used to inventory items like stories and web maps, but also the content within those items or related to those items.

• The script also creates a ‘missed_items.csv’ report. This report contains the items that the tool could not process, along with a brief description of each item and the error encountered when it was processed.

• If the script's graphing portion is enabled, the final output includes an HTML file that visualizes the item relationships in a graph. The graph’s pop-ups can be used to quickly link to an item.

What's next?

We hope this Python notebook helps you better manage and understand the connections within your ArcGIS Online content. We’d love to hear how you’re using this tool and any insights you’ve gained from it. Feel free to share your experiences, suggestions, or any creative ways you’ve adapted it to fit your workflow!