Select to view content in your preferred language

Trying to list all Feature Services in Portal and what Maps use those specific services

3181
7
12-16-2022 12:08 PM
phess_luckstone
Regular Contributor

So I am trying to create an excel document that list all of my feature services / feature collections  and what specific maps they are in, so far I have this, but when it runs it lists all my services in Column A, but then Column B is blank, any idea what could be happening?  I am very new to python.  Any help would be appreciated.

import arcgis
import openpyxl

# Set the URL of your ArcGIS Online portal
portal_url = 'https://yourportal.arcgis.com'

# Set your ArcGIS Online username and password
username = 'your_username'
password = 'your_password'

# Connect to the portal
gis = arcgis.GIS(portal_url, username, password)

# Search for all feature services and feature collections in the portal
items = gis.content.search(query='type: "Feature Service" OR type: "Feature Collection"', max_items=5000)

# Create a new Excel workbook
workbook = openpyxl.Workbook()
worksheet = workbook.active

# Add a header row to the worksheet
worksheet.append(['Feature Service/Collection', 'Maps Using Feature'])

# Iterate through the feature services and feature collections
for item in items:
    # Search for maps in the portal that use the feature service or feature collection as a layer
    maps = gis.content.search(query=f'type: "Web Map*" AND layers: {item.id}', max_items=5000)
    
    # Create a list of the map names
    map_names = [map.title for map in maps]
    
    # Add a row to the worksheet with the feature service/collection name and the list of map names
    worksheet.append([item.title, *map_names])

# Save the workbook to a file
workbook.save('feature_services_maps.xlsx')

 

Tags (3)
0 Kudos
7 Replies
Kara_Shindle
Frequent Contributor

When I tested this, I was getting an empty empty list when I printed the map_names list.  Are you sure it is populating?  I would try building in some print statements to test your results.

0 Kudos
phess_luckstone
Regular Contributor

This is what the output looks like for me, I am using Enterprise 10.8.1, it seems like it is picking up more than just services but the built in esri tools as well.

phess_luckstone_1-1671224359589.png

 

0 Kudos
Kara_Shindle
Frequent Contributor

your map_names variable is trying to populate a list with map.title, and it errors out telling me that map has no attribute 'title'

0 Kudos
JoshuaSharp-Heward
Frequent Contributor

As I think it's more reliable that way.

 I'm also not 100% sure you can query a webmap on its layers using the gis.content.search function. I wrote some code to search webmaps for all services in a portal/agol instance to find services that were unused, which you can find here https://github.com/joshsharpheward/gis-administration/blob/master/find_unused_services.py which I think you could modify to do what you need or at least use for inspiration!

phess_luckstone
Regular Contributor

So modifying a little of what you your code is, I was able to get the output of every map and every service inside of the map using this:

from arcgis.gis import GIS
from arcgis.mapping import WebMap
import arcpy
import pandas as pd

def main():
    # logs into active portal in ArcGIS Pro
    gis = GIS('pro')

    arcpy.AddMessage("Logged into {} as {}".format(arcpy.GetActivePortalURL(), gis.properties['user']['username']))

    # creates list of items of all map image, feature, vector tile and image services (up to 10000 of each) in active portal
    services = (gis.content.search(query="", item_type="Map Service", max_items=10000) +
                gis.content.search(query="", item_type="Feature Service", max_items=10000) +
                gis.content.search(query="", item_type="Vector Tile Service", max_items=10000) +
                gis.content.search(query="", item_type="Image Service", max_items=10000))

    arcpy.AddMessage('Searching webmaps in {}'.format(arcpy.GetActivePortalURL()))

    # creates list of items of all webmaps in active portal
    web_maps = gis.content.search(query="", item_type="Web Map", max_items = 10000)

    # Create an empty data frame to store the data
    df = pd.DataFrame(columns=['Web Map', 'Service URL'])

    # loops through list of webmap items
    for item in web_maps:
        # creates a WebMap object from input webmap item
        web_map = WebMap(item)
        # accesses basemap layer(s) in WebMap object
        basemaps = web_map.basemap['baseMapLayers']
        # accesses layers in WebMap object
        layers = web_map.layers

        # loops through basemap layers
        for bm in basemaps:
            # tests whether the bm layer has a styleUrl(VTS) or url (everything else)
            if 'styleUrl'in bm.keys():
                for service in services:
                    if service.url in bm['styleUrl']:
                        services.remove(service)
            elif 'url' in bm.keys():
                for service in services:
                    if service.url in bm['url']:
                        services.remove(service)

        # loops through layers
        for layer in layers:
            # Add a new row to the data frame for each service used in the web map
            if hasattr(layer, 'styleUrl'):
                df.loc[len(df)] = [item.title, layer.styleUrl]
            elif hasattr(layer, 'url'):
                df.loc[len(df)] = [item.title, layer.url]

    # Save the data frame to an Excel file
    df.to_excel('web_maps.xlsx', index=False)

if __name__ == "__main__":
    main()
LindsayRaabe_FPCWA
MVP Regular Contributor

Very useful script. I had a go at modifying it further to return the layers item ID and spatial reference as well, but failed. Anyone have any idea's on how to do that?

Lindsay Raabe
GIS Officer
Forest Products Commission WA
0 Kudos
LindsayRaabe_FPCWA
MVP Regular Contributor

Note to other tinkerers. I tweaked the code as per below to use print statements instead of arcpy.AddMessage, but essentially is just the same, exporting an .xlsx with a list of webmaps and their feature services (as feature service URL's only). I then exported an item report from ArcGIS Online and used Vlookup using the service URL to get more information on each layer. 

from arcgis.gis import GIS
from arcgis.mapping import WebMap
import arcpy
import pandas as pd

def main():
    # logs into active portal in ArcGIS Pro
    gis = GIS('pro')

    print("Logged into {} as {}".format(arcpy.GetActivePortalURL(), gis.properties['user']['username']))

    # creates list of items of all map image, feature, vector tile and image services (up to 10000 of each) in active portal
    services = (gis.content.search(query="", item_type="Map Service", max_items=10000) +
                gis.content.search(query="", item_type="Feature Service", max_items=10000) +
                gis.content.search(query="", item_type="Vector Tile Service", max_items=10000) +
                gis.content.search(query="", item_type="Image Service", max_items=10000))

    print('Searching webmaps in {}'.format(arcpy.GetActivePortalURL()))

    # creates list of items of all webmaps in active portal
    web_maps = gis.content.search(query="", item_type="Web Map", max_items = 10000)

    # Create an empty data frame to store the data
    df = pd.DataFrame(columns=['Web Map', 'Service URL'])

    # loops through list of webmap items
    for item in web_maps:
        # creates a WebMap object from input webmap item
        web_map = WebMap(item)
        # accesses basemap layer(s) in WebMap object
        basemaps = web_map.basemap['baseMapLayers']
        # accesses layers in WebMap object
        layers = web_map.layers

        # loops through basemap layers
        for bm in basemaps:
            # tests whether the bm layer has a styleUrl(VTS) or url (everything else)
            if 'styleUrl'in bm.keys():
                for service in services:
                    if service.url in bm['styleUrl']:
                        services.remove(service)
            elif 'url' in bm.keys():
                for service in services:
                    if service.url in bm['url']:
                        services.remove(service)

        # loops through layers
        for layer in layers:
            # Add a new row to the data frame for each service used in the web map
            if hasattr(layer, 'styleUrl'):
                df.loc[len(df)] = [item.title, layer.styleUrl]
            elif hasattr(layer, 'url'):
                df.loc[len(df)] = [item.title, layer.url]

    # Save the data frame to an Excel file
    df.to_excel(r'C:\temp\web_maps.xlsx', index=False)

    print("Finished")

if __name__ == "__main__":
    main()

 

I then also played with the coding posted here (Solved: Re: Where's the spatial reference information? - Esri Community) to print a list of feature services in ArcGIS Online and their WKID's which I could copy/paste into excel and also perform a Vlookup to the item report from ArcGIS Online to get more info for each layer. 

from arcgis.gis import GIS

# Connect to AGOL
agol = GIS("home")

# Search for all items in AGOL
items = agol.content.search(query="", max_items=10000)

# Iterate through each item
for item in items:
    try:
        # If it is a feature service or vector tile package
        if item.type in ("Feature Service", "Vector Tile Package"):
            # Print the WKID of the SRS
            print("Item ID: {0}, WKID: {1}".format(item.id, item["spatialReference"]))
        else:
            print("Skipping Item ID: {0}, Type: {1}".format(item.id, item.type))

    except Exception as e:
        print("Failed to process item ID: {0}, Error: {1}".format(item.id, str(e)))

LindsayRaabe_FPCWA_0-1735019171159.png

 

Lindsay Raabe
GIS Officer
Forest Products Commission WA