Filter Services by those which contain .MapServer to get Service Manifest

744
9
Jump to solution
09-29-2022 04:04 AM
JaymieCroucher
New Contributor II

Hi Everyone,

Our team are trying to create a single table which creates a link between a portal item its service URL and the feature class(s) and database it sits in. Trawling the forum has led me to look at using the service manifest as a source for this for which I've had some success however I've now encountered an issue where as I loop through each Service within a folder I hit an error if the service holds no manifest. This is because geoprocessing services (e.g. A custom data export / print service) holds no manifest and so the script fails. 

I've tried a TRY function and also various way of trying to create a test case to filter our non MapServer services but with little success. This is mainly as each service in the returned list isn't string but a custom ArcGIS Server data type which is limited. 

Does anyone have any ideas on how to filter services? 

---------------------------------------------------------------------------------------------------------------------------------------

from arcgis.gis import *
import json

gis = GIS("portal", "username", "password",verify_cert=False,)

print("Successfully logged in as: " + gis.properties.user.username)

servers = gis.admin.serverss = servers.list() #list federated servers
server1 = s[1] #isolate portal server in list
services = server1.services.list(folder="Shared") #list the services within the folder "Shared"

for service in services: #list the manifest for each service URL
     ii = service.iteminformation
     manifest = ii.manifest
     a = json.dumps(manifest)
     b = json.loads(a)
     fcnam = b['databases'][0]['datasets'] #scan the JSON and fine the list of feature classes
     dbnam = b['databases'][0]['onServerName'] # scan the JSON and find the DB name
     for x in fcnam:
          print(service,dbnam,x['onServerName']) #print service with list of db fc names 

Tags (1)
0 Kudos
1 Solution

Accepted Solutions
RobertAkroyd1
New Contributor III

Hi @JaymieCroucher 

I think all you need to do what you are asking above to filter on MapServer services is to add the following inside your "for service in services", e.g.

for service in services: #list the manifest for each service URL

     if service.type == "MapServer":
          ii = service.iteminformation

          # the rest of your code

View solution in original post

9 Replies
RobertAkroyd1
New Contributor III

Hi @JaymieCroucher 

I think all you need to do what you are asking above to filter on MapServer services is to add the following inside your "for service in services", e.g.

for service in services: #list the manifest for each service URL

     if service.type == "MapServer":
          ii = service.iteminformation

          # the rest of your code

JaymieCroucher
New Contributor II

Very grateful Rob, worked a dream 😀

0 Kudos
HaThach1
New Contributor II

This script print out a few mapserver and then return this KeyError message...

Traceback (most recent call last):
File "Q:\Aqua_ArcGIS\WebServiceInfoExtract\test\test7.py", line 23, in <module>
fcnam = b['databases'][0]['datasets'] #scan the JSON and fine the list of feature classes
KeyError: 'databases'

Have any idea?

0 Kudos
RobertAkroyd1
New Contributor III

Looks like the JSON for that service did not include a "databases" element, so you should check for that in the code first, e.g.

     b = json.loads(a)

     if b['databases']:

          fcnam = b['databases'][0]['datasets'] #scan the JSON and fine the list of feature classes

 

0 Kudos
HaThach1
New Contributor II

I got the same error after updating the script.

HaThach1_0-1668525123886.png

 

0 Kudos
JaymieCroucher
New Contributor II

Sorry I only recently managed to get this working in anger. The below provides a CSV output of the following: 

  • Each Feature Class and sourced database provided within a service 
  • The Service URL and Name 
  • The Portal URL and ID 

 

import json
import csv
import os
from arcgis.gis import *

#Step 1 - Import all libraries and setup connection to portal and location of csv

gis = GIS("Portal URL", username, password,verify_cert=False,)
outputFile=r"\\Output CSV Location"
print("Successfully logged in as: " + gis.properties.user.username)

#Step 2 - wipe the CSV and set header values

with open(outputFile, 'w', newline='') as ResultFile:
wr = csv.writer(ResultFile)
header = 'Portal Item', 'Portal Item ID','Service URL','Portal URL','DB Name', 'Feature Class Name'
wr.writerow(header)

#Step 3 - Find hosting server and list and join all services across folders

servers = gis.admin.servers
s = servers.list()
server1 = s[0] #isolate portal server in list
services_1 = server1.services.list(folder="Server Folder Name 1") #list the services within the folder "root"
services_2 = server1.services.list(folder="Server Folder Name 2")
services_3 = server1.services.list(folder="Server Folder Name 3")
services_4 = server1.services.list(folder="Server Folder Name 4")
services = services_1 + services_2 + services_3 +services_4

#Step 4a - Isolate only MapServer services and print the manifest for each service
#Step 4b - Extract service properties from the Portal
#Step 4c - Sift through JSON from manifest and extract feature class and database
#Step 5 - Write out many to one relationship of feature class to service to portal item

for service in services:
if service.type == "MapServer":
ii = service.iteminformation
manifest = ii.manifest
a = json.dumps(manifest)
b = json.loads(a)
piid = service.properties.portalProperties.portalItems
pin = service.properties.serviceName
piidi = piid[0]['itemID']
piidiurl = "Portal Map Viewer URL" + piidi
fcnam = b['databases'][0]['datasets'] #scan the JSON and fine the list of feature classes:
dbnam = b['databases'][0]['onServerName'] # scan the JSON and find the DB name
for x in fcnam:
with open(outputFile, 'a',newline='') as ResultFile:
wr = csv.writer(ResultFile)
ContentRow = pin,piidi,service,piidiurl,dbnam,x['onServerName']
wr.writerow(ContentRow)
loops = loops+1

print("Successfully wrote " +str(loops) + " records to the manifest CSV")

0 Kudos
HaThach1
New Contributor II

Hi Jamie,

I tried your script as well and got 

AttributeError: 'PropertyMap' instance has no attribute 'portalProperties'

even if I comment out variable piid, now I'm getting the same error as before relate to

KeyError: 'databases'

HaThach1_0-1668526200868.png

 

0 Kudos
JaymieCroucher
New Contributor II

Do you get a list of services from the hosting server? 

Have you manually checked the JSON manifest through Server Manager for a specific service?

(Should look like: https://resources.arcgis.com/en/help/server-admin-api/serviceManifest.html. Gained through Server Manager) Example URL here: https://portal.domain/server/admin/services/ServerFolder/ServiceName.MapServer/iteminfo/manifest/man... 

0 Kudos
HaThach1
New Contributor II

I didn't get pass this error

AttributeError: 'PropertyMap' instance has no attribute 'portalProperties'

As for the JSON manifest check, I was able to constructed the URL from your example and see the JSON manifest of a specific service.

HaThach1_0-1668535538695.png

 

 

0 Kudos