How do I find what database a map service is pulling from?

963
5
03-08-2021 01:09 PM
HaroldKrivell
New Contributor III

I am trying to determine what databases a a map service is pulling from. We are inventorying our map services and their data sources but I do not know where to look for the database addresses. How do you identify the sources of the data and where it's coming from?

Tags (1)
0 Kudos
5 Replies
FDEPAGO
New Contributor

Assuming you have access to the Server Manager interface, you can click the database icon next to the service and then expand the database name to find the specific feature classes in the service. 

CraigRussell
Esri Contributor

Another way that you could do this is to look at the manifest.xml file for each of the services in the ArcGIS Server configuration store directory, e.g. D:\arcgisserver\config-store\services.  This includes a list of all of the databases/datasets used in the service:

CraigRussell_0-1615259292767.png

You could just search across the services folder, or maybe do something to programmatically pull these details out if you're looking for a more elegant solution.

0 Kudos
HenryLindemann
Esri Contributor

Hi @HaroldKrivell,

Here is a little script that I wrote to do that just run it on the server just point line 137 to your config store connection file, it will spit out a csv file.

# Get config store path
# C:\Program Files\ArcGIS\Server\framework\etc\config-store-connection.xml

# for entry in file get
# <entry key="connectionString">\\dns\Arcgis\arcgisserver\hostname\config</entry>

# get registered data connections
# \\dns\Arcgis\arcgisserver\hostname\config\data

# get federation if exists
# \\dns\Arcgis\arcgisserver\hostname\config\security

# get list of services
# for folder in services does it have a json file if yes what type eg mapserver
# if the manifest exists get datasource details
# compile service details
import json
import wmi
# Dependency for pyinstaller
import win32
import win32.com
import win32.com.client
import zmq
import pywintypes
import pythoncom
import pkgutil

computer = wmi.WMI()

# modules used in Win32
computer_info = computer.Win32_ComputerSystem()[0]


def get_service_types(path):
    print_list = []
    path = path.split('/')
    path = '\\'.join(path)
    import os
    service_types = set()

    # {'GPServer', 'WorkspaceServer', 'GeoDataServer', 'IndexGenerator', 'IndexingLauncher', 'SearchServer',
    # 'MapServer', 'ImageServer', 'FeatureServer', 'GeometryServer', 'SceneServer'}
    for root, dirs, files in os.walk(path + '\\services', topdown=False):
        for name in dirs:
            if name.__contains__('.'):
                service_type = name.split('.')[-1]
                service_types.add(service_type)
                service_path = os.path.join(root, name)
                if service_path.__contains__('\\System\\'):
                    continue
                if service_path.__contains__('\\Utilities\\'):
                    continue

                manifest = f'{service_path}\esriinfo\manifest'

                manifest_path = f"{manifest}\manifest.json"
                service = service_path.split('\\')
                name = str(service[-1]).split('.')
                if os.path.isfile(manifest_path):
                    try:
                        if service[-2] == 'services':
                            print_list.append(f"root;{name[0]};{name[1]};True")
                        else:
                            print_list.append(f"{service[-2]};{name[0]};{name[1]};True")
                        with open(manifest_path, 'r') as manifest_file:
                            manifest = json.load(manifest_file)
                            manifest = manifest['databases']

                            for dataset in range(0, len(manifest)):
                                #Dataset
                                if dataset > 0:
                                    print_list.append(f"\n;;;;;{dataset + 1}")
                                else:
                                    print_list.append(f";{dataset + 1}")
                                for record in manifest[dataset]:
                                    data_string_list = []
                                    if record in ['onServerWorkspaceFactoryProgID', 'onServerName', 'onPremisePath', 'datasets']:
                                        continue
                                    if record in ['onServerConnectionString', 'onPremiseConnectionString']:
                                        data_string = manifest[0][record]
                                        if data_string.__contains__(';'):
                                            data = data_string.split(';')
                                        else:
                                            data = None
                                        if data is not None:
                                            for record_1 in data:
                                                data_string_list.append(record_1.split('='))
                                    else:
                                        data_string = manifest[0][record]
                                    if data_string_list.__contains__('DATABASE'):
                                        continue
                                    if len(data_string_list) > 1:
                                        data_string = ''
                                        for value in data_string_list:
                                            if len(value) != 2:
                                                continue

                                            if value[0] in ['ENCRYPTED_PASSWORD', 'INSTANCE', 'VERSION']:
                                                continue
                                            data_string += f"{value[0]}={value[1]};"
                                    if record == 'onPremiseConnectionString':
                                        print_list.append(f"\n;;;;;;;;{record};{data_string}")
                                    else:
                                        print_list.append(f"{record};{data_string}")

                            print_list.append('Line_Break')

                    except Exception as e:
                        print(e)
                        print_list.append(e)
                else:
                    if service[-2] == 'services':
                        # Folder Name Type Manifest
                        print_list.append(f"root;{name[0]};{name[1]};False")
                        print_list.append('Line_Break')
                    else:
                        # Folder Name Type Manifest
                        print_list.append(f"{service[-2]};{name[0]};{name[1]};False")
                        print_list.append('Line_Break')

    with open(f'arcgis_server_services_sql_{computer_info.DNSHostName}.csv', 'w') as ass_write_file:
        write_out = ''
        ass_write_file.write('Folder;Service;Type;Manifest;;Dataset_Count;Referenced\n')
        for record in print_list:
            if record == 'Line_Break':
                write_out += '\n'
                write_out = write_out.replace(',', '_')
                ass_write_file.write(write_out)
                write_out = ''
            else:
                write_out += f"{record};"


if __name__ == "__main__":
    import os
    import re
    default_path = r'C:\Program Files\ArcGIS\Server\framework\etc\config-store-connection.xml'
    if os.path.isfile(default_path):
        with open(default_path, 'r') as config_store_connection_file:
            file = config_store_connection_file.readlines()
            for line in file:
                if line.__contains__('key="connectionString"'):
                    path = re.sub('<entry key="connectionString">', '', line)
                    path = re.sub('</entry>\n', '', path)
                    path = path.split('\\')
                    path = '/'.join(path)
                    print(path)
                    get_service_types(path)

 

FraserHand
Occasional Contributor II

Hi,

Further to the response above re the xml - this is actually exposed in the admin REST admin. 

https://developers.arcgis.com/rest/enterprise-administration/server/servicemanifest.htm

Example: 

https://machine.domain.com/webadaptor/admin/services/Maps/California.MapServer/iteminfo/manifest/man...

You can see the response from the developers link above.

You could write a pretty simple python script using requests. If you are familiar with the Python API then this info is also exposed via the ItemInformationManager. It would be pretty easy to sketch up a Jupyter notebook that could pull this info out for a given GIS.

https://developers.arcgis.com/python/api-reference/arcgis.gis.server.html#iteminformationmanager

property manifest

The service manifest resource documents the data and other resources that define the service origins and power the service. This resource will tell you underlying databases and their location along with other supplementary files that make up the service.

 

Cheers

 

CarstenB_orsted
New Contributor II

 

I am trying to use ItemInformationManager in the Python API order to get the manifest. But I really can't make it work and I can't find any samples.

EDIT: Never mind! I found the information I was missing. This works:

 
from arcgis.gis import GIS
import pprint

gis = GIS(portal_url, username, password)

servers = gis.admin.servers.list()
server1 = servers[0]
services = server1.services.list(folder="myfolder")
service = services[0]

ii = service.iteminformation
manifest = ii.manifest
pprint.pprint(manifest)

properties = ii.properties
pprint.pprint(properties)

 

0 Kudos