rastrauch

Resolved: Testing whether a service is a geoprocessing service, AND if accessing a datasource

Discussion created by rastrauch Champion on Sep 29, 2016
Latest reply on Oct 4, 2016 by rastrauch

UPDATE: title changed to better reflect the results needed. 

Bottom line: in python, how can I check whether a data source is being used by a GP service?

 

I'm working on a script (currently run against AGS10.2.2) that looks at all running services and checks the mxd in, for example:

D:\arcgisserver\directories\arcgissystem\arcgisinput\wc_public\Sublines.MapServer\extracted\v101

to see if a specified file gdb is used in the service. I make a list of STARTED services, make a list of those that are using the datasource, and then stop those services so I can update the fgb and restart the same services.  This is all running fairly well.

 

The problem is I still get locks if there is a GP service that is accessing the fgdb. (took a while for me to track that down). Since there is no .mxd associated with the GP service in a relative folder, the one suggest (per Kevin Hibma ) would be to dig into the

C:\arcgisserver\config-store\services\wc_gp\ExtractSomeData.GPServer\esriinfo\manifest\manifest.json

file, which means doing a bit of extra work with the json file, (or possibly the .xml file in same location) to see if the path is in there. For example, the .json for the above service is similar to this (the source I'm looking for is green ...but my datapath name is actually a unc path for the D drive, but that I can adjust my search.:)

{"databases":[{"byReference":true,"onServerWorkspaceFactoryProgID":"esriDataSourcesGDB.FileGDBWorkspaceFactory.1","onServerConnectionString":"DATABASE=D:\\data\\wc\\Master_current.gdb","onPremiseConnectionString":"DATABASE=D:\\data\\wc\\Master_current.gdb","onServerName":"Master_current.gdb","onPremisePath":"","datasets":[{"onServerName":"Subunits"},{"onServerName":"UNIT"},{"onServerName":"Regions"}]}],"resources":[{"onPremisePath":"C:\\Users\\usename\\AppData\\Local\\Esri\\Desktop10.2\\Staging\\agsAdmin@myservername_6080 (publisher)\\sddraftToolboxes\\ExtractGMUData.tbx","clientName":"MYSERVERNAME","serverPath":"d:\\arcgisserver\\directories\\arcgissystem\\arcgisinput\\wc_gp\\ExtractGMUData.GPServer\\extracted\\v101\\ExtractGMUData.tbx"}]}

 

So, my questions are:

  1. has anyone written a python script that searches a GP service manifest.json (or manifest.xml) file for a <datasource> path, gdb in my case? and if so, anything that can be shared? 
  2. does anyone have another way (python script) to query a GP service for a datasource locks?  ......or....when getting a list of STARTED services, is there a way to determine that it is a GP service and not a map service?  I'm actually using a modified function (from Kevin and others) to create this list (and it works for creating my list) 
    EDIT: Think I figured out how to do this part.
    # Get list of Services based on STARTED/STOPPED/or-all
    def getServiceList(siteURL, token, statusFilter):
         # Function to list AGS services based on status: STARTED, STOPPED or all
         #   Requires token
         # Note: Will not return any services in the Utilities or System folder   
         if not statusFilter == "STARTED" and not statusFilter == "STOPPED":
              statusFilter = ""
         services = []
         filteredServices = []
         folder = ''   
         URL = "{0}/admin/services{1}?f=pjson&token={2}".format(siteURL, folder, token)   
         serviceList = json.loads(urllib2.urlopen(URL).read())

         # Build up list of services at the root level
         for single in serviceList["services"]:
              services.append(single['serviceName'] + '.' + single['type'])

         # Build up list of folders and remove the System and Utilities folder (we dont want anyone playing with them)
         folderList = serviceList["folders"]
         folderList.remove("Utilities")            
         folderList.remove("System")
         #folderList.remove("anotherFolderToRemove")

         # working on services in other folders, not root    
         if len(folderList) > 0:
              for folder in folderList:                                             
                   URL = "{0}/admin/services/{1}?f=pjson&token={2}".format(siteURL, folder, token)   
                   fList = json.loads(urllib2.urlopen(URL).read())

                   for single in fList["services"]:
                        services.append(folder + "//" + single['serviceName'] + '.' + single['type'])                   
         if len(services) == 0:
              myMsgs ("No services found")
         else:
              #print ("\nServices on {}:".format(server))   # use to test or debug
              for service in services:
                   statusURL = "{0}/admin/services/{1}/status?f=pjson&token={2}".format(siteURL, service, token)
                   status = json.loads(urllib2.urlopen(statusURL).read())
                   if statusFilter == "":
                        #myMsgs ("  {0} >  {1}".format(status["realTimeState"], service))  # use to test or debug
                        filteredServices = services
                   elif status["realTimeState"] == statusFilter:
                        #myMsgs ("  {0} >  {1}".format(status["realTimeState"], service))  # use to test or debug
                        #filteredServices.append((status["realTimeState"], service))  #returns status, good for debug
                        filteredServices.append(service)
         return filteredServices
    And my guess is I can query something in that, but haven't dug into that yet....but maybe someone already knows?

 

I'll be able to grab the correct path to the .json or .xml, in my script when needed.  BTW - I have all the clear memeory and other lock checks that I've gathered already in place.  I manually used compmgmt.msc and fsmgmt.msc to find some, but finally tracked down the final locks using Process Explorer , then stopped one GP service at a time to find the problem GPs.  But would like to get this automated in my script.

 

Bottom line: in python, how can I check whether a data source is being used by a GP service?

 

Thanks

Python python snippets

 

EDIT: I think I just figured out how to determine if it is a GP service, so one issue down.  I can do one of two things....one, if I don't find the mxd, most likely a GP service.  But even better, my returned services will end in .GPServer (vs..MapServer).  So that will reduce the number I need to look at the json.  Crossed out message is Remanent, not yet determined.

Outcomes