Start/Stop Secure Services in 10.6.1

6486
28
03-21-2019 12:42 PM
AmyRoust
Occasional Contributor III

Prior to 10.6.1, I was using the ArcGIS Server Admin Toolkit to stop and start services in bulk. That toolkit has depreciated and does not appear to work in 10.6.1. (It could be that I'm making an error with the parameters, but I don't think I am) Then I found this:

Example: Stop or start all services in a folder—ArcGIS Server Administration (Windows) | ArcGIS Ente... 

The above script assumes that you are using port 6080, but ours are on 6443. I tried to change the parameter in the same script from 6080 to 6443, but that did not work. Again, I'm not ruling out the possibility that I'm entering the wrong variables, but the error message says that the server is forcibly closing the connection to port 6443 when I run it.

In scouring the forums, I've seen comments that suggest that A) you cannot start/stop services using Python if said services are secured (https), and B) the Esri is intentionally not maintaining or developing tools to allow you to batch start/stop services anymore because they want you to do it through Server Manager/Portal/ArcCatalog. I know you can't start/stop services in Pro yet, but that it might be coming in a future version.

Can anyone help me fill in the gaps? My ultimate need is to be able to stop all web services at once and then restart them in bulk. I'm using Enterprise 10.6.1, including Portal. I published all of my services to Server using ArcMap 10.6.1.

28 Replies
MichaelVolz
Esteemed Contributor

Amy:

Were you able to stop secured services in an earlier version of AGS?  I have never gotten the port 6443 and https services to stop without a python error.  Did you ever try to modify AGS security to generate tokens as per below link:

Enable token acquisition through an HTTP GET request—ArcGIS Server Administration (Windows) | ArcGIS... 

I was testing setting up my AGS environment only to accept https protocol, but then I was no longer able to use the script you allude to to stop/start the AGS https services.

0 Kudos
AmyRoust
Occasional Contributor III

This is the first time that I've had secured services, so I don't have any prior successes to look back on. (*sad trombone*)

After reviewing my workflows that used bulk stops and starts, I guess I don't have to shut down my web services. I can just tell the database to stop accepting connections and kick out anyone who is currently connected. (This happens in the middle of the night via automated task, so that the end user impact is minimal.) The only use I'll really miss is the automated task that restarted all of my web services at 6 am, just to give us a fresh start if something crashed overnight and didn't restart properly.

0 Kudos
MichaelVolz
Esteemed Contributor

Do you have any geocode services?  In order to update the associated address locator you would still need to stop/start the service.

0 Kudos
AmyRoust
Occasional Contributor III

Good point. I do have one locator so that people can search by intersection. Esri probably wants us to move entirely to string searches within layers, but unless I generate an intersection point layer, I still need that geocoder.

0 Kudos
MichaelVolz
Esteemed Contributor

I believe the security changes in ArcGIS Server 10.7 (only TLS 1.2 compatibility) will require that all AGS services will be administered with https protocol.  You might get away with http and port 6080 at 10.6.1, but you will need https syntax to work in 10.7.

I hope someone (most preferably from ESRI) can provide guidance on the configuration needed to get https protocol to work with the link you provided above:

Example: Stop or start all services in a folder—ArcGIS Server Administration (Windows) | ArcGIS Ente... 

0 Kudos
JonathanQuinn
Esri Notable Contributor

Programmatically interacting with any administrative API is and will continue to be an integral part of the software. HTTPS doesn't necessarily mean they're secured, at least in the same sense that you'd say you need a token to reach them. It simply means that the traffic to reach them is encrypted.

In order to reach an HTTPS endpoint via Python, the certificate used for the endpoint needs to be trusted by Python or an untrusted certificate needs to be ignored. This isn't specific to Esri, ArcGIS Server, arcpy, etc. Server, even at 10.7, can be HTTPS and HTTP, but by default on a new site, it is HTTPS. Ideally, the certificate is already trusted by Python, or with a bit of work, you can configure Python to trust your own certificates:

https://stackoverflow.com/questions/39356413/how-to-add-a-custom-ca-root-certificate-to-the-ca-store...

The following script uses urllib and urllib2 instead of the httplib library to make web requests and disables certificate validation:

import urllib, urllib2, json,ssl

baseURL = 'https://server.domain.com:6443/arcgis'
username = 'admin'
password = 'admin'
whatToDo = "start"

def openURL(url,params=None, protocol=None):
    try:
        if params:
            params.update(dict(f="json"))
        else:
            params = dict(f="json")
        if protocol:
            encoded_params = str.encode(urllib.urlencode(params))
            encoded_params = encoded_params.decode("utf-8")
            url = "{0}?{1}".format(url, encoded_params)
            request = urllib2.Request(url)
            #Ignore certificate validation
            sslContext = ssl._create_unverified_context()
            request.add_header('referer',baseURL)
            response = urllib2.urlopen(request, context=sslContext)
        else:
            encoded_params = str.encode(urllib.urlencode(params))
            request = urllib2.Request(url)
            #Ignore certificate validation
            sslContext = ssl._create_unverified_context()
            request.add_header('referer',baseURL)
            response = urllib2.urlopen(request,encoded_params,context=sslContext)
        decodedResponse = response.read().decode('utf-8')
        jsonResponse = json.loads(decodedResponse)
        return jsonResponse
    except urllib2.HTTPError as e:
        return e
    except urllib2.URLError as e:
        return e
    except Exception as e:
        print(e)
        
def createToken(baseURL,username,password):
    tokenURL = "{}/tokens/generateToken".format(baseURL)
    params = {"username":username,
              "password":password,
              "client":'referer',
              "referer":baseURL}
    token = openURL(tokenURL,params)['token']
    return token

'''Function to stop or start services'''
def startStopService(serviceURL):
    serviceName = urllib2.urlparse.urlparse(serviceURL).path.split("/")[-1]
    resp = openURL("{}/{}".format(serviceURL,whatToDo),params)
    if "status" in resp and resp['status'] == 'success':
        print("Successfully {} {}".format(msg,serviceName))
    else:
        print("Unable to {} {}.\n {}".format(whatToDo,serviceName,resp))    

token = createToken(baseURL,username,password)
params = {"token":token}
servicesURL = '{}/admin/services'.format(baseURL)
root = openURL(servicesURL,params)
if whatToDo == "stop":
    msg = "stopped"
else:
    msg = "started"
for service in root['services']:
    serviceURL = '{}/{}.{}'.format(servicesURL,service['serviceName'],service['type'])
    startStopService(serviceURL)
for folder in root['folders']:
    if not folder in ['System','Utilities']:
        folderURL = "{}/{}".format(servicesURL,folder)
        folderServices = openURL(folderURL,params)
        for service in folderServices['services']:
            serviceURL = '{}/{}/{}.{}'.format(servicesURL,folder,service['serviceName'],service['type'])
            startStopService(serviceURL)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

https://community.netapp.com/t5/Software-Development-Kit-SDK-and-API-Discussions/Python-How-to-disab...

You can do this with the requests module as well:

https://stackoverflow.com/questions/15445981/how-do-i-disable-the-security-certificate-check-in-pyth...

I'll reiterate that the preferred solution isn't disabling certificate validation, though.

MichaelVolz
Esteemed Contributor

Jonathan:

Thanks for the python code.  Can you tell me which lines of code I would need to modify so that the code does not ignore certificate validation?

Does ArcGIS Server need to be configured based on the following link to properly handle tokens for the code you provided:

Enable token acquisition through an HTTP GET request—ArcGIS Server Administration (Windows) | ArcGIS... 

In the example provided in the original post and even in the code that you provided, is there a way to call the service without providing a port?  My org has a trusted certificate setup on our load balancer that is separate from the internal load-balancing of ArcGIS Server, but when I tried to input the parameters for the service with the load-balanced URL and no port it was rejected.

0 Kudos
JonathanQuinn
Esri Notable Contributor

The baseURL variable on line 3 is where you define the host and optionally port and context to the Server you want to connect to. It doesn't matter if you go through 443, 80, 6443 or 6080 as long as you can reach the Admin API via that URL. For example, you could use your web adaptor URL, https://webadaptor.domain.com/server. You need to make sure that the machine you're running the script on has access to the URL you want to use to connect to.

To validate certificates, get rid of lines 20 and 27 and update line 22 to be

response = urllib2.urlopen(request)

and 29 to be:

response = urllib2.urlopen(request,encoded_params)

You're basically just getting rid of the logic to define the SSL context as "unverified".
0 Kudos
GavinMcDade
New Contributor III

Jonathan, 

I'm working w/ Mike to revise and test our code, and I've gotten the above to work when bypassing the certificate verification. However, when I comment out and alter the code as you've indicated above, I receive the following Python error (ignore line numbers, as they differ from your raw script above): 

Traceback (most recent call last):

  File "Stop_GeocodeService_Cert.py", line 148, in <module>

    token = createToken(baseURL,username,password)

  File "Stop_GeocodeService_Cert.py", line 118, in createToken

    token = openURL(tokenURL,params)['token']

TypeError: sequence index must be integer, not 'str'

For whatever reason, it appears to no longer like the ['token'] parameter... Any ideas why this might be? 

Gavin

0 Kudos