topic Re: Publish Map Service with modified Properties in Python Questions

Publish Map Service with modified Properties

BenVan_Kesteren1 — Sun, 12 Dec 2021 01:04:18 GMT

Hi All,

I currently have a Python script which does 98% of what I require.

Currently, it picks up a local MXD file that has been modified in the last 3 days, then uses arcpy.mapping.CreateMapSDDraft() to create the Draft Service Definition, then stages the service using arcpy.StageService_server() and finally arcpy.UploadServiceDefinition_server()

Each time this file is uploaded, it is overwriting an existing map service, and these services all have Advanced Properties modified, for example the maxImageWidth and Height are modified, along with Max Number of Instanced modified to be 15.

The problem I have is that each time this script runs, it reverts those three settings back to the default, so then I have to manually set them to what we require, which in turn defeats the purpose of automating this process... I have pasted my working code below, its not pretty, but seems to do most of what I require....

Any ideas on how to set those additional parameters in python?

def publish_services(service):


    log.write_log(python_file, 'Updating Services (' + service + ') to GISServer')
    err = False
    # Define Local Variables
    wrkspc = r'\\gisserver\weave projects\Project_Files'
    doc = os.path.join(wrkspc, service + '.mxd')
    log.write_log(python_file, 'Publishing File to Service:')
    log.write_log(python_file, '    %s [doc]' % doc)
    is_edited = check_edit_date(service, doc)
    if is_edited is True:
        mapDoc = arcpy.mapping.MapDocument(doc)


        # Provide path to connection file
        # To create this file, right-click a folder in the Catalog window and
        #   click New > ArcGIS Server Connection
        con = 'GIS Servers\GisServer (Administration)'


        # Provide other service details
        sddraft = os.path.join(wrkspc, service + '.sddraft')
        sd = os.path.join(wrkspc, service + '.sd')
        summary = 'Map document displayed by the Weave Mapping System'
        tags = 'Weave, python, automated, MXD, Map Service Automation, No Man Hours Used'


        def del_sd():
            try:
                os.remove(sd)
                log.write_log(python_file, 'Removed the existing sd file - %s' % str(sd))
            except OSError:
                pass


        log.write_log(python_file, 'Creating .sddraft file')


        # Create service definition draft (.sddraft)
        arcpy.mapping.CreateMapSDDraft(mapDoc, sddraft, service, 'ARCGIS_SERVER',
            con, True, None, summary, tags)


        log.write_log(python_file, 'Analyzing the .sddraft file for errors')
        # Analyze the service definition draft
        analysis = arcpy.mapping.AnalyzeForSD(sddraft)


        # Print errors, warnings, and messages returned from the analysis
        try:
            for key in 'errors':  # 'messages', 'warnings'):    # This is commented out for clarity
                log.write_log(python_file,  "--------" + key.upper() + "--------")
                vars = analysis[key]
                for ((message, code), layerlist) in vars.iteritems():
                    log_message = "    ", message, " (CODE %i)" % code
                    log.write_log(python_file, log_message)
                    log.write_log(python_file, "       applies to:", )
                    for layer in layerlist:
                        log.write_log(python_file, '\n             ' + str(layer.name), )
                    log.write_log(python_file, "")
        except:
            log.write_log(python_file, 'Ironically, there was an error printing the errors')
            print 'Ironically, there was an error printing the errors in the publish_services() module'
            err = True


        # remove file if it already exists,
        # need the folder to be cleared before creating again
        del_sd()


        # Stage and upload the service if the sddraft analysis did not contain errors
        if analysis['errors'] == {} or err is False:
            # Execute StageService
            arcpy.StageService_server(sddraft, sd)
            # Execute UploadServiceDefinition
            arcpy.UploadServiceDefinition_server(sd, con)
            log.write_log(python_file, "Service %s successfully published" % service)
        else:
            # if the sddraft analysis contained errors, display them
            log.write_log(python_file, str(analysis['errors']))
            log.write_log(python_file, "Service %s could not be published because errors were found during analysis."
                          % service
                          )


        # remove the temporary file once tasks complete
       del_sd()
    else:
        pass

Re: Publish Map Service with modified Properties

BenVan_Kesteren1 — Sun, 12 Dec 2021 01:04:21 GMT

Hey,

So I might have found something useful here, it may work - Example: Edit service properties—Documentation (10.3 and 10.3.1) | ArcGIS for Server

This section of the code may do part of what I require:

# Edit desired properties of the service
dataObj["minInstancesPerNode"] = minInstancesNum
dataObj["maxInstancesPerNode"] = maxInstancesNum

My secondary question is, how do I find what all the options are called? The above example exactly answers part of my question, as one of the items I want to update is max instances, but how do i find out what the syntax would be for the Max image Width and height will be? How does one go about searching for this??

Thanks in advance.

Cheers

Re: Publish Map Service with modified Properties

DanPatterson_Retired — Wed, 06 Jul 2016 11:59:12 GMT

If you are in this extension, An overview of the Publishing toolset—Help | ArcGIS for Desktop

just check the code samples for each of the tools

Re: Publish Map Service with modified Properties

SubuSwaminathan1 — Wed, 13 Jul 2016 22:57:09 GMT

ArcGIS REST API - Out of box Service/extension types and properties listed here in the REST API docs

Re: Publish Map Service with modified Properties

ChrisPedrezuela — Thu, 14 Jul 2016 09:53:37 GMT

from documentation, Upload Service Definition—Help | ArcGIS for Desktop , this script creates and modifies service definition draft before uploading to server

Re: Publish Map Service with modified Properties

BenVan_Kesteren1 — Thu, 14 Jul 2016 13:28:22 GMT

Hi Everyone for your suggestions. I have looked through each of the links provided, unfortunately I am yet to see an example to help me set the MaxImageWidth and MaxImageHeight... back to the drawing board, I may have to continue to do this manually 😕

Thanks anyways.

Re: Publish Map Service with modified Properties

DavidMann1 — Tue, 19 Jul 2016 17:51:11 GMT

Ben - I was struggling with the same issue last week and came across this post along with many others to help me figure out how to do it. What I ended up doing was using the ArcGIS Server Admin URL to access the service parameters of the service to be overwritten in JSON format, went through the process you describe of creating SDDRAFT, Staging SD, and Uploading SD, and then finally using the JSON config I saved to Edit the parameters of the service after published with default parameters. No need to pick out the individual parameters this way that you want to preserve, you keep them all. Code snippets of applicable parts below (sorry don't know how to format as code block)...

adminUrl = 'https://services.gisserver.com:6443/arcgis/admin/services/'

# Start of get token function
def generateToken(username, password, portalUrl):

'''Retrieves a token to be used with API requests.'''
parameters = urllib.urlencode({'username': username,

'password': password,

'client': 'referer',

'referer': portalUrl,

'expiration': 60,

'f': 'json'})

try:

response = urllib2.urlopen(portalUrl + '/sharing/rest/generateToken?',

parameters).read()

except Exception as e:

log.write('Unable to open the url %s/sharing/rest/generateToken' % (portalUrl))

log.write(e)

responseJSON = json.loads(response.strip(' \t\n\r'))

# Log results
if responseJSON.has_key('error'😞

errDict = responseJSON['error']

if int(errDict['code']) == 498:

message = 'Token Expired. Getting new token... '
token = generateToken(username, password, portalUrl)

else:

message = 'Error Code: %s \n Message: %s' % (errDict['code'],

errDict['message'])

log.write(message)

token = responseJSON.get('token')

return token

# Generate a token
token = generateToken(portalAdminName, portalAdminPassword, portalUrl)

# Add token to service URL to authenticate access.
params = {'token': token,

'f': 'json'}

fUrl = adminUrl + sfolder + '/' + service + '.MapServer?' + urllib.urlencode(params)

# Open service URL to access parameters

openUrl = urllib2.urlopen(fUrl, '').read()

# Load json response in string serviceJson to access later
serviceJson = json.loads(openUrl)

# Check response for valid service
if '{"serviceName":' not in serviceJson:

raise Exception(('Service access error, response = {}').format(json.dumps(serviceJson)))

# After republishing service...

# Load parameters for edit service call, dumping serviceJson string back into JSON format
params = {'token': token,

'f': 'json',

'service': json.dumps(serviceJson)}

fUrl = adminUrl + sfolder + '/' + service + '.MapServer/edit?' + urllib.urlencode(params)

openUrl = urllib2.urlopen(fUrl, '')

Re: Publish Map Service with modified Properties

BenVan_Kesteren1 — Sun, 12 Dec 2021 01:04:24 GMT

Thanks very much for the reply David, I have tweaked what you have provided and been able to work out how to connect and generate a token. That works nicely.

I am now trying to connect back to the server and publish my new parameters, and I am getting this error:

Traceback (most recent call last):

File "U:/Automated/editMapServiceParameters.py", line 62, in <module>

raise Exception('Service access error, response = {}'.format(json.dumps(serviceJson)))

Exception: Service access error, response = {"status": "error", "code": 500, "messages": ["1"]}

I am unsure how to progress from here. Did you hit this hurdle in your travels by any chance?

Here is the code which causes the above error:

import json
import urllib
import urllib2
import httplib


adminUrl = 'http://gisserver:6080/arcgis/admin/services'
portalAdminName = 'gisadmin'
portalAdminPassword = PASSWORD
portalUrl = "/arcgis/admin/generateToken"
serverName = 'gisserver'
serverPort = 6080




# A function to generate a token given username, password and the adminURL.
def getToken(username, password, serverName, serverPort):
    # Token URL is typically http://server[:port]/arcgis/admin/generateToken
    tokenURL = "/arcgis/admin/generateToken"


    params = urllib.urlencode({'username': username, 'password': password, 'client': 'requestip', 'f': 'json'})


    headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}


    # Connect to URL and post parameters
    httpConn = httplib.HTTPConnection(serverName, serverPort)
    httpConn.request("POST", tokenURL, params, headers)


    # Read response
    response = httpConn.getresponse()


    if response.status != 200:
        httpConn.close()
        print "Error while fetching tokens from admin URL. Please check the URL and try again."
        return
    else:
        data = response.read()
        httpConn.close()


    # Extract the token from it
    token = json.loads(data)


    return token.get('token')


# Generate Token
token = getToken(portalAdminName, portalAdminPassword, serverName, serverPort)


print 'token:: %s' % token


# Add token to service URL to authenticate access.
params = {'token': token, 'f': 'json'}


fUrl = 'http://gisserver:6080/arcgis/admin/services/Maps/Weave_Inset.MapServer?' + urllib.urlencode(params)
print fUrl


# Open service URL to access parameters
openUrl = urllib2.urlopen(fUrl, '').read()


# Load json response in string serviceJson to access later
serviceJson = json.loads(openUrl)


# Check response for valid service
if '{"serviceName":' not in serviceJson:
    raise Exception('Service access error, response = {}'.format(json.dumps(serviceJson)))

Thanks again.

Re: Publish Map Service with modified Properties

DavidMann1 — Mon, 08 Aug 2016 12:09:56 GMT

Hey Ben - The error is caused by the code not being able to access valid json parameters for an existing service. What the code is checking for ("serviceName":) is one of the first items to appear on that parameters page when the service exists. So my guesses would be either a) the service you are trying to overwrite doesn't already exist/isn't running or b) there is a problem accessing due to the token maybe. I would try accessing the fUrl manually and see if the response looks the same as you are getting in the error.

Re: Publish Map Service with modified Properties

BenVan_Kesteren1 — Sun, 12 Dec 2021 01:04:26 GMT

So after many hours of trial and error, then implementing most of the changes as suggested in this thread, I contacted esri australia tech support.

They pointed out the errors of my ways.

A few misunderstandings from me as to how JSON works, and other simple code tweaks, the end result is below, works a treat!

service = 'Service Name'

# A function to generate a token given username, password and the adminURL.
def getToken():

    portalAdminName = 'gisadmin'  # Gisserver login
    portalAdminPassword = 'xxxxxxx'  # gisserver password
    serverName = 'gisserver'  # Server Name
    serverPort = 6080  # Port number of the server
    # Token URL is typically http://server[:port]/arcgis/admin/generateToken
    tokenURL = "/arcgis/admin/generateToken"

    params = urllib.urlencode({
        'username': portalAdminName,
        'password': portalAdminPassword,
        'client': 'requestip',
        'f': 'json'
    })

    headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}

    # Connect to URL and post parameters
    httpConn = httplib.HTTPConnection(serverName, serverPort)
    httpConn.request("POST", tokenURL, params, headers)

    # Read response
    response = httpConn.getresponse()

    if response.status != 200:
        httpConn.close()
        print "Error while fetching tokens from admin URL. Please check the URL and try again."
        return
    else:
        data = response.read()
        httpConn.close()

    # Extract the token from it
    token = json.loads(data)

    return token.get('token')


# Generate Token
token = getToken()

# Add token to service URL to authenticate access.
params = {'token': token, 'f': 'json'}

# build the URL used to connect to the web form
fUrl = 'http://gisserver:6080/arcgis/admin/services/' + service + '.MapServer?' + urllib.urlencode(params)

# Open service URL to access parameters
openUrl = urllib2.urlopen(fUrl, '').read()

# Load json response in string serviceJson to access later
servicePython = json.loads(openUrl)

# Change the maxImage size properties in the python dictionary
servicePython['properties']['maxImageHeight'] = 5200
servicePython['properties']['maxImageWidth'] = 6600
servicePython['maxInstancesPerNode'] = 15

# Convert python dictionary back into a JSON string
serviceJson = json.dumps(servicePython)

# Define params dictionary
params = {"service": serviceJson, "f": "json", "token": token}

# REST endpoint for editing the service
finalUrl = 'http://gisserver:6080/arcgis/admin/services/' + service + '.MapServer/edit?'

# URLencoded paramaters dictionary can be included as a parameter of the call to urllib2.urlopen
openUrl = urllib2.urlopen(finalUrl, urllib.urlencode(params))

‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Publish Map Service with modified Properties

thomasbales — Sun, 12 Dec 2021 01:04:29 GMT

Finally I see some discussion on this topic. I am going through a similar problem.

I have automated the overwrite of the service definition of a feature service, but every time it is overwritten it resets the 'maxRecordCount' back to the default value of 1000.

As you say, this then defeats the purpose of automating since I have to manually upload the service so it allows 50,000 records to be queried at REST.

I am doing this through AGOL -- any idea how to tailor this part:

 portalAdminName = 'gisadmin'  # Gisserver login
    portalAdminPassword = 'xxxxxxx'  # gisserver password
    serverName = 'gisserver'  # Server Name
    serverPort = 6080  # Port number of the server

...to allow for log in and edit of a service on an AGOL organizational account?

I have tried:

but I am getting an error on the httpConn request saying "getaddrinfo failed".

Any help or direction would be greatly appreciated, otherwise it looks like a call to Esri is in store. Thanks.

Re: Publish Map Service with modified Properties

JonathanQuinn — Sun, 12 Dec 2021 01:04:32 GMT

The code to generate a token is for an on-premises ArcGIS Server, not AGOL. This link has a code example for generating a token with in AGOL:

try:
    print('Generating Token')
    tokenURL = 'https://www.arcgis.com/sharing/rest/generateToken'
    params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://www.arcgis.com'}
    req = urllib2.Request(tokenURL, urllib.urlencode(params))
    response = urllib2.urlopen(req)
    data = json.load(response)
    token = data['token']
except:
    token = ''‍‍‍‍‍‍‍‍‍‍

I'll add that you'll likely want to capture the exception:

try:
    print('Generating Token')
    tokenURL = 'https://www.arcgis.com/sharing/rest/generateToken'
    params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://www.arcgis.com'}
    req = urllib2.Request(tokenURL, urllib.urlencode(params))
    response = urllib2.urlopen(req)
    data = json.load(response)
    token = data['token']
except Exception as e:
    print(e)
    token = ''‍‍‍‍‍‍‍‍‍‍‍

And also note that the token is generated using a referer, so any request that uses the token should have the same referer header.