topic Overwrite map service with ArcPy but use existing service properties? in Python Questions

Overwrite map service with ArcPy but use existing service properties?

Anonymous User — Thu, 12 Apr 2018 00:05:04 GMT

I am currently use a map service publishing script that uses CreateMapSDDraft to create the service definition draft and then I edit the service properties by treating it as an xml file. This script works for fresh publishes as well as overwrites. I'm curious if anybody has a workflow for overwrites that uses the service properties that already exist for the given map service, just like "Share As > Service... > Overwrite an existing service" in ArcMap does. I'd like to avoid having to repopulate the service properties every time I want to overwrite a map service because they differ slightly depending on the map service.

In case it's relevant, I am using ArcMap 10.2.1 and publishing to ArcGIS Enterprise 10.5.1.

Re: Overwrite map service with ArcPy but use existing service properties?

JonathanQuinn — Thu, 12 Apr 2018 19:08:46 GMT

What if you store the settings for the service as JSON by accessing the service through the Admin API (ex https://server.domain.com:6443/arcgis/admin/services/<service>.MapServer?f=json) prior to publishing, overwrite the service, and then re-apply the settings?

What is updating within the service to require you to overwrite it? Symbology, layer/layer order, etc)?

Re: Overwrite map service with ArcPy but use existing service properties?

Anonymous User — Thu, 12 Apr 2018 21:24:33 GMT

The general idea is that when a change to a map document is made (symbology, layers, whatever), I'd like to republish it without changing the service properties. For example, I'm currently working on a request that involves enabling the time properties on a handful of layers scattered across 20 map documents that have different min/max instances set. I'd like to run a script to republish them all in one go without having to worry about re-setting the min/max instances per-document.

I think your solution will do the trick. I just wish there was a better way to fetch service properties so it could function more like the manual overwrite process that just uses what's already there. Thank you!

Edit: I'll go ahead and mark your answer as correct but I'm still curious to see if anybody else has other ideas!

Re: Overwrite map service with ArcPy but use existing service properties?

ZahirIbrahim — Wed, 10 Oct 2018 14:54:51 GMT

Has there been any update on this? I am trying to complete the exact same task.

Re: Overwrite map service with ArcPy but use existing service properties?

ZahirIbrahim — Wed, 10 Oct 2018 14:55:20 GMT

How do you do this? Ie. store settings as JSON and then reapply?

Re: Overwrite map service with ArcPy but use existing service properties?

JonathanQuinn — Wed, 10 Oct 2018 17:36:48 GMT

Yes, access the service properties through the Admin API first and store them in a variable, then once the service is published, reapply the service properties through the <service URL>/edit URL.

Re: Overwrite map service with ArcPy but use existing service properties?

ZahirIbrahim — Thu, 11 Oct 2018 16:09:56 GMT

Thanks for this. I am attempting this for the first time. Do you have a script for this, or an example i can use?

Re: Overwrite map service with ArcPy but use existing service properties?

Anonymous User — Tue, 21 Jan 2020 15:29:52 GMT

Thank you so much for sharing this.

Re: Overwrite map service with ArcPy but use existing service properties?

Anonymous User — Sun, 12 Dec 2021 03:12:36 GMT

Here's the script I made after following Jonathan's advice. This script iterates through all .mxd files in a given folder and publishes a map service with each document. I modified the script to get the map service properties as JSON via the Admin API. Then I modified the properties in .sddraft to match what came back from the Admin API request. This basically works the same as overwriting a map service while keeping the same service properties.

Some details specific to this script that you may have to modify:

This script only works with map services, not feature services or hosted feature services.
This uses the 'requests' module which you would have to add to your Python installation. There's other modules out there that do similar things if you want to avoid using this one. I use this module for HTTP requests because I like the easy-to-read syntax.
You may want to modify lines 78-85 (where it obtains the service properties from the Admin API) to first write the JSON to a file, then look at the output file in a text editor so you know where the properties you care about are. For example, min instances is not nested but schemaLockingEnabled is nested under 'properties'. I'm only overwriting the properties we have that are not default.
I've added an extra check to the part of the script that checks for publishing errors, warnings, and messages that will cause the publishing job to fail if the warning "24011: Data source is not registered with the server and data will be copied to the server" appears, because for us this usually means there's something wrong with one of the registered data sources in the map document.

Hope this helps!

import arcpy, glob, os, sys, json, requests
import xml.dom.minidom as DOM
from arcpy import env

# Obtain script parameters
svrName = arcpy.GetParameterAsText(0) #string indicating which environment to publish to (I have this as a dropdown in an ArcMap tool)
username = arcpy.GetParameterAsText(1) #Portal username
password = arcpy.GetParameterAsText(2) #Portal password
wrkspc = arcpy.GetParameterAsText(3) #Filepath to a folder where all .mxd files are
serverFolder = arcpy.GetParameterAsText(4) #The ArcGIS Server folder to publish to (I have this as a dropdown in an ArcMap tool)

if svrName == 'DEV':
    server_url = 'https://dev-server-machine.domain.com/arcgis/admin/'
    token_url = 'https://dev-portal-machine.domain.com:7443/arcgis/sharing/rest/generateToken'
elif svrName == 'TEST':
    server_url = 'https://test-server-machine.domain.com/arcgis/admin/'
    token_url = 'https://test-portal-machine.domain.com:7443/arcgis/sharing/rest/generateToken'
elif svrName == 'UAT':
    server_url = 'https://uat-server-machine.domain.com/arcgis/admin/'
    token_url = 'https://uat-portal-machine.domain.com:7443/arcgis/sharing/rest/generateToken'
else: #'PROD'
    server_url = 'https://prod-server-machine.domain.com/arcgis/admin/'
    token_url = 'https://prod-portal-machine.domain.com:7443/arcgis/sharing/rest/generateToken'

#Fix string if ArcGIS Server root folder is chosen
if serverFolder == '[root]':
    serverFolder = ''

#Set parameters for server connection file function
out_folder_path = wrkspc
con = 'tmp.ags'
use_arcgis_desktop_staging_folder = True
staging_folder_path = None

#Create the temporary server connection file
arcpy.mapping.CreateGISServerConnectionFile('PUBLISH_GIS_SERVICES',
                                            out_folder_path,
                                            con,
                                            server_url,
                                            'ARCGIS_SERVER',
                                            use_arcgis_desktop_staging_folder,
                                            staging_folder_path,
                                            username,
                                            password,
                                            'SAVE_USERNAME')

#Generate token to use in request to get service properties
requests.packages.urllib3.disable_warnings()
payload = { 'username': username, 'password': password, 'expiration': '60', 'referer': server_url, 'request': 'gettoken', 'f': 'json' }
r = json.loads(requests.post(token_url, data=payload, verify=False).content)
#Confirm token was generated
if 'token' in r:
    token = r.get('token', None)
else:
    arcpy.AddError('Error: Unable to generate token')
    sys.exit()

#Change directory to given folder
env.workspace = wrkspc
os.chdir(wrkspc)

#Iterate through all MXDs in the folder
for mxd_str in glob.glob('*.mxd'):

    #Access and print name of MXD
    arcpy.AddMessage('***Publishing {0}***'.format(mxd_str))
    mapDoc = arcpy.mapping.MapDocument(mxd_str)

    #Provide other service details
    filename = os.path.splitext(mxd_str)[0] #Removes '.mxd' from filename
    service = filename
    sddraft = filename + '.sddraft'
    sd = filename + '.sd'

    #Create service definition draft
    arcpy.mapping.CreateMapSDDraft(mapDoc, sddraft, service, 'FROM_CONNECTION_FILE', con, False, serverFolder)

    #Get service properties from ArcGIS Server to add back to sddraft
    if serverFolder == '':
        url = server_url + 'services/' + service + '.MapServer'
    else:
        url = server_url + 'services/' + serverFolder + '/' + service + '.MapServer'
    payload = {'f':'json',
                'token':token}
    r = json.loads(requests.post(url, data=payload).content)

    #.sddraft can be parsed like XML. Modify the .sddraft to add existing service properties
    xml = sddraft
    doc = DOM.parse(xml)
    keys = doc.getElementsByTagName('Key')
    for key in keys:
        if key.hasChildNodes():
            if key.firstChild.data == 'maxDomainCodeCount':
                key.nextSibling.firstChild.data = r['properties']['maxDomainCodeCount']
            if key.firstChild.data == 'antialiasingMode':
                key.nextSibling.firstChild.data = r['properties']['antialiasingMode']
            if key.firstChild.data == 'maxRecordCount':
                key.nextSibling.firstChild.data = r['properties']['maxRecordCount']
            if key.firstChild.data == 'schemaLockingEnabled':
                key.nextSibling.firstChild.data = r['properties']['schemaLockingEnabled']
            if key.firstChild.data == 'MinInstances':
                key.nextSibling.firstChild.data = r['minInstancesPerNode']
            if key.firstChild.data == 'MaxInstances':
                key.nextSibling.firstChild.data = r['maxInstancesPerNode']
            if key.firstChild.data == 'IdleTimeout':
                key.nextSibling.firstChild.data = r['maxIdleTime']

    #Output to a new .sddraft
    newSddraft = filename + 'New.sddraft'
    f = open(newSddraft, 'w')
    doc.writexml(f)
    f.close()

    #Analyze the service definition draft
    analysis = arcpy.mapping.AnalyzeForSD(newSddraft)

    #Print errors, warnings, and messages returned from the analysis
    badDataSource = False
    arcpy.AddMessage('The following information was returned during analysis of the MXD:')
    for key in ('messages', 'warnings', 'errors'):
        arcpy.AddMessage('----{0}----'.format(key.upper()))
        vars = analysis[key]
        for ((message, code), layerlist) in vars.iteritems():
            #Check for "24011: Data source is not registered with the server and data will be copied to the server"
            #Cannot publish if this warning appears
            if code == 24011 or code == 24012:
                badDataSource = True
            arcpy.AddMessage('    {0} (CODE {1})'.format(message, code))
            arcpy.AddMessage('       applies to:')
            for layer in layerlist:
                arcpy.AddMessage('          {0}'.format(layer.name))

    #Stage and upload the service if the sddraft analysis did not contain errors
    if (analysis['errors'] == {} and not badDataSource):
        #Execute StageService. This creates the service definition file.
        arcpy.StageService_server(newSddraft, sd)
        #Execute UploadServiceDefinition. This uploads the service definition and publishes the service.
        arcpy.UploadServiceDefinition_server(sd, con)
        arcpy.AddMessage('Service successfully published')
        os.remove(sd)
    else:
        arcpy.AddMessage('Service could not be published because errors were found during analysis.')
        os.remove(newSddraft)

    #Delete temporary files
    os.remove(sddraft)

    #Print any error messages that may have generated
    arcpy.AddMessage(arcpy.GetMessages())

#Delete temporary connection file
os.remove(con)

#Print success message
arcpy.AddMessage('All done!')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Overwrite map service with ArcPy but use existing service properties?

BenjaminBlackshear — Thu, 01 Jul 2021 20:19:56 GMT

Thanks, this is very helpful!

I am trying to modify it to use to migrate services from one server to another and it is mostly working but I have a question.

Is there an easy way to use all the properties of the existing service for the new service or do they have to be specified one by one?

Given the ease of importing the properties when publishing in arcmap it feels like there must be an arcpy equivalent to this.