Convert ArcMap published services (provider=ArcObjects) to PRO published services (provider=ArcObjects11) without re-publishing?

[pfoppe]

Hello, 

We have a 5 machine ArcGIS Server v10.6.1 site hosting ~200 services and relying on "clustering" to manage machine VM size and isolate services from each other.  Clusters have been deprecated at v10.7.x and we are assessing upgrade options (to v10.8.1). 

Each server runs in its own cluster and ~1/5 of the services are in each cluster (so each machine is hosting ~40 services).    ~20-25% of the services are 'active' daily, while the other ~75-80% of the services have sporadic use.  We believe the Esri "Shared Instance Pool" configuration is a candidate for the majority of the services on the stack. 

Problem is, this environment was built in JULY 2010 (literally within 30 days of ArcGIS Server v10.1 general availability) and ALL services published were from ArcMap.  So... per the docs, it seems we would have to re-publish all services using ArcGIS PRO to take advantage of the new "Shared Instances" model.  

We have been looking into developing scripts to mass convert ArcMap documents (MXD) to ArcPRO projects (APRX), then re-publish with an 'overwrite' option so that we can take advantage of the shared instances.  

As part of this effort, we discovered that we can just make Admin API calls to 'changeProvider'.  More specifically, it appears that an "ArcMap" published service sets the provider to "ArcObjects" while an ArcGIS PRO published service sets the provider to "ArcObjects11".  A service running in the shared instance pool has a provider=DMaps. 

We were unable to directly update the provider from ArcObjects->DMaps (and vise versa DMaps -> ArcObjects), however...

we were successful in making an API call to

1) convert provider=ArcObjects -> ArcObjects11

2) then a second API call to convert ArcObjects11  -> DMaps.  

As part of that, we did set the min/max instances to 0 (since thats what the 'manager' GUI did as well when manually converting a "PRO Published" service to the shared instance pool.  

We completed basic business testing (map export, query, identify, dynamic layers, etc) and have found no adverse impact yet.  We are able to switch ArcMap->Pro->DMaps (and back in same direction) and can see the new ArcSOC.exe "command" path on the server change accordingly.  

So....  how viable of an approach is this?  I suspect Esri support would indicate this is not a supported workflow (as my research has yet to find this) but so far seems completely functional. This conversion would allow us to more quickly support a v10.8.1 upgrade, although there is still value in getting our ArcMap documents converted to PRO for future publishing support (if we have to make changes to existing services and re-publish). 

Thanks for any info/advise you may have.  

Good Resources - 

• Esri API Doc - "Change Provider" - https://developers.arcgis.com/rest/enterprise-administration/server/change-provider.htm
• Esri Doc - Configure Instance Settings - https://enterprise.arcgis.com/en/server/latest/administer/linux/configure-service-instance-settings....
• Esri BLOG - Move services to shared instances using python - https://www.esri.com/arcgis-blog/products/arcgis-enterprise/administration/move-services-to-shared-i...
• Esri GeoNet Post - Switch from Dedicated to Shared - https://community.esri.com/t5/arcgis-enterprise-questions/how-to-switch-all-services-from-dedicated-...

[pfoppe]

Bump.  

Any thoughts on this?  Thanks!

[pfoppe · Accepted solution]

I did have an opportunity to meet with some Esri colleagues and figured I would sum up the sentiment here - 

• The workflow is viable as long as we follow the vendor limitations.  EX -
  • only mapping/feature services with things like KML/WMS and even SOE/SOI are supported. 
  • No Image services, GP services, geodata services, etc
    
    
• We should expect to receive vendor support if we ran into issues given that the product manager (@pheede-esri)  and product engineer (@Anonymous User) crafted the referenced BLOG article and github hosted "sharedinstances.py" script
  
  
• There is still value in mass converting (or manually rebuilding) the ArcMap (.mxd) documents used to publish the services into an ArcGIS PRO project (.aprx).  
  • We are actively testing this out in our environment and will try to post back with experiences if I remember/have time

In addition to those, we are still trying to fully understand the shared instance model and implications for management on our systems.  Key areas we know some about but still trying to fully understand: 

• "Caching Service Configurations in Memory" -
  • How much 'memory' is used to 'cache' the service configurations?  How much is too much, and what is a sustainable value (defaults to 50)
  • Does the service get cached in all instances in the shared pool, or just the (1) that received the first request?
  • How long does this take to 'cache' the service in memory?  
    Note - we've done some initial testing and consistently saw an additional 1 second added to the first request response time..  subsequent requests ran faster.  
    
    
• "Service Candidates" - 
  • What is the criteria to leverage shared instance?   Or... alternatively, what criteria should indicate we should move back to 'dedicated' instances?
    • "Less than XX hits per YY time"?
      Rational: If the service is frequently accessed, move to dedicated instances
    • "Average response time less than AA per YY time"?
      Rational: If the service API calls are 'complex' (ex - lots of data, slow response times, etc) the run dedicated so it does not impact other services in the shared pool
  • How do we track service usage and response times?
    • Most likely through our web-server (ex - IIS) access logs correlating to the services in the shared instance pool. 
    • Then correlating to 'wait time' below
      
      
• "Wait Time" -
  • We have observed 'wait time' metrics (ArcGIS Server "fine" level logging) to be reported on the 'dynamic mapping host' and NOT on the actual services themselves (for those services running in the shared instance pool).  
    • I suspect that wait-time would be experienced on ALL services in the shared instance pool if under load
  • I think 'wait timeouts' were reported on the services in the shared instance pool, not the dynamic mapping host (needs to be confirmed still)

Hope some of this helps.  Thanks!

[pfoppe]

Well, quick (disappointing) follow-up to this.  

I met a few Esri colleagues at the Esri virtual Dev Summit this year, and the general concensus was that it should be ok to change the provider with the methods outlined above, however, We ended up opening an Esri support case RE: an issue in ArcGIS Server where 'export map' API calls are returning a successful HTTP 200 response, but an empty (blank) image. 

In some of these sites we have dabbled with this shared instance pool configurations.  I do not think changing out the 'provider' to arcmap<-arcgis pro->shared instances (forth/back) this is a root cause as it is happing in a few other sites that have not been touched, however Esri supports default answer was: 

Unfortunately, the only supported workflow at this time is to republish from ArcGIS Pro. 
...
Further, my point of contact did confirm that later versions of ArcGIS Enterprise will offer a workflow to support this, so it may be best to wait for a future iteration, if this is not time-sensitive.

So... even though the referenced blog article was authored by the product manager (@pheede-esri)  and product engineer (@Scott_M_MacDonald), Esri does not actually "support" this method.  

Use with caution....

I'll update this thread if Esri support decides to change their tune (or any new findings).  

Thanks!