topic Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests? in ArcGIS Enterprise Questions

How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JamalNUMAN — Sat, 01 Jun 2019 14:59:25 GMT

How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

I couldn’t figure out if there is a mathematical equation that relates the number of min\max instances (ArcSoc.exe) for a particular map service with the number of requests. In other words, is the number of min\max instances (ArcSoc.exe) a function of its number of requests?

Y=f(X), where:

Y: the number of min\max instances (ArcSoc.exe) for a map service

X: the number of requests

For example, if the number of requests for a particular map service is 100, what could be the best choice for the number of min\max instances (ArcSoc.exe)?

Thank you

Best

Jamal

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

ThomasJones1 — Mon, 03 Jun 2019 12:47:19 GMT

Hi Jamal,

The number of ArcSOC.exe is controlled by the minimum and maximum instances defined for a service. If necessary ArcGIS Server may spin up additional ArcSOC.exe to accommodated more requests.

If you encounter performance issues for a high use service you may want to increase the number of maximum instances. In general I would recommend limiting the maximum instances to n +1 (n = number of cores).

For example in a single machine site with a four core machine my maximum number of instances would be five.

I've provided links to documentation below which talks about ArcGIS Server instances and tuning and configuring service performance.

ArcGIS Server - Tune and configure services

ArcGIS Server - Configure service instance settings

If you have any additional question please let me know!

Thanks,

Thomas.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JamalNUMAN — Mon, 03 Jun 2019 14:50:49 GMT

Many thanks Thomas for the help.

I got confused.

1. In the screenshot below, I have a test machine of 4 cores. Does this mean that total number of instances (ArcSoc.exe) the machine can take is 5? Or the 5 is the maximum number a single service can take?

2. What is the difference between the instance and ArcSoc.exe in this sense? Aren't they the same?

3. To calculate the optimal number of instances (ArcSoc.exe) the service needs in order to respond to the requests, the only known value (from the statistics of ArcGIS Server Manager) is the number of requests. This suggests that there should be a mathematical formula by which one calculate the optimal number of instances (ArcSoc.exe). am I correct?

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

ThomasNoble — Mon, 03 Jun 2019 17:38:41 GMT

Hi Jamal,

- In this scenario, the maximum recommended instances per service would be 5. Some 4 core environments are able to handle a higher number of max instances per service, though some environments might require the max instances to be set at N-1 to achieve the desired performance. You'll have to play around with these settings and test to see what works best for your organization.

2. What is the difference between the instance and ArcSoc.exe in this sense? Aren't they the same?

- The minimum number of instances controls how many ArcSOC.exe processes will continuously run, while the maximum number of instances controls how many additional ArcSOC.exe processes can be spun up to support incoming requests.

- The System Design Strategies Wiki and Capacity Planning Tool contain a lot of information that will help you determine your hardware needs based on system throughput rates. Implementing ArcGIS‌ would also be another good resource.

Cheers,

Tom

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JamalNUMAN — Mon, 03 Jun 2019 20:18:16 GMT

Thank you Thomas for the input. This is very useful.

1. If (n+1) is recommended for the maximum number of ArcSoc that can be allocated for a single service, then how about the maximum number of ArcSoc the machine can take for all services? is there math formula for this?

2. All are services treated equally without considering the content of the service:

i) The number of layers contained in the service

ii) The type of layers contained in the service (point, line, polygon)

iii) How heavy the layers contained in the service (is the layer that contains 1000 features is treated as the one that contains 100,000)

3. The documentations on this topic is very general and don’t answer some very specific questions like the ones highlighted in this post

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

ModyBuchbinder — Tue, 04 Jun 2019 05:34:15 GMT

The possibilities here are endless so it is impossible to create formulas.

As usual, trying is the best way. In the statistics of the server you can create a report of maximum of running instances per service.

If your service get to the maximum often you should think about increasing the maximum.

The way the SOC is using the resources is very different. it depend on number of layers, the data source (shape file on SSD is quicker then database), the network and other things.

Mody

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JamalNUMAN — Tue, 04 Jun 2019 05:48:48 GMT

This is very useful.

How about the maximum number of ArcSOC.exe as server can take?

For example, if a server has 32 cores, then how about the total number of ArcSOC.exe that can be allocated for all services published in that machine?

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JonathanQuinn — Tue, 04 Jun 2019 16:38:41 GMT

I agree with Mody Buchbinder, these are somewhat impossible questions to answer. I assume you're talking about map services. You'll run into an absolute limit against RAM and non-interactive desktop heap size rather than CPU. If you have all of the cores in the world but limited in RAM, you'll quickly run into the upper limit of the number of services, (and really, processes), running on the machine. That's a hard limit. CPU is a bit more forgiving: you'll run into performance problems, timeouts, probably instability with the whole Server site.

In general, theoretical questions like yours have complicated answers as there are far too many variables. Run load tests on your site using JMeter, RoadRunner, or any other load testing software, (even a multi-threaded Python script can give you a minimum baseline) with your expected number of users and usage to understand how to plan for capacity and service throughput. Shared instances at 10.7 will greatly assist in minimizing the memory throughput of services.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

AhmadSALEH1 — Wed, 05 Jun 2019 18:08:16 GMT

Hi Jamal,

Have you heard about "ArcSOC Optimizer" this might be what you're looking for. It helps you to optimize the Min/Max instances for a service based on the historical usage and available memory on your server.

here is the link:

https://www.arcgis.com/home/item.html?id=a0c2dde661f14b8ca58b8b7b5ddeb379

There is no direct mathematical formula that you can use to setup the Min/Max process, all what you can do is to keep tuning and optimizing the instances to reduce the number of times that a user waits in the queue in order to use that service.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

ThomasColson — Thu, 06 Jun 2019 14:15:46 GMT

We have found 250 instances to be the max, regardless of ram/cores. That could be 5 services each with min instances set to 50, or 25 services with min instances set to 10. Coincidentally, this applies to a multi-machine site as well. 3-machine cluster? Anything more than 25 services with min instances set to 10 and it starts wobbling.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

Subu_Swaminathan_GISP — Thu, 06 Jun 2019 15:19:44 GMT

Concur. ~200 services with the default 1 min instance is the max number of services we have seen ArcGIS Server be stable at.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JonathanQuinn — Thu, 06 Jun 2019 15:23:01 GMT

You may be running into a limit with the non-interactive desktop heap:

https://support.esri.com/en/technical-article/000001218

However, I think Subramanian Swaminathan‌ is correct, there may be stability issues as you force Server to support hundreds of services. I believe that shared instances introduced at 10.7 is a solution for that, though.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JoshuaBixby — Thu, 06 Jun 2019 15:52:01 GMT

Prior to 10.6.x, I would agree with you. Starting with the changes Esri made in 10.6.x and continued with 10.7.x, we are able to push the number of services much higher.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JamalNUMAN — Sat, 08 Jun 2019 08:46:23 GMT

Many thanks guys for sharing the experience. Very much appreciated

I thought that there should be direct association between the load (map services) and machine resources (cores/ram)

Thanks Ahmad. We will be considering the “ArcSoc Optimizer Add-on” and find out if it brings a good tool to design the best fit between the load and resources. However, this tool is very costly as I got.

Thanks Jonathan. We will be configuring the Registry Editor as it is recommended. We will provide our feedback. We need also to better explore the concept of the “shared instances” of the 10.7

Thanks Thomas. I’m still wondering how the total number of ArcSoc.exe assigned for min\max for all services in a machine is not related to the recourse. Does a machine of 4 cores works the same as a machine of 32 cores?

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JonathanQuinn — Tue, 11 Jun 2019 16:17:01 GMT

Assuming no load or requests are making it to the Server, the number of services you can publish to a 4 core and 32 core machine should be the same as long as the machines have the same amount of RAM. There's a bit of internal overhead for managing services and more cores will handle that better, but in general, it should be roughly the same.

Once the machine starts taking requests, you'll need to monitor the usage and resources and make adjustments as necessary.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

MajdoleenO_A__Awadallah — Tue, 10 Sep 2019 10:56:43 GMT

Thank you Jonathan Quinn for your reply, this is what we have done before (monitoring), but now with the sharing instances option in Arcgis 10.7.1, we think it is a good approach, but why not to have the sharing option at the level of the site and not per service?y

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

AhmadSALEH1 — Tue, 10 Sep 2019 14:57:32 GMT

I think you can change the behavior by

Browse to Site > Settings. You'll access these settings from the Pooling page.
To change the default instance type for compatible map services, click the edit button for Default Instance Type. Select the instance type you want to have be the default setting:
- Select Dedicated instances if you anticipate that most of your services will receive frequent traffic, or if you will not have enough services running in your server site to present a significant load on machine memory.
- Select Shared instances if you publish many services that only receive infrequent requests, or if you are concerned about additional services being published that will strain your memory resources.

https://enterprise.arcgis.com/en/server/latest/administer/windows/configure-service-instance-settings.htm

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

Mehretab — Wed, 11 Sep 2019 10:52:20 GMT

The number of services (ArcSoc's to be precise) in your machine depends on your server resources (Particularly Memory). But in Windows environment there is limitation to run more than 200 process at the same time. you might need to tune your services based on their priority. I have around 330 ArcSoc's concurrently running in Linux and had no complaint from the server as of yet.

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

MajdoleenO_A__Awadallah — Sun, 15 Sep 2019 07:19:19 GMT

Thank you Ahmad for your reply, do you mean that changing the default instance type per site to “shared instances” will affect all the services, and all the services will share the number of instances? and in this case we don’t need to apply the pooling setting per services as shown below.

Best

Majdoleen

Re: How to determine the number of min\max instances (ArcSoc.exe) for a particular map service based on its number of requests?

JamalNUMAN — Tue, 25 Aug 2020 09:23:28 GMT

How the number of arcsoc.exe (instances, processes) is associated to the machine resources (cores, ram)?