ArcSOC Optimizer for ArcGIS Monitor

[AndrewSakowicz]

Overview

ArcSOCOptimizer is a python command line utility.  It can help administrators:

1. Reduce memory by identifying low usage services with dedicated pooling and reducing the overall number of arcsoc.exe processes:
   1. changing dedicated to shared instance pool, if supported  
   2. reducing min instances for dedicated instance pool services
2. Improve performance by identifying high usage service and allocating dedicated arcsoc.exe processes if available memory:
   1. changing shared to dedicated instance pool
   2. increasing min instances for dedicated instance pool 

Get started

To run using provided executable for windows, unzip ArcSOCOptimizer.zip from

https://prof-services.maps.arcgis.com/home/item.html?id=f9c3e2864054444fb87e54575b11a96a

and execute from command line:

ArcSOCOptimizer.exe -f SampleConfig.json

SampleConfig.json

edit:false/true

False executes in read only and produces a list of recommendation.  Start with False, review and validate the results.  Once validated, execute true. 

past_days:number     

Number of past days to be analyzed to determine low and high usage.

The recommendations are based on historical data.  You may choose to increase the 7 day default.

avg_sec/day:float       

Average number of seconds per day a service is used throughout all past_days.  For example, if during the business hours service is used 10 seconds per day and 0 seconds during the weekend, the avg_sec/day=avg (10,10,10,10,10,0,0).  This can be evaluate using the combination req/s and response time, see below

shared_to_dedicated: true/false      

Allows or disallows low usage services to be moved from dedicated to shared.

dedicated_to_shared: true/false      

Allows or disallows high usage services to be moved from shared to dedicated.

host_address:array

Each arcgis server machine should be registered with ArcGIS Monitor as a host component.  This parameter takes a list of hosts (in quotes), with the names exactly as the Address of registered host, see below. 

Use case

Always test in your non-production environment first.

edit=false

Start with edit=false (default).  This will generate a report with recommendations, without making any changes. Review the recommendation and change the inputs if needed, especially avg_sec/day for low and high usage.  Repeat the process as needed.

For example, in the following use case we have the following 4 services:

1. ID 1 and ID 2 have a low usage (<5s) and already recommended shared pool.  Therefore, there is no change.
2. ID 3 has a low usage (<5s), but dedicated pool. Therefore, pool change from dedicated to shared is recommended.
3. ID 4 has a high usage and shared pool.  Therefore, pool change to dedicated and min/max instances are recommended

edit=true

Once you validate the edit=false recommendation, test executing edit=true manullay.  This mode will make changes to instances and pooling and restart services.  If satisfied, user could schedule this as a daily job, using windows scheduled task.

Note, the minProposed is calculated as the ceiling of Instances Used Avg.  In the case below, the ceiling of 1.21 is equal 2.

Memory limit

If memory available is less then specified memory_available_GB limit, the edit will not be executed.  In that case, user should lower this limit or free up memory manually.

Note, this utility is not an ArcGIS Monitor product supported feature or a supported Esri product.

[AndrewSakowicz]

@ChrisBerryman , @ChrisTaylor2 , to clarify the proposed instance information is per machine, just like you should see in ArcGIS Server Manager.  If you see differently, please provide examples.  As far as float, thanks for pointing this.  This float was on purpose so user would know how much over the instance is.  I can change to integer in the code the next time if it helps.  In the meantime, you can use Excel to do this.

[Craig_Lunde]

@AndrewSakowicz No matter what url I try I get the same alternating errors. Nothing has changed in the address from my upgrade to 2024.0.1.

[Craig_Lunde]

@AndrewSakowicz any suggestions?

[AndrewSakowicz]

@Craig_Lunde , per doc,

in your case, it appears there is a difference:

...local:6443/ - component address in ArcGIS Monitor

...local:6443/arcgis in the config file (change to as above: ...local:6443/)

I hope it helps.

Thanks,

Andrew

[Craig_Lunde]

@AndrewSakowicz Thanks! I also had to switch username/password from server to portal.

[René_Ténière]

Hi @AndrewSakowicz 

Any chance you will be able to release the Python version like you did for v2023? I had added the ability to change reporting location and added email functionality... I can also send you my code to incorporate if you prefer.

[ChrisTaylor2]

Hi @AndrewSakowicz,

I had noticed my ArcSOC Optimizer reports hadn't been working since I upgraded to 2024.1, but hadn't been able to take the time to troubleshoot until now. I saw your comment from the Excel Report tool saying it's not expected to be able to run against 2024.1 - is the same true for ArcSOC Optimizer? 

If not, I'm getting the same E_VALIDATION_FAILURE error mentioned elsewhere in the thread:

04/22/2025 11:27:43 AM INFO: observer run {"success":false,"error":{"code":"E_VALIDATION_FAILURE","message":"Validation Exception","details":["id: required validation failed on id"]}}

While the tool does complete successfully afterwards, all of the subsequent calls after the error are returning 0/None values:

04/22/2025 11:27:49 AM INFO: <service name> requests_received 0
04/22/2025 11:27:49 AM INFO: <service name> instances_used_avg0
04/22/2025 11:27:49 AM INFO: <service name> requests_response_time_avg None
04/22/2025 11:27:49 AM INFO: <service name> rt_avg is None, setting to 0

I've tried changing some of the parameters in my config file but none have made a difference so far.

Thanks as always for your help.

Chris

Edit: This is looking like it may be an ArcGIS Monitor problem, not an ArcSOC Optimizer problem. My ArcGIS Server sites are showing request totals, but my individual services are showing all zeros for some reason. I'll investigate and follow up with Support.

[JoeGuziStarkCountyOH]

Greetings,

I would also like to request that this tool to be updated to work with the latest version of AGM (2024.1.1+704). It is very useful and has become part of our routines and procedures. Thank you for everything and have a delightful day.

Sincerely,

Joe Guzi

[AndrewSakowicz]

I'll look into refactoring the code so is less dependent on the monitor changes.

[JoeGuziStarkCountyOH]

Greetings @AndrewSakowicz ,

I'm sorry but my curiosity got the best of me. Would you, by any chance, have an update on this? Thank you for everything and have a delightful day.

Sincerely,

Joe Guzi

[AndrewSakowicz]

Joe, to clarify, did this tool stop working for you after upgrading ArcGIS Monitor? What errors do you see?

[AndrewSakowicz]

Joe,
Did this tool stop working for you? What errors do you receive?
Thanks,
Andrew

[JoeGuziStarkCountyOH]

Greetings @AndrewSakowicz ,

That is correct. When I upgraded to AGM version 2024.1.1+704, the ArcSOC Optimizer Stopped working. Below is the error. I just figured I would check in since you mentioned refactoring the code on 4/28/25, and it appears that the item has not been updated since 10/2/24. Please let me know if there is anything else I can do to help. Thank you for everything and have a delightful day.

Sincerely,

Joe Guzi