Since upgrading to 2025.0, I'm seeing a number of Instances Saturation Percent critical alerts for shared services. It appears that, if a request is received by one of these services, the Instances Saturation Percent metric goes to 100% for that interval. From what I understand, on a shared service, there are no configured service instances being used. The Instance Limits are "0 min - 0 max"
Is this normal or expected behavior? It is the cause of most of my Critical-level alerts, and I would like to minimize that.
As an example, here is the Instances Saturation Percent metric on one particular service
And here is the metric for Requests Received over that time:
And here is the metric showing Instances Used.
The intervals in which requests were received show an Instance Saturation of 100%, and an Instances Used metric of 0. You can see how they all line up in the charts. It seems like services with shared instance types should not be able to trigger this Critical alert. If I'm missing something, please let me know. Thanks.
Solved! Go to Solution.
Hey @JustinJ_DTS
It appears that this is a bug according to this post: https://community.esri.com/t5/arcgis-monitor-questions/instance-count-for-shared-services-after-arcg...
Here is the slated workaround offered by the answering user:
"
I went into ServerAdmin and just changed the max maxInstancesPerNode value to 1 and left the minInstancesPerNode at 0 and am now getting small percentage sat values (like 0.45, 0.56, etc) for these services in the shared instance:
Configured State
Started
Instance Type
Shared
Instance Limits
0 min - 16 max
"
I would raise an issue with support and bring more attention to this bug as well!
Cody
Hey @JustinJ_DTS
It appears that this is a bug according to this post: https://community.esri.com/t5/arcgis-monitor-questions/instance-count-for-shared-services-after-arcg...
Here is the slated workaround offered by the answering user:
"
I went into ServerAdmin and just changed the max maxInstancesPerNode value to 1 and left the minInstancesPerNode at 0 and am now getting small percentage sat values (like 0.45, 0.56, etc) for these services in the shared instance:
Configured State
Started
Instance Type
Shared
Instance Limits
0 min - 16 max
"
I would raise an issue with support and bring more attention to this bug as well!
Cody
Thanks, Cody,
I looked into your links and suggestion and I tested the workaround. It seems to be a fix, but not an entirely stable one.
I went to the Server Admin Directory and clicked to the shared service that I showed earlier. I stopped it, changed the maxInstancesPerNode to 1, then restarted it from the Server Admin. The change appeared in the Server Admin JSON as 1. In Monitor Server, it now shows "0 min - 16 max". The Pooling for this server has a max of 16, so that makes a bit of sense. However, in Server Manager, it still shows "min 0 - max 0".
After the change, the Instance Saturation Percent metric dropped to 0.00 and then 0.93 and 0.01 and the Alert was gone.
Everything was looking good, but when I went to Server Manager and clicked the "Save and Restart" button, it reset the Max back to 0. Server Manager never displayed a Max other than 0, so saving it there probably resets it in the Admin Directory. It's working for now, but I don't know how it will handle a restart of the entire server. Seems like a definite bug.
Good to know I'm not the only one experiencing this.