ArcGIS GIS Server Health Check determination

Question

Hi,

I have a multi-machine ArcGIS GIS Server site (v11.5) providing critical geocoding and routing services. The site has a third party load balancer configured to send the traffic to one of the servers and has health check url configured as per https://developers.arcgis.com/rest/services-reference/enterprise/health-check/ .

Is it possible to enforce health check url to return {"success": false} if any of the critical services are not operational (stopped or hung) on that machine? I understand that there is a time penalty to link individual service(s) with health check determination but I am prepared for such a penalty.

Is it possible?

Otherwise, what logic is applied to determine ArcGIS Server is healthy? Is it a public information  / documented anywhere?

Cheers,

Vish

JoshuaBixby · Answer

The risk of doing a frequent, deep-level service health check is that you will never let a service fully spin down.  For example, if you had a dedicated service set to a minimum of 0 instances and maximum of 2 instances, a deep service health check (by deep I mean querying any data and not just looking at service REST info page) will force that service to spin up a instance/SOC.  If the health checks are frequent enough, it will always have to keep an instance/SOC running.

I worked for an organization once that used minimum instance 0 dedicated services to conserve resources for low usage GIS services.  Someone on the business side created their own service health check script that checked every service deeply all at the same time.  Effectively, the monitoring script because a form of service denial because it spun up every service, and the server was never resourced to handle having every service spun up at the same time.

Make sure your monitoring doesn't become one of the biggest consumer of compute resources on your servers.

VishApte_NGIS · Answer

Thank you all for your replies. It seems to me that linking individual service(s) with health check URL is not possible.

Only two options I can think of, use of the most critical service's metadata URL as a health-check URL as suggested by @A_Wyn_Jones  or create a geoprocessing service that checks required services availability (may be using admin API) and if all are determined to be operational, return {"success": true} like health check URL.

DerekLaw · Answer

Hi @VishApte_NGIS,

> Otherwise, what logic is applied to determine ArcGIS Server is healthy?

You could use ArcGIS Monitor, an extension to ArcGIS Enterprise - which provides observability by collecting metrics on the health, performance, and usage of your enterprise GIS. This includes ArcGIS Enterprise and the underlying infrastructure.

There are plenty of ArcGIS Monitor customer success stories on this page.

Hope this helps,