Generate an alert when accessing a stopped service

[Lerman]

Imagine a common scenario: a map service has stopped, perhaps due to some accident. However, the application is still using the service, and when it accesses the service, it gets a request response of 500:[ Error: Service XXX/MapServer not started Code: 500.]  ArcGIS Server generates a log with the warning level:[ Unable to process request. Service XXX/MapServer not started.]  In this case, I think ArcGIS Monitor should generate an alert so that the administrator can check the status of the service in time to avoid serious consequences. Surprisingly, ArcGIS Monitor does not generate any alerts for this case. （Is this correct? Is there an additional setting that I didn't notice?)

I tried to analyze this problem, and I think there are two reasons, the first reason is that ArcGIS Monitor doesn't count this kind of failed requests as failed requests, so even though I added the Request Failed(req) Metrics, it still didn't generate any alert. The second reason is that ArcGIS Monitor does not generate alerts for ArcGIS Server warning level logs. I know that earlier versions such as ArcGIS Monitor 10.8.1 could generate alerts for different levels of logs, however, for version 2023.1, I did not find such a setting.

Any suggestions please? I would like ArcGIS Monitor to be able to generate alerts for this situation.

Thank you!

[IrisHadarStltd]

Hi Josh,

I am using the 2024 version and couldn’t find the new "disconnected" or "unreachable" status. Could you help me locate these statuses, or suggest a solution for setting up alerts when a service stops or has severe logging issues?

[Amarz]

I believe those settings are in the Analysis, not Notifications tab.

[Lerman]

Hello @GeoJosh ,

Thank you Josh! The solution you provided requires the use of a webhook right? I see in the help documentation that webhook supports geoprocessing and feature services, other service types are not supported. Thank you!

[GeoJosh]

@Lerman, Can you point me to what documentation you are referring to?

[Lerman]

Hi @GeoJosh , It's this document: https://enterprise.arcgis.com/en/portal/latest/administer/windows/about-arcgis-webhooks.htm

[GeoJosh]

@Lerman, Thank you for clarifying.

Enterprise webhooks are different from ArcGIS Monitor webhooks. Webhooks in ArcGIS Monitor are one of two ways a notification can be delivered (the other being via e-mail). Notifications can be set on a variety of different resources in Monitor, including registered components and metrics. This means webhook notifications in ArcGIS Monitor are not limited to specific service types. A webhook payload will be emitted whenever the notification rules you define are triggered.

Please see this documentation for more on notifications and this documentation for more on the webhook payloads that ArcGIS Monitor emits.

Josh

[Lerman]

Thank you @GeoJosh In arcgis monitor, it's all about asking me to fill in the url of the webhook, and I can't find any information about creating a webhook in the help of the arcgis monitor.

[GeoJosh]

Hi @Lerman,

The webhook URL would come from the platform that you choose to implement with (e.g. Microsoft Teams, Slack, Integromat). Since there are many different platforms on which webhooks can be processed, this is not covered in the ArcGIS Monitor documentation.

Josh

[Lerman]

Hello @GeoJosh ,

I'm having trouble finding articles on creating a webhook and using it together with a monitor. Do you have any resources on your side? The monitor supports webhooks, but there are few helpful instructions with detailed steps, which really increases the difficulty of use. Through Esri Support, I found this article (https://support.esri.com/en-us/knowledge-base/how-to-create-a-webhook-with-make-which-sends-an-email...). Is this related to ArcGIS Monitor WebHook? I don't think it is. Thank you Josh!

[PhilLarkin1]

Consider voting for this idea:
https://community.esri.com/t5/arcgis-monitor-ideas/alert-on-the-status-of-a-service-stopping-startin...