We're monitoring three ArcGIS GeoEvent Servers with ArcGIS Monitor 2023.0 since several weeks and are experiencing the following strange behaviour: the registered component (Type "ArcGIS GeoEvent Server") is disappearing in ArcGIS Monitor regulary (although the GeoEvent Servers are still running indeed).
.
We noticed two different scenarios (the observers are set to 15 minutes for Discover and Metrics):
- The GeoEvent Server component disappears and reappears 15 minutes later (occurs approx. 1 time a day)
- The GeoEvent Server component disappears continuous. Only after a restart of the "ArcGIS Monitor Server"-Service it reappears again (occurs approx. 1 time a week)
It's difficult to recognize any pattern. Sometimes only 1 of our 3 GeoEvent Server is affected, sometimes all 3 at the same time. The point in time seems randomly distributed over the whole day. It seems that the monitoring process is somehow not "robust" enough for any potentially temporarily network issues/timeouts. We've monitored those GeoEvent Server already with the old ArcGIS Monitor 10.8 - not having any similar problems.
Logs entries for scenario 1:
At the time of disappearing I found those entries for the observer of the ArcGIS GeoEvent
- Warning with Code ETIMEDOUT and Message "Run failed."
- Severe with Code null and Message "["Timeout awaiting 'lookup' for 200ms"]"
Logs entries for scenario 2:
At the time of disappearing I found those entries for the observers of the ArcGIS GeoEvent
- Warning with Code null and Message "Run failed."
- Severe with Code null and Message " ["Response code 403 (Forbidden)"] "
After restarting the "ArcGIS Monitor Server"-Service - without having changed anything in the configuration! - the components are reappearing again.
Has anybody seen any similar behaviour? Is it possible to increase the waiting timeouts for the observers?