Community
Select to view content in your preferred language

Monitoring a registered web server's service discovery

1836
0
11-05-2020 05:40 PM
RJSunderman
by Esri Regular Contributor
Esri Regular Contributor
6 0 1,836
‎11-05-2020 05:40 PM

In this blog we will take a deeper look at registering server connections with GeoEvent Server, something administrators commonly have to do to when configuring a GeoEvent Server deployment. 

Why do you have to register an ArcGIS Server connection with GeoEvent Server

There are several different configurable components which require you to select a registered server and specify which services folder, map/feature service, and often a specific feature layer exposed by that service in order to use the feature layer's schema or feature records. A few examples include:

All of the user-interfaces in the above examples populate their options from a cache of information GeoEvent Server collects when it queries a registered ArcGIS Server to discover published feature services. Service discovery is not performed the moment you click to open the panel. The cache is created and updated in the background because it can take several seconds – sometimes minutes or even tens of minutes – for GeoEvent Server to completely crawl the ArcGIS Server's REST Services Directory and query the information it needs from all of the available services.

Which leads into the topic I want to address in this blog:  Is there a way to know when service discovery is being run and how long it is expected to take?

Using the ArcGIS Server Connection component to log messages  

You can configure the following component logger to request DEBUG messages be logged:

com.esri.ges.datastore.agsconnection.DefaultArcGISServerConnection

I've included a sample of the logged messages you will be looking for at the end of this article. Requesting this component logger include DEBUG messages in its logging will allow you to see, in the system log file, when the service discovery kicks off and which map/feature services it interrogates to learn about their layers. A message will be logged for every feature service being interrogated as well as a success message when service discovery is complete. There is no indication in the GeoEvent Manager web application that service discovery is about to start or is currently running. The best way to tell that service discovery is running is to start the workflow to import a GeoEvent Definition. If the blue/white indicator displays, requesting you "please wait", you know that the GeoEvent Server is busy updating its ArcGIS Services cache. Otherwise the GeoEvent Definition user-interface will display immediately allowing you to choose a server connection, folder, feature service, and feature layer.

GeoEvent Manager does not allow you to configure, or schedule, when service discovery should take place. You are able to change the Discovery Rate for each server connection you register to specify how frequently a refresh should be performed. The default for recent software releases is every 60 minutes. I have personally found that when I use ArcGIS Pro, the Enterprise portal, or GeoEvent Manager to publish a new feature service, I want to use it now – so it is not uncommon for me to publish a feature service and immediately request GeoEvent Server run a service discovery to update its cache by clicking the refresh button on the server connection I have registered as a Data Store. This eliminates the need to have GeoEvent Server periodically refresh the cache for me, so I usually set the Discovery Rate for server connections I register to a fairly large value like 1440 minutes so that service discovery is run once once per day (or when GeoEvent Server is stopped and restarted).

Service discovery takes too long and interferes with normal operations. Is there anything I can do?

This is something the product team is working on. Refactoring GeoEvent Server to support all of the operations which use feature services, however, to interface with ArcGIS Server some other way is both high risk and high reward. Several different design options have been considered, but implementation has had to be deferred for each of the last several major releases. The best option for now, therefore, is to limit the number of features services which are discoverable each time service discovery is run.

A recommended best practice is to configure your GeoEvent Server Data Store (e.g. the server connection you are registering with GeoEvent Server) with credentials. The user credentials do not have to be an administrative user, just a user who owns and published the feature service. You can use the Enterprise portal content item manager to assign existing feature services a new owner and configure GeoEvent Server to authenticate as that user when crawling the ArcGIS Server's REST Services Directory to limit the number of discoverable services. GeoEvent Server administrators often configure their registered server connections with the ArcGIS Server or Enterprise portal primary administrative account – which naturally sees all published services.

If you can identify the feature services to which you want GeoEvent Server to write data, or from which you want GeoEvent Server to retrieve feature records or a feature layer's schema, and assign ownership of just those services to a user set aside specifically for your "real-time" data, you can improve service discovery considerably by not crawling all of the feature services being maintained by more traditional feature editing workflows. The trick is to identify the feature services your GeoEvent Server components actually care about and limit discovery to only those feature services.

Example messages logged by the ArcGIS Server Connection component logger

2020-11-05T13:55:02,124 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | sleep interrupted
java.lang.InterruptedException: sleep interrupted
        at java.lang.Thread.sleep(Native Method) ~[?:?]
        at com.esri.ges.datastore.agsconnection.DefaultArcGISServerConnection$CacheUpdater.run
(DefaultArcGISServerConnection.java:235) [56:com.esri.ges.framework.datastore.agsconnection-datastore:10.8.1]
2020-11-05T13:55:02,124 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Exiting Cache Updater run method....
2020-11-05T13:55:03,274 | DEBUG | qtp808353329-598 | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Create an instance of CacheUpdater....
2020-11-05T13:55:03,274 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating cache for DataStore Public_Esri_Hosted_Server...
2020-11-05T13:55:08,944 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Active_Hurricanes_Sampler", Type: "FeatureServer").
2020-11-05T13:55:15,296 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Active_Hurricanes_v1", Type: "FeatureServer").
2020-11-05T13:55:21,613 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Air_Quality_PM25_Latest_Results", Type: "FeatureServer").
2020-11-05T13:55:23,311 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "ASAM_events_V1", Type: "FeatureServer").
2020-11-05T13:55:25,033 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Coral_Reef_Stations", Type: "FeatureServer").
2020-11-05T13:55:27,876 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "GDELT_Health_Pandemic", Type: "FeatureServer").
2020-11-05T13:55:29,170 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "GDELT_v1_Social_Tones", Type: "FeatureServer").
2020-11-05T13:55:30,368 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "IHME_Projected_Peaks_QA", Type: "FeatureServer").
2020-11-05T13:55:31,995 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Median_Sea_Ice_Extent_for_the_Antarctic", Type: "FeatureServer").
2020-11-05T13:55:32,995 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Median_Sea_Ice_Extent_for_the_Arctic", Type: "FeatureServer").
2020-11-05T13:55:34,063 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "MODIS_Thermal_v1", Type: "FeatureServer").
2020-11-05T13:55:35,225 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "National_Farmers_Market_Directory", Type: "FeatureServer").
2020-11-05T13:55:36,445 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "ncov_cases", Type: "FeatureServer").
2020-11-05T13:55:38,421 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_Ice_v1", Type: "FeatureServer").
2020-11-05T13:55:40,579 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_Precipitation_v1", Type: "FeatureServer").
2020-11-05T13:55:43,479 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_SnowFall_v1", Type: "FeatureServer").
2020-11-05T13:55:45,920 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_WindForecast_v1", Type: "FeatureServer").
2020-11-05T13:55:49,935 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_WindGust_v1", Type: "FeatureServer").
2020-11-05T13:55:51,519 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_WindSpeed_v1", Type: "FeatureServer").
2020-11-05T13:55:53,124 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDGD_SmokeForecast_v1", Type: "FeatureServer").
2020-11-05T13:55:54,193 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NOAA_METAR_current_wind_speed_direction_v1", Type: "FeatureServer").
2020-11-05T13:55:55,683 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NOAA_short_term_warnings_v1", Type: "FeatureServer").
2020-11-05T13:55:58,401 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NOAA_storm_reports_Sampler", Type: "FeatureServer").
2020-11-05T13:56:00,844 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NOAA_storm_reports_v1", Type: "FeatureServer").
2020-11-05T13:56:06,494 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NWS_Watches_Warnings_Sampler", Type: "FeatureServer").
2020-11-05T13:56:12,942 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NWS_Watches_Warnings_v1", Type: "FeatureServer").
2020-11-05T13:56:19,622 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Recent_Hurricanes_v1", Type: "FeatureServer").
2020-11-05T13:56:21,726 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Satellite_VIIRS_Thermal_Hotspots_and_Fire_Activity", Type: "FeatureServer").
2020-11-05T13:56:22,871 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "seaice_extent_N_v1", Type: "FeatureServer").
2020-11-05T13:56:23,976 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "seaice_extent_S_v1", Type: "FeatureServer").
2020-11-05T13:56:25,081 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "SPI_recent", Type: "FeatureServer").
2020-11-05T13:56:28,250 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Standardized_Precipitation_Index_(SPI)", Type: "FeatureServer").
2020-11-05T13:56:31,492 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "US_Cases_per_county_(time)", Type: "FeatureServer").
2020-11-05T13:56:33,073 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "USA_Wildfires_v1", Type: "FeatureServer").
2020-11-05T13:56:35,074 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "USGS_Seismic_Data_v1", Type: "FeatureServer").
2020-11-05T13:56:36,595 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection    | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating cache for DataStore Public_Esri_Hosted_Server done. Success: true.‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

You can see from the above that discovery of 36 feature services hosted by an external, public server, took just over 90 seconds. The more services there are to discover, the longer service discovery will take.

 
See Also: