While the steps to administratively reset ArcGIS GeoEvent Server don't change very much from release-to-release, you are working with the system files and folders which constitute the software product. Changes to the steps are inevitable. Procedures for an administrative reset are therefore being written-up as version specific PDF files.
Each downloadable PDF starts with an identification of "What Changed" at a specific release to require a change in the scripted steps. Each document includes things you should understand before executing an administrative reset. The actual administrative reset steps are on the last page(s) of each document.
Please download the PDF document which is most appropriate to your software release.
Verizon Connect provides multiple fleet management solutions for tracking and managing your fleet of vehicles. This blog will focus specifically on Verizon Connect Reveal and how you can connect to the data feed and perform real-time ingestion and analysis with ArcGIS GeoEvent Server.
The Field Mapper and Field Calculator Processors are two of the most often used processors in ArcGIS GeoEvent Server. While the Field Mapper Processor provides the ability to map one GeoEvent Definition (schema) to another GeoEvent Definition, the Field Calculator Processors allows you to use functions to manipulate field values. The Field Calculator supports many different functions related to data type conversion, string manipulation, mathematics, creating geometry from fields, and more.
In GeoEvent Server 10.9, the Field Mapper now allows you to use the same Field Calculator functions inside each Field Map text box. This makes it possible for a single Field Mapper to potentially replace a series of Field Calculators when performing calculations on multiple event attribute values. Let’s walk through some examples to see the power of this enhancement.
Choice elements are configurable elements of GeoEvent Services that behave like Filter elements. However, unlike Filter elements, choice elements route event records to more than one destination based on an ordered list of filter operations called ‘when’ clauses. These conditional statements are applied to the event records in the order in which they are configured, much like an if-else statement in programming languages. In addition to the when clauses, an optional final route can be defined for events that do not pass the list of specified conditional statements.
ArcGIS GeoEvent Server 10.9 offers many new and exciting improvements in usability, expanded analytic capabilities, and stability enhancements. At a high-level, the user experience for designing and maintaining all of your real-time event processing workflows has been greatly simplified. In addition, several stability enhancements were made to increase event throughputs and uptime.
In this blog we will take a deeper look at registering server connections with GeoEvent Server, something administrators commonly have to do to when configuring a GeoEvent Server deployment.
Why do you have to register an ArcGIS Server connection with GeoEvent Server
There are several different configurable components which require you to select a registered server and specify which services folder, map/feature service, and often a specific feature layer exposed by that service in order to use the feature layer's schema or feature records. A few examples include:
All of the user-interfaces in the above examples populate their options from a cache of information GeoEvent Server collects when it queries a registered ArcGIS Server to discover published feature services. Service discovery is not performed the moment you click to open the panel. The cache is created and updated in the background because it can take several seconds – sometimes minutes or even tens of minutes – for GeoEvent Server to completely crawl the ArcGIS Server's REST Services Directory and query the information it needs from all of the available services.
Which leads into the topic I want to address in this blog: Is there a way to know when service discovery is being run and how long it is expected to take?
Using the ArcGIS Server Connection component to log messages
You can configure the following component logger to request DEBUG messages be logged:
I've included a sample of the logged messages you will be looking for at the end of this article. Requesting this component logger include DEBUG messages in its logging will allow you to see, in the system log file, when the service discovery kicks off and which map/feature services it interrogates to learn about their layers. A message will be logged for every feature service being interrogated as well as a success message when service discovery is complete. There is no indication in the GeoEvent Manager web application that service discovery is about to start or is currently running. The best way to tell that service discovery is running is to start the workflow to import a GeoEvent Definition. If the blue/white indicator displays, requesting you "please wait", you know that the GeoEvent Server is busy updating its ArcGIS Services cache. Otherwise the GeoEvent Definition user-interface will display immediately allowing you to choose a server connection, folder, feature service, and feature layer.
GeoEvent Manager does not allow you to configure, or schedule, when service discovery should take place. You are able to change the Discovery Rate for each server connection you register to specify how frequently a refresh should be performed. The default for recent software releases is every 60 minutes. I have personally found that when I use ArcGIS Pro, the Enterprise portal, or GeoEvent Manager to publish a new feature service, I want to use it now – so it is not uncommon for me to publish a feature service and immediately request GeoEvent Server run a service discovery to update its cache by clicking the refresh button on the server connection I have registered as a Data Store. This eliminates the need to have GeoEvent Server periodically refresh the cache for me, so I usually set the Discovery Ratefor server connections I register to a fairly large value like 1440 minutes so that service discovery is run once once per day (or when GeoEvent Server is stopped and restarted).
Service discovery takes too long and interferes with normal operations. Is there anything I can do?
This is something the product team is working on. Refactoring GeoEvent Server to support all of the operations which use feature services, however, to interface with ArcGIS Server some other way is both high risk and high reward. Several different design options have been considered, but implementation has had to be deferred for each of the last several major releases. The best option for now, therefore, is to limit the number of features services which are discoverable each time service discovery is run.
A recommended best practice is to configure your GeoEvent Server Data Store (e.g. the server connection you are registering with GeoEvent Server) with credentials. The user credentials do not have to be an administrative user, just a user who owns and published the feature service. You can use the Enterprise portal content item manager to assign existing feature services a new owner and configure GeoEvent Server to authenticate as that user when crawling the ArcGIS Server's REST Services Directory to limit the number of discoverable services. GeoEvent Server administrators often configure their registered server connections with the ArcGIS Server or Enterprise portal primary administrative account – which naturally sees all published services.
If you can identify the feature services to which you want GeoEvent Server to write data, or from which you want GeoEvent Server to retrieve feature records or a feature layer's schema, and assign ownership of just those services to a user set aside specifically for your "real-time" data, you can improve service discovery considerably by not crawling all of the feature services being maintained by more traditional feature editing workflows. The trick is to identify the feature services your GeoEvent Server components actually care about and limit discovery to only those feature services.
Example messages logged by the ArcGIS Server Connection component logger
You can see from the above that discovery of 36 feature services hosted by an external, public server, took just over 90 seconds. The more services there are to discover, the longer service discovery will take.
These days, the preferred method for monitoring a GeoEvent Server in a production system is the ArcGIS Monitor. But sometimes you want to just see/monitor the local system directly within GeoEvent to quickly evaluate how the system is performing (or alert you if it isn't performing well). Below are instructions for deploying an input connector that will collect some basic information from your system and output it to a csv file. You are free to modify the configuration once imported to write the data to a geodatabase and/or alert when certain parameters exceed certain tolerances.
Import the System Info Transport
Unzip the attached zip file below to a temporary location. You will deploy the new System Info Transport to your GeoEvent server.
Method 1: Copy to the deploy folder
Copy the .jar file from the temporary location into your GeoEvent Deploy directory. On Windows the default location for the deploy directory is:
Method 2: Import the .jar file using GeoEvent Manager
In GeoEvent Manager, navigate to Site > Components > Transports and press the Add Local Transport button. Press the Choose Files button, browse to the temporary location, select the .jar file, press Open, then press the Add button. Once the System Info Transport is deployed it will show up in your list of inbound transports within GeoEvent Manager on the Site > Components > Transports page (you may have to refresh the page).
Import the configuration
In GeoEvent Manager navigate to Site > GeoEvent > Configuration Store and press the Import Configuration button. Press the Choose File button, browse to the location of the GeoEventConfig SystemInfo.xml file, press the Open button, then press the Next button. Select Import Configuration and press the Import button. This will import the System Info Connector, GeoEvent Definition, Input, Output, and Geoevent Service for you.
Explore the connector
Navigate to Site > GeoEvent > Connectors The connector utilizes the System Info Transport, which generates information in generic JSON format. The adapter is the Generic JSON Adapter. The following fields are hidden from the user since they are not used: Expected Date Format, Construct Geometry from Fields, X/Y/Z Geometry field, Default Spatial Reference, Learning Mode, As GeoJSON, and JSON Object Name. Note that the JSON Object Name is first default hard coded to ‘OperatingSystemInformation’.
The remaining fields can be left in the Shown Properties area. You may wish to modify the Update Interval, since the default value (1 second) may be too frequent for long term monitoring.
Explore the GeoEvent Definition
A default system-info-in GeoEvent Definition is supplied in the configuration with some basic parameters. You may wish to have the input re-generate a new GeoEvent Definition for you just to be sure that the properties reported on your system are the same. To do this, follow the steps below:
Rename the system-info-in GeoEvent definition to something like system-into-in-orignal.
Navigate to Services > Inputs > system-info and make the following change and save the input:
Create GeoEvent Definition: Yes
GeoEvent Definition Name (New): system-info-in-auto
Run the input until at least one event is received, then turn the input off.
Navigate to Site > GeoEvent > GeoEvent Definitions and make a copy of the site-info-in-auto GeoEvent Definition.
Name the copy: site-info-in
Save the copy
Navigate back to Services > Inputs > system-info and make the following change and save the input:
Create GeoEvent Definition: No
GeoEvent Definition Name (Existing): system-info-in
At this point you can delete the system-info-in-auto and system-info-in-originalGeoEvent Definitions, since they are no longer used.
Explore the GeoEvent Service
The GeoEvent Service system-info using the system-info input to generate a system-info-in event every 60 seconds. These events are written directly to an output that writes the events to a csv file in the GeoEvent Backup folder.
By default this folder (on Windows) is located:
These csv files will record the status of your server over time and can be easily manipulated in Excel to create graphs of the historical system resources.
With the 2020 Esri UC coming up next week I wanted to highlight what Esri and the larger GIS community are doing this year to promote Diversity, Equity & Racial Justice. I believe that everyone should step up and participate in these activities.
Click the links below to see what is going on and how you can get involved:
With the 2020 Esri UC coming up next week I wanted to highlight the ability to schedule appointments with experts to get your questions answered, your designs reviewed, or whatever it is that you feel may need a review by an expert.
Click the link below to schedule your appointment today: