Skip navigation
All Places > GIS > Enterprise GIS > GeoEvent > Blog > 2015 > August

Another recurring question:


I've configured a 'Poll an ArcGIS Server for Features' input to 'Get Incremental Updates', is there a way to prevent the input from polling all of the features in a feature class when GeoEvent Server is restarted?


The short answer is: No.  When GeoEvent Server is restarted (or the server on which it is running is rebooted), inputs which use the out-of-the-box FeatureService transport to poll an ArcGIS Server map service (or feature service) lose whatever value they've cached which enables them to query for features which are "new" relative to the last poll conducted by the input.


The capability to 'Get Incremental Updates' is unique to the 'Poll an ArcGIS Server for Features' input connector and should not be confused with the 'Receive New Data Only' parameter, exposed by the HTTP transport, which requires event data include the HTTP "Last-Modified" header. (Refer to comments in the thread Re: Receive RSS Inbound Connector.)


The issue we're exploring here deals only with the 'Poll an ArcGIS Server for Features' input connector -- or a custom input you may have configured which uses the FeatureService transport to poll an ArcGIS Server map / feature service.


An input configured to poll a map / feature service and retrieve only incremental feature updates maintains an in-memory cache. The value in this cache depends on whether your 'Method to Identify Incremental Updates' is ObjectID or Timestamp. In either case the input incorporates the largest value observed from its last poll into a WHERE clause so that only features whose OID (or date/time) is greater than the greatest value (from the last query) are returned (by the next query).


If you stop the input the cache is maintained, so that when the input is restarted it will be able to poll for features whose specified attribute value is greater than the value in the cache. If you stop the ArcGIS GeoEvent Server Windows service, or reboot the server, the cache is destroyed and the input has no way of knowing which features were polled previously. The next poll conducted by the input will retrieve all of the items in the map / feature service.


This becomes painfully obvious when one of the notification outputs (e.g. 'Send a Text Message' or 'Send an Email') are included in a GeoEvent Service which polls a map / feature service for event data. When the GIS Server is rebooted an e-mail recipient can potentially receive hundreds of messages if event data polled by the input satisfy filtering and/or processing criteria designed into a GeoEvent Service.


The motivation behind this behavior is that a cache persisted within a system file on disk could be difficult to find, might only be editable by a user with administrative credentials, and unnecessarily involves file I/O in a potentially high-volume event processing scenario. Locating and deleting a system file-based cache was deemed more burdensome than requiring that GeoEvent Server outputs be stopped in order to prevent unwanted notifications from being sent. Basically, this behavior is by design.


As a best practice, if you find you are frequently restarting GeoEvent Server (or having to reboot your server), make sure to stop all notification outputs, or any outputs you do not want to process events based on "old" features, when GeoEvent Server is restarted.


You can also employ a strategy of writing notification messages to a secondary feature layer, rather than directly to a notification output. The secondary feature layer acts as a notification message cache. You could then design a second GeoEvent Service (or extend your original GeoEvent Service) to poll this "notification message cache" and as event messages are sent to a notification output, use an 'Update a Feature' output to flag the notification message as having been sent. This will enable a filter to discard messages which have been sent and avoid sending repeat notifications.


If you have other approaches you have developed to deal with this particular behavior, your comments are welcome.


As always, I hope this information helps.


- RJ

Morakot has been keeping a blog under his GeoNet user-id.


I'm going to reference a recent blog of his, and probably begin re-posting his content here, in the GeoEvent product's blog.


How to Create Temporal Filter in GeoEvent


- RJ

Hey All -



Normally the GeoEvent Simulator loads comma separated text from a simulation file and allows you to send this data to a GeoEvent Server tcp-text-in input to simulate a real-time data feed. Sample data in the tutorials either represents a point geometry as a quoted pair of X,Y coordinate values (e.g. "-75.175,39.991") ... or provides the coordinates of a point location as separate X and Y attribute values which the input can take and use to construct a geometry.


But what if you want to simulate a dynamic polygon, such as a series of forecast areas affected by a storm?


Here's a trick you might find handy:

  1. You do not have to use a comma to delineate your event attributes in a simulation file.
  2. Valid Esri Feature JSON can be placed in a simulation file.
  3. A GeoEvent Server input can be configured to interpret the JSON as a geometry.


Consider the following two lines of simulation input:


"AA-1234";"12/24/2015 23:59:59";{ "rings": [ [ [-75.175, 39.991], [-75.173, 39.991], [-75.173, 39.99], [-75.175, 39.991], [-75.175, 39.991] ] ], "spatialReference": { "wkid": 4326 } }


"BB-7890";"02/15/2015 12:34:56";{ "rings": [ [ [-8368449.66, 4864715.92], [-8368263.15, 4864676.62], [-8368272.04, 4864618.25], [-8368459.87, 4864645.20], [-8368449.66, 4864715.92] ] ], "spatialReference": { "wkid": 102100, "latestWkid": 3857 } }


In both examples I am sending GeoEvent Server a JSON string representation of a geometry using the Esri Feature JSON format for a polygon geometry. Please refer to the ArcGIS Developers on-line documentation for the JSON spec and samples of Point, Multipoint, Polyline, and Polygon geometries.


Notice that I have chosen to separate the event attributes using a semi-colon rather than a comma. Since both commas and literal quotation marks are part of the Esri Feature JSON syntax, using a semi-colon for field delineation simplifies my simulation file considerably. It allows me to keep the required quotes and commas without having to escape or quote them as string literals. I'm free to quote the other event attributes of the simulated event. In the examples above, I've quoted my TRACK_ID and my TIME_START values, though I probably do not need to.


Also notice that each geometry string includes the coordinate system associated with the coordinate values. The first event uses the WGS 1984 Geographic Coordinate System (its coordinate values are expressed in decimal degrees). The second event uses the Web Mercator Aux Sphere Projected Coordinate System (its coordinate values are expressed in meters).


Attached are illustrations of the GeoEvent Definition I configured my 'Receive Text from a TCP Socket' input to use. The input is still responsible for adapting the delimited text it receives from the GeoEvent Simulator, so it needs to know what characters to expect for the message separator and attribute separator ... and it relies on an event definition to tell it that the third attribute should be interpreted as a Geometry.


If you try this and run into problems, let me know. There may be limits on the raw number of bytes you can pass over a TCP socket or how many messages of a given size you can load into the GeoEvent Simulator and send each second. It is probably best to simplify string representations of your geometries when including JSON in simulated event data.


Hope you find this information useful -



GeoEvent Definition




GeoEvent input configuration