Increasing an integer field with incoming events

[Anonymous User]

Original User: tommigro

Hello

we would like to realize the following idea: with every incoming event we would like to increase the value of an integer field of a feature service.

What would be the best approach for that?

We checked how this could be done using the field enricher. I did not seem appropriate.

Thanks
Thomas

[RJSunderman]

Hello Thomas -

Using only out-of-the-box processors, this could be tricky. By design most of the configurable filters and processors used in a GeoEvent Service do not maintain any sort of cache, so your GeoEvent Servcie will never know, for example, how many events for a vehicle with the license plate ABX-339 have been received.

We might be able to work around this, though, without having to use the GeoEvent Processor's SDK to develop a custom processor.

Do your events have unique track identifiers? If so we might develop a creative GeoEvent Service which polled its own output feature service to update a non-spatial table containing event statistics with attributres being maintained on the base features.

[ATTACH=CONFIG]32896[/ATTACH]

Consider the illustration of the GeoEvent Service above (click to enlarge the thumbnail). The event flow goes something like this:

1. An event is received on a GEP input connector's REST endpoint

- The input connector constructs a Geometry from the event's attribute fields 
- A Field Reducer is used to remove the Latitude and Longitude fields from the event 
- The constructed Geometry is retained and the event is now "flat" (complient with an Esri Feature Service) 
X 

2. A Field Enricher joins the target feature service's non-spatial table to the event

- If field enrichment succeeds because a field named EventCount is found, a Field Calculator will increment the integer in the joined field and write the incremented value back into the enriched event. The enriched event is then written out to the target feature service's feature class. 
- If the field enrichment fails because the target feature service's non-spatial table has no EventCount for the event with the received TRACK_ID, then a different Field Calculator ... the one named ResetField ... writes a hard-coded '1' into a new field named EventCount. 
X 

3. In either case, a single feature in the target feature service's feature class is updated with the received event which is tracking a count of the events received for that TRACK_ID.

X 

4. A separate input connector is polling the target feature service's feature class, and when it sees a feature whose DateTime value is greater than the last date/time a feature was added/updated ... the "new" feaures are retrieved and the EventCount from each feature is transferred to the target feature service's non-spatial table.

Depending on the rate at which you are receiving new events for a given track, I've observed this approach working. My tests were only preliminary though ... for example, I was making sure that the second input connector always had a chance to poll the feature class to get updated counts for the events received before new events were POST'ed to the other input connector's REST endpoint. I'll try to make some time to work with this approach some more and identify any faults. Please reply if you are able to make this work or have any questions.

- RJ

[Anonymous User]

Original User: tommigro

Hello RJ

Thanks for your help. I tried to follow your instruction for several hours now but didn't manage to get the problem solved. Could you please give me insight into your Field Enricher Processor and your Filters (screenshots)? I would like to compare them with mine.

Thank you.

Regards
Thomas

[RJSunderman]

Thomas -

As it turns out, a couple of developers on the team are working to prototype a custom processor which might do what you're looking for, but the proof of concept is not working properly yet, so it's not something I can share.

So, let's see what we can do with what we have available out-of-the-box...

I changed my FieldEnricher to enrich events received from the feature service's feature class, rather than the non-spatial table.

[ATTACH=CONFIG]33128[/ATTACH]

The two filter elements are branching the event flow one way if the EventCount field exists in the received event (bottom branch) and the other way if the field is not included in the event's attributes. Notice that one filter specifies 'AND' while the other filter specifies 'NOT'.

[ATTACH=CONFIG]33130[/ATTACH]

I've attached a 10.2.1 product configuration I exported which defines all of the various elements I created for this exercise. I had to wrap it up in a ZIP to get the forum to allow me to attach it. You should be able to unzip and import it into your GeoEvent Processor to create the Inputs, Outputs, GeoEvent Service, and GeoEvent Definitions needed - along with a registered Data Store pointing to LOCALHOST where the feature service is assumed to have been published. The GeoEvent Definition trail is especially important to getting the event flow to work. I didn't use any Field Mapper processors to ensure that the event schemas matched what was needed to update the feature service. You will want to look at the GeoEvent Definitions being created by the Field Reducer, Field Enricher, and the Field Calculator named 'ResetField' which creates a field named EventCount if the received event does not already have that field.

I included some sample JSON data in the ZIP archive with the product configuration.

- RJ

[KDeVogelaere]

Thank you RJ for posting a solution to this, is the prototype custom processor you spoke of available in 10.2.2?  I am also interested in a solution for calculating EventCounts using multiple key fields (eg. ID + StartTimestamp + EndTimestamp) to increment the count. Thanks!