Latest Contributions by RJSunderman

Hello Ananth - Yes, looking at the TAIP0xF0 GeoEvent Definition, the DeviceId is being retrieved from the event payload by the adapter. In fact it is being used as the TRACK_ID. What output are you using to review the event data? Are you receiving the other fields (TimeStamp, Shape, Speed, Heading, Source, and Age)? - RJ ... View more

Hello Brian - The inbound connector being discussed in this thread - Receive Features on a REST endpoint - has a program name / identifier esri-receive-features-rest-in. You are correct, this input requires that an external party POST content to the GeoEvent Processor. The adapter being used by this inbound connector expects that Esri Feature JSON, not generic JSON, will be sent. If you wanted an external party to be able to POST generic JSON to the GeoEvent Processor you would use the Receive JSON on a REST endpoint inbound connector (which has a program name / identifier esri-receive-json-rest-in). You've indicated that you want to periodically poll an ArcGIS feature service to obtain features and process those as events. For this you would want to use the Poll an ArcGIS Server for Features inbound connector (which has a program name / identifier esri-in-poll-feature-service). This connector allows you to specify the registered ArcGIS Server connection, the services folder, the feature service or map service, and the layer from which features should be retrieved. You also specify the Refresh Interval (how often you want GeoEvent Processor to poll the service). When you say "... an ArcGIS server feature for updates, one that is not my own", I'm not sure if you mean another publisher's feature/map service running on your ArcGIS for Server ... or if you mean some external server. You may need to use the Poll an external website for JSON inbound connector if you are polling a site/server external to your organization. Be aware that this input expects generic JSON, not Esri Feature JSON, and you may have to build the Geometry using values returned in specific event fields. Hope this answers your question - RJ ... View more

Simon - Polling the feed you indicated:  [url=http://pipes.yahoo.com/pipes/pipe.run?_id=7892c39c1778e81c27aeb4a0ba265623&_render=json]http://pipes.yahoo.com/pipes/pipe.run?_id=...&_render=json[/url] it appears to me that all of the items event fields are being expressed as JSON strings, with the first item in the list advertising the expected field names. : : items: [ { Site: "Site", LastSampleTime: "Last Sample Time", RiverLevel: "River Level", FlowRate: "Flow Rate", RainfallIntensity: "Rainfall Intensity", WaterTemp: "Water Temperature", Conductivity: "Conductivity", description: null, title: null }, { Site: "210001-Singleton", LastSampleTime: "30/04/2014 05:57:56", RiverLevel: "0.652", FlowRate: "400.2", RainfallIntensity: " ", WaterTemp: " ", Conductivity: " ", description: null, title: null }, ... The GeoEvent Definition generated for me by the GeoEvent Processor input specifies that all fields are expected as String: [ATTACH=CONFIG]33454[/ATTACH] When I send the poll'd JSON to a GeoEvent Cache, I can review the event data as either Text or JSON by visiting the appropriate REST endpoint for the cache: localhost:6180/geoevent/rest/cache/cache-out?f=text localhost:6180/geoevent/rest/cache/cache-out?f=generic-json The GeoEventCache adapter used by the Publish GeoEvents on a REST endpoint appears to be handling the unicode string/character \u00a0 being reported when values are missing/unreported as part of the unicode extended character set. [ATTACH=CONFIG]33455[/ATTACH] The Text adapter used by the Publish text to a TCP Socket, on the other hand, appears to be converting the "non breaking space" character ... which in HTML is &# 160; ... to an ASCII extended character equivalent:  á. What you could do, to avoid having this unicode character being interpreted by different adapters, is edit the generated GeoEvent Definition to specify that the metric values are expected as type Double. The input will then attempt to cast the JSON string values it receives to Double, and set the field's value to null if the type cast fails. Might as well include the description and title event fields from the provided JSON, currently being reported as null strings, while you're at it: [ATTACH=CONFIG]33457[/ATTACH] Notice in the following illustration that in a "text" adaption of the event data, the null fields appear "empty", while in a JSON adaption of the event data the null fields are not reported. [ATTACH=CONFIG]33459[/ATTACH] I think this will better allow you to apply filter criteria, process events, and eventually send event data to a feature service. Better, in my opinion, to send "null" data as null than as a "non-breaking space" unicode character. - RJ ... View more

Hello Simon - I think the problem is with the Field Calculator (Regular Expression) processor's configuration. Looking at your screenshot, the Field Name has been specified as:  Site.  When using the "regular" Field Calculator processor, field names are expressed without decoration. When using the Field Calculator (Regular Expression) processor you need to enclose field names in curl-braces and prepend a '$' ... so your configuration needs to specify ${Site} rather than just Site. This is a common source of confusion. With the 10.2.2 product release the "regular" Field Calculator processor will accept Java String functions - some of which accept RegEx expressions. Hopefully this will reduce the need to use the Field Calculator (Regular Expression) processor ... and help eliminate the confusion in how the two processors expect event field names be specified. For example, if you are using 10.2.2 you could configure a "regular" Field Calculator processor with an expression like replaceAll(fieldName,'[-].*','') to identify and replace all characters following a literal '-' with an empty string. That would effectively trim SiteId-SiteName expression down to just the 6-digit SiteId.  (You could also use the substring() function to grab just the first six characters of the SiteId-SiteName expression ... something like substring(fieldName,0,6).) If you are using 10.2.1 you won't have support for Java String functions. In that case you'll have to stick with the Field Calculator (Regular Expression) processor, specify the field name ${Site}, use a RegEx pattern like ^[0-9][0-9]*, and either place the substring into a new field (allowing you to specify that it is an Integer or Long) ... or overwrite the existing Site field value with the substring as a string representation of the SiteId. ... View more

Hello Elias - The Field Enricher processor was designed around a one-to-one relationship where each received event can be linked to one (and only one) row/feature in a secondary enrichment table. Sorry, but I've consulted with the development team, and I don't have a ready work around to suggest. I've entered this as an enhancement request - to extend the Field Enricher to support a one-to-many relationship with the enrichment table. We will consider this along with other product enhancements post 10.3. Thanks - RJ ... View more

Hello Vitor - You are correct. When GeoFences overlap and an event is INSIDE the overlapping area the GeoTagger processor's default GeoTag Format, DelimitedValue, will output a comma separated list of GeoFence names. Because the IN and MATCHES operators within expressions do not support variable substitution, filter expressions such as SearchKey IN ${TaggedValuesList} or TaggedValuesList MATCHES .*${SearchKey}.* which include the name of a field on the right hand side of the expression will not work. The parsing of the second expression in fact - the one using MATCHES with .* regular expression atoms on either end - ends up allowing all of the events through the filter when the parser encounters the unsupported ${SearchKey} in the expression; the filter's implementation is handling the unsupported expression by replacing the entire expression with .* (which matches everything). If you must handle the case in which GeoFences overlap, I would suggest that you review the forum thread Overlapping" rel="nofollow" target="_blank">http://forums.arcgis.com/threads/105319-Overlapping-geofences-with-different-attributes]Overlapping geofences with different attributes which presents an alternative to using the default GeoTag Format, DelimitedValue. You might find working with the GeoTagged values as a List will allow you some flexibility in your GeoEvent Service design. ...I used a GeoTagger to add a field GeoFenceNames containing the names of each GeoFence the received event's Geometry is INSIDE as a List. Note that I did not use the default DelimitedValue. If the received event's Geometry is within an area overlapped by two GeoFences, the List appended to the event by the GeoTagger will have GeoFence names at the first two index positions: GeoFenceNames[0] and GeoFenceNames[1]. (Note that the list's index is zero-based.) By sending the geotagged event through a series of Field Calculator and Field Enricher processors, I can isolate the individual GeoFence names from the list and use those individual names as the key to the join needed to further enrich the events with attributes from a uniquely named polygon feature. I have reviewed the need for supporting variable substitution for the IN and MATCHES operators within filter expressions with the development team and have created an enhancement request. No promises, but we will evaluate this against other work currently in our backlog and see if we can get it into a future product release. Best Regards - RJ ... View more

Hello Vitor - Sorry, but what you are trying to do is not supported by a filter at any of the 10.2.x product releases. The filter will not expand a field name such as ${ValuesList} to build a list of comma separated values. When configuring a filter with an expression and selecting the IN operator, the Value portion of the expression must be a comma separated list of values. These can be integer values ... 1,16,32 as you discovered. They can also be unquoted strings ... for example abc,def,ghi. You might refer to the thread Spatial" rel="nofollow" target="_blank">http://forums.arcgis.com/threads/102155-Spatial-filter-Different-Geofences-depending-on-a-field]Spatial filter - Different Geofences depending on a field which describes a way to use a filter to compare a geotagged value against another field's value using = and != operators rather than the IN operator. Hope this information helps - RJ ... View more

The development team is prototyping a generic XML adapter for the 10.3 product release. If you have specific XML examples you would like to make sure GeoEvent Processor can handle, please post them as attachments to this thread so that we can evaluate them against the requirments we've developed and discuss specifics with you as needed. ... View more

Hey Adam - You are spot on.  With what we've exposed, you really don't have a true if/then/else programming construct. You only have filters which will send the event through for processing if the filter's criteria are satisfied. So you can split the event stream to feed three different filters, and as long as the filters' criteria are mutually exclusive, you can feed events falling in different categories to separate Field Calculators to place a status into either an existing field or a new field. Beware that if an event satisfies two or more filters' criteria you will end up duplicating events as the full event stream is independently considered by each filter. [ATTACH=CONFIG]33282[/ATTACH] You will want to be careful, if working with non-integer values, to designate a sufficiently small epsilon value that you can test less-than or greater-equal ... don't test equal-to when using non-integer values as a value '0' might be actually 0.000000000013 behind the scenes (which of course would fail if the filter was testing strictly Speed = 0. Your only other option is to crack open the SDK and develop a custom processor which accepts an event, performs its own custom business logic, and then outputs a new event which has been stamped with an appropriate status. For such simple attribute tests, I'm not convinced that any gain in performance would be worth the development effort. - RJ ... View more

Hello - I had some problems reaching out to the URL you provided (http://api.metro.net/agencies/lametro/routes/207/vehicles). I was seeing some HTTP 503 "Service Unavailable" errors returned. However, I was able, intermittently, to obtain JSON from the site and I've included a sample below. You are using the correct input connector - Poll an external website for JSON. Notice in the sample JSON below that the response contains a number of items in a list. If you specify items for the input's JSON Object Name parameter, GeoEvent Processor will parse the JSON and ingest a separate GeoEvent for each "item" in the list returned from the external site. {     "items": [         {             "seconds_since_report": 6,             "run_id": "207_240_0",             "longitude": -118.308899,             "heading": 360,             "route_id": "207",             "predictable": true,             "latitude": 34.003338,             "id": "9479"         },         {             "seconds_since_report": 244,             "run_id": "207_240_0",             "longitude": -118.309067,             "heading": 360,             "route_id": "207",             "predictable": true,             "latitude": 34.065708,             "id": "9556"         },         {             "seconds_since_report": 6,             "run_id": "207_236_1",             "longitude": -118.309166,             "heading": 180,             "route_id": "207",             "predictable": true,             "latitude": 34.056084,             "id": "9543"         },         {             "seconds_since_report": 6,             "run_id": "207_236_1",             "longitude": -118.309296,             "heading": 180,             "route_id": "207",             "predictable": true,             "latitude": 34.091366,             "id": "9425"         },         {             "seconds_since_report": 6,             "run_id": "207_240_0",             "longitude": -118.309036,             "heading": 330,             "route_id": "207",             "predictable": true,             "latitude": 34.021954,             "id": "9431"         },         {             "seconds_since_report": 64,             "run_id": "207_240_0",             "longitude": -118.308998,             "heading": 360,             "route_id": "207",             "predictable": true,             "latitude": 33.965965,             "id": "9554"         },         {             "seconds_since_report": 6,             "run_id": "207_236_1",             "longitude": -118.309036,             "heading": 180,             "route_id": "207",             "predictable": true,             "latitude": 34.00177,             "id": "9504"         },         {             "seconds_since_report": 65,             "run_id": "207_240_0",             "longitude": -118.308968,             "heading": 220,             "route_id": "207",             "predictable": true,             "latitude": 33.932251,             "id": "9438"         },         {             "seconds_since_report": 6,             "run_id": "207_236_1",             "longitude": -118.309395,             "heading": 220,             "route_id": "207",             "predictable": true,             "latitude": 34.104317,             "id": "9427"         },         {             "seconds_since_report": 6,             "run_id": "207_236_1",             "longitude": -118.308998,             "heading": 315,             "route_id": "207",             "predictable": true,             "latitude": 33.931149,             "id": "9424"         }     ] } You will have to work some to first generate, and possibly refine, a GeoEvent Definition for the event data you expect to receive. For example, if you do not specify a node name in the JSON Object Name parameter, and you configure the input to generate a GeoEvent Definition, GeoEvent Processor will generate an event definition which looks something like the illustration below (click the thumbnail to see a larger view): [ATTACH=CONFIG]33251[/ATTACH] When the input is reconfigured with items specified as the input's JSON Object Name, the generated GeoEvent Definition does not include a 'Group' element since each "item" is ingested as a separate event: [ATTACH=CONFIG]33252[/ATTACH] Now that you have a GeoEvent Definition which specifies the format of each vehicle report, you need to reconfigure the input to Construct Geometry From Fields. The events you are receiving contain latitude and longitude values, and if you want to update a feature class through a feature service you will need to construct a Geometry. To do that you must tell the input which fields (specified in the event definition) contain the coordinate values. My final configuration of the Poll an external website for JSON is illustrated below. Notice that I've changed the Create GeoEvent Defintion parameter to 'False' and selected the previously created GeoEvent Definition as the one to use to interpret the received events. (This is assuming, of course, that the structure of the data is not going to change...) I've also specified that my X Geometry Field value should be taken from 'longitude' and my Y Geometry Field should be taken from 'latitude'. Because I am requesting the input build a Geometry for me, I need a field in which the Geometry can be placed, so I also needed to edit the generated LA-Metro-Vehicle-Report event definition to include a new field of type Geometry. [ATTACH=CONFIG]33254[/ATTACH] [ATTACH=CONFIG]33253[/ATTACH] Hope this information helps - RJ ... View more

Hey Simon - The latest Twitter" rel="nofollow" target="_blank">http://www.arcgis.com/home/item.html?id=041138094e5348eb902f4b71175eeb6f]Twitter Connector on the Product" rel="nofollow" target="_blank">http://links.esri.com/geoevent-gallery]Product Gallery should match the latest twitter-for-geoevent source in the GitHub" rel="nofollow" target="_blank">https://github.com/Esri/twitter-for-geoevent]GitHub repo. When we went to present Twitter at the DevSummit (March 2014) we had a problem sending tweets. This was addressed by rebuilding the latest from GitHub and updating the offering on the product gallery. I think Cris pulled the offering from the Gallery to verify this (refer to thread Twitter" rel="nofollow" target="_blank">http://forums.arcgis.com/threads/106353-Twitter-connector-Gallery-vs.-Github-version]Twitter connector, Gallery vs. Github version. I had thought that the Twitter API specified that we could filter tweets for specified hash-tags or a bounding box ... but Morakot corrected me and indicated that, per the Twitter Dev Website, the bounding box doesn't act as a filter ... it provides spatial context, so you should get the union of tweets filtered-by-hashtag and all tweets within the specified bounding box. If this is not what you are seeing, please let us know. You might consider using the Twitter adapter to filter for �??interesting�?� tweets by hash tag, but use GeoEvent Processor GeoFences to provide the spatial restriction by configuring a filter in a GeoEvent Service which discards any event not inside the GeoFence. I think you will find this approach more flexible as you don't have to reconfigure an input to change or update your area of interest ... you can instead synchronize the GeoFence(s) in GEP with a dynamic feature service. Please let us know how this is working for you... - RJ ... View more

Thomas - As it turns out, a couple of developers on the team are working to prototype a custom processor which might do what you're looking for, but the proof of concept is not working properly yet, so it's not something I can share. So, let's see what we can do with what we have available out-of-the-box... I changed my FieldEnricher to enrich events received from the feature service's feature class, rather than the non-spatial table. [ATTACH=CONFIG]33128[/ATTACH] The two filter elements are branching the event flow one way if the EventCount field exists in the received event (bottom branch) and the other way if the field is not included in the event's attributes. Notice that one filter specifies 'AND' while the other filter specifies 'NOT'. [ATTACH=CONFIG]33130[/ATTACH] I've attached a 10.2.1 product configuration I exported which defines all of the various elements I created for this exercise. I had to wrap it up in a ZIP to get the forum to allow me to attach it. You should be able to unzip and import it into your GeoEvent Processor to create the Inputs, Outputs, GeoEvent Service, and GeoEvent Definitions needed - along with a registered Data Store pointing to LOCALHOST where the feature service is assumed to have been published. The GeoEvent Definition trail is especially important to getting the event flow to work. I didn't use any Field Mapper processors to ensure that the event schemas matched what was needed to update the feature service. You will want to look at the GeoEvent Definitions being created by the Field Reducer, Field Enricher, and the Field Calculator named 'ResetField' which creates a field named EventCount if the received event does not already have that field. I included some sample JSON data in the ZIP archive with the product configuration. - RJ ... View more

Hello David - Sorry, but the text adapter does not support events which have a variable amount of whitespace (or other characters) between attribute values. You can specify a single character or a multi-character sequence using HTML encoding (&#32 for an ASCII space for example), but the attribute separator cannot be set to a RegEx pattern. If you knew that attributes were delimited by exactly three ASCII spaces you could specify &#32&#32&#32 as the attribute separator. Syntax should actually be unicode format: \u003B (semi-colon) \u0020 (ASCII space) ... see illustration below. Also, multi-character delimiters do not seem to be working in 10.2.2 ... a bug has been entered to address this. [ATTACH=CONFIG]33172[/ATTACH] You might be able to design a GeoEvent Service, using the support for Java String functions available in the 10.2.2 release, to reduce the variable delimiters between event attributes to a single, standard delimiter ... but I think a better approach would be to explore a JavaScript solution to pre-process your incoming events, replace the variable delimitation with a single delimiter, and then write the event data to a TCP socket or WebSocket for a GeoEvent Service to ingest. - RJ ... View more

You will want to configure a Field Calculator whose Expression looks something like substring(DateAsString,5,7) to pull a two character substring out of a date/time field received as a String and write the retrieved sub-string to a new field. The index values 5 and 7 in the substring( ) syntax specify the starting index and ending index of a zero-based array of characters. The downside of this is that the date/time value must be specified, by the GeoEvent Definition, as a String and the service elements in the GeoEvent Service will only be able to work with the date/time as a String ... not a Date. You can work around this by feeding the event data back into a second input and allowing the second input to handle the conversion of the String representation of the date/time to an actual Date value. Details are included in the attached PDF. Please note that support for Java String functions from within a Field Calculator is supported in the 10.2.2 release of GeoEvent Processor, which should be publically available on your Customer Care Portal tomorrow 15-April-2014. Esri Distributors should have been given access to download the new release last week. - RJ ... View more

Hello Simon - We have an item in our backlog to develop a processor which will do what I think you are describing - take X/Y (or lat/lon) points from an asset's last reported position, treat them as vertices, and use the vertex to update a polyline feature to build a track line. The Add a feature and Update a Feature outbound connectors will only work with the geometry of the target feature service's feature layer. If the service's feature layer has point geometry, the X/Y (or lat/lon) values received will either add new or update existing point features. If you wanted to use these GEP outputs to update a polyline or polygon the Geometry received with the event would need to be a polyline feature or a polygon feature. With the 10.2.x releases you might be able to use an Incident Detector processor to build a track line. You would select Polyline for the processor's Geometry Type to configure the processor to build and update a polyline rather than point features. You will also need to make sure that the Incident Type is Cumulative (not PointInTime) ... and set the processor's Expiry Time to a sufficiently high value that the incident will not be automatically closed when no updates are received for a period of time; the default is 300 seconds (5 minutes). Update: March 2019 I am retracting my advice above to possibly use an Incident Detector processor to build track lines for assets whose positions are being periodically reported as point locations. This approach would rely on the Incident Manager – a server-side component to which you have read-only access through the GeoEvent Server administrative API – to maintain an audit history with a guid, timestamp, and point geometry, for every event record received which satisfies the Incident Detector's opening condition. The polyline geometry output from the Incident Detector processor is built from this audit history. I didn't think about it at the time, but it obvious to me now that maintaining an open/ongoing incident record for all assets you are tracking in order to have Incident Detector output an updated polyline geometry is going to consume more and more RAM until the JVM heap is full. Unless you include a way of periodically closing incidents so that GeoEvent Server can use its ability to maintain a fixed number of "ongoing" and "closed" incidents with a reasonable audit history, using an Incident Detector to build a track line is a very bad idea.  Sorry ... The correct approach to this problem is to use GeoEvent Server to receive and persist locations reported in real-time, allow feature records to accumulate in a feature class, then use GeoAnalytics Server to periodically run a batch analysis and reconstruct tracks. You can read more about that capability here:  Reconstruct Tracks Hope this information helps - RJ ... View more