Latest Contributions by RJSunderman

Hello James - I tried the 'Construct Geometry from Fields' option as shown below... ...but when I did, the input connector was no longer streaming any data through. I think the reason that the input stopped showing data records being received (when you toggled its 'Construct Geometry from Fields' to 'Yes') is that the GeoEvent structure already has a field named 'Geometry' which is preventing the input's adapter from creating a field it wants to name 'Geometry' into which it can place the value it has created. Assuming that you have no influence over the data provider's choice of naming the group element, here's something you can do to work around the issue. Bring the event data in, as provided, without attempting to build a Geometry Use a 'Field Mapper' processor to remap the event structure: Geometry.X  =>  Longitude Geometry.Y  =>  Latitude Geometry.spatialReference.wkid  =>  WKID Send the remapped event to a 'Push JSON to an External Website' output The 'URL' parameter will be the REST endpoint of a new 'Receive JSON on a REST Endpoint' input you create e.g.   http://mymachine.mydomain.com:6180/geoevent/rest/receiver/rest-json-in-repost Configure the rest-json-in-repost input you create to construct a Geometry from the Latitude, Longitude, and WKID field values When configuring the  'Push JSON to an External Website' output, I elected to clear the parameter field 'Header Parameter Name:Value List' (since I wasn't actually providing a list of name:value pairs...) What I ended up doing was placing both the rest-json-in and rest-json-in-repost inputs in the same GeoEvent Service. This might lead to some confusion with your event counts on the Monitor page as every event received increments the GeoEvent Service's event count by 2 (since it has two inputs and is essentially sending a copy of the event record to itself). My solution uses three GeoEvent Definitions ... an original matching the data provider's structure, a remap of that structure, and a copy of the remapped structure with a Geometry field added (used by the rest-json-in-repost input constructing the Geometry, so it has somewhere to place the Geometry it constructs). Please let me know how this works for you - RJ ... View more

Hello Erik - Erik Nilsson wrote: The date format I am trying to read is formatted like this: 2015-12-05T21:09:29+01:00 How do I avoid the errors? I was able to configure a 10.3.1 Poll an External Website for JSON input to retrieve event data from the http://opendata-download-warnings.smhi.se/api/alerts.json site. My input successfully generated a GeoEvent Definition whose date/time fields were handled using the Date data type (refer to smhi-alerts.png illustration attached). When I display a single response from that page in a browser, it looks like all of the date/time values conform to the ISO 8601 standard:  YYYY-MM-DDThh:mm:ssTZD (e.g. 1997-07-16T19:20:30+01:00) GeoEvent's inputs prefer date/time values represented in the ISO 8601 standard. Inputs such as Poll an External Website for JSON make an attempt to parse other common string representations - which is why date/time strings common in the U.S. (such as 3/16/2012 02:25:31) ingest without error. The error message you illustrate above looks like GeoEvent is receiving a String representation of a date/time which it does not recognize, specifically:  Sat, 05 Dec 2015 21:09:21 GMT I'm not sure that these date/time values are (or were) coming from the SMHI open data. You can configure a GeoEvent input to interpret such representations by specifying a Java SimpleDateFormat pattern as the input's Expected Date Format parameter. In order to illustrate this, I configured a Receive JSON on a REST Endpoint and sent the following simple data record to the input's endpoint: [{   "TrackID": "AA-1234",   "LastUpdated": "Sat, 05 Dec 2015 21:09:21 GMT" }] Initially, I had the input generate a GeoEvent Definition without specifying an Expected Date Format. GeoEvent selected a data type String to represent the LastUpdated value. I then updated the GeoEvent Definition to specify that LastUpdated should be handled as a Date, not a String, and updated my input's configuration to specify EEE, dd MMM yyyy HH:mm:ss z as the Expected Date Format. Again, you should not need to do this if the data being polled already represents its date/time values using the ISO 8601 standard. But in case you encounter uncommon date/time representations, the illustrations below should help you accommodate them. Hope this information helps - RJ Click on the images below to see them display in a pop-up. Pattern letters used to construct a Java SimpleDateFormat pattern. Hover over the (?) next to Expected Date Format to see these. Also, refer to SimpleDateFormat (Java Platform SE 6) The ISO 8601 specification is laid out here, Date and Time Formats​, on the World Wide Web Consortium (W3C) site. ... View more

Hello Elliott - Creating a filter expression to compare two date values is difficult. The product team has an issue open in the product backlog to address this. For now, as you suggest, you could use a Field Calculator to "cast" the date/time from an event's Date field to a field whose type is Long, and then use a Filter to compare two long integer (epoch millisecond) values. You could also use date/time algebra notation supported by filter expressions ... please see the links below. Hope this information helps - RJ See Also: How to Create Temporal Filter in GeoEvent​ Re: Filter out Time Anomalies​ ... View more

Hello Alexander -- I checked with Morakot, the developer who created the Motion Calculator processor. He indicated that what you describe sounds like a bug. We are working to create a public repository for this processor on GitHub. If you have development experience and want to review the processor's implementation, you will be able to shortly. In the meantime, if this is still an issue for you, would you please open a case with Esri Support so that we have something to work against? You can ask that the case be elevated to [email protected] in support; he has worked with the Motion Calculator recently. You can also follow-up with Morakot Pilouk​ on the GeoEvent product team. Apologies for the delay in getting back with you on this issue - RJ The repository is now available on GitHub: https://github.com/Esri/motion-calculator-for-geoevent ... View more

Hey K - What I think you're asking for is a ODBC / JDBC connector capable of retrieving data directly from an RDBMS table. No, there are no plans to offer such a connector. Connectors such as these would be database specific and we are working to keep GeoEvent database agnostic. The GeoEvent existing inbound connector which polls an Esri feature/map service to retrieve data from a feature layer uses the feature service as an interface to the database. There are a couple of alternatives described in the threads cross-linked below. - RJ See Also: Direct connection to RDBMS​ GeoEvent process  Oracle/SQL connector​​ ... View more

Hello Steven - My apologies that your question went so long without an answer or reply. I don't have a Sierra Wireless setup I can use to truly replicate your workflow, but I can assume a data provider is sending invalid date/time values and choose another way of ingesting the data. I configured a 'Receive JSON on a REST Endpoint' and sent the following three records into a GeoEvent Service [ {"TrackID":"AA-1234", "DT":"11/30/1999 12:34:56"}, {"TrackID":"BB-3456", "DT":"01/22/2016 12:34:56"}, {"TrackID":"CC-5678", "DT":"11/30/2099 12:34:56"} ] The date/time values are received as string values, but I made sure my GeoEvent Definition specifies that the attribute values be handled as Date. I used a stream stream service to broadcast the unfiltered output so that I could see the epoch milliseconds value GeoEvent was handling: {"attributes":{"TrackID":"CC-5678","DT":4099754096000}} {"attributes":{"TrackID":"AA-1234","DT":943994096000}} {"attributes":{"TrackID":"BB-3456","DT":1453494896000}} I've been told that attempting to configure a Filter with any of the following expressions will not work: DT >= ${EffectiveFrom} Comparing an event attribute against another field of type Date DT >= 2/15/2015 13:26:45 Comparing an event attribute against a String literal TIME_START <= ${MyEventTime} Using a GeoTag to identify a field to compare against another field But using the date/time algebra you tried (e.g. TimeStamp >= 2015y 11M 19d) should have worked. I tested the following expression in a simple Filter element:  DT >= 2010y 01M 01d That expression successfully filtered/removed the invalid '1999' value. When I changed my filter to have two expressions (with a logical AND):  DT >= 2010y 01M 01d AND DT <= 2030y 01M 01d both invalid dates were filter/removed. Only the event with the TrackID "BB-3456" passed through to my stream service output. You may want to check your GeoEvent Definition to make sure that your TimeStamp field is being handled as a Date. You could also use a File/JSON output or a stream service output to confirm the date/time value associated with the events you are receiving. I confirmed that a Filter element's expression will not accept a literal integer value representing epoch milliseconds. A Field Calculator processor performs implicit data type conversions as part of its expression evaluation (which is why we can add/subtract some number of milliseconds from a Date value) ... but a Filter is limited to the date/time algebra I used above. The product team has an issue open in the product backlog ("The date format does not work on filter in GeoEvent Services - Issue #581"). If applying filters to date/time values continues to be an issue for you and the date/time algebra is not a workable solution, please submit an incident with Esri Technical Support and request the incident be escalated and linked to the existing issue in the product team's backlog. Hope this information was helpful - RJ See Also:  How to Create Temporal Filter in GeoEvent ... View more

Cross Reference: ArcGIS GeoEvent Twitter Module: How to get USA-Only Tweets? ... View more

Satyanarayana - Your question is not specific to the GeoEvent Extension for Server. You might have more luck posting to one of the forums beneath All Places > GIS > Applications such as: Collector for ArcGIS or Web AppBuilder for ArcGIS​ Best Regards - RJ ... View more

Hello Daniel - My apologies that this reply is coming so long after you posted your question. In case you are still facing a workflow in which a GeoTagger processor (out-of-the-box) is producing a list of GeoFences which satisfy the spatial criteria (e.g. GeoEvent Geometry INTERSECTS .*/.*) when GeoFence areas overlap, please take a look at the custom processor below which has been uploaded to the product gallery: Field Splitter Processors for GeoEvent​ This custom processor is intended to be placed in-line with a GeoTagger, immediately following the GeoTagger, to split a single GeoEvent whose GeoTags attribute field contains a list of values into multiple events. This should eliminate the need to incorporate multiple Field Calculator processors in your GeoEvent Service to manually pull the names of GeoFences out of the list by hard-coded index. Please let us know how this custom processor works for you. Best Regards - RJ ... View more

Hello Richardson – The ‘Send a Text Message’ output is really just a special case of the ‘Send an Email’. GeoEvent relies on SMTP for both e-mail and SMS. It does not support the SMPP protocol. When you specify the ‘Carrier’ and ‘Recipients’ (which is just a numeric phone number) you’re allowing GeoEvent to construct an e-mail address for the recipient. So you provide: Carrier:    AT&T Recipients:   9095551234 And GeoEvent constructs [email protected] for you and sends the SMS text to that e-mail address via the SMTP server you specified. So you could use the ‘Send an Email’ output instead and just enter in the known SMS / SMTP address for the recipient. The Carrier / Gateway values are configurable, by the way, from Site > Settings as illustrated above. Sorry for the delay in response. Hope this information is helpful. - RJ ... View more

Hello Jerry - Yes, there is a RabbitMQ management console you can access to check the internal queues, but we normally don't advertise how. The administrative endpoints are undocumented because our dependency on the RabbitMQ platform service is considered an implementation detail. The GeoEvent product team could change this at any release, although it did not change for 10.4 and is looking (for now) like it won't be changing for the 10.4.1 release. I've been told that enabling the plug-in increases the overall memory footprint of the product, so you should probably only leave it enabled when you plan on actually using it, disabling it on a production system. Finally, you can really muck things up from within the management console, so if you do enable it, be careful not to make any changes to the exchanges or message queue bindings. Here's how to enable and access the management console: If RabbitMQ is running, you must first stop it Log-out of GeoEvent Manager and stop the GeoEvent Windows Service Log-in to the ArcGIS Server Administrator Directory:  https://<server>:6443/arcgis/admin/login Navigate to the platform services endpoint:  http://<server>:6080/arcgis/admin/system/platformservices?f=json This is an undocumented REST endpoint within the Server Admin API Note the <id>, <user> and <password> for the RabbitMQ Message Bus displayed on the platform services page Navigate to the following endpoint to stop the RabbitMQ platform service http://<server>:6080/arcgis/admin/system/platformservices/<id>/stop?f=json You can replace ‘stop’ in the URL above with either ‘status’ or ‘start’ to start, stop, and status the service In a Windows File browser, open the folder C:\Users\…\AppData\Roaming\RabbitMQ on the server machine for the user account used to run the ArcGIS Server and GeoEvent Windows Services. Create a file named:  enabled_plugins The filename has no extensions (make sure Windows isn’t hiding a .txt extension from you) Edit the file to contain a single line of text:  [rabbitmq_management]. Note that the ‘.’ Is part of the line of text you enter into the file Use the ArcGIS Server administrative endpoint to start the RabbitMQ platform service http://<server>:6080/arcgis/admin/system/platformservices/<id>/start?f=json Refer to:  http://mediawiki/index.php/RabbitMQ_Integration#Enable_RabbitMQ_Management_plugin Restart your GeoEvent Windows Service You do not need to be logged-in to GeoEvent Manager, but GeoEvent needs to be running Browse to the following URL to log-in to the RabbitMQ management console (using <user> and <password> noted above) https://<server>:27274 From here you should be able to click on ‘Exchanges’ to review the exchanges managing communication for the GeoEvent ‘default’ cluster. You should see several exchanges which identify GeoEvent  (e.g.  geoevent.<cluster>.<type>:<guid> is how they appear in the 10.4 release). If you log-in to the GeoEvent administrative API (https://<server>:6143/geoevent/admin) you can click to list the inputs you have running, for example, and look at the GUID assigned as the Name for an input. Back in the RabbitMQ management console you can click on ‘Queues’ and identify the queue associated with a particular GeoEvent input. Good luck, and hope this information is helpful - RJ ... View more

Hello Guillaume - I don't think you will be able to use GeoEvent, out-of-the-box, to do what you are trying to do. While the SOAP protocol does use XML to structure its requests and responses, SOAP messages are not the type of event structure GeoEvent is expecting when XML is used to format event data provided by a real-time data stream. You might configure a 'Poll an External Website for XML' to poll a SOAP service, but the GeoEvent's input will only make a single attempt to parse the response it receives. When a GeoEvent input polls an external server, it expects a single response to its request, consistent with the architectural style of REST. The data structure, or byte payload, sent to GeoEvent in response to its request is parsed by the input's adapter (a transport is responsible only for delivering the byte payload to the adapter). The adapter expects a specific format (JSON, XML, Delimited Text) and uses a GeoEvent Definition to take attribute values from the response and construct a GeoEvent object. There is no provision in the event ingest workflow described above to perform multiple actions or operations with a data provider's service. Integrating a SOAP toolkit into your solution to handle the token acquisition and data request is one approach. You also might be able to use the GeoEvent SDK to implement a custom transport which will handle the token acquisition, data request, and session closure. This might be preferable if the SOAP toolkit uses Java. If you plan on using an external batch script to handle the service request and write the data to a system file for GeoEvent to then ingest, please be aware that there are several problems inherent when working with file-based input. One example is that GeoEvent may attempt to acquire a handle to a file (and begin reading the file) before the external process is finished writing the file to disk. Another example is related to the issue above; GeoEvent needs to read the contents of a file into memory in “chunks” and may encounter a problem emulating a data stream when retrieving blocks of event data from a system file. Inputs which watch a system folder for files are generally intended to prove that event filtering and real-time analytics you have designed into a GeoEvent Service behave as intended. Files give you the ability to repeatedly send a small sample of event data to GeoEvent to test the behavior of a GeoEvent Service before transitioning to a production configuration in which real-time data feeds arrive via a stream - such as an HTTP/POST from an external server or as a reply to a query made on an external server's URL. Sorry this reply is coming to you so late. Hope that it is of some help. - RJ ... View more

Hello Britt - I downloaded the sample file.xml you attached.  I had to remove some literal dash characters ('-') from several lines before an XML editor I normally use would validate and display the file's content for me to review. My first question is if I can extract all the location information defined at the lowest hierarchical position along with high level details from the top level xml hierarchical nodes ... Let me make sure I understand what you want to do. Given a structure like the one illustrated below, you want to bring each <Issue> element in from the <Issues> list as separate events, but you want the events to incorporate the information from the <LocalTime> and/or the <UtcTime> elements found outside the <Issues> structure. The quick answer is no, you cannot do this. When specifying the 'XML Object Name' parameter for either the 'Poll an External Website for XML' or 'Receive XML on a REST Endpoint' GeoEvent inputs, you are telling GeoEvent that it should scan forward (through the XML structure), skipping past elements and attributes until it locates the specified 'XML Object Name' node. GeoEvent will then bring in elements listed beneath that root node as separate events. The trade-off to identifying a list deep within a structure and having GeoEvent parse that list is that you have told GeoEvent to skip past and disregard elements outside of the list. You might want to take a look at the discussion thread XML can I use this source?​  The problem presented in that thread is similar - the date on which each event occurred is outside of the list of events Since each record in the list only specifies a time, you need a way to enrich each of the individual events with the date recorded as an attribute in the list's root node. In your case your challenge is compounded as it looks like each <Issue> can have more than one <Location>. ... I selected 'Location' as my 'XML Object Name'. Can we somehow extract all the location objects with all the elements and attribute information from this xml? Looking at the XML structure, <Issues> appears to be a list.  I would expect I could use GeoEvent to parse the <Issues> structure in order to bring each <Issue> in as a discrete event. Because the various <Location> records are not organized beneath a single root node (e.g. thay are children associated with a particular issue) I don't think the data structure will support GeoEvent bringing each <Location> in as a separate event. The idea behind the 'XML Object Name' parameter is that you can specify a single node as a "root" node and have GeoEvent only bring in data found beneath that "root" node as discrete (separate) events. Hope this information is helpful - RJ ... View more