Latest Contributions by RJSunderman

Hey Ben - I love your idea of using a 'Poll an ArcGIS Server for Features' input, whose ‘Query Definition’ has been set to poll features whose TO_DELETE attribute has been set to 1, and having the input delete any features it has polled. I’d never considered using an input to perform a data cleansing task this way, after updates have been made the features by an output. This illustrates that inputs, as runnable components, can manipulate a dataset (by deleting data) even if the input has not been incorporated into a GeoEvent Service. Simply creating the input and allowing it to run will result in records flagged for deletion to be deleted; you do not have to route the input’s events through any filtering or processing or send them to an output. You mention that configuring an output to do this deletion would be difficult if the event data didn’t include a timestamp. You could configure a Field Calculator with an expression which invokes the function receivedTime() to write the date/time an event was received into a new field – or overwrite the value in an existing field if you cared more about the date/time GeoEvent received the event than the date/time a provider associated with an event. On that latter point, using a Field Calculator to overwrite a value in an event’s Date field, you could artificially mark a feature as “old” by having the Field Calculator subtract a number of milliseconds from receivedTime(), then update features with the artificially aged date/time. Now when the output polls to look for features which ought to be deleted, it will find features which look “old” and delete them. You are correct, though. Forcing clients to figure out how to determine “now” in order to filter features which are some number of minutes older than “now” so that they are not displayed is harder than configuring a map layer to filter features whose TO_BE_DELETED attribute has been set to 1. That’s another reason I liked your use of the input to delete “old” features… - RJ ... View more

Hello Mike - I've branched your question to a new thread, as you are asking for advice on bringing data into GeoEvent rather than using a stream service to broadcast data out from GeoEvent. When working internally, behind your firewall, data you HTTP/POST to a GeoEvent hosted REST endpoint would need to be sent to the default port 6180. For example: http //server-name:6180/geoevent/rest/receiver/rest-json-in On another thread - Geoevent Processor and Verizon Networkfleet​ - Daniel mentions that your IT department might be able to create a specific tunnel for your device to reach the default port 6180 on the server running your GeoEvent Instance. I also describe, on that thread, how you might use a pair of GeoEvent instances, one running in the cloud, to relay data from a provider into your organization. One of Esri's solution engineers asked what sounds like a similar question to the one you pose. He indicated that he had a public facing Web Server with a JavaScript application submitting event data to GeoEvent via a ‘Receive JSON on a REST Endpoint’ inbound connector. The public facing server was actually a Reverse Proxy (Apache) to an ArcGIS/GeoEvent Server (IIS) communicating with GeoEvent using port 6180 in the httpd.conf file: ProxyPass /geoevent http://10.61.3.5:6180/geoevent ProxyPassReverse /geoevent http://10.61.3.5:6180/geoevent With a reverse proxy in place the URL for the JSON request does not contain the port. So rather than sending data to http//host:6180/geoevent/rest/receiver/<name> you would send it to something more like http//proxy/geoevent/rest/receiver/<name> . The advice provided was to install the Apache Module mod_proxy_wstunnel to enable the reverse proxy to handle websocket tunneling. One of the developers on the GeoEvent team evaluated the Nginx reverse proxy, so I don't have the exact syntax needed to configure the Apache reverse proxy. You would need to create a rule something like the following so that websocket connections from the client would get routed to the web server listening for those connection requests: ProxyPass /arcgis/ws  ws://10.61.3.5:6180/arcgis/ws ArcGIS Server’s REST handler would need to be configured to provide the correct Websocket address to clients, without the port number embedded in them. To do this you need to use the AGS Admin Directory to add a property to the Server to register your proxy of choice. (Refer to the attached illustration...) If you are working with a machine where the HTTP port for the GeoEvent Extension has been changed from the defaults (6180 / 6143) to something else (e.g. 8080 / 8443) … be sure to specify the correct port when updating the ArcGIS Server properties above. Hope this information is helpful. - RJ ... View more

You do have a few options available, however, without relying on an ODBC / JDBC connector: Here's another trick that a customer described to me. If you know an RDBMS wizard, ask them to help you create a spatial view of your data. ArcGIS can use a spatial view in the same way as it uses a feature class, so you can publish a spatial view as an ArcGIS feature service. This can provide you a great deal of flexibility. You have complete control over your spatial view, such as joins you want to perform with other tables in your database or limiting the data exposed through the spatial view to the "most recent five minutes". The feature service will appear to be automatically updating to display the most recent data - but the back end database is doing all the work. And you now have the ability to query the feature service using endpoints exposed through the ArcGIS REST Services Directory. I'm not an RDBMS wizard and thought this approach was absolutely brilliant. - RJ ... View more

Hello Allen - The ‘Poll an ArcGIS Server for Features’ input makes the assumption that the Geometry for your events will come from the features you are polling. So that particular input does not offer a configuration option to ‘Construct Geometry From Fields’. I'd recommend you try using the ‘Poll an External Website for JSON’ input. That input connector uses the product's generic JSON adapter (vs. the Esri Feature JSON adapter used by the  ‘Poll an ArcGIS Server for Features’ input). When configuring your ‘Poll an External Website for JSON’ input you will have to specify the URL that GeoEvent should poll. You will have to include all of the required parameters for polling a Esri feature service as generic JSON. The easiest way to see what these parameters are is to use the ArcGIS REST Services Directory to query your feature service, then copy/paste the URL generated for your query and examine the parameters. Many of the parameters will be unspecified. Examples include:   &text= &objectIds= &time=  You can omit many of the parameters. For example: <code>http://localhost:6080/arcgis/rest/services/SampleWorldCities/MapServer/0/query<br/>?where=1%3D1&geometryType=esriGeometryEnvelope&spatialRel=esriSpatialRelIntersects<br/>&outFields=*&returnGeometry=false&returnTrueCurves=false&returnIdsOnly=false<br/>&returnCountOnly=false&returnZ=false&returnM=false&returnDistinctValues=false<br/>&f=json</code> You would need to specify the above query as a single line of text ... as the ‘Poll an External Website for JSON’ input's URL property. You will also need to configure the input to specify 'features' (no quotes) for the input's ‘JSON Object Name’ parameter so the input knows to look for an array containing features and bring each feature in as a separate event. In my example above, I elected to specify returnGeometry=false since you indicated that you intend to have the input use coordinate values returned from the feature query to construct a Point Geometry. Hope this information helps - RJ ... View more

Hello Michael - A stream service output's 'Related Feature' property is a URL which is passed along to clients subscribing to the stream service. GeoEvent does not actually do anything with this URL (other than pass it along to clients). Clicking 'Subscribe' on the stream service's HTML page in the ArcGIS REST Services Directory, for example, will not show you any enrichment. The intent is that a real-time data stream, rich with attributes but missing a Geometry, can link to a related feature service which contains *only* the location (Geometry) for sensors in a network so that the real-time sensor data can be displayed on a web map. I've attached a PDF with several steps which illustrate how the 'Related Features' capability is supposed to work when using Stream Services. I shared these with you off-forum already ... I'm posting them here for the benefit of the community. Hope this information helps - RJ ... View more

Shane - I've moved your question to GeoNet's space for GIS > Enterprise GIS > ArcGIS for Server discussions. I think your question on scripting geodatabase administration is better positioned here than in the GeoEvent area. Best Regards - RJ ... View more

Marc - Is it possible that your system has a Locale other than 'English'? Numeric values in the simulation file use ‘.’ as the decimal separator, rather than a ‘,’ as is typical in many European locales. The GeoEvent Extension will assume your server’s locale if the Language for Number Formatting parameter is unspecified in the TCP/Text input you've configured to receive the simulated data. If this is the case, you need to specify en_US for the Language for Number Formatting parameter (see attached illustration). Module 2 in the Introduction to GeoEvent tutorial was updated July 30th (2015) to clarify this. Please download the latest tutorial and refer to pages 29 - 31 which discusses configuring the TCP/Text input for the exercise. Hope this information helps - RJ ... View more

The http://brgov.com/reports/public/brtrxml.xml feed provides the current date as an attribute of the trafficincidentlist structure. Since each inbound traffic record only specifies a local time, it is difficult for GeoEvent to determine a proper epoch timestamp in milliseconds to associate with each event. We can work with this. By polling the same feed, in a different way, we can cache the header information in a feature service’s feature class, an then use that cache to enrich each of the individual events we receive from the feed. Consider the attached GeoEvent Definition illustration. This event definition specifies that only the trafficincidentlist.time and trafficincidentlist.date attributes should be pulled from the feed. Further, the trafficincidentlist.time is to be handled as a Date, not as a String. If we configure a new ‘Poll an External Website for XML‘ input, we can poll the same feed. This time, however, I leave the 'XML Object Name' property unspecified, so this new input will retrieve the base date for every traffic incident. I need to specify an ‘Expected Date Format’ so that the input knows not to expect a time value as part of the received date string. (Refer to attached illustration of the ‘Poll an External Website for XML‘ input.) I can then incorporate this second input into my GeoEvent Service, using a Field Mapper to map the retrieved Date value to a schema consistent with a feature class in my geodatabase. I use a Field Calculator to hard-code the URL being polled for data as a TRACK_ID for these “features”. I use an ‘Update a Feature’ output to update the base date, in epoch milliseconds, as a feature in the feature class. This updates our cached epochbase attribute as a feature in a feature service. (Refer to illustration detailing the use of Field Mapper and Field Calculator with the fs-out output updating the feature service.) Now we can retrieve the epochbase attribute from the feature service and use it to enrich each incoming traffic event from the city’s feed. To do this, we add additional fields (feedidentifier, hours, and minutes) to our BatonRouge-TrafficIncidents event definition. These fields are not provided by the feed, but adding them to the event definition provides us attributes into which we can write data. We also add a Field Enricher to enrich the incoming traffic events with the Date from the feature service ... and a Field Calculator to extract the hours and minutes as substrings from each event’s time attribute. An additional Field Calculator can then be used to add the hours and minutes to the epochbase – creating a fully-qualified date/time value for each event. (Refer to illustration ‘Final GeoEvent Service’). Admittedly, this is a lot to go through to pull information from a XML feed’s header and incorporate it into individual event records extracted from a feed’s list. But it illustrates several concepts of what you can do using a combination of processors to manipulate data obtained from a feed. Also, it appears that the GeoEvent input, configured to expect only MM/dd/yy for the Date, assumes that the date must be local. But when the Date is written to a feature service, clients retrieving the value will assume that the value has been expressed in epoch milliseconds GMT, so displaying the manufactured date in a web map will probably be artificially offset from GMT to your server's local time. To complete the solution we really should correct the locally reported Baton Rouge time by adding +5 hours to push the value forward and represent it as a GMT / UTC value, when caching it in a feature service. Hope this information is helpful – RJ ... View more

Hello William – I had no problems bringing in your traffic incidents feed from the City of Baton Rouge. I had to remove the input’s default 'application/xml' specification from my configured GeoEvent input's 'Acceptable MIME Types’ parameter. I also had to specify trafficincident as the 'XML Object Name' so that the input would know how to extract individual events from the feed's structure. I cannot be sure from the feed whether using the location_number attribute as a TRACK_ID is appropriate. Identifying a field within the feed's data which can be used to uniquely identify each event will be fairly important. The input is receiving all available event records every polling cycle, and if we elect to broadcast the event data out a stream service, or update features in a feature service, we'll need a TRACK_ID in order to visualize the data properly on a map. Because the feed’s structure includes the date as an attribute of the trafficincidentlist structure, rather than as part of each trafficincident item, we have a small challenge to overcome. More on that in a moment. Attached are illustrations of my configuration of the ‘Poll an External Website for XML‘ input and the GeoEvent Definition I configured my input to use. I allowed the input to create an initial GeoEvent Definition for me, then copied the generated one and configured my input to use my copy of the event definition. This is a fairly simple best practice. Hope this information helps – RJ ... View more

Please be aware that when using the Twitter for GeoEvent connector you are registering with Twitter as a developer and are using their public API and are subject to Twitter's terms and conditions. For example, the twitter input for GeoEvent is only going to receive a sample of the tweets being sent by Twitter's users since that is what their public API provides to developers. The GeoEvent team conducted an extended proof-of-concept to demonstrate that, if the twitter connector is configured to follow a specific list of registered users, you *will* receive all of the tweets from those users. The "sampling" is enforced as a threshold, so following a dozen specific people you are unlikely to find that group tweeting above the threshold established by the public API. If you were to filter all tweets for a currently trending (popular) hashtag, on the other hand, you would likely receive only a sample of the tweets being sent. Additional information is in the following FAQ. - RJ ... View more