Latest Contributions by RJSunderman

Hey Darryl �?? When configuring a �??Subscribe to an external Web Socket for JSON�?? Input, I think the pattern you want to use for the �??Remote server web socket URI�?? value is:  ws://<server><domain>:<port>/<service> So for the ExactEarth sample data on geoeventsample1 the value would be: ws://geoeventsample1.esri.com:8080/exactearthservice Morakot on our team helped me track this down. Here�??s the rough steps we followed to figure it out�?� Browse to http://geoeventsample1.esri.com/demosite in a Chrome Browser Select the �??ExactEarth Vessel Tracking�?? sample service Ctrl-Shift-I to open Chrome�??s Developer tools Under the �??Sources�?? tab, expand to view the HTML code for the �??exactearthservice�?? Set a breakpoint in the init : function() to pause script execution as the WebSocket is about to be created Clicking the �??Connect�?? button in the displayed webpage allowed us inspect the value of the location variable and see the URI being used. Plugging that value into our websocket-json-in input connector, we immediately began receiving JSON event data from the sample service (allowing GEP to create the GeoEvent Definition for us based on its discovery of the event data). Hope that helps �?? RJ ... View more

Hello Darryl - Thank you for checking the product gallery, and apologies for not having up there what you needed. We did prepare a tutorial Working with WebSockets, but didn't post it because it wasn't a very complete user story. The tutorial, as it stands, does not reference a really good input data stream. That said, there is still information in the tutorial you might find useful. I am, in fact, incorporating information from it as I rework the tutorial for the Twitter connector. I've attached what we have for Working with WebSockets ... please let me know if you find in there what you need and/or discover something missing that I can help provide. - RJ ... View more

Hey Ingeborg, Sorry for the confusion ... using a stream layer to visualize the geographic locations contained in tweets should be compliant with the Twitter "Rules". We are reworking the tutorial exercise to not collect the twitter content in a feature class; this I feel is a violation of the terms of service. I've attached a file 'WebSocketsSupportFiles.zip' which contains the supporting files from a draft tutorial on working with WebSockets. - RJ ... View more

Omar - You will want to create an Input using the 'Poll an external website for JSON' connector. The workflow is very similar to polling an RSS feed for events. The difference is that the connector is polling a website for data, rather than an RSS feed, and is using a JSON adapter rather than an RSS adapter. You might find the workflow documented on the thread esri-receive-features-rest-in helpful. But keep in mind that you will want to adapt the workflow to configure an Input to poll an external website for generic JSON, rather than receiving JSON posted to GeoEvent Processor. Hope this information helps - RJ ... View more

Omar - There was a tutorial prepared for working with RSS feeds, but it did not get uploaded with the 10.2.0 product release when we discovered some issues with consistently updating feature services hosted by ArcGIS Online. The Working with RSS tutorial is being reviewed and updated for release with the 10.2.1 product. However, to hopefully provide you some immediate help, I've attached what I have as the latest draft of the tutorial with its supporting map package. Please keep in mind that the attached material is still a draft. An updated tutorial package should be posted to the GeoEvent Processor product gallery by the end of the year. - RJ ... View more

Omar - Please visit our ArcGIS GeoEvent Processor product gallery for the latest available tutorials and connectors. We will be updating some of the tutorials, gallery items, and adding new tutorials/connectors as we finish our documentation efforts for the 10.2.1 release. Note that if you visit the gallery and do not see anything listed, to check the box 'Show ArcGIS Desktop Content'. This should be checked by default, but I've seen the checkbox cleared with the page first loads in some browsers. - RJ ... View more

Hello Ingeborg - Your best option for viewing Twitter posts on a map will be to push the GeoEvent data as Feature JSON (as opposed to Generic JSON) out to a web socket and use either Chrome or Firefox to create a "stream layer". This leverages JavaScript and WebSocket technology to stream the twitter events in near real time to a map for visualization. The Twitter tutorial is being reworked for the 10.2.1 product release. The approach above will be reflected in the tutorial. Twitter's terms of service prohibit developers from caching or storing data harvested from tweets - which means that you cannot use the GEP Cache or Feature Service outputs and remain compliant with the Twitter TOS. Attached is a PDF with draft material from the tutorial which is currently being developed. Hope this helps you in the near term. - RJ ... View more

Hello Dennis - Interpreting a JSON structure as simple comma separated text when the JSON contains groups and fields whose cardinality is greater than one can be confusing. This is the reason for the known limit when non-trivial JSON event data is output to either TCP/Text or CSV/File output. The behavior will be the same at the 10.2.1 release as it was at the 10.2.0 release. As Adam suggests, the correct way to view your output would be to use a JSON/File output or write the event data to GEP"?s cache using the "?Publish GeoEvents on a REST endpoint"? and then query the endpoint using the f=json formatting specification. Support for editing a GeoEvent Definition generated by GEP when the event definition has fields of type group and/or fields whose cardinality is greater than one is limited at the 10.2.0 release. Product enhancements at 10.2.1 provide the functionality I believe you are looking for. The event definition generated by GEP will depend on the structure of the first JSON event received. As you observed, fields whose value is null are omitted from the field definition. At 10.2.1 you will be able to edit the event definition to explicitly add fields and specify data types for the fields whose published values are occasionally null. Event data which contains fields of type group and/or fields whose cardinality is greater than one cannot be used to update a feature service without first flattening the data structure. This is the reason the Field Mapper processor element was originally developed. Since the fields in a feature class must be discrete types (e.g. String, Long, Double, Date, etc.) you would use a GEP field mapper to map the data from the JSON field loc.longitude to a field Longitude of type Double. Your last question, to explain "field groups and fields with multiple cardinality"� is probably best addressed with an illustration. In the image below (click to enlarge), the JSON structure has a single field, a group, named FlightGroup. FlightGroup has nested within it two other groups, Pilot and FlightInfo, as well as simple/flat fields (Flight and Altitude). Within the group FlightInfo you"?ll see an element named geometry which is actually a list containing two elements. Thus you see that groups can contain groups, and sub-groups can contain lists (fields with multiple cardinality) [ATTACH=CONFIG]29122[/ATTACH] The value for the flight"?s origin airport would be:  FlightGroup.FlightInfo.OriginAirportCode The geometry value of the second list item, the one with the non-zero "?z"? value, would be: FlightGroup.FlightInfo.geometry[1] Hope this information helps - RJ ... View more

Adam - NIM095738 has been addressed in the 10.2.1 product release which should be publicly available this December. What we found was that GEP was clearing its cache before it found that it was unable to reach the feature service containing the enrichment data. Unable to refresh its now empty cache, the processor in the GeoEvent Service had no enrichment data and was therefore failing to enrich any events it received. The fix was to mark the cache as stale, rather than clearing it. If the feature service containing the authoritative enrichment data cannot be reached, the processor will continue using the "stale" data from its cache until the cache can be refreshed. ... View more

Sergio / Adam - Thanks for the feedback and the using the ideas portal to vote up product enhancements. We are actually working on this capability as part of this development iteration for the 10.2.2 product, which should be out for Dev Summit (Spring 2014). Best Regards - RJ ... View more

Hello Darryl - When questioning the availability and reliability of systems or servers external to GeoEvent Processor, please keep in mind that GEP was developed as a processor of real-time event data. Considering your first example, when a data provider goes off-line, recovery depends on the actions taken by the data provider when they come back on-line. GEP listens for and polls data providers in real-time, receiving and processing immediately whatever a provider sends or has published. GEP does not manage its inputs with a record of the date/time a particular input last received event data; it is not designed to observe a lapse and issue a request for data it thinks it might have missed. If your GEP input were watching a system folder for files and a provider elected to send multiple files to account for a period of time the provider was offline, then GEP will receive the files and process their data as "new" events. If your GEP input were binding to a TCP or UDP port listening for event data, then GEP will receive whatever data the provider sends to that network port. If the provider is offline and not broadcasting then there is nothing for GEP to receive. If your GEP input were polling an external HTTP or RSS feed and the site/feed was unavailable when GEP conducted its poll, GEP simply receives no events for that polling interval. On the next polling interval it will poll and receive whatever data the site/feed has published once the provider comes back online. Considering your second example, when a network outage prevents a data sink from receiving data sent by GEP, the only recovery would be for GEP to resend event data it previously sent -- but you would have to assume that the receiver would be able to identify and discard duplicate event it received from GEP since there is no way for GEP to know whether the data it previously sent was received successfully or for GEP to determine whether data it has sent should be resent. There is no acknowledgement returned to GEP by a data receiver. For example, suppose you were using GEP to push JSON to an external website, but the server hosting your IIS went down. The GEP output would attempt to POST the data over HTTP, but the requests containing the data from GEP will not be received by the external server and there is no way for GEP to know this. I suppose you could design some redundancy into your GeoEvent Services by having GEP cache a copy of events it outputs, using the 'Publish GeoEvents on a REST endpoint' output. Then you could have a second GeoEvent Service poll this cache to re-ingest the cached data as new events and periodically re-send the data ... but there's no mechanism to determine whether resending the data was actually necessary, or whether the second time you sent the data was any more successful than the first time you sent it. Also, you'd have to manage the items in the GEP cache so that you didn't send a particular event more than 'N' times. Worse, in high volume cases, you would have to worry about GEP's cache maintenance discarding events as the cache fills up such that a particular set of events are not available to be resent. Best Regards - RJ ... View more

There is a pretty wide field of potential with regard to GPS and AVL using GeoEvent Processor. For example, we have developed a connector for the RAP protocol to support SierraWireless devices. We also have supported clients with Zonar Systems and Network Fleet. We've conducted some testing with TomTom WebFleet and our DC Regional office staff has developed a connector for InReach (the Iridium satellite-based service for areas where other types of communication are not available). We've also performed some testing with GPS Insight whose data services can be polled as CSV Text over HTTP - you will need to massage the data from Insight some to get rid of the header row. Finally, we have a connector for the GPS variant of NMEA which some folks have used to turn an iPad / iPhone into a simple asset location monitoring device. There's not a lot of detailed documentation on integrating these solutions as they're all very new. I would recommend contacting your Esri Sales Representative to discuss the possibilities of working with a technical adviser or getting a quote for support from Esri Professional Services. Hope this information is helpful - RJ ... View more

Hello Vassilo �?? The development team has not researched development of a connector for the specific package you mention (the AP5000 from Vaisala). We typically look to the community to build out these types of connectors. In the case of the Universal ASCII Lightning Format (UALF) however, you may be able to leverage out-of-the-box components. Briefly looking over the documents at the links below, it looks like UALF is simple ascii text, tab delimited, numeric fields. This would suggest that you could pair the existing Text adapter with an appropriate transport to design a custom connector without necessarily needing to leverage the GeoEvent Processor API to develop any actual code. Link:  AP5000" rel="nofollow" target="_blank">http://www.hobeco.net/pdf/ap5000v1.7_users__book.pdf]AP5000 User Guide Link:  CP7000/CP8000" rel="nofollow" target="_blank">http://somemanuals.com/VAISALA%2CCP7000%2CIBAHJDG.PDF]CP7000/CP8000 User Guide In general, this is the workflow we follow when custom connectors (or other GEP service components such as custom processors) are needed: Step 1:  We typically look for partners to build the components needed to support their respective systems. A couple of examples: - Exact Earth is building a connector for their AIS Vessel data - We have AGI building a connector to output to their CESIUM web client. Partners will be making custom connectors available to the community via the GeoEvent Processor gallery via the �??Partner Gallery�??. Step 2:  If that does not make sense, we look to Esri solution teams to build out the needed components for their respective domains: A couple of examples - The Emergency Response team is building out a CAP (Common Alerting Protocol) connector - The Defense team is planning on building out connectors for military formats/systems that are being demand by their customer base These solution teams would make their connectors available as part of their solution and/or via the GeoEvent Processor gallery. Step 3:  If that does not make sense or cannot be supported then Esri Professional services could be contracted to build the needed components as a consulting project. If it could be agreed to, the development would ideally be made available to the community via the product gallery.  I would imagine in most cases though that the components built would only be available to the customer since they are funding the work as a consulting project. Step 4:  If none of the above are feasible, then the customer (or someone else) can build a connector that meets their needs using the GeoEvent Software Development Kit (SDK). I hope you find this information helpful. Best Regards �?? RJ ... View more

Yaser - if you are referring to the sort of analysis available through GP Tools such as those in the system tools Spatial Statistics toolbox of ArcGIS Desktop, then no, this is not supported at the 10.2.0 release of GeoEvent Processor. Keep in mind that when we refer to "processing" and "analysis" within GEP we are referring to calculations performed on field values, enriching events with data pulled from external data table(s), tagging events with data values based on their spatial relationship with GeoFences, and incident detection based on attribute and spatial conditions ... we are not talking about the sort of processing and analysis performed on feature data using ArcGIS Desktop. Work is being conducted to develop a custom processor element which enable a published GP service to be invoked when event data is received by GeoEvent Processor. This custom processor would be an element a GeoEvent Service designer would incorporate into a GeoEvent Service to send attribute data from a received GeoEvent to an external GP Service. However, the work being conducted is not generic for any GP service; it is for specific types of GP services within one of the Solutions verticals here within Esri. I encourage you to submit ideas for functional enhancements to the ArcGIS" rel="nofollow" target="_blank">http://ideas.arcgis.com/]ArcGIS Ideas portal. If you tag your suggestion with 'GeoEvent Processor' and submit it under the 'ArcGIS Server' category it will help get the idea routed to the GEP team. Ideas which receive votes have a better chance of being prioritized within our product backlog of functionality and enhancements. ... View more

Attached is a summary of the steps needed to use "?Receive Features on a REST endpoint"? input connector as a single PDF. ... View more