Latest Contributions by RJSunderman

Omar - There was a tutorial prepared for working with RSS feeds, but it did not get uploaded with the 10.2.0 product release when we discovered some issues with consistently updating feature services hosted by ArcGIS Online. The Working with RSS tutorial is being reviewed and updated for release with the 10.2.1 product. However, to hopefully provide you some immediate help, I've attached what I have as the latest draft of the tutorial with its supporting map package. Please keep in mind that the attached material is still a draft. An updated tutorial package should be posted to the GeoEvent Processor product gallery by the end of the year. - RJ ... View more

Omar - Please visit our ArcGIS GeoEvent Processor product gallery for the latest available tutorials and connectors. We will be updating some of the tutorials, gallery items, and adding new tutorials/connectors as we finish our documentation efforts for the 10.2.1 release. Note that if you visit the gallery and do not see anything listed, to check the box 'Show ArcGIS Desktop Content'. This should be checked by default, but I've seen the checkbox cleared with the page first loads in some browsers. - RJ ... View more

Hello Ingeborg - Your best option for viewing Twitter posts on a map will be to push the GeoEvent data as Feature JSON (as opposed to Generic JSON) out to a web socket and use either Chrome or Firefox to create a "stream layer". This leverages JavaScript and WebSocket technology to stream the twitter events in near real time to a map for visualization. The Twitter tutorial is being reworked for the 10.2.1 product release. The approach above will be reflected in the tutorial. Twitter's terms of service prohibit developers from caching or storing data harvested from tweets - which means that you cannot use the GEP Cache or Feature Service outputs and remain compliant with the Twitter TOS. Attached is a PDF with draft material from the tutorial which is currently being developed. Hope this helps you in the near term. - RJ ... View more

Hello Dennis - Interpreting a JSON structure as simple comma separated text when the JSON contains groups and fields whose cardinality is greater than one can be confusing. This is the reason for the known limit when non-trivial JSON event data is output to either TCP/Text or CSV/File output. The behavior will be the same at the 10.2.1 release as it was at the 10.2.0 release. As Adam suggests, the correct way to view your output would be to use a JSON/File output or write the event data to GEP"?s cache using the "?Publish GeoEvents on a REST endpoint"? and then query the endpoint using the f=json formatting specification. Support for editing a GeoEvent Definition generated by GEP when the event definition has fields of type group and/or fields whose cardinality is greater than one is limited at the 10.2.0 release. Product enhancements at 10.2.1 provide the functionality I believe you are looking for. The event definition generated by GEP will depend on the structure of the first JSON event received. As you observed, fields whose value is null are omitted from the field definition. At 10.2.1 you will be able to edit the event definition to explicitly add fields and specify data types for the fields whose published values are occasionally null. Event data which contains fields of type group and/or fields whose cardinality is greater than one cannot be used to update a feature service without first flattening the data structure. This is the reason the Field Mapper processor element was originally developed. Since the fields in a feature class must be discrete types (e.g. String, Long, Double, Date, etc.) you would use a GEP field mapper to map the data from the JSON field loc.longitude to a field Longitude of type Double. Your last question, to explain "field groups and fields with multiple cardinality"� is probably best addressed with an illustration. In the image below (click to enlarge), the JSON structure has a single field, a group, named FlightGroup. FlightGroup has nested within it two other groups, Pilot and FlightInfo, as well as simple/flat fields (Flight and Altitude). Within the group FlightInfo you"?ll see an element named geometry which is actually a list containing two elements. Thus you see that groups can contain groups, and sub-groups can contain lists (fields with multiple cardinality) [ATTACH=CONFIG]29122[/ATTACH] The value for the flight"?s origin airport would be:  FlightGroup.FlightInfo.OriginAirportCode The geometry value of the second list item, the one with the non-zero "?z"? value, would be: FlightGroup.FlightInfo.geometry[1] Hope this information helps - RJ ... View more

Adam - NIM095738 has been addressed in the 10.2.1 product release which should be publicly available this December. What we found was that GEP was clearing its cache before it found that it was unable to reach the feature service containing the enrichment data. Unable to refresh its now empty cache, the processor in the GeoEvent Service had no enrichment data and was therefore failing to enrich any events it received. The fix was to mark the cache as stale, rather than clearing it. If the feature service containing the authoritative enrichment data cannot be reached, the processor will continue using the "stale" data from its cache until the cache can be refreshed. ... View more

Sergio / Adam - Thanks for the feedback and the using the ideas portal to vote up product enhancements. We are actually working on this capability as part of this development iteration for the 10.2.2 product, which should be out for Dev Summit (Spring 2014). Best Regards - RJ ... View more

Hello Darryl - When questioning the availability and reliability of systems or servers external to GeoEvent Processor, please keep in mind that GEP was developed as a processor of real-time event data. Considering your first example, when a data provider goes off-line, recovery depends on the actions taken by the data provider when they come back on-line. GEP listens for and polls data providers in real-time, receiving and processing immediately whatever a provider sends or has published. GEP does not manage its inputs with a record of the date/time a particular input last received event data; it is not designed to observe a lapse and issue a request for data it thinks it might have missed. If your GEP input were watching a system folder for files and a provider elected to send multiple files to account for a period of time the provider was offline, then GEP will receive the files and process their data as "new" events. If your GEP input were binding to a TCP or UDP port listening for event data, then GEP will receive whatever data the provider sends to that network port. If the provider is offline and not broadcasting then there is nothing for GEP to receive. If your GEP input were polling an external HTTP or RSS feed and the site/feed was unavailable when GEP conducted its poll, GEP simply receives no events for that polling interval. On the next polling interval it will poll and receive whatever data the site/feed has published once the provider comes back online. Considering your second example, when a network outage prevents a data sink from receiving data sent by GEP, the only recovery would be for GEP to resend event data it previously sent -- but you would have to assume that the receiver would be able to identify and discard duplicate event it received from GEP since there is no way for GEP to know whether the data it previously sent was received successfully or for GEP to determine whether data it has sent should be resent. There is no acknowledgement returned to GEP by a data receiver. For example, suppose you were using GEP to push JSON to an external website, but the server hosting your IIS went down. The GEP output would attempt to POST the data over HTTP, but the requests containing the data from GEP will not be received by the external server and there is no way for GEP to know this. I suppose you could design some redundancy into your GeoEvent Services by having GEP cache a copy of events it outputs, using the 'Publish GeoEvents on a REST endpoint' output. Then you could have a second GeoEvent Service poll this cache to re-ingest the cached data as new events and periodically re-send the data ... but there's no mechanism to determine whether resending the data was actually necessary, or whether the second time you sent the data was any more successful than the first time you sent it. Also, you'd have to manage the items in the GEP cache so that you didn't send a particular event more than 'N' times. Worse, in high volume cases, you would have to worry about GEP's cache maintenance discarding events as the cache fills up such that a particular set of events are not available to be resent. Best Regards - RJ ... View more

There is a pretty wide field of potential with regard to GPS and AVL using GeoEvent Processor. For example, we have developed a connector for the RAP protocol to support SierraWireless devices. We also have supported clients with Zonar Systems and Network Fleet. We've conducted some testing with TomTom WebFleet and our DC Regional office staff has developed a connector for InReach (the Iridium satellite-based service for areas where other types of communication are not available). We've also performed some testing with GPS Insight whose data services can be polled as CSV Text over HTTP - you will need to massage the data from Insight some to get rid of the header row. Finally, we have a connector for the GPS variant of NMEA which some folks have used to turn an iPad / iPhone into a simple asset location monitoring device. There's not a lot of detailed documentation on integrating these solutions as they're all very new. I would recommend contacting your Esri Sales Representative to discuss the possibilities of working with a technical adviser or getting a quote for support from Esri Professional Services. Hope this information is helpful - RJ ... View more

Hello Vassilo �?? The development team has not researched development of a connector for the specific package you mention (the AP5000 from Vaisala). We typically look to the community to build out these types of connectors. In the case of the Universal ASCII Lightning Format (UALF) however, you may be able to leverage out-of-the-box components. Briefly looking over the documents at the links below, it looks like UALF is simple ascii text, tab delimited, numeric fields. This would suggest that you could pair the existing Text adapter with an appropriate transport to design a custom connector without necessarily needing to leverage the GeoEvent Processor API to develop any actual code. Link:  AP5000" rel="nofollow" target="_blank">http://www.hobeco.net/pdf/ap5000v1.7_users__book.pdf]AP5000 User Guide Link:  CP7000/CP8000" rel="nofollow" target="_blank">http://somemanuals.com/VAISALA%2CCP7000%2CIBAHJDG.PDF]CP7000/CP8000 User Guide In general, this is the workflow we follow when custom connectors (or other GEP service components such as custom processors) are needed: Step 1:  We typically look for partners to build the components needed to support their respective systems. A couple of examples: - Exact Earth is building a connector for their AIS Vessel data - We have AGI building a connector to output to their CESIUM web client. Partners will be making custom connectors available to the community via the GeoEvent Processor gallery via the �??Partner Gallery�??. Step 2:  If that does not make sense, we look to Esri solution teams to build out the needed components for their respective domains: A couple of examples - The Emergency Response team is building out a CAP (Common Alerting Protocol) connector - The Defense team is planning on building out connectors for military formats/systems that are being demand by their customer base These solution teams would make their connectors available as part of their solution and/or via the GeoEvent Processor gallery. Step 3:  If that does not make sense or cannot be supported then Esri Professional services could be contracted to build the needed components as a consulting project. If it could be agreed to, the development would ideally be made available to the community via the product gallery.  I would imagine in most cases though that the components built would only be available to the customer since they are funding the work as a consulting project. Step 4:  If none of the above are feasible, then the customer (or someone else) can build a connector that meets their needs using the GeoEvent Software Development Kit (SDK). I hope you find this information helpful. Best Regards �?? RJ ... View more

Yaser - if you are referring to the sort of analysis available through GP Tools such as those in the system tools Spatial Statistics toolbox of ArcGIS Desktop, then no, this is not supported at the 10.2.0 release of GeoEvent Processor. Keep in mind that when we refer to "processing" and "analysis" within GEP we are referring to calculations performed on field values, enriching events with data pulled from external data table(s), tagging events with data values based on their spatial relationship with GeoFences, and incident detection based on attribute and spatial conditions ... we are not talking about the sort of processing and analysis performed on feature data using ArcGIS Desktop. Work is being conducted to develop a custom processor element which enable a published GP service to be invoked when event data is received by GeoEvent Processor. This custom processor would be an element a GeoEvent Service designer would incorporate into a GeoEvent Service to send attribute data from a received GeoEvent to an external GP Service. However, the work being conducted is not generic for any GP service; it is for specific types of GP services within one of the Solutions verticals here within Esri. I encourage you to submit ideas for functional enhancements to the ArcGIS" rel="nofollow" target="_blank">http://ideas.arcgis.com/]ArcGIS Ideas portal. If you tag your suggestion with 'GeoEvent Processor' and submit it under the 'ArcGIS Server' category it will help get the idea routed to the GEP team. Ideas which receive votes have a better chance of being prioritized within our product backlog of functionality and enhancements. ... View more

Attached is a summary of the steps needed to use "?Receive Features on a REST endpoint"? input connector as a single PDF. ... View more

Hello Q �?? Here�??s a summary of the steps you need to follow to use �??Receive Features on a REST endpoint�?? input connector. The forum only allows five images to be uploaded per post, so I've included a PDF of the remarks below with additional screenshots as a separate reply. 1) Create an Input using the provided �??Receive Features on a REST endpoint�?? input connector template     a.  I�??d recommend configuring the input connector to create a GeoEvent Definition for you: [ATTACH=CONFIG]28448[/ATTACH] 2) Create an Output using the provided �??Write to a .json file�?? output connector template     a.  You�??ll need to register a system folder with GEP as a Data Store     b.  I registered C:\arcgisserver\GeoEvent_Files as �??GeoEvent_Files�?? and have my connectors         write to an �??output�?? sub-folder (see screenshot below) [ATTACH=CONFIG]28449[/ATTACH] 3) Design and publish the GeoEvent Service which uses the input and output connectors to receive     the feature JSON and write the created GeoEvents out to a system file as a *.json text file. [ATTACH=CONFIG]28450[/ATTACH] 4) Obtain some valid feature JSON for posting.    a.  An easy way to do this is to query your local ArcGIS for Server sample service �?� [ATTACH=CONFIG]28452[/ATTACH] Note:   You can copy the entire block of feature JSON, or you can locate and copy just one of the features: [ATTACH=CONFIG]28451[/ATTACH] 5) Download an HTML Poster application for your preferred browser    a.  http://code.google.com/p/chrome-poster is one such app for Chrome 6) Configure the poster to identify the MIME type in its header     a.  �??Content-Type�??  / �??application/json�?? (see screenshot below) 7) Post the JSON block to the GeoEvent Processor�??s REST endpoint     a.  http://localhost:6180/geoevent/rest/receiver/<name-of-the-input> (e.g. rest-features-in) 😎 Locate your output *.json file in your registered system folder. You should be able to format the JSON as a list by adding an opeingin �??[�?? and closing �??]�?? set of brackets around the text in the file �?� and replacing every �?? {�?? with �??,{�?? so that the individual JSON blocks are comma separated.  Here�??s what I got for a query of the World Cities whose OBJECTID is less-or-equal to 10. [     {         "CITY_NAME": "Brasilia",         "POP": 2207718,         "POP_RANK": 2,         "POP_CLASS": "1,000,000 to 4,999,999",         "LABEL_FLAG": 0,         "shape": {             "x": -47.897705078125,             "y": -15.7921142578125,             "z": 0,             "spatialReference": {                 "wkid": 4326             }         }     },     {         "CITY_NAME": "Goiania",         "POP": 1171195,         "POP_RANK": 2,         "POP_CLASS": "1,000,000 to 4,999,999",         "LABEL_FLAG": 0,         "shape": {             "x": -49.2550048828125,             "y": -16.72698974609375,             "z": 0,             "spatialReference": {                 "wkid": 4326             }         }     },     {         "CITY_NAME": "Cuiaba",         "POP": 521934,         "POP_RANK": 3,         "POP_CLASS": "500,000 to 999,999",         "LABEL_FLAG": 0,         "shape": {             "x": -56.093017578125,             "y": -15.614990234375,             "z": 0,             "spatialReference": {                 "wkid": 4326             }         }     },     {         "CITY_NAME": "Campo Grande",         "POP": 729151,         "POP_RANK": 3,         "POP_CLASS": "500,000 to 999,999",         "LABEL_FLAG": 0,         "shape": {             "x": -54.61590576171875,             "y": -20.45098876953125,             "z": 0,             "spatialReference": {                 "wkid": 4326             }         }     },     {         "CITY_NAME": "Salto del Guaira",         "POP": 7385,         "POP_RANK": 7,         "POP_CLASS": "Less than 50,000",         "LABEL_FLAG": 0,         "shape": {             "x": -54.2833251953125,             "y": -24.04998779296875,             "z": 0,             "spatialReference": {                 "wkid": 4326             }         }     },     {         "CITY_NAME": "Encarnacion",         "POP": 74983,         "POP_RANK": 6,         "POP_CLASS": "50,000 to 99,999",         "LABEL_FLAG": 0,         "shape": {             "x": -55.85101318359375,             "y": -27.37701416015625,             "z": 0,             "spatialReference": {                 "wkid": 4326             }         }     },     {         "CITY_NAME": "Posadas",         "POP": 312060,         "POP_RANK": 4,         "POP_CLASS": "250,000 to 499,999",         "LABEL_FLAG": 0,         "shape": {             "x": -55.906005859375,             "y": -27.39801025390625,             "z": 0,             "spatialReference": {                 "wkid": 4326             }         }     },     {         "CITY_NAME": "Puerto Maldonado",         "POP": 37543,         "POP_RANK": 7,         "POP_CLASS": "Less than 50,000",         "LABEL_FLAG": 0,         "shape": {             "x": -69.1920166015625,             "y": -12.60302734375,             "z": 0,             "spatialReference": {                 "wkid": 4326             }         }     } ] ... View more

Hey Bradley - When you indicate "whenever my server gets restarted", are you saying whenever the server machine itself is rebooted? Do you see this same behavior (opening the Service Designer and finding no services) when you stop and restart the ArcGISGeoEventProcessor windows service? Do you typically exit/close the Service Designer application and the GeoEvent Processor Manager web page before the restart occurs? I believe you are describing a different manifestation of an issue we"?ve seen before. Let"?s say that for some reason a published GeoEvent Service is not valid when the GeoEvent Processor starts up. Maybe an input component was deleted or failed to load; maybe the bundle supporting a processor component of the service is missing; maybe the GEP configuration was not persisted to disk correctly due to an abrupt system shutdown "? there could be several possible reasons... If GeoEvent Processor encounters an invalid service on startup, at 10.2.0 the service will not be loaded into the framework. There have been improvements made with 10.2.1 which handle this situation more defensibly. Also with 10.2.1, the application you know as the Service Designer is being retired and its functionality incorporated into the GeoEvent Processor Manager "? so you won"?t have two separate application which you need to "?refresh"? or use to configure and manage your inputs, outputs, and services. If you"?d like, send us your GeoEventConfig.xml and we can review your configuration. Regardless, I"?m comfortable saying that when the 10.2.1 release is available "? sometime in the next several weeks "? you won"?t continue to experience this issue. Best Regards - RJ ... View more

Referring to your screenshot, the username discovered by the GEP installer, arcgis, does not contain .\ The .\arcgis you see in the dialog indicates that the user arcgis is a local user, not a domain or AD user (e.g. MyDomain\user1234). I have worked to recreate your issue, using a local user arcgis on Windows 2012, installing ArcGIS Server 10.2 and the 10.2 release of GEP. I made sure that the local user"?s password included an "?_"? character. Unfortunately, I was unable to reproduce the issue. The GEP installer is discovering the user configured to run the "?ArcGIS Server"? service. GEP will create and configure its service ("?ArcGIS GeoEvent Processor"?) to run using the same account. That is why you are unable to change the username from within the GEP installation wizard; you must authenticate by providing the discovered user"?s password and allow GEP to create and configure the Windows service used to run the GEP product. The two Windows system services are run using the same operating system account as part of the prerequisite that GEP is run as an extension to ArcGIS Server. At this point your best option is to open an incident with Esri Technical Support. They have the ability to remotely view your system and work with you to resolve the product installation issue. Best Regards - RJ ... View more

You would not normally need to do this, but to convince myself that I do indeed have the correct username and password for the ArcGIS Server windows service I have gone so far as to stop the AGS service, edit the service"?s properties to change the "?Log On"? to an invalid password, verify that the service will not start, then restore the "?Log On"? properties to what I believe the username and password ought to be and verify that I can restart the ArcGIS Server service. Once I have verified that I have the correct username and password, I"?ve always been able to authenticate as part of the GeoEvent Processor installation wizard. My own experience and that of folks I"?ve helped is normally that we think we have the correct password when in fact we don"?t. Is the user running your ArcGIS Server windows service a local user (default is "?arcgis"?)? Does the local user"?s password have any special characters:  < > / : ! @ # $ % ^ & * ( )  ? ... View more