Latest Contributions by RJSunderman

‎05-22-2015

Please refer to the thread Error log entry on the GeoEvent subspace. Responses to this question are being posted there.

‎05-21-2015

Beginning with the 10.3 product release you have some additional options available. Enhancements to the Text adapter -- used by the 'Push Text to an External TCP Socket' and 'Write to a CSV File' out-of-the-box connectors -- allow you to specify that you want a custom String representation of a Date: The configuration is usually found beneath the 'Advanced' properties of the connector, as illustrated above. This feature is not supported on the MessageFormatter adapter, used by outbound connectors such as 'Send a Text Message' or 'Send an Email'. The enhancement was only made for the Text adapter. Hope this information helps - RJ

‎05-21-2015

Prior to the 10.3 release, if you wanted to output a custom string representation of a date/time value, you would have to follow a process like K DeVogelaere suggests above. Polling event data from a feature service for input, as G Tiemans indicates, and bringing the date/time values into GeoEvent using the Date data type prevents you from later obtaining the days / months / years or hours / minutes / seconds values from the Date. GeoEvent does not provide an interface like you might find in Java (illustrated below) to pull discrete values out of a Date: Thus, you would have to broadcast the Date value out a GeoEvent output and re-ingest it as a String in order to use the substring() function supported by the Field Calculator processor for string manipulation. You cannot bring the date/time in from the feature service as a String because the feature service's JSON represents the date/time as a long integer in epoch milliseconds.

‎05-21-2015

If you suspect that ArcGIS Server has failed to correctly configure the RabbitMQ platform service: In a Windows file explorer, browse to the ArcGIS Server configuration store. The default location of the AGS config store is: C:\arcgisserver\config-store The path to your configuration store is reported by Server Manager. Navigate to Site > Configuration Store to observe the path reported by Server Manager. In your Windows file explorer, browse to the …\config-store\locks folder. The only file you should find here should be a file named site.rlock If you discover any files in …\config-store\locks whose names match the pattern machine-<MACHINE_NAME>.wlock or machine-<MACHINE_NAME>.wlock.info: Stop the GeoEvent Windows Service Stop the ArcGIS Server Windows Service Examine the running tasks on the GIS Server Force stop any processes still running that were launched from the ArcGIS Server Installation directory. Click the illustration below to see a larger representation of the illustration. The illustration above only shows a couple of the processes frequently found to still be running when the RabbitMQ platform service failed to initialize correctly and the ArcGIS Server Windows Service has been shut down. With AGS stopped, there should be no processes running whose command line path indicates they were launched from either: - C:\Program Files\ArcGIS\Server - C:\PROGRA~1\ArcGIS\Server Back in your Windows file explorer, in the …\config-store\locks folder Delete any *.wlock or *.wlock.info files you find. Again, site.rlock is the only lock file you should need to keep. Restart ArcGIS Server Windows Service on your GIS Server. Once you are able to launch and log-in to the ArcGIS Server Manager web application, go ahead and restart the GeoEvent Windows Service.

‎05-21-2015

Sharon - The log messages you provide above look to me like the Rabbit MQ message broker service (provided by ArcGIS Server) has failed or is not running. Are you able to launch the GeoEvent Manager web application? Can you create a simple TCP/Text input -> TCP/Text output GeoEvent Service? If you send event data to the TCP input socket using GeoEvent Simulator, does the input's event count increment, but the 'In'/'Out' event count for the GeoEvent Service and event count on the output remain at zero (do not increment)? There are some trouble shooting steps I can share with you that are found in the product tutorial for clustering. Technically even if you are configuring a Server site with only a single server machine, you are still running a cluster - just with only one machine. Before we go into that, however, I'd like to get your feedback on the questions above. Thanks - RJ

‎05-21-2015

Hello Al / Alexander - My apologies that no one has replied to you on this post. The ArcGIS GeoEvent extension does not have an out-of-the-box connector for the Motorola ICC Pro. Generally GeoEvent does not provide connectors for specific commercial devices. When considering GeoEvent for real-time event processing the questions we need to ask are more general. How is the data being broadcast such that GeoEvent could receive it? Can data from the device be sent as an HTTP/POST to a REST endpoint hosted by GeoEvent, for example? Can the data be streamed over a TCP socket or WebSocket? Once we've figured out how the data will be delivered to GeoEvent, we need to know what format the data will be in. Will the data be broadcast in a generic XML structure, as generic JSON, or as comma-separated text, for example? There are adapters for the above data formats available out-of-the-box. If the ICC Pro were broadcasting its data in some other data format, then a custom adapter would need to be developed to interpret the data being sent by the device. A developer would use the provided GeoEvent SDK to develop the needed adapter. Al indicated that he was successful in getting the ICC Pro data written out to SQL Server tables. You might want to take a look at the thread GeoEvent process Oracle/SQL connector which describes some options for retrieving this data. Generally speaking, the Esri Feature Service is the interface GeoEvent relies on to access data in enterprise database tables. GeoEvent does not support direct database table queries out-of-the-box. We are preparing a response to a recent post How can I put a depedency in a custom processor. The response will likely be very technical and intended for developers. The developer in this case needs advice on incorporating a 3rd party JDBC component into the Apache Karaf OSGi container GeoEvent uses to manage the various components it uses during runtime. Hope this information is of some use to you - RJ

‎05-20-2015

This blog has been updated as part of a new series describing debugging techniques you can use when working to identify the root cause of an issue with a GeoEvent Server deployment or configuration. The original blog's text is included below, however, please consider the new blogs which can be accessed by clicking any of the quick links below: Debug Techniques - Configuring the application logger Debug Techniques - Add/Update Feature Outputs Debug Techniques - Application logging tips and tricks Debug Techniques - Geofence synchronization deep dive How to debug the Add a Feature and Update a Feature Output Connectors is probably the question I have been asked the most over the last couple of years working on GeoEvent Extension development team – so it’s appropriate that my inaugural blog to GeoNet address it. The scenario: An input appears to be successfully receiving and adapting the data from the stream and creating GeoEvents. The Filters and/or processors incorporated in a GeoEvent Service are handling the GeoEvents as expected. The event count on an Add a Feature or Update a Feature Output Connector is incrementing, but no features are being added or updated in the targeted feature layer. So, how do you start debugging to determine what the problem might be? My advice is to enable DEBUG logging on the feature service outbound transport to see if we can capture the JSON request being sent to ArcGIS Server and the response GeoEvent Extension receives from ArcGIS Server. I’ve attached an image below (FeatureServiceUpdate.png) of a karaf.log I created a while ago which shows the transactions taking place when a Output Connector performs an HTTP/POST to update features in a feature service. Don’t be concerned that the illustration identifies the 10.2.2 version – the concepts and workflows are the same for the GeoEvent 10.3 and 10.4 product releases when using a traditional RDBMS. To enable DEBUG logging on a single component in GeoEvent Manager: Navigate to the Logs page and click the Settings button. Enter the logging component in the text field Logger and select the DEBUG log level. Click Save. In this case you want to log DEBUG messages for the com.esri.ges.transport.featureService.FeatureServiceOutboundTransport component only. Setting a logging level of DEBUG on the ROOT component is not recommended. Doing this will produce a very verbose set of log messages and can cause the Logs page in the GeoEvent Manager to 'hang' as it tries to refresh the rapidly updating logs. With DEBUG logging enabled for the specified component, the GeoEvent Extension will produce more detailed logs when the feature service outbound transport handles event data. The DEBUG logging statements will include the JSON being sent and the ArcGIS Server’s response. I prefer looking at the log in a text editor, rather than using the log manager in GeoEvent Manager. You can find the karaf.log in the default folder C:\Program Files\ArcGIS\Server\GeoEventProcessor\data\log. In the FeatureServiceUpdate.png that is attached, find the two messages time stamped 2014-03-28 14:06:09,074. The “querying for missing track id XXXX” messages indicate the GeoEvent Extension has discovered that it has not cached any information on features with the TRACK_ID “SWA1568” or “SWA510”. Looking at the third message in the series, it shows the SQL where clause used to query the …/FeatureServer/0/query REST endpoint to discover the necessary OBJECTID values. The response from the ArcGIS Server includes a JSON array features[ ] with the geometry, OBJECTID, and unique identifier field (flightNumber in this example) for the features with the flight identifiers “SWA1568” and “SWA510”. Notice that it took 175 milliseconds for the GeoEvent Extension to receive the ArcGIS Server response: 14:06:09,250 - 14:06:09,75 = 75 ms You might find this query/response latency information valuable when profiling / debugging your GeoEvent Services which are adding or updating features through a feature service. Once the GeoEvent Extension has the necessary OBJECTID values for the features it wants to update, it posts a block of JSON to the …/FeatureServer/0/updateFeatures REST endpoint. ArcGIS Server responds with “success”:true 325 milliseconds later: (577 - 252 = 325). If you find too many JSON event records being included in the transactions, making the log file difficult to read, you can try configuring your Update a Feature Output Connector to specify that the ‘Maximum Features Per Transaction’ should be limited to 1 (the default is 500). This obviously not something you would do in a production environment, but while debugging it can make the log file much easier to read. If you find that the log is rolling over too frequently, you can edit settings in the following configuration file to allow the karaf.log to grow larger than 1MB and to keep more than 10 archival log files: ...\Program Files\ArcGIS\Server\GeoEvent\etc\org.ops4j.pax.logging.cfg (Click the thumbnail above to open a larger view) The logging package changed with the 10.6.0 release to use version 2.x of Log4J. Settings applicable for the log4j2.appender are illustrated in the attached ops4j.pax.logging.cfg.png file. You can also edit the message formatting specified by the layout.ConversionPattern in this configuration file to reformat the messages being written to the karaf.log - more information on that can be found here: http://www.codejava.net/coding/common-conversion-patterns-for-log4js-patternlayout Hope this information helps – RJ

‎05-20-2015

Hello Brian – Yes, one of the limitations of the File transport used by the ‘Watch a Folder for New CSV Files’ inbound connector is that it does not monitor file size or the date/time a file was last modified. The inbound connector will cache the name of any file it has previously read and will not re-read the contents of a file with that filename. This is by design; the connector does this to prevent ingesting event data which has already been processed. Edit: 23-Jan-2018 Behavior for the ‘Watch a Folder for New CSV Files’ inbound connector was changed at 10.5.1 to no longer require that a file’s name be changed for the input to consider it a new file. The mechanism watching the folder for new files still does not consider file properties such as changes to a file’s “last updated” timestamp or file size. However, if you want an input to re-read files you’ve placed in a folder, you can simply stop and restart your input connector and each file’s content will re-read with its content processed as newly received event records. The work your service is doing to download event content as a system file, uniquely name that file using a GUID, and place it in a system folder GeoEvent is watching should be sufficient to work around this limitation. We have an item open in our product backlog related to watching a system folder for JSON files in which files larger than about 5100 bytes are not ingested by the inbound connector watching the system folder. The root issue here may be similar to what you are observing - that some CSV files your service has copied to a system folder are not being picked-up by GeoEvent. There are several problems inherent when working with file-based input. One example is that GeoEvent may attempt to acquire a handle to a file before an external process is finished writing the file to disk. Another example is related to the issue above; GeoEvent needs to read the contents of a file into memory in “chunks” and may encounter a problem emulating a data stream when retrieving blocks of event data from a system file. Inputs which watch a system folder for files are generally intended to prove that event filtering and real-time analytics you have designed into a GeoEvent Service behave as you intend. They give you the ability to repeatedly send a small sample of event data to GeoEvent to test the behavior of your GeoEvent Service before transitioning to a production configuration in which real-time data feeds arrive via a stream (e.g. as an HTTP/POST from an external server, or as a reply to a query you make on an external server's URL). Are you able to identify specific CSV file content which GeoEvent is not retrieving? Is GeoEvent ever able to successfully retrieve data from those files? Or is it only occasionally skipping a file, and if you were to stop and restart the ‘Watch a Folder for New CSV Files’ input it would read the file the “second” time? If you can consistently reproduce the issue, please open an incident with Esri Technical Support. It would be helpful if you could provide samples of the data or help Tech Support reproduce the issue so that we can determine if you’re encountering a known limitation of the GeoEvent File transport or if there’s a bug we can address. Hope this information helps – RJ

‎05-07-2015

Hello Daniel - I see that from your post that you're working with Esri on mapping the fields from the feed you are receiving from Network Fleet in order to update features in a feature class. Esri Technical Support should be able to help you with this, but as a reference, you might want to take a look at Module 4 of the product introduction tutorial. Specifically pages 30-31 and 36-38 which illustrate how a Field Mapper is used to change the fields / schema of a GeoEvent. Mapping a GeoEvent from one schema or event definition to another so that you can update features through a feature service is primarily what the Field Mapper processor is used for. You can download the product tutorial from our Product Gallery. Here's a link to the tutorials: http://links.esri.com/geoevent-tutorials Hope this information helps - RJ

‎05-07-2015

Hello Thomas - What Daniel describes is certainly viable, and probably the preferred way to allow Verizon to HTTP / POST data to your ArcGIS Server on-premises. To answer your second question, GeoEvent is an extension to ArcGIS Server and requires ArcGIS Server be installed. So, no, you cannot install GeoEvent on a server which does not have ArcGIS Server. An alternative to having your IT staff create tunnels for Verizon IPs would be to stand-up an ArcGIS Server and GeoEvent Extension in the cloud, say on Amazon EC2. This server would be public facing. Because Web Sockets use port 80 you could then have your primary on-premises server connect to a Web Socket hosed by the public facing server in the cloud ... and then relay event information from the GeoEvent running in the cloud to the GeoEvent you have running behind your firewall. The downside to this approach is that you have to license ArcGIS Server and the GeoEvent Extension for both the server you have stood-up in the cloud and the server you have on-premises. That's why I think Daniel's approach would be preferred - but if opening a tunnel for Verizon to use to access your on-premises server is not an option, then at least you have an alternative. Hope this information helps - RJ

‎05-01-2015

Hello Larry - The incident and TrackGap GeoEvent Definitions are owned by their respective core processors - the Incident Detector and Track Gap Detector. While you were able to edit these event definitions prior to 10.3, it is not recommended. Each processor's implementation expects the GeoEvent Definition structure you see immediately following product installation. You can copy the out-of-the-box event definition and make any changes you need to your copy of the GeoEvent Definition: Just be sure to direct the GeoEvent output from your Incident Detector or Track Gap Detector through a Field Mapper processor to map the event data to a structure you want to use. Hope this information helps - RJ

‎05-01-2015

So why is it that the geoevent Extension stops after 4-6 hours? There was an issue we observed, primarily with updating feature services hosted via ArcGIS Online in the 10.3 release. A few users also observed the issue using local feature services they had published to their on-premises Server. The symptom was that GeoEvent would appear to be receiving and processing data (its event counters would increment), but updates to features through a feature service would stop after a variable length of time. We usually found messages in the karaf.log with the text Error: null response or Error Posting to URL ... java.io.IOException: null respons when this specific issue would manifest. The issue was addressed with the 10.3 Patch 1 for GeoEvent, available here from Esri Support. I am not sure if the issue was actually present in earlier releases (10.2.x). In any case, I'd encourage you to upgrade to the 10.3.1 release which should be publicly available in just a couple of weeks (13-May 2015). The focus for 10.3.1 was on improving the product's stability, however, we did complete a product enhancement with regard to registering an ArcGIS Server connection as a GeoEvent Data Store. You can now specify whether you want to use a token for authentication or if you want to use web-tier authentication and provide a username / password when registering a GeoEvent Data Store. This is new at the 10.3.1 release. If we connect to server 2 with the directy URL then we the error below: properly due a certificate error. Invalid URL: Please provide a valid url for example: http://<hostname>:<port>/arcgis There were enhancements in the area of security certificates introduced in both the 10.3 and 10.3.1 releases. GeoEvent should trust the ArcGIS Server certificate presented by the local server on which it was installed at the 10.3 release (rather than relying on a GeoEvent self-signed certificate). With the 10.3.1 release GeoEvent will also trust any SSL certificates you have imported using the ArcGIS Server Administrator Directory. After you've upgraded to the latest release, if you find that, after working for a period of time, feature updates stop working, you see log messages you believe are security or SSL certificate related, or need advice on configuring your enterprise's reverse proxy or web-tier authentication ... please go ahead and open new GeoNet discussion threads for the separate issues. Questions on enterprise system architecture and security configuration I will probably have to refer to Esri Tech Support. I'm better positioned to offer advice on questions concerning GeoEvent product functionality. Best Regards - RJ

‎04-30-2015

Hello Greg - I do not believe GeoEvent will support what you are trying to do for a couple of reasons. First, GeoEvent does not provide any sort of node-level cache which you can access.The "nodes" in a GeoEvent Service (inputs, outputs, filters, processors) generally do not cache information from events they receive. Every node handles the GeoEvents it receives independently. There are a few exceptions, but the information being cached in these cases is very specific to the node's function and is not accessible for you to extract or use. Second, I don't believe there is any syntax for variable substitution built-in to an inbound connector's configuration. If you take a look at my reply to Thread_510827 (which illustrates how to include an event field name in a spatial filtering expression), you'll see that there are a few places within GeoEvent that you can specify ${fieldName} in place of a literal string and GeoEvent will dynamically replace the "macro" ${fieldName} with a value taken from the GeoEvent currently being handled by that node. Because there is no variable substitution syntax built-in to an inbound connector's configuration, whatever text you enter to configure an input must be the literal text value. I'm trying to think if recommending you explore using the GeoEvent SDK to develop a custom transport would support what you are trying to do. I suppose it would be possible to implement a self-modifying transport which would specify, say, the URL used initially to poll an external server. Then, after data is received from a poll, if the transport can retrieve data values from the adapter, the transport could use one or more of the received data values values to - possibly conditionally - reconfigure itself before the next poll is conducted. Keep in mind, though, that it is the responsibility of an adapter to take the byte stream from the connector's transport and "adapt" it to create a GeoEvent. You are looking at using the SDK here to develop a custom adapter and transport, functionality found right up front before a GeoEvent is even passed to a GeoEvent Service for filtering / processing ... and I don't know if member variables can be accessed between a transport object's classes an adapter object's classes. The SDK may not even support what you are trying to do. Hope this information helps - RJ

‎04-30-2015

Hello Undral – I believe the functionality you are looking for is an enhancement being introduced in the 10.3.1 product. Let’s say you design a GeoEvent Service with a filter as illustrated below: In this example the GeoFence category is still specified as a constant String, AdminAreas. But the GeoFence name is dynamic: ${TrackID}. Notice the required decoration around the value indicating to GeoEvent that TrackID is the name of a field, not a literal string. In this example, only GeoEvents whose TrackID attribute value matches the name of the GeoFence the event’s Geometry is found to be INSIDE will be allowed through the filter. Rather than a tcp-text-out you would use an output suitable for notification such as Send an Email. Hope this information helps – RJ

‎04-29-2015

Felix, A colleague mentioned that while the Java AIS library https://github.com/tbsalling/aismessages is open source, it does require a commercial license. Since TB Salling is what James referenced in his post, I don't think James will be able to share the custom adapter he built - it's probably licensed for use by their client / customer. There are a few other Java AIS libraries out there: https://github.com/dma-ais/AisLib (Java 😎 https://github.com/schwehr/libais (built on a core C++ lib) Some other tools / hints you might find helpful: Use telnet to check the AIS stream. This downloads page from nmearouter.com provides a desktop application which can connect to an AIS stream and decode messages. The site http://catb.org/gpsd/AIVDM.html has good descriptions of all AIS messages. Some Python and Java AIS parsers built on a fast C++ core are available from https://github.com/schwehr/libais Some additional parsers can be found here: https://github.com/bcl/aisparser It was also mentioned that serial to TCP/IP hardware adapters were a big help to the development team. They allowed access the AIS transponders remotely. A big thank you to the Esri Australia team for the information above. You can reach out to them via their distributor page: https://esriaustralia.com.au/contact-us

Online Status	Offline
Date Last Visited	‎05-21-2026 12:39 PM

My Ideas

Latest Contributions by RJSunderman

Re: GeoEvent Processor question

Re: Formatting the date in the MessageFormatter adapter (outbound)

Re: Formatting the date in the MessageFormatter adapter (outbound)

Re: Error log entry:

Re: Error log entry:

Re: GeoEvent server as OPC client?

Debugging the Add a Feature / Update a Feature Output Connectors

Re: Watching for new CSV Files

Re: GeoEvent Update Feature

Re: Geoevent Processor and Verizon Networkfleet

Re: Incident Detector Processor 10.3 GeoEvent Definition

Re: GEP 10.2.1 Datastore - Cannot register AGS w/ Windows AD & non-public services

Re: Poll External Website for JSON with variables

Re: One to One relationship between GeoFence and GeoEvent

Re: Ais decoder

Re: Support for string manipulation and pattern ma...

Support for string manipulation and pattern matchi...

Re: GeoEvent input connector using token authentic...

Re: Updating the Status of a Fire Unit

Re: Is there a way to delete records from a Featur...

ArcGIS Arcade

ArcGIS Analyse