Latest Contributions by RJSunderman

The Expected Date Format property is only applicable to a single attribute field whose data type, as specified by the event record's GeoEvent Definition, is a Date. An inbound connector, like the one you are using (SystemFile/Text) would have to specify an existing GeoEvent Definition to use (vs. allowing the input to create one for you) in order to specify which attribute field(s) should be handled as Date values. However, you can only specify a single Expected Date Format mask, which means that both your "date" and your "time" would have to be part of a single attribute. An inbound adapter cannot handle the conversion of a "date" and a "time" when the values are in separate attribute fields. None of the configurable processors in the GeoEvent Server releases up through the current release (10.8.1) provide a way to "cast" data from a String to a Date. This means your only opportunity to adapt a string representation of a date/time and create an actual Date value is as part of an inbound connector's adapter . Once data has passed from an inbound connector into a GeoEvent Service you won't be able to "cast" a value to a Date unless the value happens to be an epoch value such as 784041330000. The good news, looking forward, is that the 10.9 release will have a toDate( ) function which both the Field Mapper and the Field Calculator will be able to use. The new function will allow two or more string values from separate attributes to be combined with a literal string to produce an ISO 8601 formatted date/time string. You will be able to send data to GeoEvent Server with a "time" and a "date" in separate fields ... for example: "dateString": "1994-11-05" and "timeString": "08:15:30-05:00" ... and write an expression such as toDate(dateString + 'T' + timeString) to create a single date/time string and write the result out to a field of type Date. - RJ reference: https://www.w3.org/TR/NOTE-datetime

Hello Ctal - I would ask that you please submit an incident with Esri Technical support so that someone can be assigned specifically to follow up with you. If you would please include a sample of the tab-delimited data your File/Text inbound connector is receiving, that will help reproduce the issue. I'm going to take a guess and assume that you have configured a Watch a Folder for New CSV Files input using the input's Expected Date Format property to specify a Java SimpleDateFormat string. Data coming from your SQL database has only the "time" portion of a date/time string and thus is not one of the few well-known string formats GeoEvent Server recognizes for specifying a date/time value. GeoEvent Server uses epoch values (in milliseconds) to specify Date values. Both date and time are inherently part of every Date value since the values are not a time-of-day string but rather a number of milliseconds since the Unix Epoch (Midnight Jan 1 1970). GeoEvent Server's inbound adapter will not be able to cast a string value such as "23:45:15" to a Date value "(today) 11:45:15 PM (time zone) 2020" unless the string representation of the date/time incorporates a date as well as a time. Hope this information is helpful – RJ

‎11-05-2020

In this blog we will take a deeper look at registering server connections with GeoEvent Server, something administrators commonly have to do to when configuring a GeoEvent Server deployment. Why do you have to register an ArcGIS Server connection with GeoEvent Server There are several different configurable components which require you to select a registered server and specify which services folder, map/feature service, and often a specific feature layer exposed by that service in order to use the feature layer's schema or feature records. A few examples include: The Poll an ArcGIS Server for Features input used to query a service for feature records. When you want to import a GeoEvent Definition from a feature service's schema . The Field Enricher (Feature Service) processor used to load a feature record set to use for enrichment. The Add a Feature and Update a Feature outputs used to persist data from processed event records as feature records in a geodatabase. All of the user-interfaces in the above examples populate their options from a cache of information GeoEvent Server collects when it queries a registered ArcGIS Server to discover published feature services. Service discovery is not performed the moment you click to open the panel. The cache is created and updated in the background because it can take several seconds – sometimes minutes or even tens of minutes – for GeoEvent Server to completely crawl the ArcGIS Server's REST Services Directory and query the information it needs from all of the available services. Which leads into the topic I want to address in this blog: Is there a way to know when service discovery is being run and how long it is expected to take? Using the ArcGIS Server Connection component to log messages You can configure the following component logger to request DEBUG messages be logged: com.esri.ges.datastore.agsconnection.DefaultArcGISServerConnection I've included a sample of the logged messages you will be looking for at the end of this article. Requesting this component logger include DEBUG messages in its logging will allow you to see, in the system log file, when the service discovery kicks off and which map/feature services it interrogates to learn about their layers. A message will be logged for every feature service being interrogated as well as a success message when service discovery is complete. There is no indication in the GeoEvent Manager web application that service discovery is about to start or is currently running. The best way to tell that service discovery is running is to start the workflow to import a GeoEvent Definition. If the blue/white indicator displays, requesting you "please wait", you know that the GeoEvent Server is busy updating its ArcGIS Services cache. Otherwise the GeoEvent Definition user-interface will display immediately allowing you to choose a server connection, folder, feature service, and feature layer. GeoEvent Manager does not allow you to configure, or schedule, when service discovery should take place. You are able to change the Discovery Rate for each server connection you register to specify how frequently a refresh should be performed. The default for recent software releases is every 60 minutes. I have personally found that when I use ArcGIS Pro, the Enterprise portal, or GeoEvent Manager to publish a new feature service, I want to use it now – so it is not uncommon for me to publish a feature service and immediately request GeoEvent Server run a service discovery to update its cache by clicking the refresh button on the server connection I have registered as a Data Store. This eliminates the need to have GeoEvent Server periodically refresh the cache for me, so I usually set the Discovery Rate for server connections I register to a fairly large value like 1440 minutes so that service discovery is run once once per day (or when GeoEvent Server is stopped and restarted). Service discovery takes too long and interferes with normal operations. Is there anything I can do? This is something the product team is working on. Refactoring GeoEvent Server to support all of the operations which use feature services, however, to interface with ArcGIS Server some other way is both high risk and high reward. Several different design options have been considered, but implementation has had to be deferred for each of the last several major releases. The best option for now, therefore, is to limit the number of features services which are discoverable each time service discovery is run. A recommended best practice is to configure your GeoEvent Server Data Store (e.g. the server connection you are registering with GeoEvent Server) with credentials. The user credentials do not have to be an administrative user, just a user who owns and published the feature service. You can use the Enterprise portal content item manager to assign existing feature services a new owner and configure GeoEvent Server to authenticate as that user when crawling the ArcGIS Server's REST Services Directory to limit the number of discoverable services. GeoEvent Server administrators often configure their registered server connections with the ArcGIS Server or Enterprise portal primary administrative account – which naturally sees all published services. If you can identify the feature services to which you want GeoEvent Server to write data, or from which you want GeoEvent Server to retrieve feature records or a feature layer's schema, and assign ownership of just those services to a user set aside specifically for your "real-time" data, you can improve service discovery considerably by not crawling all of the feature services being maintained by more traditional feature editing workflows. The trick is to identify the feature services your GeoEvent Server components actually care about and limit discovery to only those feature services. Example messages logged by the ArcGIS Server Connection component logger 2020 - 11 - 05T13 : 55 : 02 , 124 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | sleep interrupted java . lang . InterruptedException : sleep interrupted at java . lang . Thread . sleep ( Native Method ) ~ [ ? : ? ] at com . esri . ges . datastore . agsconnection . DefaultArcGISServerConnection$CacheUpdater . run ( DefaultArcGISServerConnection . java : 235 ) [ 56 : com . esri . ges . framework . datastore . agsconnection - datastore : 10.8 . 1 ] 2020 - 11 - 05T13 : 55 : 02 , 124 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Exiting Cache Updater run method . . . . 2020 - 11 - 05T13 : 55 : 03 , 274 | DEBUG | qtp808353329 - 598 | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Create an instance of CacheUpdater . . . . 2020 - 11 - 05T13 : 55 : 03 , 274 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating cache for DataStore Public_Esri_Hosted_Server . . . 2020 - 11 - 05T13 : 55 : 08 , 944 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Active_Hurricanes_Sampler" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 15 , 296 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Active_Hurricanes_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 21 , 613 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Air_Quality_PM25_Latest_Results" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 23 , 311 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "ASAM_events_V1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 25 , 033 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Coral_Reef_Stations" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 27 , 876 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "GDELT_Health_Pandemic" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 29 , 170 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "GDELT_v1_Social_Tones" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 30 , 368 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "IHME_Projected_Peaks_QA" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 31 , 995 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Median_Sea_Ice_Extent_for_the_Antarctic" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 32 , 995 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Median_Sea_Ice_Extent_for_the_Arctic" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 34 , 063 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "MODIS_Thermal_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 35 , 225 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "National_Farmers_Market_Directory" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 36 , 445 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "ncov_cases" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 38 , 421 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NDFD_Ice_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 40 , 579 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NDFD_Precipitation_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 43 , 479 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NDFD_SnowFall_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 45 , 920 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NDFD_WindForecast_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 49 , 935 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NDFD_WindGust_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 51 , 519 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NDFD_WindSpeed_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 53 , 124 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NDGD_SmokeForecast_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 54 , 193 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NOAA_METAR_current_wind_speed_direction_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 55 , 683 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NOAA_short_term_warnings_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 55 : 58 , 401 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NOAA_storm_reports_Sampler" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 00 , 844 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NOAA_storm_reports_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 06 , 494 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NWS_Watches_Warnings_Sampler" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 12 , 942 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "NWS_Watches_Warnings_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 19 , 622 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Recent_Hurricanes_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 21 , 726 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Satellite_VIIRS_Thermal_Hotspots_and_Fire_Activity" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 22 , 871 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "seaice_extent_N_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 23 , 976 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "seaice_extent_S_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 25 , 081 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "SPI_recent" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 28 , 250 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "Standardized_Precipitation_Index_(SPI)" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 31 , 492 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "US_Cases_per_county_(time)" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 33 , 073 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "USA_Wildfires_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 35 , 074 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating layers for Service ( Folder : "/" , Name : "USGS_Seismic_Data_v1" , Type : "FeatureServer" ) . 2020 - 11 - 05T13 : 56 : 36 , 595 | DEBUG | Public_Esri_Hosted_Server - Updater | DefaultArcGISServerConnection | 56 - com . esri . ges . framework . datastore . agsconnection - datastore - 10.8 . 1 | Updating cache for DataStore Public_Esri_Hosted_Server done . Success : true . ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ You can see from the above that discovery of 36 feature services hosted by an external, public server, took just over 90 seconds. The more services there are to discover, the longer service discovery will take. See Also: GeoEvent Server > Administer > Data stores in the GeoEvent Server on-line help Debug Techniques - Configuring the application logger GeoEvent Server blog series GeoEvent Configuration: Data Store Connections w/Tokens blog by Eric Ironside

‎10-12-2020

Hello jess neuner – You're correct that clicking Publish Stream Service a "second" time is not going to show you the existing service's configuration. All that does is open the dialog to allow you to begin selecting (again) the properties you want to specify to the service publication mechanism in order to publish a service. I'm assuming you toggled the Store Latest capability when originally publishing the stream service, and want to know which feature service the stream service is using to store latest observations? Your best bet, if I'm following your question, is to browse to the stream service in the ArcGIS REST Services Directory. At the top of the service's web page, in the upper-left hand corner, you should see a JSON link; clicking this will open a browser tab with the service's raw JSON specification. In there you will find things like which field is being used as a track identifier (derived from the field tagged TRACK_ID in the selected GeoEvent Definition) and the URL of any supporting feature services for capabilities like "Store Latest" and "Related Features". If you open an incident with Esri Technical Support, we can probably publish a stream service with store latest similar to how you have in your environment, then deliberately use the stream service to broadcast some data which the feature service would not be able to accept or persist as a feature record in the geodatabase and see how that condition gets logged in the GeoEvent Server's karaf.log ... Hope this information is helpful – RJ

‎08-27-2020

Matej – Using my test data above, I can configure three MCFS (multicardinal field splitter) processors to split the data first on the group element metric_collection , then on the group element Level1_Group3 , and finally on the array Group3_Items . Following this pattern I think you'll be able to achieve what you need to do. Above, I've illustrated the three event records routed from the final field splitter to the output. Please make sure you have downloaded the latest release of the field splitter processor bundle (Release 8 - June 24, 2020) for release 10.8.1 of ArcGIS. This version includes changes which allow multiple MCFS processors to be arrayed in series. If you are using an earlier release of ArcGIS, please use Release 7 (February 27, 2020). Hope this information is helpful – RJ

‎08-26-2020

Hello Matej – Unlike a Field Mapper or Field Calculator which needs you to specify the full "path" into a data structure, the Field Splitter and in particular Multicardinal Field Splitter processors only expect the name of the element at which you want the split to be applied. So the Field to Split parameter's value should only specify the element's base name, linearWithinLinearGNElement , not the full path to the element (e.g. groupOfLocations.globalNetworkLinear.linearWithinLinearGNElement ). That said, you may have found a limit of what the processor is able to handle in terms of a deeply nested, hierarchical, multicardinal, data structure. To test what you are trying to do I created some sample JSON data which adapts using the GeoEvent Definition illustrated below. Note that it does not matter whether the data received is formatted as XML or JSON – the inbound connector has to adapt whatever data is received to create an event record which can be routed to a GeoEvent Service for processing. After receiving the JSON above and allowing the inbound connector to adapt it using the illustrated GeoEvent Definition, I can use a Field Mapper to extract, for example, the data value "Delta" (line 21 in the sample JSON) by specifying the full path down through the data structure to the "E301" element in the array: metric_collection.Level1_Group3.Group3_Items[1].E301 If I specify the same full path when configuring the Multicardinal Field Splitter, I see the error message you were able to capture from the system log: Field name <name> is invalid . However, if I specify only Group3_Items (the name of the array) as the field on which to conduct the split, I see a different error message in the log indicating a null pointer exception was encountered. I am in contact with one of our developers to review the source code of this processor. We will see what we can do to better handle the case where Group3_Items (the name of the array) is specified and why a null pointer is not being caught. The work around, for now, is to use multiple Multicardinal Field Splitter processors in series and "walk down" the hierarchy from the top to the multicardinal element on which you want the final split applied. I'll reply to this post with what I tested so you can see what I had to do to implement the work around. Please make sure you have downloaded the latest release of the field splitter processor bundle (Release 8 - June 24, 2020) for release 10.8.1 of ArcGIS. This version includes changes which allow multiple MCFS processors to be arrayed in series. If you are using an earlier release of ArcGIS, please use Release 7 (February 27, 2020). Hope this information is helpful – RJ

‎08-18-2020

Hello Jitendrudu – When replies were written for the original thread Re: GeoEvent process Oracle/SQL connector I don't think the GeoEvent Manager supported publishing a feature service. The functional server role for licensing hadn’t been invented yet and GeoEvent Server was still being referred to as the GeoEvent Processor Extension for ArcGIS Server. In recent releases, however, you can use GeoEvent Manager to publish Hosted Feature Layers to your Enterprise portal or map/feature services to a stand-alone ArcGIS Server. So on the one hand, nothing has changed with regard to the interface between GeoEvent Server and an RDMBS table. The interface is still a feature service since GeoEvent Server can only make REST requests against an Esri ArcGIS Server feature service to add or update feature records. But on the other, you can now publish feature services which do not necessarily have Geometry. A feature service is typically published using a client, like ArcMap, which will not allow you to publish a feature service without a geometry. ArcMap defines a feature class as a collection of feature records and a feature record is a data structure with both geometry and attributes describing some object in the real-world. GeoEvent doesn’t make the same assumptions regarding Geometry that ArcMap does though. Real-time data streaming from a sensor whose position does not change wouldn’t necessarily include a geometry or coordinate values with its data. GeoEvent Server doesn’t assume that all data records will have geometry. To GeoEvent Server, “geometry” is just another type of attribute like a String or a Date. Say, for example, you were to create a GeoEvent Definition with two attribute fields – one a Date the other a String – and didn’t include a Geometry field type. You can use any release of GeoEvent after 10.6.1 to publish a feature service to the ArcGIS Server beneath which GeoEvent Server is running. Without a Geometry, to any other client, the feature class's table looks like a non-spatial table. But ArcGIS Server provides a feature service interface to the table allowing you to write the Date and String data you want to it without having a Geometry. You would still configure an Update a Feature output, and made sure to use a Field Mapper to map whatever schema your inbound connector uses to adapt data you receive to the feature service’s schema (PostGRE, for example, insists on using only lower-case field names; Oracle only upper-case field names). You shouldn't have any problem ingesting some sample data that has no geometry and using GeoEvent Server to add/update “feature records” via the feature service. The data records in the RDBMS simply have no geometry … so you won’t be able to add them to a web map as a feature layer for example, but I can query the data at REST. I assume if I had some other database client able to query from the data table I’d be able to retrieve the data that way, rather than going through the feature service’s REST interface to query the data. This would be a lot easier that developing a custom outbound transport which understood how to connect to the RDBMS using ODBC. This is possible, but I don’t know how feasible it is, really, given the inter-dependency of bundles in the Java system framework that underlies GeoEvent Server. GeoEvent Server’s whole design, in this case, assumes that you’ll be able to work through a feature service to access feature records. Hope this information is helpful – RJ

‎08-18-2020

‌ Hello René – You are correct. Documentation newly updated with the 10.8.1 release, particularly the on-line help topics Load Balancing and Strategies for scalability, reliability, and resiliency, no longer discuss what I have referred to as a 'site' deployment approach (where multiple ArcGIS Server instances, each with a GeoEvent Server, are organized in a single ArcGIS Server site). Please take a look at the newly prepared on-line documentation; we've added quite a bit and have plans for more by the end of the year. Moving forward, the product team is recommending that all GeoEvent Server deployments follow a 'silo' deployment pattern. The distinction comes when you decide whether you have to deploy the same configuration to each machine, so you can use an external mechanism to route or distribute event records to two otherwise identical instances in parallel, or whether you can deploy a different configuration to each machine (allowing one instance to ingest event records of one type and another instance to handle event records of some other type). Nothing in the product's implementation changed with regard to allowing system architects to pursue the previously documented strategies of deploying multiple ArcGIS Server instances, each with a GeoEvent Server, in a single ArcGIS Server site. However, the added complexity in system administration and occasional recovery required when failures in hardware or software components occur tend to outweigh system design objectives for resiliency and high-availability. If you would like an informal write-up I prepared looking at some concerns surrounding this, please e-mail me directly: rsunderman .at. esri.com Hope this information is helpful – RJ

‎08-18-2020

Hello nasser hinai – I think what you want to do is use an Incident Detector with an opening condition which determines an asset has "entered" and area of interest (registered as a geofence) and set the closing condition to evaluate true when the asset's location has "exited" the geofence. The event record output from the Incident Detector will be an incident which has a very different schema from the event record you routed to the processor. While you'll lose a bunch of attributes from the original record, you will retain whatever field you tagged TRACK_ID and also GEOMETRY. The incident event record will also have a duration which you can pass through a filter. If the asset were to dwell within an area of interest for some time you could use a filter to check that: (a) the incident's status is 'Ended' (b) the incident's duration is greater than some value. Then whatever event records the filter allows to pass you send to an output configured to send an e-mail notification. Hope this information is helpful – RJ

‎08-18-2020

Hello Jose – To be clear, you are developing a custom web application which complements or reproduces what you can do using the GeoEvent Manager? So you're authenticating with the GeoEvent Server's administrative REST API to (probably automate) the creation of a bunch of processors, publishing a workflow connecting these processors as a GeoEvent Service. Does the service publish, albeit in an error state? If the service publishes you can export an XML representation of the configuration and inspect that to look for where the GeoEvent Service publication and validation is detecting a loop. There is no hard limit I'm aware of for how many processors you can place in a GeoEvent Service or how many different event flow connections you bring into or extent out from a node (e.g a processor or filter). There are certainly practical limits, and you should understand that every time a node handles an event there is some non-zero latency associated with that operation. Complicated real-time analytics applied through complicated event processing workflows through GeoEvent Services can severely impact the overall event records you can expect a GeoEvent Server instance to be able to handle. I have attached a simple illustration of an XML snapshot from an export of a GeoEvent Server configuration. Reading the XML, you'll be able to review the <flow> of a GeoEvent Service and see the <from> and <to> connections between every node. Hopefully the illustration helps with an audit / inspection of your more complicated GeoEvent Service. – RJ

Online Status	Offline
Date Last Visited	Wednesday

My Ideas

Latest Contributions by RJSunderman

Re: Geoevent Erroneously Imports Date Field

Re: Geoevent Erroneously Imports Date Field

Monitoring a registered web server's service discovery

Re: Stream Service - Store latest, review configuration

Re: Question on Multicardinal Field Splitter Processor

Re: Question on Multicardinal Field Splitter Processor

Re: JDBC connector for Oracle/SQL connector

Re: GeoEvent Server High Availability

Re: Send email notification by calculating the time difference

Re: Loop Errors in GeoEvent

ArcGIS Analyse