Latest Contributions by RJSunderman

‎04-30-2021

hey @Ctal_GISsquad The default time that geoevent creates is going to be UTC. So if your server's time is not set to UTC, the date will be offset according to the timezone you are in. To get around you can add/subtract your timezone offset from the date value. But that only returns the time assigned to the date back to midnight (or noon) and doesn't really get rid of it. To not display the time, you will need to rely on the web map to format the date field without a time. Alternatively, you could add the time you have from the call_created_time field (call_created_date - timeZoneOffset + call_created_time). This would result in a date in call_created_date field that had the proper date and time. That assumes the call_created_time is a local time (if it is in UTC you can omit the timeZoneOffset).

‎04-21-2021

Philip -- Your solution using an outbound connector which is essentially an No Operation component is a bit orthogonal to GeoEvent Server's design. What you're doing is one reason that we don't offer out-of-the-box processors with capabilities to invoke a GP Service, for example. We certainly could, as a GP service is as RESTful as other web services GeoEvent Server is interfacing with. But the question becomes do we want to block a processor node's flow as it waits on a response from a GP Service? This wouldn't be feasible when trying to process hundreds of event messages per second. Or do we allow the processor to invoke an asynchronous GP service task/job and have the processor send the logical equivalent of "nothing" or "process pending" along to an outbound connector? That's not consistent with GeoEvent Server's design. GeoEvent Server is fundamentally accepting data, adapting the data to produce individual event records, then processing each of those event records atomically (without retaining or caching data from an event record unless absolutely necessary), so that data from a processed event record can be routed along to an outbound connector for dissemination. Your solution appears to be developer-centric and highly customized. If I understand what you're saying you have a custom inbound adapter, a custom processor, and now a custom outbound connector. If the GeoEvent Server's Java SDK allows you to develop a solution using GeoEvent Server as a platform for event record processing -- that's great -- but I'm not sure that the product team can be of much help moving forward. I will offer that the multicardinality and hierarchy supported by a GeoEvent Definition is not specific to JSON. How data is ingested and adapted is not tied to a specific data format (e.g. JSON object format). Every event record has a GeoEvent Definition which describes the event record's data structure. This event definition applies only to the interior of an event record object. There is no mechanism which allows you to define a group, list, or hierarchy of multiple event records. A GeoEvent Definition can specify a data structure which includes a list of Java primitive values (e.g. Date, Double, Long, String, ...) and/or incorporate a non-primitive type Group which includes multiple primitive values as a sub-structure within the overall data structure. But this all sill describes the data structure of a single event record object. The hierarchy and multicardinality concepts discussed in the article you found do not apply to collections of multiple event records.

‎04-20-2021

Hey Adam -- The Field Mapper processor was designed to flatten a schema to make the event record's data structure compatible with the ArcGIS REST Services API used when sending processed event data as JSON to a feature service's addFeatures or updateFeatures endpoint. You cannot use Field Mapper ... or any of the out-of-the-box processors ... to write data to a hierarchical structure. It appears, from your illustration, that event data being ingested is already adapted using flattened data structure (e.g. the cardinality of every event record attribute is '1' and the data type is Date, Double, Long, String, (etc.) ... not Group). I think you'll want to consider developing a custom outbound adapter which is able to take a flat data structure and adapt it into a hierarchical data structure expected by the Web Hook you want to receive data you've processed through a GeoEvent Service.

‎04-14-2021

Adding one more reference to this question: You can use the Timetree Processor to create lines out of N points or Y period of time: https://community.esri.com/t5/arcgis-geoevent-server-blog/geoevent-delaying-and-or-time-sorting-events/ba-p/901908 If your events sometimes arrive out of order, causing your lines to be incorrect, you can add the Delay Processor to the mix before your Motion Calculator or Timetree to properly order your events: https://community.esri.com/t5/arcgis-geoevent-server-blog/geoevent-delaying-and-or-time-sorting-events/ba-p/901908

‎04-12-2021

Hello Dinesh -- The first part of your question is relatively easy. If you were to poll a feature service to obtain a set of feature records whose associated geometry were a polygon modeling a "project boundary" you could route each event record through a GeoTagger processor to enrich the event record with the (unique identifier) names of point geofence imported from a feature service providing the point locations of towers. What you have now is a comma delimited list of towers that fall within a project boundary. The difficulty is three-fold. First, GeoEvent Server does not provide any sort of iterator to inspect individual items in a list. You don't know how many towers are expected to be in any given project boundary, so you cannot further enrich the "project boundary" event record with the "alert status" for each tower ... because you cannot iterate across the list of towers to query their alert status. You could use a Field Splitter processor from the GeoEvent Gallery to split a comma delimited list of towers in an enriched (geotagged) project boundary event record. This would produce separate event records, one for each tower in the project boundary. You could then enrich a second time to get the tower's alert status and add it to the project boundary event record. But each event record emitted from a Field Splitter is processed atomically (individually). This is the second challenge / limitation ... you cannot compare attributes from one event record with attributes in another event record. The third challenge, as I see it, is there is no easy way to compare one tower's alert status to another and pick the greater of the two, especially when you don't know how many towers there are. I've never tried, for example, to design bitwise arithmetic into a GeoEvent Service to logically OR two bit sequences 0x0100 and 0x0010 to produce 0x0110 and then determine the highest-order bit set in the sequence. The logical operations GeoEvent Server supports are much more general (e.g. determining if an event record's string is empty or null to set a Boolean result to 'true' or 'false' and then comparing that 'true' / 'false' value against another Boolean to determine what to do with the singular event record being processed. You might approach the problem using a GeoTagger as described above to get the name of point geofences in an area of interest, splitting the event record using a Field Splitter to produce several independent event records, and then use a Field Enricher to look-up the alert status for each event record's associated tower. You could then use an Update a Feature output to have GeoEvent Server make a REST request on a feature service to update the alert for an entire area (or project boundary) ... but you'll need some sort of database trigger to catch that request and only allow it to proceed if the alert value is equal-to or greater-than the feature record's current alert value. Otherwise, as I'm sure you realize, GeoEvent Server's serialized event processing stream will simply overwrite the project boundary feature record's alert status with the most recently processed tower's status. Hope this information helps you think through the analysis you want to perform. -- RJ

‎02-26-2021

If data you are receiving contains only a date value (e.g. 12/31/2021 ) without a time, this is not a pattern GeoEvent Server recognizes without you specifying a an Expected Date Format the inbound adapter can use to figure out how to parse a String as a Date. You would have to specify a value like MM/dd/yyyy when configuring your inbound connector. The connector will apply this pattern to all event record attributes whose data type is Date in the GeoEvent Definition used by the inbound connector. When I send the String value "12/31/2021" to my GeoEvent Server with the Expected Date Format configuration described above, the Date value my inbound adapter constructs for me from the received string is 1640937600000. This is an epoch value used by Java to represent date/time values. GeoEvent Server uses millisecond epoch values, which is why the value has 13 digits rather than only 10. If I ask GeoEvent Sever to cast its Date to a String I get a representation of the date which looks like "Fri Dec 31 00:00:00 PST 2021". Notice that the string has both a "date" and a "time" and includes the Time Zone for the expressed date/time value. In this case, the Date is expressed in the Pacific time zone. This is because an Expected Date Format pattern was specified -- which is required to handle the inbound string which does not match one of the few built-in expected patterns for a date/time value. The time zone handling is important to note because, in this case, the date/time is not in UTC units. GeoEvent Server assumes that the non-standard date/time must be a date/time local to my solution, so it uses the locale of my server (whose clock is configured to use the Pacific Time Zone). Focusing on your question, if you are receiving a string value which is somehow being adapted to produce the epoch date value 1640908800000 (which could also be represented as "Thursday, December 30, 2021 4:00:00 PM GMT-08:00" or "Thu Dec 30 16:00:00 PST 2021") and you need to truncate the value to be simply "Thursday, December 30" ... you have a couple of options. I strongly recommend you make sure you understand how the received data is actually being adapted, and check to verify how client applications are representing the value in web map pop-ups or web forms. A client application will likely represent a Java Epoch date/time value it receives, when querying a feature service for feature records for example, in whatever time zone the client web application is running. The value 1640908800000 already represents the date/time "Friday, December 31, 2021 12:00:00 AM" when a UTC value is assumed and web clients are likely going to try and represent an assumed UTC date/time in whatever time zone the web application is running. If you were to add or subtract some number of milliseconds from the epoch to drop the "time" portion and keep only the whole "date" value, your effort is likely going to have unintended consequences client-side. You could use a RegEx pattern match on a value toString(myDate) to isolate the whole hours portion of the "time" and then multiply this by 3,600,000 (which is 60 min x 60 sec x 1000 ms) and then subtract that from your Date using a Field Calculator. The eventual expression would be something like: myDate - (16 * 3600 * 1000) This assumes you are able to extract the value "16" from a string "Thu Dec 30 16:00:00 PST 2021" to know that you wanted to subtract 16 hours worth of milliseconds from the myDate attribute value. You also might want to look at some of the supported expressions for the Field Calculator processor. The function currentOffsetUTC() specifically computes the millisecond difference between your GeoEvent Server's locale and UTC. Since my server is configured to use the Pacific Time Zone, which is currently -08:00 hours behind UTC, the currentOffsetUTC() function returns a value -28800000, which is (8 hours x 60 minutes x 60 seconds x 1000 milliseconds). You might scale the computed value by some constant when performing date/time adjustment arithmetic, or more likely, shift an epoch Date from an assumed local time zone so that the value represents a UTC value. The advantage of using currentOffsetUTC() is that the function automatically recognizes changes in daylight savings, so you don't have to rely memory to update GeoEvent Services twice a year when a fixed constant value you might have hard-coded in an expression no longer reflects the observance of daylight savings time. See Also: What time is it? Well That Depends...

‎02-20-2021

Thank you. That indeed helped. Instead of connecting field calculators along different paths, I stacked them one after another and that seems to have done the trick. Regards, Shital

‎02-09-2021

Thank you, RJ! It's taken me awhile to get back to this, but did submit an incident into technical support. Being able to review the configuration of the stream service through the json is great! gives confidence for what is expected and can confirm that I didn't fat finger something in the set up. We've abandoned the idea of storing events at this point, just trying to get the related features to work as we have that scenario with a status being sent for a fixed asset - so would like to pull the location from the feature class. I'm optimistic we'll be able to get to the bottom of it and will report here for others that may run into similar -- jess

‎01-14-2021

Hi @GaryBowles1 , I was running into the same problem. I was able to get it to add the definitions after I published and ran the new GeoEvent Service. I stopped it quickly and then added the filter, but that was the only way I could get it to work. Thanks, Chris

‎12-04-2020

Very cool, Checking it out now. Thanks.

‎11-05-2020

In this blog we will take a deeper look at registering server connections with GeoEvent Server, something administrators commonly have to do to when configuring a GeoEvent Server deployment. Why do you have to register an ArcGIS Server connection with GeoEvent Server There are several different configurable components which require you to select a registered server and specify which services folder, map/feature service, and often a specific feature layer exposed by that service in order to use the feature layer's schema or feature records. A few examples include: The Poll an ArcGIS Server for Features input used to query a service for feature records. When you want to import a GeoEvent Definition from a feature service's schema. The Field Enricher (Feature Service) processor used to load a feature record set to use for enrichment. The Add a Feature and Update a Feature outputs used to persist data from processed event records as feature records in a geodatabase. All of the user-interfaces in the above examples populate their options from a cache of information GeoEvent Server collects when it queries a registered ArcGIS Server to discover published feature services. Service discovery is not performed the moment you click to open the panel. The cache is created and updated in the background because it can take several seconds – sometimes minutes or even tens of minutes – for GeoEvent Server to completely crawl the ArcGIS Server's REST Services Directory and query the information it needs from all of the available services. Which leads into the topic I want to address in this blog: Is there a way to know when service discovery is being run and how long it is expected to take? Using the ArcGIS Server Connection component to log messages You can configure the following component logger to request DEBUG messages be logged: com.esri.ges.datastore.agsconnection.DefaultArcGISServerConnection I've included a sample of the logged messages you will be looking for at the end of this article. Requesting this component logger include DEBUG messages in its logging will allow you to see, in the system log file, when the service discovery kicks off and which map/feature services it interrogates to learn about their layers. A message will be logged for every feature service being interrogated as well as a success message when service discovery is complete. There is no indication in the GeoEvent Manager web application that service discovery is about to start or is currently running. The best way to tell that service discovery is running is to start the workflow to import a GeoEvent Definition. If the blue/white indicator displays, requesting you "please wait", you know that the GeoEvent Server is busy updating its ArcGIS Services cache. Otherwise the GeoEvent Definition user-interface will display immediately allowing you to choose a server connection, folder, feature service, and feature layer. GeoEvent Manager does not allow you to configure, or schedule, when service discovery should take place. You are able to change the Discovery Rate for each server connection you register to specify how frequently a refresh should be performed. The default for recent software releases is every 60 minutes. I have personally found that when I use ArcGIS Pro, the Enterprise portal, or GeoEvent Manager to publish a new feature service, I want to use it now – so it is not uncommon for me to publish a feature service and immediately request GeoEvent Server run a service discovery to update its cache by clicking the refresh button on the server connection I have registered as a Data Store. This eliminates the need to have GeoEvent Server periodically refresh the cache for me, so I usually set the Discovery Rate for server connections I register to a fairly large value like 1440 minutes so that service discovery is run once once per day (or when GeoEvent Server is stopped and restarted). Service discovery takes too long and interferes with normal operations. Is there anything I can do? This is something the product team is working on. Refactoring GeoEvent Server to support all of the operations which use feature services, however, to interface with ArcGIS Server some other way is both high risk and high reward. Several different design options have been considered, but implementation has had to be deferred for each of the last several major releases. The best option for now, therefore, is to limit the number of features services which are discoverable each time service discovery is run. A recommended best practice is to configure your GeoEvent Server Data Store (e.g. the server connection you are registering with GeoEvent Server) with credentials. The user credentials do not have to be an administrative user, just a user who owns and published the feature service. You can use the Enterprise portal content item manager to assign existing feature services a new owner and configure GeoEvent Server to authenticate as that user when crawling the ArcGIS Server's REST Services Directory to limit the number of discoverable services. GeoEvent Server administrators often configure their registered server connections with the ArcGIS Server or Enterprise portal primary administrative account – which naturally sees all published services. If you can identify the feature services to which you want GeoEvent Server to write data, or from which you want GeoEvent Server to retrieve feature records or a feature layer's schema, and assign ownership of just those services to a user set aside specifically for your "real-time" data, you can improve service discovery considerably by not crawling all of the feature services being maintained by more traditional feature editing workflows. The trick is to identify the feature services your GeoEvent Server components actually care about and limit discovery to only those feature services. Example messages logged by the ArcGIS Server Connection component logger 2020-11-05T13:55:02,124 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | sleep interrupted java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) ~[?:?] at com.esri.ges.datastore.agsconnection.DefaultArcGISServerConnection$CacheUpdater.run (DefaultArcGISServerConnection.java:235) [56:com.esri.ges.framework.datastore.agsconnection-datastore:10.8.1] 2020-11-05T13:55:02,124 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Exiting Cache Updater run method.... 2020-11-05T13:55:03,274 | DEBUG | qtp808353329-598 | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Create an instance of CacheUpdater.... 2020-11-05T13:55:03,274 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating cache for DataStore Public_Esri_Hosted_Server... 2020-11-05T13:55:08,944 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Active_Hurricanes_Sampler", Type: "FeatureServer"). 2020-11-05T13:55:15,296 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Active_Hurricanes_v1", Type: "FeatureServer"). 2020-11-05T13:55:21,613 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Air_Quality_PM25_Latest_Results", Type: "FeatureServer"). 2020-11-05T13:55:23,311 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "ASAM_events_V1", Type: "FeatureServer"). 2020-11-05T13:55:25,033 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Coral_Reef_Stations", Type: "FeatureServer"). 2020-11-05T13:55:27,876 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "GDELT_Health_Pandemic", Type: "FeatureServer"). 2020-11-05T13:55:29,170 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "GDELT_v1_Social_Tones", Type: "FeatureServer"). 2020-11-05T13:55:30,368 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "IHME_Projected_Peaks_QA", Type: "FeatureServer"). 2020-11-05T13:55:31,995 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Median_Sea_Ice_Extent_for_the_Antarctic", Type: "FeatureServer"). 2020-11-05T13:55:32,995 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Median_Sea_Ice_Extent_for_the_Arctic", Type: "FeatureServer"). 2020-11-05T13:55:34,063 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "MODIS_Thermal_v1", Type: "FeatureServer"). 2020-11-05T13:55:35,225 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "National_Farmers_Market_Directory", Type: "FeatureServer"). 2020-11-05T13:55:36,445 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "ncov_cases", Type: "FeatureServer"). 2020-11-05T13:55:38,421 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_Ice_v1", Type: "FeatureServer"). 2020-11-05T13:55:40,579 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_Precipitation_v1", Type: "FeatureServer"). 2020-11-05T13:55:43,479 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_SnowFall_v1", Type: "FeatureServer"). 2020-11-05T13:55:45,920 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_WindForecast_v1", Type: "FeatureServer"). 2020-11-05T13:55:49,935 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_WindGust_v1", Type: "FeatureServer"). 2020-11-05T13:55:51,519 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDFD_WindSpeed_v1", Type: "FeatureServer"). 2020-11-05T13:55:53,124 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NDGD_SmokeForecast_v1", Type: "FeatureServer"). 2020-11-05T13:55:54,193 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NOAA_METAR_current_wind_speed_direction_v1", Type: "FeatureServer"). 2020-11-05T13:55:55,683 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NOAA_short_term_warnings_v1", Type: "FeatureServer"). 2020-11-05T13:55:58,401 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NOAA_storm_reports_Sampler", Type: "FeatureServer"). 2020-11-05T13:56:00,844 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NOAA_storm_reports_v1", Type: "FeatureServer"). 2020-11-05T13:56:06,494 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NWS_Watches_Warnings_Sampler", Type: "FeatureServer"). 2020-11-05T13:56:12,942 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "NWS_Watches_Warnings_v1", Type: "FeatureServer"). 2020-11-05T13:56:19,622 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Recent_Hurricanes_v1", Type: "FeatureServer"). 2020-11-05T13:56:21,726 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Satellite_VIIRS_Thermal_Hotspots_and_Fire_Activity", Type: "FeatureServer"). 2020-11-05T13:56:22,871 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "seaice_extent_N_v1", Type: "FeatureServer"). 2020-11-05T13:56:23,976 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "seaice_extent_S_v1", Type: "FeatureServer"). 2020-11-05T13:56:25,081 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "SPI_recent", Type: "FeatureServer"). 2020-11-05T13:56:28,250 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "Standardized_Precipitation_Index_(SPI)", Type: "FeatureServer"). 2020-11-05T13:56:31,492 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "US_Cases_per_county_(time)", Type: "FeatureServer"). 2020-11-05T13:56:33,073 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "USA_Wildfires_v1", Type: "FeatureServer"). 2020-11-05T13:56:35,074 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating layers for Service (Folder: "/", Name: "USGS_Seismic_Data_v1", Type: "FeatureServer"). 2020-11-05T13:56:36,595 | DEBUG | Public_Esri_Hosted_Server-Updater | DefaultArcGISServerConnection | 56 - com.esri.ges.framework.datastore.agsconnection-datastore - 10.8.1 | Updating cache for DataStore Public_Esri_Hosted_Server done. Success: true.‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ You can see from the above that discovery of 36 feature services hosted by an external, public server, took just over 90 seconds. The more services there are to discover, the longer service discovery will take. See Also: GeoEvent Server > Administer > Data stores in the GeoEvent Server on-line help Debug Techniques - Configuring the application logger GeoEvent Server blog series GeoEvent Configuration: Data Store Connections w/Tokens blog by Eric Ironside

‎08-06-2020

Hello Geoffrey West – Looking at the JSON you illustrated it looks to me like the data object is an array of JSON objects: "data": [ { "id": "b2c2626d-ff0b-4892-8262-5acae60946a4", "name": "JUPITER II", "mmsi": 701000533, ...‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ You probably need to add an index into your hierarchy parsing string to change data.last_known_position.geometry.coordinates[0] to data[0].last_known_position.geometry.coordinates[0] Also, I noticed that all of the negative double values seem to have an extra space between the '-' and the actual value. Quick experiments with GeoEvent Server seem to show that the extra white-space is being removed by the library used to parse the data ... I noticed https://jsonlint.com does the same thing ... but a different validator I often use triggered on these values and refused to validate the JSON to show it in a tree with collapsible nodes. Hope this information is helpful – RJ

‎07-10-2020

Questions and Answers from the Expo Floor: What is ArcGIS GeoEvent Server GeoEvent Sever is one of several license roles for ArcGIS Enterprise. You license and install GeoEvent Server to add real-time capabilities to your Enterprise. GeoEvent Server, out-of-the-box, provides configurable inputs for connecting to a variety of data streams from virtually any data provider. What is a GeoEvent Service A GeoEvent Service is an event processing workflow you design and publish using the GeoEvent Manager web application. You drag, drop, and configure an input to specify how data will be received. You add and configure an output to specify what will be done with the data the GeoEvent Service processes. You then add one or more filters and processors to the service, connect the nodes to define an event processing workflow, and publish the GeoEvent Service. See Also: GeoEvent Server inputs GeoEvent Server filters GeoEvent Server processors GeoEvent Server outputs What's New in GeoEvent Server 10.8 / 10.8.1 Significant work went into simplifying common user workflows enabling users to maintain their focus on configuring and publishing GeoEvent Services. Most of what you need to do is now done from within the Service Designer rather than switching to a different web page in the GeoEvent Manager. The status, catalog, and option to create, edit, and delete primary service components (inputs, outputs, etc.) is now available from a consolidated Monitor page in the GeoEvent Manager. There is also a new GeoEvent Sampler utility built into the Service Designer allowing you to see the effect filters and processors you configure have on event records as they are received in real-time. For details, please see the PDF attached to this article. Question Answer Question Answer Question Answer

‎03-24-2020

RJ - You are always there to make sure that I remember the finer points. Thank you very much my friend. --Adam

‎03-06-2020

I sent the following to one of our contractors today. The information on configuring SSL certificates, administrative tips for multi-machine deployments following a 'site' model, and things to check when GeoEvent Server fails to load its ArcGIS Server's configured certificates and instead uses its own SelfSignedCertificate might be of more general use, so I'll leave this here in case it helps someone working with GeoEvent Server deployments. With a multi-machine ‘site’ configuration it is critical that all machines trust one another. That means that not only do I have to configure an SSL certificate on Box#1 and configure that machine’s ArcGIS Server to use that certificate as its Web Server Certificate … I have to import certificates for Box#2, Box#3, … Box#N into the ArcGIS Server so that it trusts all the other machines participating in the site. I have to do this “fan-out” on every server, setting *that* server’s Web Server Certificate and importing certificates from all the *other* machines onto that server. I’ve captured what I do that works for me when setting up a couple of machines. But to be honest, SSL certificate configuration is not something I understand at a deep, technical level. Likely there is a “better” way of doing what I propose in the attached, maybe using a wild-card certificate, but I don’t know how to set that up. I’d also like to break the problem you’re seeing into two pieces. The first being SSL certificate configuration, for which I’ll capture some screenshots (see attached PDF). The second piece involves things I look at when GeoEvent Server seems unable to locate and load the certificates its ArcGIS Server is configured to use. The second part probably has more to do with why GeoEvent Server completes a fail over to use its SelfSignedCertificate rather than the certificate its ArcGIS Server is configured to use. I’ll apologize if anything I share is overly pedestrian … like I said, SSL certificates are not my cup of tea, so all I can do is show you what works for me and hope that your experience will allow you to iterate and adapt what I have to share. The first part, SSL certificate configuration, is attached. For the second part … I would caution against opening the Java Keystore using a command like keytool. I’ve watched developer’s do this, but I’ve never seen that administratively editing the JKS do anything to resolve a problem. GeoEvent Server, when it launches for the first time, interrogates its ArcGIS Server for information on its site and SSL certificates. If you would like to see some evidence for this, you can request DEBUG logging on the com.esri.ges.security.arcgis.sslconfig GeoEvent Server logger component. GeoEvent Server will attempt to copy the certificate configuration of the ArcGIS Server is it running beneath. If GeoEvent Server cannot obtain the certificates from the ArcGIS Server configuration, it will fail over to use its own SelfSignedCertificate. The fail over is intended to at least allow GeoEvent Server to complete its startup – but if GeoEvent Server does not trust machines the same way as its ArcGIS Server does, lots of stuff is probably not going to work. By the way, it is precisely because GeoEvent Server interrogates its ArcGIS Server for information that it is best to have your ArcGIS Enterprise (Portal for ArcGIS, hosting ArcGIS Server, ArcGIS Data Store) fully configured with a site created, federated and all SSL certificates configured before you introduce GeoEvent Server to the Enterprise. Installing – or at least starting the GeoEvent Gateway and GeoEvent Server – before ArcGIS Server and Portal for ArcGIS are fully configured means that the initial interrogation fails. Security topology may change … you may later decide to federate for example, or SSL certificates have to change … in which case resetting your GeoEvent Server configuration from within GeoEvent Manager (e.g. not an “administrative reset”) should force GeoEvent to pick-up changes made to the Enterprise configuration. Worst case you have to stop and restart GeoEvent Server after resetting its configuration then import your inputs, outputs, …etc. You don’t always have to re-install, but installation order can make your life easier administratively when deploying all this s/w for the first time. There are a few things I check when I find that GeoEvent Server is using its own SelfSignedCertificate rather than the certificate its ArcGIS Server specifies as its Web server SSL certificate. Did I accurately follow the certificate configuration laid out in the attached PDF? Sometimes a machine gets re-imaged, or a something else invalidates a certificate I had previously generated, applied, and imported using the attached procedure. That is when I have to walk through that whole process again. Sometimes it is just that a certificate has expired. They do that, and rarely when it’s convenient. ArcGIS Server maintains two different certificate stores – do their contents match? Seriously, this has bitten us more than once. There’s a certificate store beneath …\ArcGIS\Server\framework used, I think, by web clients. ArcGIS Server maintains a copy of these certificates in its configuration store for each machine in the site. This second key store is used, I think, by thick client applications. C:\Program Files\ArcGIS\Server\framework\etc\certificates C:\arcgisserver\config‑store\machines\MYMACHINE.ESRI.COM The two certificate stores should be identical. I’ve found once or twice that files had not been copied from the Server framework into its configuration store. When this happened I had to stop ArcGIS Server, manually create the folder named for the machine (e.g. CARMON.ESRI.COM beneath …\config-store\machines) and copy the files from the framework into the configuration store folder. When I restarted ArcGIS Server and administratively reset GeoEvent Server, it adopted its Server’s certificates and began working as expected. ArcGIS Server maintains both JSON and XML copies of its SSL configuration – do they match? When debugging we’ve found a couple of times that the SSL configuration reported by ArcGIS Server by its Admin API did not match an XML file’s content that GeoEvent Server was using to retrieve certificate information. Specifically a file D:\arcgisserver\config-store\machines\10.0.0.131.json specified a webServerCertificateAlias which did not match what should have been the same information in a C:\Program Files\ArcGIS\Server\framework\etc\machine-config.xml file. When this happens you might try stopping GeoEvent Server (and GeoEvent Gateway) and reconfiguring the ArcGIS Server’s certificates. If the files match after ArcGIS Server completes a restart, then you can administratively reset GeoEvent Server and it should pick-up the correct certificate configuration. Does the GeoEvent Gateway have its correct hostname / IP Address in its com.esri.ges.gateway.cfg file? Part of the GeoEvent Server administrative reset is to delete this file and make sure that it gets regenerated automatically when GeoEvent Gateway (or maybe its when GeoEvent Server) comes up for the “first” time. If you look at the file’s content in a text editor you’ll see that it instructs the Gateway as to which server and port it should use for connecting to the Zookeeper distributed configuration store which manages your GeoEvent Server’s configuration. It also specifies the Apache Kafka topic partitions, replication and how to reach the broker. If the machine information in this file designates a machine which does not exist – like when you use cloud image utilities to push a machine image out to multiple virtual machine instances – when GeoEvent Gateway launches it never reaches a stable state and cannot support its GeoEvent Server. The procedures to administratively reset GeoEvent Server are in a blog: Administratively Reset GeoEvent Server You can follow the procedures for 10.6.x as they will be the same for 10.7.x and 10.8 deployments. These are the steps, by the way that you have to run on each server when following a multi-machine deployment with a ‘site’ configuration and one of the machines drops out of the configuration and does not automatically re-integrate. Resetting a multi-machine ‘site’ configuration is both tedious and error prone. You basically have to work as if you’re installing all of the s/w for the first time: Install ArcGIS Server, create site, configure certificates, install GeoEvent Server Install ArcGIS Server, join site, configure certificates, install GeoEvent Server Install ArcGIS Server, join site, configure certificates, install GeoEvent Server (lather, rinse, repeat) When you already have an ArcGIS Server site with, say, three machines things get messy. I think what you do is use ArcGIS Server Manger to ‘STOP’ two of the machines – you’ll want to stop GeoEvent Gateway and GeoEvent Server on those machines first. The idea is that as far as the ArcGIS Server site is concerned it only has one machine. Complete the admin reset for GeoEvent Server on that machine then start its Gateway, wait a couple minutes, then start its GeoEvent Server. Then, back in ArcGIS Server Manger to ‘START’ a second machine. The site now thinks it has two machines, only one of which is running GeoEvent Server. Complete the admin reset for GeoEvent Server on the second machine then start its Gateway, wait a couple minutes, then start its GeoEvent Server. As the GeoEvent Gateway and GeoEvent Server come up they’ll discover and coordinate with the running GeoEvent Server, through the AGS site, and work out among themselves how to balance the kafka topics and brokers. Finally, in ArcGIS Server Manager, ‘START’ the third machine. The site now thinks it has thee machines, only two of which are running GeoEvent Server. Complete the admin reset for GeoEvent Server on the third machine then start its Gateway, wait a couple minutes, then start its GeoEvent Server. As the GeoEvent Gateway and GeoEvent Server come up on this final machine they’ll integrate with the other two. If you try to bring all three machines on-line at the same time and they were not properly integrated / balanced when they were taken down … they’ll likely not integrate correctly with one another. You have to stage their startup so that the ArcGIS Server site never has more than one machine ‘STARTED’ which does not have a fully initialized and integrated GeoEvent Server. When two or more GeoEvent Server’s try to integrate at the same time things tend to fail. It is precisely this sort of fragility, and the fact that it is so administratively difficult to determine if the machines were not properly integrated / balanced in the first place, that I feel a ‘site’ configuration really doesn’t provide the resiliency it was designed to provide. Sure, when everything is working it works beautifully. But when a machine falls out of configuration … getting the ‘site’ back to nominal is difficult (to say the least). Hope this information is helpful – RJ

Online Status	Offline
Date Last Visited	‎03-07-2024 07:12 PM

My Ideas

Latest Contributions by RJSunderman

Re: Default Timestamp Automatically Gets Added To The Date Field

Re: Sending multicardinal data in a custom adapter

Re: Sending JSON as a Post Request to a Microsoft Teams Webhook?

Re: Geoevent: Create Line from Points

Re: How to select the group of features in feature class using geofence in geoevent service?

Re: Converting between Date Formats

Re: Geoevent Erroneously Imports Date Field

Re: Stream Service - Store latest, review configuration

Re: Geoevent Filter to ensure a date field is within a tolerance of the current time

Re: Ais decoder

Monitoring a registered web server's service discovery

Re: GeoEvent Server - parsing many cardinality for array of coordinates to create geometry

UC 2020 FAQ

Re: CSV in a Shared Folder

Configuring SSL Certificates for a Machine Using Domain Enrollment

Re: GeoFence sync is not detecting deleted feature...

Re: GeoTab Connector with Multiple GeoTab Database...

Re: alert when 2 points are close to each others

Re: What time is it? Well That Depends...

Re: Shapefile as Input to GEP

ArcGIS Analyse