|
BLOG
|
I sent the following to one of our contractors today. The information on configuring SSL certificates, administrative tips for multi-machine deployments following a 'site' model, and things to check when GeoEvent Server fails to load its ArcGIS Server's configured certificates and instead uses its own SelfSignedCertificate might be of more general use, so I'll leave this here in case it helps someone working with GeoEvent Server deployments. With a multi-machine ‘site’ configuration it is critical that all machines trust one another. That means that not only do I have to configure an SSL certificate on Box#1 and configure that machine’s ArcGIS Server to use that certificate as its Web Server Certificate … I have to import certificates for Box#2, Box#3, … Box#N into the ArcGIS Server so that it trusts all the other machines participating in the site. I have to do this “fan-out” on every server, setting *that* server’s Web Server Certificate and importing certificates from all the *other* machines onto that server. I’ve captured what I do that works for me when setting up a couple of machines. But to be honest, SSL certificate configuration is not something I understand at a deep, technical level. Likely there is a “better” way of doing what I propose in the attached, maybe using a wild-card certificate, but I don’t know how to set that up. I’d also like to break the problem you’re seeing into two pieces. The first being SSL certificate configuration, for which I’ll capture some screenshots (see attached PDF). The second piece involves things I look at when GeoEvent Server seems unable to locate and load the certificates its ArcGIS Server is configured to use. The second part probably has more to do with why GeoEvent Server completes a fail over to use its SelfSignedCertificate rather than the certificate its ArcGIS Server is configured to use. I’ll apologize if anything I share is overly pedestrian … like I said, SSL certificates are not my cup of tea, so all I can do is show you what works for me and hope that your experience will allow you to iterate and adapt what I have to share. The first part, SSL certificate configuration, is attached. For the second part … I would caution against opening the Java Keystore using a command like keytool. I’ve watched developer’s do this, but I’ve never seen that administratively editing the JKS do anything to resolve a problem. GeoEvent Server, when it launches for the first time, interrogates its ArcGIS Server for information on its site and SSL certificates. If you would like to see some evidence for this, you can request DEBUG logging on the com.esri.ges.security.arcgis.sslconfig GeoEvent Server logger component. GeoEvent Server will attempt to copy the certificate configuration of the ArcGIS Server is it running beneath. If GeoEvent Server cannot obtain the certificates from the ArcGIS Server configuration, it will fail over to use its own SelfSignedCertificate. The fail over is intended to at least allow GeoEvent Server to complete its startup – but if GeoEvent Server does not trust machines the same way as its ArcGIS Server does, lots of stuff is probably not going to work. By the way, it is precisely because GeoEvent Server interrogates its ArcGIS Server for information that it is best to have your ArcGIS Enterprise (Portal for ArcGIS, hosting ArcGIS Server, ArcGIS Data Store) fully configured with a site created, federated and all SSL certificates configured before you introduce GeoEvent Server to the Enterprise. Installing – or at least starting the GeoEvent Gateway and GeoEvent Server – before ArcGIS Server and Portal for ArcGIS are fully configured means that the initial interrogation fails. Security topology may change … you may later decide to federate for example, or SSL certificates have to change … in which case resetting your GeoEvent Server configuration from within GeoEvent Manager (e.g. not an “administrative reset”) should force GeoEvent to pick-up changes made to the Enterprise configuration. Worst case you have to stop and restart GeoEvent Server after resetting its configuration then import your inputs, outputs, …etc. You don’t always have to re-install, but installation order can make your life easier administratively when deploying all this s/w for the first time. There are a few things I check when I find that GeoEvent Server is using its own SelfSignedCertificate rather than the certificate its ArcGIS Server specifies as its Web server SSL certificate. Did I accurately follow the certificate configuration laid out in the attached PDF? Sometimes a machine gets re-imaged, or a something else invalidates a certificate I had previously generated, applied, and imported using the attached procedure. That is when I have to walk through that whole process again. Sometimes it is just that a certificate has expired. They do that, and rarely when it’s convenient. ArcGIS Server maintains two different certificate stores – do their contents match? Seriously, this has bitten us more than once. There’s a certificate store beneath …\ArcGIS\Server\framework used, I think, by web clients. ArcGIS Server maintains a copy of these certificates in its configuration store for each machine in the site. This second key store is used, I think, by thick client applications. C:\Program Files\ArcGIS\Server\framework\etc\certificates C:\arcgisserver\config‑store\machines\MYMACHINE.ESRI.COM The two certificate stores should be identical. I’ve found once or twice that files had not been copied from the Server framework into its configuration store. When this happened I had to stop ArcGIS Server, manually create the folder named for the machine (e.g. CARMON.ESRI.COM beneath …\config-store\machines) and copy the files from the framework into the configuration store folder. When I restarted ArcGIS Server and administratively reset GeoEvent Server, it adopted its Server’s certificates and began working as expected. ArcGIS Server maintains both JSON and XML copies of its SSL configuration – do they match? When debugging we’ve found a couple of times that the SSL configuration reported by ArcGIS Server by its Admin API did not match an XML file’s content that GeoEvent Server was using to retrieve certificate information. Specifically a file D:\arcgisserver\config-store\machines\10.0.0.131.json specified a webServerCertificateAlias which did not match what should have been the same information in a C:\Program Files\ArcGIS\Server\framework\etc\machine-config.xml file. When this happens you might try stopping GeoEvent Server (and GeoEvent Gateway) and reconfiguring the ArcGIS Server’s certificates. If the files match after ArcGIS Server completes a restart, then you can administratively reset GeoEvent Server and it should pick-up the correct certificate configuration. Does the GeoEvent Gateway have its correct hostname / IP Address in its com.esri.ges.gateway.cfg file? Part of the GeoEvent Server administrative reset is to delete this file and make sure that it gets regenerated automatically when GeoEvent Gateway (or maybe its when GeoEvent Server) comes up for the “first” time. If you look at the file’s content in a text editor you’ll see that it instructs the Gateway as to which server and port it should use for connecting to the Zookeeper distributed configuration store which manages your GeoEvent Server’s configuration. It also specifies the Apache Kafka topic partitions, replication and how to reach the broker. If the machine information in this file designates a machine which does not exist – like when you use cloud image utilities to push a machine image out to multiple virtual machine instances – when GeoEvent Gateway launches it never reaches a stable state and cannot support its GeoEvent Server. The procedures to administratively reset GeoEvent Server are in a blog: Administratively Reset GeoEvent Server You can follow the procedures for 10.6.x as they will be the same for 10.7.x and 10.8 deployments. These are the steps, by the way that you have to run on each server when following a multi-machine deployment with a ‘site’ configuration and one of the machines drops out of the configuration and does not automatically re-integrate. Resetting a multi-machine ‘site’ configuration is both tedious and error prone. You basically have to work as if you’re installing all of the s/w for the first time: Install ArcGIS Server, create site, configure certificates, install GeoEvent Server Install ArcGIS Server, join site, configure certificates, install GeoEvent Server Install ArcGIS Server, join site, configure certificates, install GeoEvent Server (lather, rinse, repeat) When you already have an ArcGIS Server site with, say, three machines things get messy. I think what you do is use ArcGIS Server Manger to ‘STOP’ two of the machines – you’ll want to stop GeoEvent Gateway and GeoEvent Server on those machines first. The idea is that as far as the ArcGIS Server site is concerned it only has one machine. Complete the admin reset for GeoEvent Server on that machine then start its Gateway, wait a couple minutes, then start its GeoEvent Server. Then, back in ArcGIS Server Manger to ‘START’ a second machine. The site now thinks it has two machines, only one of which is running GeoEvent Server. Complete the admin reset for GeoEvent Server on the second machine then start its Gateway, wait a couple minutes, then start its GeoEvent Server. As the GeoEvent Gateway and GeoEvent Server come up they’ll discover and coordinate with the running GeoEvent Server, through the AGS site, and work out among themselves how to balance the kafka topics and brokers. Finally, in ArcGIS Server Manager, ‘START’ the third machine. The site now thinks it has thee machines, only two of which are running GeoEvent Server. Complete the admin reset for GeoEvent Server on the third machine then start its Gateway, wait a couple minutes, then start its GeoEvent Server. As the GeoEvent Gateway and GeoEvent Server come up on this final machine they’ll integrate with the other two. If you try to bring all three machines on-line at the same time and they were not properly integrated / balanced when they were taken down … they’ll likely not integrate correctly with one another. You have to stage their startup so that the ArcGIS Server site never has more than one machine ‘STARTED’ which does not have a fully initialized and integrated GeoEvent Server. When two or more GeoEvent Server’s try to integrate at the same time things tend to fail. It is precisely this sort of fragility, and the fact that it is so administratively difficult to determine if the machines were not properly integrated / balanced in the first place, that I feel a ‘site’ configuration really doesn’t provide the resiliency it was designed to provide. Sure, when everything is working it works beautifully. But when a machine falls out of configuration … getting the ‘site’ back to nominal is difficult (to say the least). Hope this information is helpful – RJ
... View more
03-06-2020
04:53 PM
|
0
|
0
|
3595
|
|
POST
|
Seth – I think I've confirmed that third party libraries we incorporated into GeoEvent Server to handle XML to JSON translation changed how "root" elements and arrays are recognized and handled. I've created some XML which I think follows the sample you provided above and successfully configure a GeoEvent Server inbound connector and GeoEvent Definition capable of ingesting the XML data. Your original XML data structure collects several Unit elements and organizes them beneath an element named Units. There's nothing wrong with this; I can mock up JSON which represents Units as an array of Unit objects which passes on-line validation. I do not agree 100% with how the third party (internal) libraries we're using translate the XML to JSON – they seem to want both Units and Unit to be handled as arrays of values (multicardinal). I've attached a write-up which shows the XML I used to test and discusses my approach for accomplishing what I think you are trying to do. I ended up allowing GeoEvent Server to create a GeoEvent Definition for me, altering that GeoEvent Definition to accommodate multicardinal data structures, editing the input to use my tailored GeoEvent Definition, then using a pair of Multicardinal Field Splitters to split out each responding vehicle's data joining it to the common data for an incident. From there I can use normal Field Mappers to copy data from the processed event record schema(s) to match the schema expected by a pair of feature services so that each XML incident record received ends up adding/updating an incident's common data along with potentially several vehicle feature records ... which you can relate within the back end geodatabase using a 1:M relationship. I hope the attached is helpful – RJ Note: I indicate in the attached PDF that "the current implementation of the Multicardinal Field Splitter processor does not allow two or more instances of the processor to be placed in series." Please make sure you have the latest version of this processor downloaded from the processor's page in the GeoEvent Server Gallery. I was using an older release of the processor. Eric Ironside has updated the processor so that its addition of a childIndex no longer interferes with placing two or more Multicardinal Field Splitter processors in series. (RJ)
... View more
02-19-2020
06:44 PM
|
2
|
1
|
2790
|
|
POST
|
Hello Seth Lewis – I suspect that you're running into trouble with GeoEvent Server's internal translation from XML to JSON. I worked some with a sample XML structure similar to what you show above, but I'm unable (yet) to figure out how to configure my input such that Units is an array (cardinality 'Many') of individual Unit elements (cardinality 'One'). I'll be back in touch if I can figure out a work around or get this to work – RJ
... View more
02-14-2020
06:11 PM
|
2
|
0
|
2790
|
|
POST
|
Chris – I don't know if this is a requirement for 3D geometry, but I always take care that the last vertex in my "ring" is the same as the first vertex so that the ring is closed. Looking at the Feature JSON spec for Polygon, it seems their 3D example follows this "best practice". Note that the polygon JSON which follows has two rings, the first ring has 'M' values as well as 'Z' values, while the second ring has only 'Z' values: ( Click image to enlarge... ) Hope this information is helpful – RJ
... View more
01-21-2020
05:21 PM
|
0
|
0
|
2541
|
|
POST
|
Nathan Koski / Michelle Duncan – I noticed your post when searching for something unrelated and wanted to quickly reply. I don't have any first-hand experience with benefits or challenges to developing or using a 3rd party webhook service over GeoEvent Server, but Jeffrey Scarmazzi may – he posted a question recently on an internal forum looking for advice on creating a web hook using our REST API. For others who may be looking for information, please see Eric Ironside's blog, https://community.esri.com/people/eironside-esristaff/blog/2019/11/21/geoevent-consuming-an-arcgis-enterprise-webhook?sr=search&searchId=3811b4e9-3d3e-43c0-9887-2074fa9ed7f5&searchIndex=0 in addition to the blog you've mentioned (Joel Jeyarajah's — Webhooks - Don't call us, we'll call you). Hope this information is helpful – RJ
... View more
01-21-2020
04:57 PM
|
2
|
0
|
1914
|
|
POST
|
Hello Eduardo Fernandez – The reason we recommend a maximum of five machines has more to do with system administrative burden than any hard technical limit in the GeoEvent Server implementation. Inherent administrative challenges with a 'site' deployment (where you deploy multiple ArcGIS Server instances, each running GeoEvent Server, in a single ArcGIS Server site) multiply with the number of machines you add. Also, testing has shown that GeoEvent Server does not scale linearly and you approach the point of diminishing returns when trying to architect a solution with more than five machines. Hope this information is helpful – RJ
... View more
01-21-2020
04:31 PM
|
2
|
1
|
1909
|
|
POST
|
James Madden – GeoEvent Server's JSON receivers do not perform any authentication. The out-of-the-box Receive JSON on a REST Endpoint inbound connector you copied and modified to use the HTTP-BasicAuthentication transport needs to be configured to run in Client mode with its HTTP Method parameter configured for HTTP/GET. When configured as a polling input the connector's transport will send whatever text you have entered as the User and Password property values to the external web server / web service so that it can authenticate GeoEvent Server's request. I suspect that you have left the input configured as a receiving input, so the connector's transport is ignoring the credentials you've specified. Hope this information is helpful – RJ
... View more
01-15-2020
03:31 PM
|
1
|
0
|
1688
|
|
POST
|
James Madden – Nothing we are discussing here will be included in the 10.8 release due out next week (Jan 20th). The product team continues to actively consider options for additional security and refactoring the product to implement features like authentication for REST requests or moving GeoEvent Server's open REST endpoints to its administrative API. I cannot say, however, which future release such work would target. – RJ
... View more
01-15-2020
02:31 PM
|
0
|
0
|
1457
|
|
POST
|
Hello James Madden – Yes. By design GeoEvent Server REST receiver inputs allow an unauthenticated client / server to send POST requests to a running inbound connector. Recent releases ensure that such requests occur over HTTPS (not HTTP). We have not considered this particularly troubling for several reasons. Production systems usually secure their servers with an authenticating proxy and ACLs. If they want to grant access to a specific data provider, they configure a tunnel through their firewall for that specific provider. Also, any data sent must pass through an inbound adapter which uses a strict GeoEvent Definition to interpret the data. The event definition cannot be modified without authenticating with the administrative API, so potentially malicious code will not survive adaption to create an event record which can actually be processed – malicious code or data will be discarded as unrecognized by the inbound adapter. Over time, some users have voiced concern that information about a GeoEvent Server's configuration can be obtained by an unauthenticated user, via public REST endpoints, if they are able to reach a server machine via the machine's fully-qualified domain name and port. That's why production servers are secured using authenticating proxies and firewalls. Dan Wade has referenced an effort the product team is considering to move many endpoints reachable today via host.domain:6143/geoevent/rest by moving the endpoints beneath host.domain:6143/geoevent/admin so that authentication is required to reach them. There is some hesitation to secure the REST receiver endpoints. You cannot POST malicious XML, SQL, etc. to a GeoEvent Server receiver – the receiver's inbound adapter will not recognize the data's structure / schema and will discard the data. In your opinion, should an external client / server application be required to authenticate before being allowed to send a POST request to a GeoEvent Server input? Given that communication is secured using HTTPS, access to the server can be secured using an authenticating proxy and firewall, and adaption requires a predefined and recognizable data structure / schema ... we don't want to unnecessarily inhibit inbound data flow. – RJ
... View more
01-14-2020
05:15 PM
|
0
|
2
|
1457
|
|
POST
|
Hello Mody Buchbinder – In an older post, Generating Tracks from Events, I had suggested perhaps using a GeoEvent Server Incident Detector to output a polyline rather than point output to allow point position reports to be drawn with a track line. My experience since has suggested that at any real velocity / volume this is a bad idea. A more recent thread, Geoevent: Create Line from Points, I mention that a "reconstruct tracks" processor we had considered implementing for GeoEvent Server ended up being implemented as part of GeoAnalytics Server.You can read more about that capability here: Reconstruct Tracks Eric Ironside has some additional comments in that thread on using stream services, or perhaps the GeoEvent Server's Motion Calculator. These would be more traditional GeoEvent Server integration with an ArcGIS Enterprise portal web map (or ArcGIS Online web map) than ArcGIS Pro. I mention them only as alternatives to consider. I suspect that what Kory Kramer and Tanu Hoque have offered using stream layers and query layers in ArcGIS Pro is what you are looking for. – RJ
... View more
01-14-2020
11:41 AM
|
2
|
3
|
5432
|
|
POST
|
Hello Chris – Essentially you need to perform some string manipulation to coerce the JSON array you have, which contains your coordinate value pairs, into a Esri Feature JSON string representation of a polygon geometry. You are correct that using (a few) Field Calculator processors is probably the best approach. I think comments in the thread How to switch positions on coordinates show how to use Field Calculators to do what you need to do. You should not need to create a bunch of new attribute fields for each vertex and then somehow recombine these fields later. Please take a look at the linked thread and let me know if you have additional questions. The Esri Feature JSON specification and samples for Point, Multipoint, Polyline, and Polygon geometries can be found in the ArcGIS Developers on-line documentation. You might take a look at the thread https://community.esri.com/community/gis/enterprise-gis/geoevent/blog/2015/08/01/can-i-use-the-geoevent-simulator-to-simulate-events-with-geometries-other-than-point?sr=search&searchId=c1da863a-7cb1-4f8d-b8a9-f10dc5ac71ec&searchIndex=2 for examples of what an "Esri Feature JSON string representation" of geometry looks like, if you were to try using GeoEvent Simulator to send the data to GeoEvent Server. Hope this information is helpful – RJ
... View more
01-06-2020
06:15 PM
|
0
|
5
|
2541
|
|
BLOG
|
Overview Polygons which model areas of interest – counties, national parks, or property boundaries for example – are generally static. A new area of interest might be established requiring a geofence to be added, or an existing area’s geographic extent might occasionally change requiring a geofence to be updated, but in general the geofences don't change very often. This scenario fits well with GeoEvent Server’s ability to synchronize its geofences with a feature record set maintained as part of a feature service. The areas of interest can be maintained as feature records and occasionally imported to establish or update the relatively static geofences. A synchronization rule can periodically poll the feature service to obtain updates. This blog explores a different scenario. Suppose you need geofences to be created dynamically, managed for only a short period of time, and then frequently and automatically destroyed when no longer needed. Constantly polling a feature service to check and see if there have been any changes is impractical. In a dynamic scenario, we need to push changes to GeoEvent Server immediately as the changes are received. A video attached to this blog will show how a GeoEvent Service can be used to receive attributes describing an area of interest, compute an effective date/time range during which the area of interest is considered relevant, and generate a polygon to model the area of interest. A stream service will be used to broadcast dynamically generated polygons and computed date/time values as a feature records allowing them to be registered with GeoEvent Server as new or updated geofences via a synchronization rule. Objectives Import and review a pair of GeoEvent Services configured to process a tracked asset's current location and dynamic geofences constructed for a given center point of geographic interest. Review how stream services are published and configured outbound connectors updated to use the published stream services to broadcast processed event records as feature records. Use the GeoEvent Simulator to send simulated vehicle location observations to GeoEvent Server and display those locations, live, on a web map. Configure a synchronization rule to subscribe to a stream service and receive polygon feature records as they are broadcast (rather than relying on a feature service which must be frequently polled for updates). Demonstrate how information can be sent to GeoEvent Server, on demand, via HTTP/POST to drive the generation of dynamic areas of interest (e.g. geofences). Demonstrate the display and update of dynamic geofences both on a web map and in GeoEvent Server. Extend a GeoEvent Service with an analytic which detects when a tracked asset's location intersects a dynamic geofence and produce an alert message which can be displayed using the GeoEvent Logger. Discuss the temporal relevance of geofences, how analytics you configure will ignore geofences which are not temporally relevant, and how the GeoEvent Server AOI Manager automatically purges geofences which are no longer being used to clean-up its registry. Demo Resources I have included demonstration resources with this blog post so you can recreate this demonstration in your own environment. An attached ZIP archive includes an XML snapshot of a GeoEvent Server configuration which includes a couple of GeoEvent Services as well as inbound and outbound connectors. The configuration file was taken from a 10.7.1 deployment, but should work in the upcoming 10.8 and 10.8.1 releases. A pre-configured GeoEvent Service Trackpoints connects a TCP/TEXT input with a stream service output to broadcast point feature records and report the location of a simulated tracked asset. A second pre-configured GeoEvent Service, AOI_Centerpoint, connects an HTTP/JSON input with a stream service output to buffer received point locations and broadcast each buffer as a polygon feature record suitable for use as a geofence. A video will walk you through the import of the provided configuration, necessary stream service publication, and the configuration of a geofence synchronization rule to subscribe to receive feature records broadcast from the second stream service. I hope you find the video tutorial and information useful – RJ
... View more
12-13-2019
06:29 PM
|
6
|
0
|
2444
|
|
POST
|
Hello Markus - I was able to reproduce the issue you describe above. I'm not entirely sure that this is something that can be fixed server-side in GeoEvent Server. It looks to me like the client's filter specification is being applied to the stream service subscription without error ... but I'm seeing the web map client-side dropping the stream layer from the web map's content whenever I apply a filter to a stream layer whose stream service specifies Related Features. I don't think that the stream service's Store Latest plays into this issue. We are investigating the issue and will be in touch through the partner / distributor to work with you on this issue. Best Regards - RJ
... View more
12-03-2019
06:12 PM
|
0
|
0
|
1066
|
|
POST
|
Hello Eric – I'll check with Eric Ironside on our team. I thought that he had an update for the Verizon Connect (Telogis) Connector which leveraged dynamic templates to provide a more reliable method for users who want to update both driver and vehicle metadata from the Verizon feed. I mention this to make sure you are aware that there are two different connectors available: Verizon Connect (Telogis) Connector Verizon Networkfleet for GeoEvent Server The "Verizon Connect" connector is newer and supports data request polling (vs. the older "Networkfleet" connector which I thought required you to allow Verizon to push XML data to your GeoEvent Server). In the interim, have you considered establishing a feature service whose schema has attributes for both the "sensor" and "location" messages and using a partial geoevent definition to update first one part then another part of the feature record as messages of different types are received? You also might consider using an Event Joiner processor, which is a new processor introduced in the 10.7.1 release. –RJ
... View more
11-11-2019
11:11 AM
|
0
|
1
|
1158
|
|
POST
|
Hello Bing – Glad you got what you needed figured out. I noticed, though, that your JSON representation of the polygon geometry has no spatial reference. You will want to include that to "complete" the feature record specification if you want to be able to compare an event record's geometry to, say, a geofence to test spatial coincidence. You might find the following references helpful: Core Concepts - Features and Geometries - Points Core Concepts - Features and Geometries - Polylines Core Concepts - Features and Geometries - Polygons Also, GeoEvent Server recently updated its documentation for inbound connectors which includes some relevant illustrations, and there is a blog on using GeoEvent Simulator to pass JSON string representations of geometry which enables you to simulate geometries other than point: Input Connectors - Receive Features on a REST Endpoint Can I use the GeoEvent Simulator to simulate events with geometries other than Point Hope these are helpful– RJ
... View more
11-11-2019
10:33 AM
|
2
|
0
|
2193
|
| Title | Kudos | Posted |
|---|---|---|
| 1 | 01-05-2023 11:37 AM | |
| 1 | 02-20-2025 03:50 PM | |
| 1 | 08-31-2015 07:23 PM | |
| 1 | 05-01-2024 06:16 PM | |
| 1 | 01-05-2024 02:25 PM |
| Online Status |
Offline
|
| Date Last Visited |
Thursday
|