GeoEvent has no valid keystore after config-store move

GillPaterson · ‎07-31-2019

We are on GeoEvent 10.6.1 (no patches) and have been required to move the config-store and directories to a new share. I used the ArcGIS Server admin>system>configstore and directories edit functions to do this. Upon completion the ArcGIS Server Manager opens correctly as a verified site with the correct certificates being shown. However, opening the GeoEvent Manager the browser warns that "Your connection is not secure. The owner of xxx has configured their web site improperly". Prior to the config-store and directories move, the Manager opened correctly. I restarted the server and upon start up the following was in the karaf logs

019-07-31T14:46:00,121 | ERROR | CM Configuration Updater (ManagedService Update: pid=[org.apache.cxf.osgi]) | HttpServiceStarted | 443 - org.ops4j.pax.web.pax-web-runtime - 6.0.3 | Could not start the servlet context for context path []
java.lang.IllegalStateException: no valid keystore

The certificates do exist where the error is pointing to and from what we can tell are all ok.

The arcgis.keystore matches the certificates that are installed on the machine (Windows 2012 under the Personal certificate folder, not the Trusted Root Certification Authorities folder - is this an issue?? not sure why moving the config-store would cause this to be an issue if it worked before)

Following the suggestions in 206700-geoevent-server-1051-no-service-was-found and RJs admin reset (which has been my go to geoevent fix until now) did not resolve the issue.

Am I correct in thinking that because the ArcGIS Server Manager is verified correctly that the arcgis.keystore under Program Files\ArcGIS\Server\framework\etc\certificates\arcgis.keystore is ok. However, GeoEvent is somehow not creating the C:\ProgramData\ESRI\GeoEvent\certs\geoEventSSLCertificate.jks correctly? The answer is probably not important, but how to fix it if it is the issue.

Any ideas on where to go to from here please???

Anonymous User · ‎07-31-2019

Hi Gill,

I've successfully resolved a GeoEvent SSL issue at 10.5.1 by using keytool.exe to import the certificate into carcerts locates at

C:\Program Files\ArcGIS\Server\framework\runtime\jre\lib\security\cacerts

hopefully it works for you as well.

Cheers,

Minbin

GillPaterson · ‎08-04-2019

Thank you Minbin Jiang‌ unfortunately that didn't help in our instance. Appreciate the suggestion though.

GillPaterson · ‎08-06-2019

We have a verified site again after a couple of re-installs. Yay.However a couple of things we are raising as a support ticket to determine whether we need to be worried about in terms of the long term stability and resiliency of the site.

The no valid keystore error is still present in the logs, but we now think that is potentially a trigger for it to create the geoevent keystore, not 100% sure.
We still are getting the "signed field invalid" error too. But given that the site is verified in the browser and is processing data to expectations, we are not sure whether this is a true error or if it will have any bearing on the functioning or stability of the site.
One concern is the number of bad http warnings we are getting as well as connection to the http 6080 help page.
1. not sure what the bad http warnings are for, they are present when there are no services.
2. how do we turn off the software trying to connect to the http help page (the site is configured to https only)

2019-08-07T10:08:21,971 | WARN | qtp1787802408-35708 | HttpParser                       | 407 - org.eclipse.jetty.util - 9.3.14.v20161028 | Illegal character 0x16 in state=START for buffer HeapByteBuffer@3afa250b[p=1,l=176,c=8192,r=175]={\x16<<<\x03\x01\x00\xAb\x01\x00\x00\xA7\x03\x03\xEf\x8f\xAd\xC2\x1eP\x9d...\x02\x04\x03\x03\x01\x03\x02\x03\x03\x02\x01\x02\x02\x02\x03>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}
2019-08-07T10:08:21,971 | WARN | qtp1787802408-35708 | HttpParser                       | 407 - org.eclipse.jetty.util - 9.3.14.v20161028 | bad HTTP parsed: 400 Illegal character 0x16 for HttpChannelOverHttp@589160c6{r=0,c=false,a=IDLE,uri=null}
2019-08-07T10:08:23,367 | ERROR | qtp1787802408-35565 | Http                             | 53 - com.esri.ges.framework.httpclient - 10.6.1 | Second attempt failed. Giving up. (http://LB + domain :6080/arcgis/help/en/cxhelp.xml --- Connect to LB + domain :6080 [LB + domainl/ip address] failed: Connection timed out: connect)
2019-08-07T10:08:23,367 | INFO | qtp1787802408-35565 | Http                             | 53 - com.esri.ges.framework.httpclient - 10.6.1 | Connect to LB + domain:6080 [gis-uat-ge.vicpolice.internal/ip address] failed: Connection timed out: connect