Select to view content in your preferred language

Data store fails validation

4778
7
Jump to solution
03-01-2022 01:25 PM
RobStauder
Occasional Contributor

Hi all,

I'm looking for some suggestions to help me resurrect our ArcGIS Enterprise. Here's what happened - 

  1. Errors occurred while publishing some very large scene layers.
  2. The output services did not start, so services were manually restarted and perhaps the entire site was restarted. This also caused errors due to an expired certificate.
  3. The expired certificates were replaced with self-signed certificates.

However, now:

  1. The site will not validate through the settings section in the portal
  2. The relational data store will not validate in the server manager (but the tile cache does). 
  3. The data store does validate through the server's rest validate operation (../admin/data/items/enterpriseDatabases/AGSDataStore_ds_blah/server_blah/validate)
  4. Some of the services will not shutdown or restart. Eg - the system SceneCachingTools service will not start, and throws error java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: java.net.ConnectException: Connection refused: connect, while our network routing service (map server) does not respond, won't render in the map and will not restart.
  5. We cannot publish new feature layers or scene layers, but we can view scene layers and feature layers published before the crash. 
  6. describedatastore.bat returns a status of started for both relational and tile cache. Mode = readwrite
  7. The windows application log has a few errors
    1. Notification of policy change from LSA/SAM has been retried and failed. Error 4312 to save policy change for account
    2. The description for Event ID 0 from source PostgreSQL cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer. The following information was included with the event: Is server running?

Any ideas? Should I try updatesslcertificate.bat with the self-signed certificate (as this is the only certificate I can't view)? 

Thanks!

Rob

0 Kudos
1 Solution

Accepted Solutions
RobStauder
Occasional Contributor

Thanks to everyone who replied. I resolved it. I followed the suggestions in this document - 

Problem: Unable to start or stop services in ArcGIS for Server (esri.com)

I found the word undeploy in one of the logs, which led me to that article. 

Once the system was back up and running, and I was able to publish feature and scene layers to the portal, I had a conversation with someone from Esri tech support. The analyst confirmed that the certificates and permissions were fine. 

I appreciate all the help from this forum and Esri tech support. 

Thanks all!

Rob

View solution in original post

0 Kudos
7 Replies
GeoJosh
Esri Regular Contributor

Rob, anything of interest in Data Store logs? C:\arcgisdatastore\logs\<machineName>\server

Did you apply a new license to ArcGIS Server by any chance?

0 Kudos
RobStauder
Occasional Contributor

Thanks for the response. The only thing in the logs is Validate: Failed to check layers extent. No new licensing. I just checked the software authorization in the server manager and we are good through 11/22.

0 Kudos
Ikebana
New Contributor II

Hello, did you only change cert for the datastore?

0 Kudos
RobStauder
Occasional Contributor

Hi. Thanks for the response. That's the only one I can't confirm was changed. I know that the ssl certificates for the portal and server were updated to use the self-signed certificates. I don't know how to view the data store's certificate, so I'm not sure if it was updated. 

0 Kudos
Ikebana
New Contributor II

Sounds to me like a Cert issue. But I have also experince that only tile cache datastore worked. My solution was to uninstall datastore --> reinstall --> import backup.

Is it a single machine deployment?

0 Kudos
HenryLindemann
Esri Contributor

Hi @RobStauder,

You can check you datastore cert  by opening the url https://localhost:2443/arcgis/datastore then looking at the cert info.

You have two problems your ArcGIS Server is not functioning correctly and your Datastore is not validating.

First make a backup of the system

Then can you go to services and look which service account is running your ArcGIS server then reapply the security rights on the folder structure at c:\program files\arcgis\server and the location of you configstore c:\arcgisserver or d:\arcgisserver...

Service account

HenryLindemann_0-1646279953996.png

config-store

HenryLindemann_1-1646280023629.png

Program Files

HenryLindemann_2-1646280055657.png

 

right click and open server properties then security and your service account make sure it has full control on these folders, there is a new Windows policy that changes the service account to read only. 

HenryLindemann_3-1646280121199.png

HenryLindemann_4-1646280232054.png

 

after you have applied this you can check if your ArcGIS server is functioning.

 

Then there might be a partial file that has been created during your failed publishing go to  C:\arcgis\arcgisserver\config-store\services\Hosted and see if you can see anything also your rest and services will not work if this is the case 

https://dns:6443/arcgis/rest

 

Datastore
if you can make a backup then @Ikebana suggestion will work and do a nice clean rebuild of datastore but your ArcGIS server must be in a working state.



You can also try the following go to server manager and unregister the datastore then re-register it on https://dns:2443/arcgis/datastore

you only have to unregister relational NB did you make the backups

HenryLindemann_6-1646280739834.png

 

 

using the https://dns:6443 admin url

HenryLindemann_5-1646280689115.png

 

if it still fails then go to the tool 

"C:\Program Files\ArcGIS\DataStore\tools\unregisterdatastore.bat"

  

HenryLindemann_7-1646280889933.png

and try the 2443 url again

If it still fails then the only option left is to restore the backup

to create the backups run
"C:\Program Files\ArcGIS\DataStore\tools\backupdatastore.bat" --store relational

and 

"C:\Program Files\ArcGIS\DataStore\tools\backupdatastore.bat" --store tileCache

HenryLindemann_8-1646281213637.png

 

please make sure that  they did actual make a backup in the backup location

HenryLindemann_9-1646281320318.png

and 

something like this for relational

HenryLindemann_10-1646281351883.png

 

to restore uninstall ArcGIS datastore and remove all the files except backup in c:\arcgisdatastore or your location
 

relational then edit and run these commands
"C:\Program Files\ArcGIS\DataStore\tools\restoredatastore.bat" --store relational --target most-recent --source-loc D:\arcgisdatastore\backup\relational --data-dir D:\arcgisdatastore --server-url https://dns:6443/arcgis --server-admin adminuser --server-password adminpassword --loaddata true --prompt no

tileCache
"C:\Program Files\ArcGIS\DataStore\tools\restoredatastore.bat" --store tileCache --target most-recent --source-loc D:\arcgisdatastore\backup\tilecache --data-dir D:\arcgisdatastore --server-url https://dns:6443/arcgis --server-admin adminuser --server-password adminpassword --loaddata true --prompt no

don't copy and past the above string type it out please you might get encoding problems

Regards

Henry

0 Kudos
RobStauder
Occasional Contributor

Thanks to everyone who replied. I resolved it. I followed the suggestions in this document - 

Problem: Unable to start or stop services in ArcGIS for Server (esri.com)

I found the word undeploy in one of the logs, which led me to that article. 

Once the system was back up and running, and I was able to publish feature and scene layers to the portal, I had a conversation with someone from Esri tech support. The analyst confirmed that the certificates and permissions were fine. 

I appreciate all the help from this forum and Esri tech support. 

Thanks all!

Rob

0 Kudos