topic Re: ArcGIS Data Store won't validate in ArcGIS Enterprise Questions

ArcGIS Data Store won't validate

FredSpataro — Thu, 14 Sep 2023 20:54:28 GMT

Hello,

I've got a simple all-in-one enterprise portal setup for development/testing but the Relational data store won't validate.

Thanks for any suggestions.

From the datastore config wizard:

From the server manager:

From the server rest admin:

From the server manager logs:

SEVERE Sep 14, 2023, 2:47:44 PM The connection property set was missing a required property or the property value was unrecognized. Bad login user[ ] DataValidator.GPServer
WARNING Sep 14, 2023, 2:47:14 PM Validate: null Data Store
WARNING Sep 14, 2023, 2:41:44 PM Validate: null Data Store
WARNING Sep 14, 2023, 2:19:09 PM ArcGIS Data Store is attempting recovery. Data Store
WARNING Sep 14, 2023, 2:18:27 PM ArcGIS Data Store encountered too many problems; however, the standby machine 'PORTAL.XXXXX.COM' will not proceed with failover due to validation check failures. Data Store
WARNING Sep 14, 2023, 2:18:27 PM Validate: Could not connect to server on machine 'PORTAL'. ArcGIS Data Store or ArcGIS Server on that machine may not be running or the machine is not reachable at this time. Data Store
WARNING Sep 14, 2023, 2:18:26 PM Validate: null Data Store
WARNING Sep 14, 2023, 2:18:26 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store
WARNING Sep 14, 2023, 2:18:18 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store
WARNING Sep 14, 2023, 2:18:11 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store
WARNING Sep 14, 2023, 2:15:16 PM ArcGIS Data Store is attempting recovery. Data Store
WARNING Sep 14, 2023, 2:14:26 PM ArcGIS Data Store encountered too many problems; however, the standby machine 'PORTAL.XXXXX.COM' will not proceed with failover due to validation check failures. Data Store
WARNING Sep 14, 2023, 2:14:26 PM Validate: Could not connect to server on machine 'PORTAL'. ArcGIS Data Store or ArcGIS Server on that machine may not be running or the machine is not reachable at this time. Data Store
WARNING Sep 14, 2023, 2:14:24 PM Validate: null Data Store
WARNING Sep 14, 2023, 2:14:24 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store
WARNING Sep 14, 2023, 2:14:17 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store
WARNING Sep 14, 2023, 2:14:09 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store
WARNING Sep 14, 2023, 1:54:18 PM Failed to validate the data store machine 'PORTAL'. 'Server machine 'https://PORTAL:2443/arcgis/datastoreadmin/machines/PORTAL/validate' returned an error. 'The requested resource could not be found in the system. Please check the parameters and try again.'' Admin

Re: ArcGIS Data Store won't validate

Scott_Tansley — Thu, 14 Sep 2023 21:06:55 GMT

have you used the describedatastore command line tool:

https://enterprise.arcgis.com/en/portal/latest/administer/windows/data-store-utility-reference.htm#ESRI_SECTION1_5ECEA8B5F835454CB7D0DBC942BB4FDB

it can give you some useful diagnostic information.

Re: ArcGIS Data Store won't validate

A_Wyn_Jones — Fri, 15 Sep 2023 08:14:15 GMT

The message "ArcGIS Data Store encountered too many problems" can sometimes relate to resource contention. Is the spec of the machine sufficient?

Have you tried restarting the ArcGIS Datastore service?

Re: ArcGIS Data Store won't validate

FredSpataro — Fri, 15 Sep 2023 15:53:04 GMT

@Scott_Tansley @A_Wyn_Jones thanks for the suggestions:

1. machine is just installed so no load or anything resource hogging at this point. Basic performance stats from Task Manager: < 10% CPU utilization & < 50% Memory usage

Also yes, tried a datastore service restart.

2. Output from describedatastore.bat ... nothing seems out of the ordinary expect the tile cache validates and it's member machine is PORTAL while the relational datastore's member machine is PORTAL.{domain}.com .... the FQDN has been a recurring issue for me, especially on machine that aren't joined to the domain ie dev machine and DMZ machines ...

C:\Program Files\ArcGIS\DataStore\tools>describedatastore.bat Available ArcGIS Data Stores ================================================== Relational Data Store Tile Cache Data Store Spatiotemporal Big Data Store Graph Store Object Store General information of ArcGIS data stores on PORTAL.XXXX.COM ================================================== Data store release..................11.1.0.42869 Staging location....................C:\ArcGIS\arcgisdatastore\staging Log location........................C:\ArcGIS\arcgisdatastore\logs Free disk space.....................153.00GB Threshold for READONLY mode.........1024MB Information for relational data store ds_w9mc1edt ================================================== Backup location.....................C:\ArcGIS\arcgisdatastore\backup\relational Is backup folder shared.............false Backup schedule.....................{"schedule-starttime":"00:00:00","schedule-frequency":"Every 4 DAYS"} Days backup retained................7 Data store status...................Started SSL enabled.........................true member machines.....................PORTAL.XXXX.COM Maximum connections.................150 Owning system URL...................https://portal.xxxx.com/server Portal for ArcGIS URL...............https://portal.xxxx.com/portal Number of connections...............0 connection(s) to managed database Data Store mode.....................READWRITE Is Point-in-time recovery enabled...No Query optimizer enabled.............Yes Information for tile cache data store tcs_ou0nflv3 ================================================== Data location.......................C:\ArcGIS\arcgisdatastore\nosqldata Data store status...................Started Backup location.....................C:\ArcGIS\arcgisdatastore\backup\tilecache Mode................................primaryStandby member tile cache machines..........PORTAL Owning system URL...................https://portal.xxxx.com:6443/arcgis/admin Portal for ArcGIS URL...............https://portal.xxxx.com/portal Operation completed successfully.

Re: ArcGIS Data Store won't validate

Scott_Tansley — Sun, 17 Sep 2023 22:14:15 GMT

I notice you've enabled SSL. It may make sense to disable it and then revalidate. I wonder if the portal/hosting server are unable to trust the cert:

https://enterprise.arcgis.com/en/data-store/latest/install/windows/data-store-utility-reference.htm#:~:text=ArcGIS%20Enterprise%20deployment.-,enablessl,-%E2%80%94Used%20with%20the

or test this:

https://enterprise.arcgis.com/en/portal/latest/administer/windows/update-ads-ssl-certificate.htm#ESRI_SECTION1_53A72B95A3A44470A25687B4BD32256C

Re: ArcGIS Data Store won't validate

Brian_Wilson — Mon, 11 Mar 2024 17:32:09 GMT

Doing "allowconnection" will make an entry into pg_hba.conf, to allow a different machine to connect. For example if I want to run postgres tools from my desktop. Connections on localhost should always be allowed by default on the hostssl port.

Regarding self-signed certificates, I don't think Postgres cares at all about that. The page you reference says "Similarly, when the hosting server communicates with the data store, or individual machines within the data store communicate with each other, a self-signed SSL certificate is used. This is sufficient for most organizations."

The postgres docs say they run both host and hostssl on the same port and clients negotiate. So if you want to skip hostssl you need to allow host (unencrypted) connections. I don't see an Esri tool for that, I think they have gone all in on hostssl? I mean, I probably don't want my password auth flying over a network without encryption but I'd be okay with localhost connections. The pg_hba.conf file on my datastore even has ssl turn on for localhost, for example

grep 127.0 pg_hba.conf hostssl all adm_o77v5 127.0.0.1/32 scram-sha-256 hostssl all sde 127.0.0.1/32 scram-sha-256 hostssl db_0eoas hsu_9vn8j 127.0.0.1/32 scram-sha-256 hostssl db_0eoas ins_nb0tr 127.0.0.1/32 scram-sha-256 hostssl db_0eoas sde 127.0.0.1/32 scram-sha-256 hostssl dsadmindb adm_o77v5 127.0.0.1/32 scram-sha-256 hostssl dsadmindb dsadmin 127.0.0.1/32 scram-sha-256

Esri is saying that you may run CA certificates but it's a network policy decision, the database servers won't care.

I am chasing a similar problem as @FredSpataro I think -- my two DataStores are having "issues" and won't work with each other. Same error messages as Fred but separate machines. The primary is unhappy but does not tell me why and it won't switch to the standby because they have a validation problem. Really with the language they are using ("issues", "validation", "relations") I think I need to bring in a therapist.