topic Re: Enterprise database connections dropping in ArcGIS Pro Questions

Enterprise database connections dropping

Ona_Tim — Wed, 26 Jun 2024 17:45:38 GMT

I'm not sure if this is the right forum but, I am having an issue with enterprise Database connections dropping to my computer. If I work with ArcPro on the server the connections do not drop. We also made a connection directly from my computer using pgAdmin and will not lose connection even when I loose it in Pro.

My system is as follows:

ArcGIS Enterprise 11.2 Single machine deployment. Server has 32GB of memory
ArcgisPro 3.3.0
Postgres 15.4 Downloaded from ESRI and on a separate server from enterprise. Server has 12 cores and 32GB of memory
Both servers are running Windows Server 2022 datacenter Version 21H2.

When the connection drops I get the following message:

Underlying DBMS error [no connection to the server::SQLSTATE=å] [**unspecified**.**unspecified**.GDB_Items][STATE_ID = 0]

X's replace actual names.

Working with ESRI we found the following information in the logs :

"An existing connection was forcibly closed by the remote host" where host is the PostgreSQL Server and "FATAL: terminating connection due to administrator command". The following PostgreSQL documentation provides more information about the same (https://www.postgresql.org/message-id/5665.1202148155%40sss.pgh.pa.us [postgresql.org]).

At this time IT says it is not a fire wall issue as there are no rules setup for dropping connections. We are at a loss as to what the culprit is.

I will also be posting this in PostgreSQL community to get their input.

Thank you for your help with this.

Re: Enterprise database connections dropping

George_Thompson — Wed, 26 Jun 2024 17:54:31 GMT

How are you accessing the Postgres EGDB, via VPN or directly in your office?

Does the timeout happen after a specific amount of time, like 15 or 30 minutes?

Does it drop the connection when in the middle of interacting with data or after a period of "inactivity"?

What are the resources on the Postgres VM reporting when the issue occurs? RAM / CPU utilization at that time?

Re: Enterprise database connections dropping

Ona_Tim — Wed, 26 Jun 2024 19:17:34 GMT

George,

In our office through standard geodatabase connection in pro.

If Idle it seems like 20 minutes tops.

I have had it drop while working in the database too. I'm not sure how long I was working in it when it happened.

I have the RAM/CPU Utilization up right now and will take a screen shot when disconnects. I have attached is screen shot when I opened the project.

Re: Enterprise database connections dropping

Ona_Tim — Wed, 26 Jun 2024 19:33:57 GMT

Here is the screen shot after loosing connection.

Re: Enterprise database connections dropping

George_Thompson — Wed, 26 Jun 2024 20:02:10 GMT

Ok, thanks for those items it does not appear to be resource related.

I am wondering if the PostgreSQL DB is running out of available connections?

I cannot find what I am looking for in order to check the # connections allowed.

Re: Enterprise database connections dropping

Ona_Tim — Wed, 26 Jun 2024 20:09:08 GMT

Here are the logs from ESRI highlighting, in yellow, the issues. It seems the authentication for the password for the postgres user is failing.

Edit: I forgot to mention that we are not using the postgres user to connect to the GDB.

Re: Enterprise database connections dropping

ISM — Wed, 03 Jul 2024 11:25:38 GMT

Hi!

I have the same issue and was wondering if you were able to resolve it.
Any help to resolve the problem would be greatly appreciated.

Thanks!

Re: Enterprise database connections dropping

Ona_Tim — Wed, 03 Jul 2024 14:14:22 GMT

ISM, we have not yet found a solution to this.

Re: Enterprise database connections dropping

BenjiMillard — Sun, 14 Jul 2024 01:42:19 GMT

I am having the same problems and I believe they are leading to ArcGIS Server crashing. I'm currently looking at the TCP Keepalive settings but haven't moved forward with any experimentation.

https://www.postgresql.org/docs/current/runtime-config-connection.html

Re: Enterprise database connections dropping

BenjiMillard — Mon, 22 Jul 2024 18:49:03 GMT

My suspicions were confirmed about the Keep Alive settings of PostgreSQL. After PostgreSQL 14 there were changes made to the way that PostgreSQL connections were made and kept open that close connections in unexpected ways if you're used to the older versions.

Some settings that should change in postgres.conf are:

tcp_keepalives_idle=120

tcp_keepalives_interval=20

tcp_keepalives_count=6

# 5 second interval in milliseconds

client_connection_check_interval=5000

And my ArcGIS Server connection is no longer crashing regularly.

Re: Enterprise database connections dropping

Ona_Tim — Wed, 24 Jul 2024 13:39:01 GMT

Thank you Ben!

I tested it yesterday into to today and the connection has not dropped.

Re: Enterprise database connections dropping

BenjiMillard — Wed, 24 Jul 2024 19:21:05 GMT

I'm happy to hear that. Glad it worked for you!

Re: Enterprise database connections dropping

eterra — Thu, 26 Feb 2026 22:05:43 GMT

Is this what you changed them to, or was this your default? I am currently experiencing the OP issue and my Keep Alive settings are as follows: idle=120, interval=30, count=9. The client_connection_check_interval=0 but documentation lists it as default/read-only parameter. I'm thinking of lowering my keep alives interval and count numbers and seeing if that helps. I am using postgresql 14.19