Strange DataStore Connection Issue... then automatically fixes?

1115
7
04-13-2021 05:44 AM
JohnSteed1
New Contributor III

We were unable to see content in our WebApplications this morning.  Generic "The layer .... cannot be added to the map."

We looked at the logs and saw a ton of the following errors coming from both of our Servers (two in our deployment):

SEVERE Apr 13, 2021, 7:56:58 AM FATAL: remaining connection slots are reserved for non-replication superuser connections Rest

SEVERE Apr 13, 2021, 7:56:58 AM FATAL: sorry, too many clients already Rest

During this time the DataStore would not validate with the hosting Server.

And then it magically resolved (maps and WebApplications load content fine), but left us with constant logs from our hosting Server like this:

SEVEREApr 13, 2021, 8:14:22 AMUpdate for the object was not attempted. Object may not exist.Rest

 

Any ideas?

0 Kudos
7 Replies
berniejconnors
Occasional Contributor III

This is the type of question I send directly to Esri's user support.  Your are paying for user support as part of your software licensing so you should use it.  I hope this community forum is not your primary source for support. 

It is helpful to the community to share information like this in the forum but to get the fastest solution to your problem you should contact Esri user support and then post your problem and the resolution from Esri here so we can all benefit.

Just my $0.02,

Bernie.

JohnSteed1
New Contributor III

Hi Bernie-

I also sent this to Esri support, but sometimes I get better information here.

I'll make sure to provide feedback when this is resolved.

Thanks!

John

0 Kudos
berniejconnors
Occasional Contributor III
Good to know John. I presumed you were also contacting Esri Support but I thought it was worth mentioning for the Community.
Bernie.
0 Kudos
AmberKelly
New Contributor III

Hi John,

 

Just wondering if you got any answers from Esri on this one?

 

Cheers

Amber

0 Kudos
JohnSteed1
New Contributor III

Hi @AmberKelly 
Thank you for your reply on this.
We did NOT get any answers on this.  It seemed to magically resolve.. which is unfortunate.

 

0 Kudos
AmberKelly
New Contributor III

Hi John,

Yes the "magic fixes" always leave me with uncertainty. Ill post here if I get any resolutions from Esri.

 

Thanks for replying!

Amber

0 Kudos
Vagish_Anand
New Contributor

@JohnSteed1 @AmberKelly @berniejconnors 

ArcGIS Data Store uses postgresql as it's underlying database. The application(ArcGIS Data store) is designed in a way that it will allow max 150 connections to posgresql. You can check that by running describedatastore command line tool on data store machine. In the output you will see value for maxconnections as well as Number of connections which are active. Not all 150 connections will be allocated by ArcGIS Data Store to fetch data for ArcGIS Server. Some connection slots to postgresql are reserved for super users, in this case, ArcGIS Data Store applications. Hence the message,  SEVERE Apr 13, 2021, 7:56:58 AM FATAL: sorry, too many clients already.

 

If this is happening quite often then you can consider increasing the upper limit of Max Connections but remember that more connections will require a beefier machine in terms of memory and CPU processing power.