Upgrade Enterprise Geodatabase from 10.8.1

[ZachBodenner]

Hello, I have some questions about upgrading our egdb. Background:

Currently, egdb is on 10.8.1 on a SQL Server 2019, Enterprise deployment (server/portal clients) are on 11.1. All desktop users are using Pro 3.2, with a few exceptions that have been tough to completely pull away from ArcMap. From what I understand, it isn't inherently bad to have different egdb and client versions, though it is recommended to keep them in step. I was reading about upgrading egdb and came across this article: Upgrade Enterprise Geodatabase in SQL Server In particular, this passage caught my eye:

"When you upgrade a geodatabase in SQL Server to 11.0 or a later release, the fully qualified names of tables and feature classes will no longer contain the database name. For example, a feature class named productdata.dataowner.inventory in a 10.9.x geodatabase will be named dataowner.inventory after you upgrade the geodatabase."

This seems very important, but I'm wondering what the knock-on consequences of this change are. For example, if I have web services that contain a feature class joined to a table, will the removal of the database name affect the join? Any other concerns that I should be aware of but can't see? Finally, any other issues you may have run into upgrading your egdb? This will be my first time going through the process.

[ZachBodenner]

It does, thank you.

[ChristopherBowering]

We recently upgraded several EGDBs from 10.8.1 directly to 11.1 and are now experiencing significant issues with some of our web services.  Our deployment is comprised of 3 servers with the GIS Enterprise version at 11.1.  The Enterprise upgrade to 11.1 occurred last fall.  We are using SQL 2017 and Windows Server 2019 Standard (10.0).

We have multiple database connections/users, each uniquely set up, which are used to access the aforementioned databases.  Some services published thru certain database connections remain perfectly functional...services published thru one specific database connection only partially appear when viewing the REST URL (some layers are there, some are missing)...services published thru a couple different database connection are not even viewable in REST anymore (URL times out after a couple minutes) and corresponding layers already in WebMaps fail to load.  Everything was working fine prior to upgrading.

Are there any known issues - or steps to be taken we may have missed - related to database connections and web publishing when upgrading EGDBs?  For reference, I am able to load data from all database connections into Pro without issue.  The web services are what is problematic.

[MarceloMarques]

@ChristopherBowering - You might have to republish your services using ArcGIS Pro, see the documentation below.

What's new in ArcGIS Enterprise 11.0—ArcGIS Enterprise | Documentation for ArcGIS Enterprise

Migrating services to the ArcGIS Pro service runtime—ArcGIS Server | Documentation for ArcGIS Enterp...

Client and geodatabase compatibility—ArcGIS Server | Documentation for ArcGIS Enterprise

You do not have to keep your geodatabase and ArcGIS clients at the same release, but it is recommended that you do so. Geodatabases and client software are designed to work together, and if you let one get too many releases away from the other, you risk encountering problems or unexpected behavior.

This is especially true when you use a mix of client versions at your site. A newer client can create newer dataset types in the geodatabase that older clients cannot access. For enterprise geodatabases, waiting too long between geodatabase upgrades may mean you have to upgrade the underlying database more than once before you can upgrade the geodatabase.

I hope this helps.

[ChristopherBowering]

Hi there.  All of our services have already been published in Pro.  We have no remaining ArcMap published services.  We are using Pro 3.2.x as well.

[MarceloMarques]

@ChristopherBowering - if all services were published with Pro, then check the SQL Server machine cpu and memory utilization, in SQL Server Activity Monitor you can see things like waits and active expensive queries, something might show up there, you can also enable SQL Server Query Store on the database to help diagnose database performance further, see SQL Server documentation, next you can bring each featureclass into ArcGIS Pro and make sure the performance to draw is acceptable, make sure the geodatabase maintenance is executed often, e.g., sde compress (if working with traditional version), gather new statistics, rebuild indexes (attribute columns and spatial). If this test passes, then the issue is not with the SQL Server Instance nor with the SQL Server Database nor with the Geodatabase. It must be something in ArcGIS Server Services, next I would try to reboot the ArcGIS Server machine to see if things clear up, and monitor, for the services affected, I would check the cpu/memory utilization in the ArcGIS Server machine, and then tune the min/max number of SOC processes of each ArcGIS Server service according to the workload, I would also look into the ArcGIS Server Logs in debug mode to see if any useful info show up.
I hope this helps.

[ChristopherBowering]

Thank you for your in-depth responses and suggestions.  We did reboot the ArcGIS Server machine last night with no change.  Our services were running smoothly prior to the EGDB upgrade so there were no changes in pooling, memory usage or general database configurations. The 'trigger' for these issues was the upgrade from 10.8.1 to 11.1. 

I do not believe the issues lies with the data.  The inconsistency in the behavior of the services depending on the user from which data is published is telling, I just can't figure it out.  There are no issues when actually publishing the services from Pro - no warnings or errors.  Only when we attempt to view in REST and/or load/view services in WebMaps do the problems become apparent.  Have you heard of any user/connection specific problems after performing a EGDB upgrade in regards to publishing?  We have not altered our database connections and they function fine in Pro as I had mentioned.  I feel as though the root of the problem lies within these database connections by the varying degrees that the services work - or don't work - depending on which connection a service is published thru.

[MarceloMarques]

@ChristopherBowering  

Question: Have you heard of any user/connection specific problems after performing a EGDB upgrade in regards to publishing? 

Answer: I did not hear of any issues.

You can trace the service request using Fiddler, that might help diagnose further.

Monitoring web service requests using Fiddler - Esri Community

How To: Use Fiddler to Capture Https Connections and Decrypt Https Traffic (esri.com)

You shall open a ticket with Esri Technical Support to investigate the issue as well.

[ChristopherBowering]

Thank you for those resources.  I will check that out!

I opened a ticket with ESRI yesterday.  Not much progress is being made which is why I posted here in an attempt to figure things out.

[MarceloMarques]

@ChristopherBowering - please share the esri support case number, if I learn anything new about your problem, I let you know.

[ChristopherBowering]

The case # is 03601725.  Thank you for keeping an eye on this!