Enterprise 10.9 token error logging for hosting, federated sites connecting to Portal

[DavidColey]

Hello - our 10.8.1 upgrade to 10.9 went well.  We have a distributed deployment with separate web, portal, hosting and federated sites on Windows 2016 VM machines.

I have re-shared everything I can think of from Living Atlas, Routing, Geocoding services.  No issues with our web adaptors, no 'bounce' from DNS alias to machine names, so problems with the web context url.  No problem with the 'arcgisonline' redirect uri's. The Monitor Administrator returns valid tokens when testing the portal and the hosting and federated sites, whether using the web adaptor site names or the internal DNS names.

Everything that is supposed to be publically accessible is.  However, we are getting an error that I had not seen since 10.6 from the hosting site:

 Rest - Exception in get user privileges Server machine 'https://myDNSServerName:7443/arcgis/sharing/rest/community/self' returned an error. 'Invalid token.'

Honestly, if I did not have Monitor installed I probably would not have noticed since there is no impact to any of my hosted services that I can tell so far, but the logging is excessive (100s and 1000s of entries) .  Very similar to the reply from @danweiss from this old post here:

https://community.esri.com/t5/arcgis-enterprise-portal/arcgis-portal-server-exception-setting-owners...

Any suggestions anyone?

@JonEmch 

@JonathanQuinn 

@Kathleen_Crombez 

[JonEmch]

@DavidColey  Noted, thanks!

[DavidColey · Accepted solution]

Hi @JonEmch  - with the 10.9.1 upgrade, the excessive logging no longer occurs. 

[JasonHansel1]

David,

Are you also seeing an increase in the amount of RAM the java processes are consuming? On my end I am seeing a significant increase in RAM being consumed after upgrading from 10.8.1 to 10.9

Thanks,

Jason

[DavidColey]

Hi @JasonHansel1 , no I am not seeing any real memory increases.  Monitor is showing both the machines in my hosting site hovering around 23-25 % memory utilization - typical for what I have seen since 10.6 for our environment.  If I do I will be sure to let know . . .

[JasonHansel1]

Unregistering and re-registering ArcGIS Datastores has fixed some issues I was seeing with 10.9

[HamishMills]

I am experiencing this same issue after upgrading from 10.8.1 to 10.9. Interestingly we have a test environment that does not replicate this issue. Very interested to see where you get with it. 

[rcgarrett]

I am experiencing this same issue also after upgrading from 10.8.1 to 10.9. Very interested in a resolution. Hard to separate cause and effect when other problems are occurring. Thanks.

[JRaymond]

We are also seeing this issue since upgrading from 10.7.1 to 10.9.  It sure makes it messy when reviewing logs to troubleshoot other issues.  Like was mentioned, this doesn't seem to affect performance at all, but it's a constant stream of warning messages in the logs.

[Natalie_Runyan]

I'm getting same errors on a new 10.9 build (Datastore, Server, Portal on separate machines; federated, hosting; OS Windows 2019). Identity store is SAML Azure AD. Domain service account on all servers have full control to relevant program and content directories.

[JasonHansel1]

You check out the Firewall? I’d also try to unregister the DS and re-register. I have run into a scenario or two where I’ve had to restore the DS.