Portal prompting user login to view item details for Hosted items

[Guinevere]

Our Portal is prompting logged in users to login, again,  for only Hosted items when trying to view in maps or view their items details page. This does not happen for our items that have been published via referencing. Items are properly shared to users within the organization.

This is annoying our users and we cannot figure out why this is happening, nor how to stop it.

System information:
Enterprise 10.8.1 multi-machine deployment - disconnected
Portal is Federated with Server and the GIS Server and Portal have separate web adaptors which are configured correctly. 

Any advice would be appreciated. 

[AmandaDeMedeiros]

1. When did this start occurring?
   1. Does this happen to all hosted items?
   2. Does this happen to every user?
2. What kind of logins do you have? (Built in, AD, ADFS or some other SAML)
3. Are there any Portal or Server logs?
4. Would you be able to provide the Federation information?
   1. Take a screenshot from the ArcGIS Server Administrator Directory
   2. Home > security > config
      1.         Portal properties
   3. Does the describe datastore cmd show any errors?

[Guinevere]

Here are the answers to your questions:

1. This appears to have started after we ramped up our publishing into the Portal. So a few months now. 
   1. Yes, this happens to all Hosted Items only. Items published via referencing works fine.
      • New Development: Users can no longer sign in with prompt. Shows "Unable to access the authentication service." error.
        (see screenshot below)
        
   2. Yes, all users see this issue, including Admins.
2. We have built-In Portal Users.
3. I have checked the server logs, they show "Error getting service" and "unable to process request. Service hosted/<service name>/MapServer not found" errors with WARNING level.
4. Due to our companies security policies I cannot share a screenshot as the URLS and IDs in that section would expose our environment and is thus a security risk.
   However, I can say that the URLs listed are correct for the environment, pointing to the correct machines, with the correct Port IDs and the Portal Mode there is listed as "ARCGIS_PORTAL_FEDERATION"
5. Looking at the DataStore in server manager > Site > GIS Server > Data Stores  I am now no longer able to validate it with the server. The status shows a red exclamation point when I select it and click "Validate". This is a recent new development.

Let me know if you need additional information, I will try to see what I can provide.

[tigerwoulds]

Check if the Relational DataStore validates with your Hosting ArcGIS Server. If not, that's your smoking gun.

I would start by rebooting the DataStore service. Server might need one too. I've seen odd behavior like this after Windows OS patching and rebooting the entire ArcGIS Enterprise stack fixed the issue.

[Guinevere]

The DataStore is not validating with the GIS Server, so I will try rebooting the services and see if that works.

[Guinevere]

So I tried rebooting and it did nothing to allow the ArcGIS Datastore to connect again. Still not validating. 😭

[Guinevere]

So I fixed the datastore not validating, turns out it had wiped out the json in its config file.
BUT, I am now still being prompted to sign in not only for hosted data, but referenced as well. Even if I sign in as the data owner, I get the prompt.
I tested with my colleague who signed into the same machine with the same credentials, but did not get the sign in prompt at all.... 

So I am guessing that it might be related to my windows credentials???? I am going to check that out, but I am still lost as to why this is happening.