Linux ArcGIS Server 10.4.1 Map Services Won't Start After Being Stopped

[JustinGreco]

We just started noticing an issue with one of our ArcGIS Server environments that is at 10.4.1.  After stopping a map services, we are unable to start the map service again.  The only way we can get it to start again is to restart ArcGIS Server.  When AGS comes back, the map service is in the stopped state and we are able to start it again.  Though if we repeat stopping and then starting, it never starts again.  This is the case with any service, including all of the system services (caching, publishing, etc.).  We have worked with customer support, thought I would check here as well.

[JayantaPoddar]

Make sure the ArcGIS Account and the current user account has READ/WRITE access to config-store folder. Also stop the ArcGIS Server service, and delete all the files inside (<ArcGIS Server Installation Directory>\config-store\locks). Restart ArcGIS Server service.

Allow some time for all the GIS services to restart in ArcGIS Server.

[JustinGreco]

The ArcGIS account definitely has R/W access to the config-store.  Though I do not see a locks directory, which I do see on my other environments...

[JonathanQuinn]

Any errors that you can provide?  What's the initial error after stopping and then trying to start the service?  Do you have any debug logs?

[JustinGreco]

That's the interesting thing, nothing really appears in the logs. Even have it set to debug and you can see where it tries to start the service, but that's the last message in the logs. Have an incident with premium support. My guess is that there is some package that got updated that is causing a conflict.

[JonathanQuinn]

Eek, without any logs it makes it that much more difficult.  What error are you seeing within the UI/API?  I'm surprised that debug logs wouldn't return at least the failure you see within the Admin API or wherever you're trying to start the service.

[JustinGreco · Accepted solution]

Ended up being an issue with having ArcGIS Server installed on mounted NFS storage.  All our storage in our Linux environment is NFS, but some reason ArcGIS Server didn't like that to be mounted, had to be local to the server.  The config-store and directories are still installed on the mounted shared storage, since they need to be shared among servers on the site.  Esri Premium support helps us identify this issue.