All Services Show as Stopped

[mpboyle]

Our ArcGIS Server services all show at stopped when viewing through ArcCatalog, and if I view them through Server Manager, they show as either stopping or stopped.  However, if I go to our REST endpoint the services show as normal, and all our apps appear to be functioning as normal (there are plenty of SOC instances running, which seems to support this...).

If I try to start a service, either through ArcCatalog or Server Manager, it doesn't seem to start, and stays in a 'stopping' mode.

What would cause this behavior?  Any ideas on how to fix it?  I am trying to publish a service using caching, but can't view any of our existing cached services because they are in a stopping / stopped mode, and publishing probably won't work correctly because those tools are stopped as well.

System Environment

OS: Windows 2012 R2

Server Version: 10.4.1

ArcGIS Server service is running

Server RAM: 32GB

Server Processors: 4

[JayantaPoddar]

Could you check the memory usage of the machine hosting the ArcGIS Server? 

One of the issues could be caused due to low virtual memory or low heap size.

Please check

Unable to publish new or start existing services when a large number of ArcSOC.exe processes are run... 

How To: Increase virtual memory beyond the recommended maximum 

Take caution while making changes in the Registry

[JonathanQuinn]

This can happen if there's a network change, for example the machine for whatever reasons gets a new IP address or something to that effect.  If you were to restart the ArcGIS Server Windows service, does that clear things up?  The services should either be Stopped or Started, not Stopping after the restart.

[RebeccaStrauch__GISP]

Just in case this is your development server and you are using an EDN/Dev-license. 

Ours just expired this morning on our dev box, and we had the same behavior. I did a reboot, and then none of the services showed up (but the folders still did...strange).  I had thought I had re-authorized using the 10.1-10.4 authorization already (a month ago), but for some reason, it/I had only grabbed the Portal install.  Keep in mind that the 10.1-10.4 authorization for AGS/Portal is different than the 10.5.x Enterprise authorization.

Going into myesri, Developer, I created a new authorization file (after viewing what has been provisioned) for 10.1-10.4 AGS. Then, using the .prvc file to reauthorize fixed the issue (but restarting my AGS again just to make sure).  And now that I think about it, after the reboot, the Windows Server Manager did not show that is start.  Now it does.

BTW - when I looked at the logs in Server before reauthorizing, I had a bunch of errors like:

Funny, I always know the expiration is coming (and thought I had already fixed it) but it still catches me off guard sometimes....makes me thing my privileges were changed by IT or the likes.

BTW - the reason I even looked/researched is I was trying to use "View cache status" on a service that appeared to be started, but it was grayed out.  That is, just weird behavior, but no real red flag.

[Anonymous User]

Hello 

as @Jayanta Poddar mentioned it will due to many causes or issues system server memory and CPU utilization goes high and ArcGIS server trying to reset self but it's not completed successfully.  

Reset it through services.msc and check server memory disk & CPU utilization and also check for database connectivity.

If all looks good then you have to dig deep and check n/k and other related applications installed on the system server.