AGS 10.1 SP1 won't start in Windows 7

Question

Perfectly fine installation of ArcGIS for Server 10.1 SP1 suddenly failed to start. Problems started after system boot due some MS updates. OS is Windows 7 x64 (active directory -domain, OS installed about two months ago). AGS is originally installed from "ArcGIS for Server 10.1 SP1" -installation media.  I have tried several tricks to solve this.

I have uninstalled those MS updates one by one. Booting system after those uninstalls. Trying then to start "ArcGIS Server" -service. AGS is also "repair installed" and fully uninstalled and then reinstalled. Also "configure arcgis server account" -tool is runned several times.  All these lead to same  result. Server just wont start.

Machine is even restored to earlier state with Windows Restore point. No luck.

Common for all tryouts when starting server are the events. Events 4,5,4096 (twice) and 4098 are registered in windows event log in that order. There is no explanation of those events in Event log. I also find out that AGS have not wrote anything to logs located at C:\arcgisserver\logs\<machinename>\server after the first failure in startup.

Does anyone have similar issues?

RichardWatson · Answer

I am glad that you were able to sort this out.  That was a good find that will help others.

When you uninstall (or repair) a program it typically only addresses files and setting which were made during the installation process itself.  For example, files and setting generated after installation are typically not addressed.

When I have problems what I do is:* Uninstall the ESRI program(s)
* Manually find and delete any left over files on the file system
* Manually remove any left over registry entries
*

File system examples:* C:\arcgisserver
* C:\Program Files\Common Files\ArcGIS
* C:\Program Files\ArcGIS
* C:\Program Files (x86)\ArcGIS
* C:\Program Files (x86)\Common Files\ArcGIS
* C:\Users\*\ESRI

Registry examples:* HKEY_CURRENT_USER\Software\ESRI
* HKEY_LOCAL_MACHINE\SOFTWARE\ESRI
* HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ESRI
*

JarkkoTiirikainen · Answer

4) Run the SysInternals ProcMon tool and see if you can spot where the ArcGIS Server service is having problems

This was actually quite helpful hint. Thanks!

Using Procmon I was able to trace this issue to files at C:\Program Files\ArcGIS\Server\framework\etc. At server startup there was problems when using/accessing arcgis-logsettings.json and arcgis-logsettings.json.rlock. Those files were modified at the time when server was last time in working condition. Content of the .json -file was rubbish (some empty lines and special characters. I made an assumption that some json-formatted content would be preferred).

I removed those two files and then server started normally. I had to make server configuration from scratch because original configuration was already lost in previous cleaning attempts. Anyway, that was not a big issue. I think that server was shutdown somehow 'unclean' manner when MS updates were installed. Then those files became corrupted.

It is weird that those files mentioned were still in place. Server was repair-installed, and also fully uninstalled/reinstalled several times when solving this issue. I would expect that all folders inside C:\Program Files\ArcGIS\Server would be cleaned when uninstalling/reinstalling.

RichardWatson · Answer

I think that what you are looking for is an understanding of what is wrong with your system and how do you fix it.

Suggestions:
1) Look at the NT event logs
2) Look at the ArcGIS Server logs
3) Try is running the server program manually, i.e. try the Java command line that the service is running and see if that assists you in debugging the issue.
4) Run the SysInternals ProcMon tool and see if you can spot where the ArcGIS Server service is having problems

Sometimes the system gets so messed up that trying to sort it out isn't productive.  In this case it is often better to just reload the OS and the products.  That particularly works if you use Virtual Machines and even better if you have save points which let you recover the entire system to a known good state.

Best of luck!