"Failed to start the server machine. Premature end of file"
This article suggested some sort of corruption in the installation... which we followed, and compared to a working deployment. The specified file (config.xml) had no obvious issues.. it had content and appeared comparable to a working installation.
We found a few log file in the C:\Program Files\ArcGIS\Server\framework\etc\service\logs directory and specifically found the following in the service.log:
Nov 10, 2016 9:21:05 AM org.apache.catalina.startup.Catalina stopServer
org.xml.sax.SAXParseException; systemId: file: C:/Program Files/ArcGIS/Server/framework/runtime/tomcat/conf/server.xml; lineNumber: 1; columnNumber: 1; Premature end of file.
and sure enough... the server.xml file was empty (0 bytes)
<Connector SSLEnabled="true" clientAuth="false" keyAlias="SelfSignedCertificate" keystoreFile="\\server\share\config-store\machines\servername\arcgis.keystore" keystorePass="<OBSCURED> maxHttpHeaderSize="65535" maxPostSize="10485760" maxThreads="150" port="6443" protocol="org.apache.coyote.http11.Http11Protocol" scheme="https" secure="true" sslEnabledProtocols="TLSv1.2,TLSv1.1,TLSv1" sslProtocol="TLS" />
I hope someone finds this useful if they run into a similar issue. We will be watching the site for stability, but a quick cursory test shows success and high confidence of this fix.
just want to add that if you have another machines in a cluster you can just copy the "server.xml" file from another machine in the cluster without any modifications.
Thanks P F, that was very helpful. We had server.xml blanked out on two separate clusters at the same time, but only on the site machine. Has anyone found the cause of this? I suspect a windows update or some sort of IT crapware installation...
This it us again today, and I forgot the fix... so had to re-discover. GAR
The root cause was our C:\ hosting the install location ran out of disk space. So I think the product tried to write to the file and couldn't and ended up writing a 0 byte file....
Had that issue as well. In our case the cause of issues seemed to be machine running out of memory. Lately application service running on same machine had some OutOfMemory errors and also in the ArcGIS logs. I am suspecting it crashed the ArcGIS and tomcat maybe tried to load config to memory, update the ports according to configs and then save it back to file before shutting down, but being out of memory meant 0kb were saved to the file.
Copying the server.xml from another working server is a good solution here. Glad it worked for you. Remember to re-import any certs you had on the ArcGIS Server machine. It may be also good to do a restart of ArcGIS Server first so that a new keystore gets created for this server, and then re-import any certificates as necessary.