Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

[DavidColey]

Hello - in prep for upgrade of our federated server site to 11.1 from 10.9.1, I removed the ArcMap based runtime after finally being able to remove my remaining ArcMap authored services.  When I did this, like 30K +  log files got dumped to the C:\ root. 

I deleted the log files, restarted arcgis server service, went into server admin, re-edited the log output location (even though this did not change) to C:\arcgisserver\logs\

Now, on both machines, I am seeing .log files being written to-added to the C:\ root 

Why is this happening?  What can I do?  There is nothing in this help step that says this will happen:

https://enterprise.arcgis.com/en/server/10.9.1/administer/windows/disable-arcmap-runtime.htm

@ChristopherCounsell 

@VinceAngelo 

Thanks,

David

[ChrisUnderwood]

Can you tell us something about the names and contents of these files ?

[DavidColey]

Sure @ChrisUnderwood  - they all are named similar to this:

.-20230505.100807-15284-10624.0.log  (date.time - some index number.log)

and all seem to be related to the ImageServer role:

<?xml version="1.0" encoding="utf-8" ?>
<Msg time='2023-05-05T10:21:59,894' type='WARNING' code='80020' target='(null).(null)' methodName='ImageServerIP' machine='FullyQualifiedMachineName' process='16012' thread='7656' requestId='4ddde4e9-dc7a-4921-96ee-ece22a381a59'>Calling SOAP on REST service</Msg>
<Msg time='2023-05-05T10:22:00,573' type='WARNING' code='80020' target='(null).(null)' methodName='ImageServerIP' machine='FullyQualifiedMachineName' process='16012' thread='7656' requestId='4ddde4e9-dc7a-4921-96ee-ece22a381a59'>Calling SOAP on REST service</Msg>

[DavidColey]

If I stop the ArcGIS Server service I can read and delete the files, but otherwise they are locked and access is denied. 

[JalesM]

That is interesting. I wonder if it's related to a specific service. Is your Image Server's log setting set to "WARNING" level? Can you set it to "DEBUG" and see if there are calls to a specific service before or after the errors you see above?

[DavidColey]

Thanks for the reply @JalesM  - it's not the log message itself, that is fine.  I get error messages all the time about the image services from this federated server site - request size exceeds, stuff like that - usually related to users in Pro or Map trying to make large plots with the image services.

It's that after I removed the ArcMap runtime, these log files are being written to both the C:\ root directory and to the defined C:\arcgisserver\logs\ directory on each server.  

I can tell you that the log file name I listed above, now found at the C:\ root:

.-20230505.100807-15284-10624.0.log  

is a truncated name of a log file like this:

SC2022.ImageServer-20230505.102653-11952-4324.0.log

that is found in the proper directory in C:\arcgisserver\logs\ at:

C:\arcgisserver\logs\FullyQualifiedMachineName\services\ImageServices\SC2022.ImageServer

That did not happen until the ArcMap runtime was removed.  So again, it's the location at the C:\ root of a name-truncated log name....

@ChrisUnderwood 

[DavidColey]

I don't know - I'm going to try a full server reboot at the lunch hour, see if that helps

[JalesM]

A reboot isn't a bad idea. My thought process was that maybe there was something corrupt with a particular service which caused the log files to write to the wrong location.

You could also try checking the arcgis-logsetting.json file and see if it lists the correct path as well:

• "C:\Program Files\ArcGIS\Server\framework\etc\arcgis-logsettings.json"

[DavidColey]

Yeah unfortunately no help with the reboots.  The json reads what I have set in my server admin - i.e. 

{
"logDir": "C:\\arcgisserver\\logs\\",
"logLevel": "WARNING",

.....}

Thanks for that.... It certainly is possible something corrupted with an image service (or other map service) at the moment I removed the runtime but there is no indication of that in any other way - they've all been authored in Pro long ago. 

Like I said above, when I did remove the runtime I found 30 - 40 thousand name-truncated logfiles dumped out to the root.  So, you know?

I can tell you I would have NEVER removed the runtime early (early meaning I wanted to remove the runtime before I run the upgrade from 10.9.1 to 11.1on this site) if I knew that that this would happen - - No one would.   Yet another unexpected behavior.

I'm probably going to have to set up a net stop "ArcGIS Server" and a net start as bat files to stop and start the arcgis server windows service until I can upgrade in 12 days.

@ChrisUnderwood 

[JalesM]

Darn, that's a shame. Do you have snapshot backups that you can revert to? What about webgisdr backups?

You could try taking a backup from the ArcGIS Server Admin endpoint (exportSite) and import it right back in (importSite). I've seen that fix weird issues w/ Portal. 

You'll want to make sure you take a snapshot backup prior to importing, just in case.