topic Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C in ArcGIS Enterprise Questions

Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Fri, 05 May 2023 20:17:13 GMT

Hello - in prep for upgrade of our federated server site to 11.1 from 10.9.1, I removed the ArcMap based runtime after finally being able to remove my remaining ArcMap authored services. When I did this, like 30K + log files got dumped to the C:\ root.

I deleted the log files, restarted arcgis server service, went into server admin, re-edited the log output location (even though this did not change) to C:\arcgisserver\logs\

Now, on both machines, I am seeing .log files being written to-added to the C:\ root

Why is this happening? What can I do? There is nothing in this help step that says this will happen:

https://enterprise.arcgis.com/en/server/10.9.1/administer/windows/disable-arcmap-runtime.htm

@ChristopherCounsell

@VinceAngelo

Thanks,

David

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

ChrisUnderwood — Fri, 05 May 2023 13:59:08 GMT

Can you tell us something about the names and contents of these files ?

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Fri, 05 May 2023 14:28:54 GMT

Sure @ChrisUnderwood - they all are named similar to this:

.-20230505.100807-15284-10624.0.log (date.time - some index number.log)

and all seem to be related to the ImageServer role:

<?xml version="1.0" encoding="utf-8" ?>
<Msg time='2023-05-05T10:21:59,894' type='WARNING' code='80020' target='(null).(null)' methodName='ImageServerIP' machine='FullyQualifiedMachineName' process='16012' thread='7656' requestId='4ddde4e9-dc7a-4921-96ee-ece22a381a59'>Calling SOAP on REST service</Msg>
<Msg time='2023-05-05T10:22:00,573' type='WARNING' code='80020' target='(null).(null)' methodName='ImageServerIP' machine='FullyQualifiedMachineName' process='16012' thread='7656' requestId='4ddde4e9-dc7a-4921-96ee-ece22a381a59'>Calling SOAP on REST service</Msg>

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Fri, 05 May 2023 14:31:55 GMT

If I stop the ArcGIS Server service I can read and delete the files, but otherwise they are locked and access is denied.

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

JalesM — Fri, 05 May 2023 15:21:54 GMT

That is interesting. I wonder if it's related to a specific service. Is your Image Server's log setting set to "WARNING" level? Can you set it to "DEBUG" and see if there are calls to a specific service before or after the errors you see above?

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Fri, 05 May 2023 15:42:04 GMT

Thanks for the reply @JalesM - it's not the log message itself, that is fine. I get error messages all the time about the image services from this federated server site - request size exceeds, stuff like that - usually related to users in Pro or Map trying to make large plots with the image services.

It's that after I removed the ArcMap runtime, these log files are being written to both the C:\ root directory and to the defined C:\arcgisserver\logs\ directory on each server.

I can tell you that the log file name I listed above, now found at the C:\ root:

.-20230505.100807-15284-10624.0.log

is a truncated name of a log file like this:

SC2022.ImageServer-20230505.102653-11952-4324.0.log

that is found in the proper directory in C:\arcgisserver\logs\ at:

C:\arcgisserver\logs\FullyQualifiedMachineName\services\ImageServices\SC2022.ImageServer

That did not happen until the ArcMap runtime was removed. So again, it's the location at the C:\ root of a name-truncated log name....

@ChrisUnderwood

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Fri, 05 May 2023 15:51:17 GMT

I don't know - I'm going to try a full server reboot at the lunch hour, see if that helps

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

JalesM — Fri, 05 May 2023 17:32:09 GMT

A reboot isn't a bad idea. My thought process was that maybe there was something corrupt with a particular service which caused the log files to write to the wrong location.

You could also try checking the arcgis-logsetting.json file and see if it lists the correct path as well:

"C:\Program Files\ArcGIS\Server\framework\etc\arcgis-logsettings.json"

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Fri, 05 May 2023 18:03:03 GMT

Yeah unfortunately no help with the reboots. The json reads what I have set in my server admin - i.e.

{
"logDir": "C:\\arcgisserver\\logs\\",
"logLevel": "WARNING",

.....}

Thanks for that.... It certainly is possible something corrupted with an image service (or other map service) at the moment I removed the runtime but there is no indication of that in any other way - they've all been authored in Pro long ago.

Like I said above, when I did remove the runtime I found 30 - 40 thousand name-truncated logfiles dumped out to the root. So, you know?

I can tell you I would have NEVER removed the runtime early (early meaning I wanted to remove the runtime before I run the upgrade from 10.9.1 to 11.1on this site) if I knew that that this would happen - - No one would. Yet another unexpected behavior.

I'm probably going to have to set up a net stop "ArcGIS Server" and a net start as bat files to stop and start the arcgis server windows service until I can upgrade in 12 days.

@ChrisUnderwood

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

JalesM — Fri, 05 May 2023 21:45:12 GMT

Darn, that's a shame. Do you have snapshot backups that you can revert to? What about webgisdr backups?

You could try taking a backup from the ArcGIS Server Admin endpoint (exportSite) and import it right back in (importSite). I've seen that fix weird issues w/ Portal.

You'll want to make sure you take a snapshot backup prior to importing, just in case.

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Sun, 07 May 2023 15:20:29 GMT

Thanks for the suggestions @JalesM , but at this point I will just manage the logs at the root location as I'm updating to 11.1. If this site were to remain at 10.9.1 I'd go through restore attempts, but everything is working fine, there are no other issues. Monitor shows the 2 federated site machines remain balanced in soc count, memory use, cpu etc.

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

JalesM — Mon, 08 May 2023 16:39:09 GMT

Sounds good @DavidColey. Yeah, if it's not causing major issues, probably best to leave it as is for now. However, I would recommend opening a support ticket prior to upgrading. I've heard it said that we shouldn't perform upgrades to fix an issue unless it's a documented bug. It's a 50/50 chance that the issue will compound or fix itself.

All the best!

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Mon, 08 May 2023 18:27:15 GMT

Yes thanks @JalesM Yeah I will not open a support ticket for this. We're upgrading because we have to move forward with the UN at version 6, and for documented bugs that 11.1 fixes... Still this federated site is a mature install, it's supported Server and ImageServer since 10.6. So this will mark it's 9th overall upgrade - but if this issue persists at 11.1, then I'll open a support ticket.

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

ChristopherPawlyszyn — Tue, 09 May 2023 00:06:26 GMT

@DavidColey there's a defect logged for this behavior, BUG-000152126. I see that it was reproducible in 11.0 via one attached case but I'm unsure whether it persisted at 11.1. I'll try to get in touch with the ArcGIS Server framework team and see if we can prioritize this fix if so.

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Tue, 09 May 2023 01:56:36 GMT

Interesting @ChristopherPawlyszyn , thank you for the feedback. This may not be relevant, but I have been working in a mixed deploy since the 11.0 release.

Our Portal, hosting site, and datastores are at 11.0, but we had to leave the federated site at 10.9.1 as we had to support a couple of services with Geometric networks and so had to have the ArcMap runtime.

With the upgrade to UN 6 we cam finally move everything forward to 11.1

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Mon, 22 May 2023 18:19:11 GMT

Hi @ChristopherPawlyszyn - I can report that our migration to 11.1 has so far been successful, I have not encountered (thus far) any issues with web adaptor(s), IIS, portal, hosting site, datastore(s), federated site.

Unfortunately, reported BUG-000152126 " truncated *.log files being written to/added to the C:\ root " does persist at 11.1 for Federated sites with an Image Server role.....

Re: Removal of ArcMap based runtime from ArcGIS Server now leaves .log files at root of C

DavidColey — Tue, 23 May 2023 14:18:11 GMT

Ok @ChristopherPawlyszyn - I got myself attached to this bug / defect through support....