topic Re: Python: how can I run a system command as administrator and capture the command output? in Python Questions

Python: how can I run a system command as administrator and capture the command output? Not answered, but provided summary.

RebeccaStrauch__GISP — Fri, 21 Oct 2016 02:07:30 GMT

Just to summarize what I need right away....

Bottom line: I need help running the "wmic" OS command or the "psutil.Process(<number>).cmdline()" with administrator credentials, in my script, capturing the output to a variable.

In what seems like an endless pursuit of discovering and dealing with .lock file preventing replacement of a full gdb, I've now narrowed it down to trying to discover the .lock files that are being created by an ArcGIS for Server‌ gp service. My script gives me a list of these .lock files named in this manner

Servicename.SEERVERNAME.16716.11140.sr.lock

parcing this, I'm able to get the PID (process ID) 11256, in this case. With tech support help, using the Windows Task Manager "Processes" tab, and turning on the PID and "Command Line" columns (thru View->Select Columns), I can see the service for the PID.

I have tested a number of commands to try to capture this "Command Line" information so I can parse out the service I need to stop. Some of these commands include "net file", "tasklist", "wcim", "psutil" (a python mod), fsmgmt.msc, and compmgmt.msc.

The most promising of the OS commands is the wcim, but only if I can run it "As Administrator". Testing it manually, in a cmd.exe window started as administrator, I get the fun "Command line" info. Running in a non-admin cmd.exe window does not return the info (i.e. blank)

Using psutil module also very promising, but returns an AccessDenied error when trying to return the cmdline info. So I think that would work if I could pass-it or run-it with admin credentials.

what it should return is a long string that includes "-Dservice=servicefolder.servicename.MapService" (in this test case). I want to grab the value for -Dservice so I can stop it, therefore releasing the .lock files.

I'm been researching and trying subprocess (mainly...a few other found suggestions too), and have had no luck. My script will have variables with valid user pass, but I do not know how to get this command to run properly.

My preference would be getting the info using the psutil.cmdline() for the PID. But would also be happy if I could get

wmic process where "ProcessID = 11140" get CommandLine

with whichever saving the return string to a variable (then I can parse it myself).

Bottom line: I need help running the "wmic" OS command or the "psutil.Process(<number>).cmdline()" with administrator credentials, in my script, capturing the output to a variable.

Thanks to all you python gurus that want to take a shot at this. If you have other suggestions for capturing the AGS service information base just on the .lock filename, within my script, I'm open to suggestions. I've already tagged the AGS group (near top), so maybe some ArcGIS Server administrators have a trick on where I can grab this info. (??)

BTW - Once I get my script running with all my .lock tests, I hope to summarize in a blog post or document. not there yet.

Re: Python: how can I run a system command as administrator and capture the command output?

DanPatterson_Retired — Fri, 21 Oct 2016 02:42:53 GMT

Don't have it, don't use it, but I always recommending checking out versions and version history

psutil 1.2.1 : Python Package Index

https://pythonhosted.org/psutil/ and 'issues' usually can be found on github too

psutil/INSTALL.rst at master · giampaolo/psutil · GitHub

good luck

Re: Python: how can I run a system command as administrator and capture the command output?

JonathanQuinn — Sun, 12 Dec 2021 05:37:41 GMT

I assume you don't want to stop the GP service to clear the lock? Are you able to delete the lock manually? If not, then I doubt a script would be able to either. I see your other posts, seems like you're just finding that information to know which services to stop.

In Googling, though, I came across this answer:

List all processes and their Command-line parameters

WMIC PROCESS where 'caption="ArcSOC.exe"' get Caption,Commandline,Processid > C:\arcsocprocesses.txt

returns all of them and

WMIC PROCESS where 'ProcessID=39280' get Caption,Commandline,Processid > C:\pid39280.txt

returns the process with ID 39280. I use subprocess for most Windows commands in Python, and by adding this in and parsing the results, you can pick out the service name:

import subprocess
result = subprocess.check_output("WMIC PROCESS where 'ProcessID=381960' get Caption,Commandline,Processid")
for line in result.split("\n"):
 if "-Dservice=" in line:
 serviceName = line.split("-Dservice=")[1].split(" \"")[0]
 print(serviceName)‍‍‍‍‍‍‍‍‍‍‍‍

Re: Python: how can I run a system command as administrator and capture the command output?

Luke_Pinner — Fri, 21 Oct 2016 21:13:43 GMT

Easiest way is to just run the entire python script with Admin privileges.

Re: Python: how can I run a system command as administrator and capture the command output?

RebeccaStrauch__GISP — Wed, 23 Nov 2016 02:48:20 GMT

Sorry, I was out with the flu for a while, and this got lost in my emails. Just looking back at this now.

I like where you where heading with this Jonathan Quinn , but unfortunately, it still drops the critical infomartion from the WMIC if I am not running it as an administrator. I'm not sure how well this will look, but running is in the command window....

first showing the Run-As-Administrator window, ..... I've removed a bunch of paths to .jar files to clean it up a little (otherwise very long).... Trying to sort it out, I've added colors

the headers most of the values and the part I need

C:\Windows\system32>wmic PROCESS where 'ProcessID=892'
Caption     CommandLine
      CreationClassName CreationDate               CSCreationClassName   CSName
         Description ExecutablePath                                 ExecutionSt
ate Handle HandleCount InstallDate KernelModeTime MaximumWorkingSetSize Mi
nimumWorkingSetSize Name        OSCreationClassName    OSName
                                                              OtherOperationCoun
t OtherTransferCount PageFaults PageFileUsage ParentProcessId PeakPageFileU
sage PeakVirtualSize PeakWorkingSetSize Priority PrivatePageCount ProcessId
QuotaNonPagedPoolUsage QuotaPagedPoolUsage QuotaPeakNonPagedPoolUsage Quota
PeakPagedPoolUsage ReadOperationCount ReadTransferCount SessionId Status Te
rminationDate ThreadCount UserModeTime VirtualSize WindowsVersion WorkingSe
tSize WriteOperationCount WriteTransferCount
ArcSOC.exe "C:\Program Files\ArcGIS\Server\bin\ArcSOC.exe" -XX:-CreateMinidumpOnCrash -Xmx64M -Dservice=wc_public.GMUSubunits.MapServer "-Djava.class.path=C:\Program Files\ArcGIS\Server\framework\lib\server\arcgis-admin.jar; .......bunch of stuff here" -Djava.security.policy=file:/C:/Program%20Files/ArcGIS/Server/framework/etc/arcsoc.policy "-Djava.rmi.server.codebase=file:/C:/Program%20Files/ArcGIS/
Server/framework/lib/server/arcgis-servicelib.jar file:/C:/Program%20Files/ArcGIS/Server/framework/lib/server/arcgis-common.jar " sun.rmi.server.ActivationGroupInit Win32_Process      20161122000020.890422-540 Win32_ComputerSystem DFGANCGISDEV3 ArcSOC.exe   C:\Program Files\ArcGIS\Server\bin\ArcSOC.exe
     892     962                       28750000        1380                   200                    ArcSOC.exe Win32_OperatingSystem Microsoft Windows Server 2008 R2 Enterprise |C:\Windows|\Device\Harddisk0\Partition2 115308   ....bunch of numbers at the end...

then as a regular command window...

H:\>wmic PROCESS where 'ProcessID=892'
Caption CommandLine CreationClassName CreationDate CSCreatio
nClassName CSName Description ExecutablePath ExecutionState Handle
HandleCount InstallDate KernelModeTime MaximumWorkingSetSize MinimumWorking
SetSize Name OSCreationClassName OSName
OtherOperationCount OtherTran
sferCount PageFaults PageFileUsage ParentProcessId PeakPageFileUsage PeakVi
rtualSize PeakWorkingSetSize Priority PrivatePageCount ProcessId QuotaNonPa
gedPoolUsage QuotaPagedPoolUsage QuotaPeakNonPagedPoolUsage QuotaPeakPagedPoo
lUsage ReadOperationCount ReadTransferCount SessionId Status TerminationDat
e ThreadCount UserModeTime VirtualSize WindowsVersion WorkingSetSize Write
OperationCount WriteTransferCount
ArcSOC.exe Win32_Process 20161122000020.890422-540 Win32_Com
puterSystem DFGANCGISDEV3 ArcSOC.exe 892
962 28750000
ArcSOC.exe Win32_OperatingSystem Microsoft Windows Server 2008 R2 Ent
erprise |C:\Windows|\Device\Harddisk0\Partition2 115309 3747715
56240 186456 1500 194816 615538
688 141268 8 190930944 892 72
530 82 531
13273 45997341 0
31 88750000 608038912 6.1.7601 140316672 5
5017

H:\>

As you should be able to see, the information I need is not give at all.

If I look at creating the result in the way you show, as you can see, the "result" is basically empty

>>>import psutil
>>result = subprocess.check_output("WMIC PROCESS where 'ProcessID=892' get Caption,Commandline,Processid")
>>>print(result)
Caption     CommandLine ProcessId
ArcSOC.exe               892

>>>result
'Caption     CommandLine ProcessId \r\r\nArcSOC.exe               892        \r\r\n\r\r\n'

(blank lines left there on purpose).

So, I'm still at the problem that I need to run the subprocess as "RunAs Administrator", and I can't figure out how to do that in a script.

I'll leave this thread open until next week in case anyone has a solution, then I think I'm going to mark it as "Assumed answered" .....even though it may be "no answer".

Re: Python: how can I run a system command as administrator and capture the command output?

ClintonDow1 — Sun, 12 Dec 2021 05:37:44 GMT

The win32 module has a shell.ShellExecute method which can accept the 'runas' verb. You can use a snippet like below to relaunch the current script as admin:

import os
import sys
import win32com.shell.shell as shell
ASADMIN = 'asadmin'

if sys.argv[-1] != ASADMIN:
    script = os.path.abspath(sys.argv[0])
    params = ' '.join([script] + sys.argv[1:] + [ASADMIN])
    shell.ShellExecuteEx(lpVerb='runas', lpFile=sys.executable, lpParameters=params)
    sys.exit(0)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

edit: Just wanted to add that this will prompt for UAC at this point so unless you have set your system to automatically accept UAC requests, there will need to be someone present to click accept. Also, since you're launching the process in a second shell, you won't get the StdOut piped to your current shell - so if you need to see the output, write it to a file instead of relying on print statements.

Re: Python: how can I run a system command as administrator and capture the command output?

RebeccaStrauch__GISP — Wed, 23 Nov 2016 21:41:42 GMT

Thanks Clinton. It took me a while to figure out how to download and install the win32com package, but I have that now.

I've been reading thru documents trying to figure out the rest of what you have in the sample, but not getting very far. IF I run your sample in a python window (I use WingPro), I'm getting an error...

Traceback (most recent call last):
Python Shell, prompt 17, line 5
SystemExit: 0

but I admit, I'm not sure what your sample is trying to do, or how I would insert my command.

theCommand = "wmic process where \"ProcessID = 4132\" get CommandLine"

If I could get the information from this that I ge only if run "asadmin" and then I could try to parse out Dservice. I'm not really wanting to run the rest of the tool "as admin", or at least not wanting to have the person have to be prompted for a password (for the OS admin rights)

My mind is starting to get fuzzy on how I was going to use this in my script anyway. The rabbit hole I was starting to go down was trying to do is find every which way the fgdb could be locked, and then trying to resolve it within the tool, but I'm beginning to think that something may have to be manual checks. The reason for finding locks, is if you try to rename or delete a FGDB and their are lock on any of it, i will be corrupted (i.e.enough files may be deleted so it's just a folder instead of a fgdb....and I do have a lot of try...except... blocks).

Re: Python: how can I run a system command as administrator and capture the command output?

ClintonDow1 — Mon, 28 Nov 2016 17:17:38 GMT

Hi Rebecca,

Win32 is a bit of a frustrating install with multiple Python versions installed, glad you were able to get it. I'll admit this is an old code snippet and I haven't used it for anything complex so it may be of limited usefulness in your situation.

Essentially all that snippet is doing is re-launching a python script with the UAC dialog, if it doesn't have admin rights on the first attempt. I misunderstood that it was only part of your script to run as admin and not the whole thing. There's no easy solution to launching an admin-level sub-process from a user-level program without prompting for UAC - even using a Microsoft language like C#.

My best advice in that situation is to see if you can break the workflow up into multiple tools and then chain them together in a .bat file perhaps?

Re: Python: how can I run a system command as administrator and capture the command output?

RebeccaStrauch__GISP — Mon, 28 Nov 2016 17:38:21 GMT

Thanks for the follow-up Clinton. My attempt was to make this tool cover, in theory, all possible tests available (via ArcGIS and OS options) for locked FGDB files, but at this point I think these last couple tests are not worth the effort. I may do as you mentioned and maybe make them separate tools, but better yet, I just may need to put the info in a readme file for possible solutions if the tool crashes. I've been the only one running this process for years, and just trying to make it more foolproof for others. But with newer network security restrictions, we have to learn to live with the newer, tighter rules I guess.

So, in summary....this is not possible in the way I had hoped. Work around would be as Luke and Clinton pointed out, although neither is 100% practical in my case. Best I will be able to do is attempt to use try..except to exit out of the tool before it corrupts anything, then have a separate tool or instructions (to do it manually) to get past the last hurdle. Of course, simplest and most reliable fix in this case would be to just shut down ArcGIS Server and/or reboot...but, the original thought was to only deal with the services that need to be stopped (and not touch those that don't).

Thanks all for your suggestions.

Re: Python: how can I run a system command as administrator and capture the command output? Not answered, but provided summary.

JonathanQuinn — Tue, 06 Dec 2016 19:42:16 GMT

Hey Rebecca, so I was playing around and found that by creating a shortcut to a .bat file, setting the shortcut to run as an admin, I can call on the shortcut in a command prompt that isn't launched as an administrator and it opens a new command prompt as an admin to run the script. In the screenshots below, the first command in the first command prompt isn't run as an admin, and I can't return the name of the service. The second command I call on the shortcut, which launches a new command prompt as an admin and runs the script and prints the service name. Not sure if you're already beyond this now or if this will work for you, but I figured I'd pass it along.