topic Re: Using subprocess to call another script but get an error in Python Questions

Using subprocess to call another script but get an error

ClaireParsons — Tue, 27 Sep 2011 10:28:17 GMT

OK - I'm basically trying to batch process a number of shapefiles in one directory, first clipping them, then multiple bufferring.
I thought I would use the subprocess module to call a second script to do the buffering; however, i get the following error;

WindowsError: [Error 193] %1 is not a valid Win32 application

I've posted my code below;
#This is the initial script that calls the second
import arcpy
from arcpy import env
import subprocess

# Local variables:
env.workspace = "C:\\Users\\Claire Parsons\\Documents\\Projects\\site_Selection_Testing\\Clip"
fcs = arcpy.ListFeatureClasses()
clipout = "_clip.shp"
clipFeature = "C:\\Users\\Claire Parsons\\Documents\\Projects\\site_Selection_Testing\\Clip\\East_Mids.shp"

for fc in fcs:
    dsc = arcpy.Describe(fc)

    arcpy.Clip_analysis(dsc.catalogpath, clipFeature, dsc.catalogpath.rstrip(".shp") + clipout,"")


    subprocess.Popen([r"C:\Users\Claire Parsons\Documents\Projects\Python\Multiple_buffer.py"])

Re: Using subprocess to call another script but get an error

AndrewChapkowski — Sat, 11 Dec 2021 23:22:10 GMT

It looks like you are missing some information:

 p = subprocess.Popen([pythonExe,# path to python exe
                              workerScript, # path to script
                             scriptVar1,... #variables as strings
                              ],\
                               cwd=os.path.dirname(scratchWrksp),\
                               shell=False,\
                               stdout=subprocess.PIPE,\
                               stderr=subprocess.PIPE)

Try adding that information and see what happens.

Here is the help on subprocessing: http://docs.python.org/release/2.6/library/subprocess.html

Re: Using subprocess to call another script but get an error

ClaireParsons — Tue, 27 Sep 2011 14:29:12 GMT

Hi Andrew

Thanks for you're reply but I still can't get this to run - could you post the subprocess line with exactly how you would use it.

Thanks
Claire

Re: Using subprocess to call another script but get an error

AndrewChapkowski — Sat, 11 Dec 2021 23:22:13 GMT

You can write all your functions in a simple python script to do what your doing.

But if you want to use subprocessing, here is how you can use it:

def spawn_function(pythonExe, # exe of python 
                  workerScript, # script that does the work (path to .py file)
                  inFC,# source feature class 1st variable in script
                  scratchWrksp, # scratch workspace 2nd variable in script, etc...
                  count # Thread ID number defined by user
                  ):
        p = subprocess.Popen([pythonExe,
                              workerScript,
                              str(inFC),
                              str(scratchWrksp),
                              str(count)
                              ],\
                               cwd=os.path.dirname(scratchWrksp),\
                               shell=False,\
                               stdout=subprocess.PIPE,\
                               stderr=subprocess.PIPE)
        return p

Now you have a function that will return an instance of your subprocess. In the __main__ you call the subprocessing in a loop of some other fashion based on some requirement like number of CPUs.


if __name__ == "__main__":
   pList = []
   processCount = (int(os.environ["NUMBER_OF_PROCESSORS"]) - 1) # to account for main thread
   pythonExe = os.path.join(sys.exec_prefix,'python.exe')
   for r in range(0,processCount):
      pList.append(spawn_function(ALL THE VARIABLES))
    # then you have to check if the process completed

I didn't test the code, but it will give you a general idea on how this works. You can also look at multiprocessing as well.

Enjoy

Re: Using subprocess to call another script but get an error

StacyRendall1 — Sat, 11 Dec 2021 23:22:16 GMT

Do you need to use subprocess? Perhaps you should look into writing Python modules, which you can import and then run. They seem a bit more robust than Subprocess (I think subprocess is actually designed for running other programs from Python, not other Pythons), and have the advantage that you can easily pass arguments (paths, buffer sizes, etc.) to them. I have included a basic description of what you would need to do for that, but let me know if you need more info.

In Multiple_buffer.py you need to indent all the code and in the top line have a def statement, like so:

#Multiple_buffer.py
def doBuffer():# empty brackets as no arguments are passed
    ###
    # whatever your buffer code was all goes here - all indented...

Then your main script could import it (both .py files must be in the same folder):

import arcpy
from arcpy import env
#import subprocess # no longer required
import Multiple_buffer

# Local variables:
env.workspace = "C:\\Users\\Claire Parsons\\Documents\\Projects\\site_Selection_Testing\\Clip"
fcs = arcpy.ListFeatureClasses()
clipout = "_clip.shp"
clipFeature = "C:\\Users\\Claire Parsons\\Documents\\Projects\\site_Selection_Testing\\Clip\\East_Mids.shp"

for fc in fcs:
dsc = arcpy.Describe(fc)

arcpy.Clip_analysis(dsc.catalogpath, clipFeature, dsc.catalogpath.rstrip(".shp") + clipout,"")

Multiple_buffer.doBuffer() # calling the doBuffer module in Multiple_buffer - this will run the contents of the def statement (basically your Multiple_buffer script)

Let me know how you get on...