csny490

using mutiprocessing module - can't find python.exe anymore?

Discussion created by csny490 on Aug 24, 2011
Latest reply on Aug 25, 2011 by csny490
I finally had a good reason to start writing an arcpy script that uses the multiprocessing module (new in v2.6). Great! I start coding following this example: http://pythongisandstuff.wordpress.com/2011/07/11/using-arcpy-with-multiprocessing-%E2%80%93-part-3/

When I run my script it throws this weird error when it gets to the multiprocessing part:

A new PythonWin window pops up with an error "Could not load the file from C:\Python26\ArcGIS10.0\Lib\site-packages\pythonwin"

Oh ****...

I go to a coworkers machine and try to run my script.... Same error!!!

After a long while I deduce that for whatever reason the mutiprocessing module can't find python.exe anymore. I debug it untill I finally get to the problem: line 179 in the "forking.py" script (C:\Python26\ArcGIS10.0\Lib\multiprocessing\Forking.py). Here is that code:

if sys.executable.lower().endswith('pythonservice.exe'):
   _python_exe = os.path.join(sys.exec_prefix, 'python.exe')
else:
   _python_exe = sys.executable


Problem is that the executable I am using DOES NOT end with "pythonservice.exe" (cause I am using PythonWin.exe!), so it reverts to using whatever the sys.executable variable points to (which is pythonwin.exe). Ah ah!

So, of course when I hardcode the offending lines of code to be:

if sys.executable.lower().endswith('pythonservice.exe'):
   _python_exe = os.path.join(sys.exec_prefix, 'python.exe')
else:
   _python_exe =  os.path.join(sys.exec_prefix, 'python.exe')


Everything works fine!!! AWESOME!

So my questons are:

1. If my coworker and I are having this issue, is anyone else have this issue?
2. Why can't the multiprocessing module find python.exe? Could it be related to arcpy "taking over" the Python install directory???
3. Other than hardcoding the correct path to python.exe in the "forking.py" script, what can I do to make a permanent fix for this?

P.S. Setting the PYTHONPATH system variable doesn't do the trick...

....Also, running ArcGIS v10.0 SP2, Windows7

Any ideas???

Outcomes