I've got a situation where I have a data preparation script that I've been using for years that was written in Python 2.7. We now want to add some functionality to that script that is only available in version 3.5 that comes along for the ride with ArcGIS Pro 1.4.1. So I am doing something like this inside the 2.7 script:
p = subprocess.Popen("D:/Program Files/ArcGIS/Pro/bin/Python/envs/arcgispro-py3/python.exe ../ArcGISPro/GenerateVectorTpks.py -rt all -vt electric -oc " + operCo, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
The 3.5 script runs great on its own when it knows it's using the 3.5 interpreter, but it seems like some wires are getting crossed trying to run stuff across both versions. I am getting this error:
from variables import File "../Common\\variables.py", line 21, in <module>
import arcpy File "d:\\program files (x86)\\arcgis\\desktop10.5\\ArcPy\\arcpy\\__init__.py", line 22, in <module>
from arcpy.geoprocessing import gp
File "d:\\program files (x86)\\arcgis\\desktop10.5\\ArcPy\\arcpy\\geoprocessing\\__init__.py", line 14, in <module>
from _base import *\ImportError: No module named _base
Can someone point me in the right direction on this? Thanks a ton ...
ArcGISPro/GenerateVectorTpks.py
Can you show the contents of the script you are trying to change? since pro and map's python distributions are in different locations as is their arcpy's
You're calling a conda python without activating the conda environment.
Use "D:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat" instead of 'etc.../envs/arcgispro-py3/python.exe'.
More info:Python, conda, and ArcGIS Pro—ArcPy Get Started | ArcGIS Desktop
Hi Luke,
Thanks for the tip. That got me a bit farther along. Now it appears to call the ArcGIS Pro based script, but doesn't. It does not bomb out with an error either. I can take the same command I am using in the subprocess.Popen call and run it in a command line with success. I did some googling and I think it's related to spaces being in the various components of the Popen call. I've tried everything I've seen suggested out there and the script never gets called successfully. This is where I'm at right now. Any other suggestions?
cmd = ["D:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat", "../ArcGISPro/GenerateVectorTpks.py -rt all -vt electric -oc " + operCo]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
output, error = p.communicate()
You're passing the arguments to Popen incorrectly, you need to pass each argument as a list element not a single string.
Replace
cmd = ["D:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat", "../ArcGISPro/GenerateVectorTpks.py -rt all -vt electric -oc " + operCo]
With
cmd = ["D:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat", "../ArcGISPro/GenerateVectorTpks.py", "-rt", "all", "-vt", "electric", "-oc", operCo]
For example:
#test27.py
import sys, subprocess
operCo = "blahblah"
cmd = ["C:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat", "C:/Temp/testpro.py", "-rt", "all", "-vt", "electric", "-oc", operCo]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
output, error = p.communicate()
print('*'*10)
print(sys.version)
print(' '.join(sys.argv))
print('*'*10)
print(output)
#testpro.py
import sys
print(sys.version)
print(*sys.argv)
Outputs:
**********
2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (Intel)]
C:/Temp/test27.py
**********
3.5.2 |Continuum Analytics, Inc.| (default, Jul 5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)]
C:/Temp/testpro.py -rt all -vt electric -oc blahblah
I tried that and every other combination of stuff you could think of and nothing has worked for me other than this batch file methodology. Based on what you provided it looks like it is doing the right thing for you. These scripts I'm trying to link together are part of a much bigger Visual Studio python project so it's nowhere near as simplistic as your example. Maybe there are some import statements somewhere in the chain that are causing things to not work properly. At this point I'm going to keep using the batch file methodology, but I think I will try to set up a simple example like this and see if it works for me.
Thanks for your responses on this Luke ...
Have you tried passing the full path to "../ArcGISPro/GenerateVectorTpks.py"?
My example above is simplistic, but I use this method a lot for some pretty complex stuff. Mainly loading non-arcgis conda environments to do some heavy lifting with specific native libraries that don't play nicely with ArcGIS Desktop (we're stuck at 10.2 for the foreseeable future), but running from 10.2 ArcToolbox script tools/python toolboxes.
Yeh I did try that, and like I said that exact command works great from the command prompt. Something is getting swallowed because it does not error out, it just seems to skip right over the subprocess.Popen command. I must have some object somewhere in the pipe that it doesn't like. I'll have to dive into this a little deeper when I get the chance. Thanks again for your help Luke ...
Luke: Are you able to import arcpy from your v3 script? That's where I have the issues.
I think that I found the solution for me: pass `-E` (clears PYTHONPATH) to the python invocation command. Something like this:
command = ['propy', '-E', vector_module, basemap, summary, tags]
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
Ref: 1. Command line and environment — Python 2.7.14 documentation