How do I call a Python 3.5 script from a Python 2.7 script?

7114
10
02-14-2017 03:27 PM
AndyWright
Occasional Contributor

I've got a situation where I have a data preparation script that I've been using for years that was written in Python 2.7.  We now want to add some functionality to that script that is only available in version 3.5 that comes along for the ride with ArcGIS Pro 1.4.1.  So I am doing something like this inside the 2.7 script:

 p = subprocess.Popen("D:/Program Files/ArcGIS/Pro/bin/Python/envs/arcgispro-py3/python.exe ../ArcGISPro/GenerateVectorTpks.py -rt all -vt electric -oc " + operCo, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)

The 3.5 script runs great on its own when it knows it's using the 3.5 interpreter, but it seems like some wires are getting crossed trying to run stuff across both versions.  I am getting this error:

from variables import File "../Common\\variables.py", line 21, in <module>

   import arcpy  File "d:\\program files (x86)\\arcgis\\desktop10.5\\ArcPy\\arcpy\\__init__.py", line 22, in <module>

   from arcpy.geoprocessing import gp  

   File "d:\\program files (x86)\\arcgis\\desktop10.5\\ArcPy\\arcpy\\geoprocessing\\__init__.py", line 14, in <module>

   from _base import *\ImportError: No module named _base

Can someone point me in the right direction on this?  Thanks a ton ...

0 Kudos
10 Replies
DanPatterson_Retired
MVP Esteemed Contributor
ArcGISPro/GenerateVectorTpks.py 

Can you show the contents of the script you are trying to change? since pro and map's python distributions are in different locations as is their arcpy's

0 Kudos
Luke_Pinner
MVP Regular Contributor

You're calling a conda python without activating the conda environment.

Use "D:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat"  instead of 'etc.../envs/arcgispro-py3/python.exe'.

More info:Python, conda, and ArcGIS Pro—ArcPy Get Started | ArcGIS Desktop 

AndyWright
Occasional Contributor

Hi Luke,

Thanks for the tip.  That got me a bit farther along.  Now it appears to call the ArcGIS Pro based script, but doesn't.  It does not bomb out with an error either.  I can take the same command I am using in the subprocess.Popen call and run it in a command line with success.  I did some googling and I think it's related to spaces being in the various components of the Popen call.  I've tried everything I've seen suggested out there and the script never gets called successfully.  This is where I'm at right now.  Any other suggestions?

cmd = ["D:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat", "../ArcGISPro/GenerateVectorTpks.py -rt all -vt electric -oc " + operCo]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)

output, error = p.communicate()

0 Kudos
Luke_Pinner
MVP Regular Contributor

You're passing the arguments to Popen incorrectly, you need to pass each argument as a list element not a single string.

Replace

cmd = ["D:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat", "../ArcGISPro/GenerateVectorTpks.py -rt all -vt electric -oc " + operCo]

With

cmd = ["D:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat", "../ArcGISPro/GenerateVectorTpks.py", "-rt", "all", "-vt", "electric", "-oc", operCo]

For example:

#test27.py
import sys, subprocess

operCo = "blahblah"
cmd = ["C:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy.bat", "C:/Temp/testpro.py", "-rt", "all", "-vt", "electric", "-oc", operCo]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)

output, error = p.communicate()

print('*'*10)
print(sys.version)
print(' '.join(sys.argv))
print('*'*10)
print(output)


#testpro.py
import sys
print(sys.version)
print(*sys.argv)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Outputs:

**********
2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (Intel)]
C:/Temp/test27.py
**********
3.5.2 |Continuum Analytics, Inc.| (default, Jul  5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)]
C:/Temp/testpro.py -rt all -vt electric -oc blahblah
AndyWright
Occasional Contributor

I tried that and every other combination of stuff you could think of and nothing has worked for me other than this batch file methodology.  Based on what you provided it looks like it is doing the right thing for you.  These scripts I'm trying to link together are part of a much bigger Visual Studio python project so it's nowhere near as simplistic as your example.  Maybe there are some import statements somewhere in the chain that are causing things to not work properly.  At this point I'm going to keep using the batch file methodology, but I think I will try to set up a simple example like this and see if it works for me.

Thanks for your responses on this Luke ...

0 Kudos
Luke_Pinner
MVP Regular Contributor

Have you tried passing the full path to "../ArcGISPro/GenerateVectorTpks.py"?

My example above is simplistic, but I use this method a lot for some pretty complex stuff.  Mainly loading non-arcgis conda environments to do some heavy lifting with specific native libraries that don't play nicely with ArcGIS Desktop (we're stuck at 10.2 for the foreseeable future), but running from 10.2 ArcToolbox script tools/python toolboxes.

0 Kudos
AndyWright
Occasional Contributor

Yeh I did try that, and like I said that exact command works great from the command prompt.  Something is getting swallowed because it does not error out, it just seems to skip right over the subprocess.Popen command.  I must have some object somewhere in the pipe that it doesn't like.  I'll have to dive into this a little deeper when I get the chance.  Thanks again for your help Luke ...

0 Kudos
ScottDavis
Occasional Contributor

Luke: Are you able to import arcpy from your v3 script? That's where I have the issues.

0 Kudos
ScottDavis
Occasional Contributor

I think that I found the solution for me: pass `-E` (clears PYTHONPATH) to the python invocation command. Something like this:

command = ['propy', '-E', vector_module, basemap, summary, tags]
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)

Ref: 1. Command line and environment — Python 2.7.14 documentation 

0 Kudos