Execute Python 2 Code via Python 3

8654
26
Jump to solution
12-26-2019 11:58 AM
WilliamCraft
MVP Regular Contributor

I am trying to wrap some Python 2 code into Python 3 but having trouble getting the process to work correctly.  From ArcGIS Pro, I'm running a custom GP tool with the following code as its script.  My goal is to run my Python 2 code (stored in a separate script) against a series of MXDs in order to list the data source for each layer or table.  In my case, doing it this way yields no results.  The code runs in about 2 seconds and then terminates.  Any ideas?  

PYTHON 3 SCRIPT (WRAPPER)

import subprocess, os, winreg, sys, arcpy

try:
   arcpy.AddMessage("Finding Python 2.7 installation directory...")
   hKey = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE,    "SOFTWARE\\WOW6432Node\\Python\\PythonCore\\2.7\\InstallPath")
   result = winreg.QueryValueEx(hKey, "")[0]
except:
   arcpy.AddWarning("Python 2.7 installation directory was not found.")
   arcpy.AddError("Script failed.")
   sys.exit()

if os.path.exists(result + "\\python.exe"):
   arcpy.AddMessage("Launching Python 2.7 executable...")
   CREATE_NO_WINDOW = 0x8000000
   process = subprocess.Popen([result + "\\python.exe", "C:\\temp\\python\\ListMXDDataSources.py"],    stdout=subprocess.PIPE, stderr=subprocess.PIPE, creationflags = CREATE_NO_WINDOW, shell=True, stdin=None)
   stdout, stderr = process.communicate()
   arcpy.AddMessage('{}'.format(stdout.decode("utf-8")))
   arcpy.AddWarning('{}'.format(stderr.decode("utf-8")))

PYTHON 2 SCRIPT

import arcpy, glob

rootDirectory = 'C:/temp'
fileExtension = '.mxd'

def main():
   for f in glob.glob(rootDirectory + '/*' + fileExtension):
      mxd = arcpy.mapping.MapDocument(f)
      for df in arcpy.mapping.ListDataFrames(mxd, ''):
         for lyr in arcpy.mapping.ListLayers(mxd, '', df):
            if not lyr.isGroupLayer:
               if lyr.isRasterLayer:
                  print "Raster Layer, {}".format(lyr)
               elif lyr.isFeatureLayer:
                  print "Feature Layer, {}, {}".format(lyr, lyr.serviceProperties)
               else:
                  print "Layer, {}".format(lyr)
         for tbl in arcpy.mapping.ListTableViews(mxd, '', df):
            print "Table, {}".format(tbl.dataSource)

if __name__ == '__main__':
   main()

Tags (1)
1 Solution

Accepted Solutions
JoshKalov
New Contributor III

You have already set the PATH in env so this may not be your issue, but I have gotten a python2 call from python3 to work by doing a os.environ.copy() first before updating the path.

replica_sync_command = f"C:\\Python27\\ArcGIS10.4\\python.exe c:\\syncReplica_python2.py"
python2_env = os.environ.copy()
python2_env.update({"PATH": "C:\\Python27\\ArcGIS10.4"})
run_sync = subprocess.run(replica_sync_command.split(), env=python2_env, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)‍‍‍‍‍‍‍‍‍‍‍‍

View solution in original post

26 Replies
JoeBorgione
MVP Esteemed Contributor

I've taken the liberty of copying and pasting your code sample into the syntax highlighter; you'll get more responses in this format.

Sorry , don't have any suggestions for you...

import subprocess, os, winreg, sys, arcpy

 

try:
   arcpy.AddMessage("Finding Python 2.7 installation directory...")
   hKey = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE,    "SOFTWARE\\WOW6432Node\\Python\\PythonCore\\2.7\\InstallPath")
   result = winreg.QueryValueEx(hKey, "")[0]
except:
   arcpy.AddWarning("Python 2.7 installation directory was not found.")
   arcpy.AddError("Script failed.")
   sys.exit()

 

if os.path.exists(result + "\\python.exe"):
   arcpy.AddMessage("Launching Python 2.7 executable...")
   CREATE_NO_WINDOW = 0x8000000
   process = subprocess.Popen([result + "\\python.exe", "C:\\temp\\python\\ListMXDDataSources.py"],    stdout=subprocess.PIPE, stderr=subprocess.PIPE, creationflags = CREATE_NO_WINDOW, shell=True, stdin=None)
   stdout, stderr = process.communicate()
   arcpy.AddMessage('{}'.format(stdout.decode("utf-8")))
   arcpy.AddWarning('{}'.format(stderr.decode("utf-8")))
That should just about do it....
0 Kudos
WilliamCraft
MVP Regular Contributor

Thanks Joe.  The syntax highlighter isn't working properly for me so I was unable to do this myself.  Also FYI, I've just now made some revisions to my code as well.  

0 Kudos
StephanieWendel
Esri Contributor

Hey William Craft! Have you tried running the python 2 portions of the script in ArcMap's python window or in IDLE that comes with the Python2.7 install with Desktop? That could help you determine if it is a problem with python 3 or python 2.7. The other thing you could do is add a simplistic logging system within the script to record where things are failing. This always helps me narrow down if it is my code or the setup I'm running the script in. Below is my favorite logging system to add to scripts which I've added to yours as an example. Note: You could instead of using the script file location in the create_log function hard code a path in the location variable.

import subprocess, os, winreg, arcpy
import os.path, sys
from time import localtime, strftime

def create_log(name, headers=None, ftype=".txt", subfolder=None):
    location = sys.path[0]
    if subfolder is not None:
        location = os.path.join(sys.path[0], subfolder)
    time_setup = strftime("%m_%d_%Y", localtime())
    logfile = os.path.join(location, name + "_" + time_setup + ftype)
    if not os.path.exists(logfile):
        f = open(logfile, 'wb')
        if headers is not None:
            f.write(headers)
            f.write("\n")
        f.close()
    return logfile


def Log(logfile, message):
    f = open(logfile, 'ab')
    f.write(message)
    f.write("\n")
    f.close()

def main():
    logfile = create_log("Monitor", subfolder="Monitor_Logs")
    Log(logfile, "{} Created Log.\n".format(strftime("%m_%d_%Y_%H:%M:%S", localtime())))

    try:
        arcpy.AddMessage("Finding Python 2.7 installation directory...")
        Log(logfile, "Finding python 2.7 installation directory")
        hKey = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE,  "SOFTWARE\\WOW6432Node\\Python\\PythonCore\\2.7\\InstallPath")
        result = winreg.QueryValueEx(hKey, "")[0]
    except:
        arcpy.AddWarning("Python 2.7 installation directory was not found.")
        Log(logfile, "Python 2.7 installation directory was not found.")
        arcpy.AddError("Script failed.")
        sys.exit()

    if os.path.exists(result + "\\python.exe"):
        arcpy.AddMessage("Launching Python 2.7 executable...")
        Log(logfile, "Launching Python 2.7 executable...")
        CREATE_NO_WINDOW = 0x8000000
        process = subprocess.Popen([result + "\\python.exe", "C:\\temp\\python\\ListMXDDataSources.py"],    stdout=subprocess.PIPE, stderr=subprocess.PIPE, creationflags = CREATE_NO_WINDOW, shell=True, stdin=None)
        stdout, stderr = process.communicate()
        arcpy.AddMessage('{}'.format(stdout.decode("utf-8")))
        Log(logfile,'{}'.format(stdout.decode("utf-8")))
        arcpy.AddWarning('{}'.format(stderr.decode("utf-8"))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

if __name__ == '__main__':
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
WilliamCraft
MVP Regular Contributor

Thank you Stephanie Wendel for your help.   The failure seems to be associated with the site.py file of Python 3.  The Python 2 script works perfectly fine on its own.  Here is the underlying error after printing the stderr output from the subprocess.Popen line in the Python 3 code (line 45 above):

JoshuaBixby
MVP Esteemed Contributor

The screenshot is helpful.  The answer to your question/issue is that an ArcGIS Pro Python module is being called with a Python 2.7 interpreter.  In line 177 of site.py, the code uses a syntax for print that is only valid in Python 3 and not Python 2, hence the invalid syntax error.

DanPatterson_Retired
MVP Esteemed Contributor
from __future__ import print_function
print("hello")  # not   .... print "hello"

if you want to fix up the 2.7 script

curtvprice
MVP Esteemed Contributor

Dan, I don't think this will make any difference. Note that in 2.7 print() works just fine without the from __future__ import... I always use it now in my ArcMap code so my scripts will run in both - of course the OPs script can't run in both because is uses ArcMap arcpy methods.

William, it looks to me from your error message that the PATH environment is causing the Pro site.py to get executed when Python 2.7 starts up.  The way to fix this is to make sure the shell you start with Popen has a PATH environment that doesn't include the Pro folders. This is done using the env parameter to Popen.

WilliamCraft
MVP Regular Contributor

Thank you everyone for your help.  Curtis, the suggestion you made seemed to get me past the error I was experiencing.  I set the PATH, PYTHONHOME, and PYTHONPATH variables to only reference the Python 2.7 folder (C:\Python27\ArcGIS10.6).  However, I'm now getting a new error when the 2.7 code executes.  Note that that 'result' variable is equivalent to the value of "C:\Python27\ArcGIS10.6".  

process = subprocess.Popen(result + "\\python.exe C:\\temp\\python\\ListMXDDataSources.py", stdout=subprocess.PIPE, stderr=subprocess.PIPE, creationflags = CREATE_NO_WINDOW, shell=True, env={'PATH': result, 'PYTHONHOME': result, 'PYTHONPATH': result})

0 Kudos
Arne_Gelfert
Regular Contributor

Okay, i don't typically partake in troubleshooting discussions this deep below the surface (and yet so way above my head). But this thread is producing some "keeper" bits of information.

Looks like your code fails at the use of seed() at _hexlify(). Never heard of it but when you compare random.py in 2.7 and 3.6 (my machine), you notice that hexlify shows up in 2.7...

from __future__ import division
from warnings import warn as _warn
from types import MethodType as _MethodType, BuiltinMethodType as _BuiltinMethodType
from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
from math import sqrt as _sqrt, acos as _acos, cos as _cos, sin as _sin
from os import urandom as _urandom
# =======================================
from binascii import hexlify as _hexlify
# =======================================
import hashlib as _hashlib‍‍‍‍‍‍‍‍

but not in 3.6

from warnings import warn as _warn
from types import MethodType as _MethodType, BuiltinMethodType as _BuiltinMethodType
from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
from math import sqrt as _sqrt, acos as _acos, cos as _cos, sin as _sin
from os import urandom as _urandom
from _collections_abc import Set as _Set, Sequence as _Sequence
from hashlib import sha512 as _sha512
import itertools as _itertools
import bisect as _bisect

So maybe you still have 2.x code searching for something in 3.x environment?