topic Re: Using Enthought Canopy with ArcGIS/ArcPy in Python Questions

Using Enthought Canopy with ArcGIS/ArcPy

JordanWeaver — Mon, 22 Apr 2013 16:30:52 GMT

What is the best method for developing Python scripts for ArcGIS in a third-party installation of Python and IDE? I have tried using the suggested workflow from ESRI... that being the included installation of Python and PythonWin for the IDE. Unfortunately, these will not be sufficient or enjoyable so I was hoping to use the Canopy (formerly EPD) IDE (utilizing PyLab, etc.) for development with access to those elements that are available in the ArcGIS "preferred" environment.

I'm sure there are multiple steps to accomplishing this, and one problem may be that I have the 64-bit installation of Canopy (which I can uninstall and install the 32-bit version if necessary), but I was wondering if anyone has attempted to tackle this. I didn't want to start experimenting with environment variables until I was sure that I had the appropriate installations, as well as an understanding of what I was trying to accomplish.

Thanks in advance for any help with this problem.

Setup:
Windows 7
ArcGIS Desktop 10.1
Enthought Canopy (Python 2.7.3 64-bit)

I apologize if this has been answered previously, but I couldn't find an existing good explanation on the forum. Please redirect if solution already exists.

Re: Using Enthought Canopy with ArcGIS/ArcPy

curtvprice — Sun, 12 Dec 2021 03:41:24 GMT

Here's my response to Jordan copied from my email earlier today, with some edits for clarity:

Jordan, I totally agree that it's tricky to integrate the two environments. Haven't tried Canopy yet, but I understand Canopy is pretty much the same with a juiced up UI and improved setup tools.

I really do believe that the best way to handle this is to install python from the ArcGIS distribution, and separately install Canopy. (You need EPD 32-bit to import 32-bit arcpy and EPD 64-bit to import 64-bit arcpy.) If you install EPD 32 and 64 and ArcGIS Desktop x64 background geoprocessing, this adds up to no less than four python distributions to deal with.

Jason Pardy from Esri suggested I just try putting the site-packages folders in the python path. From ArcGIS I have been successful accessing EPD packages by adding a similar .pth file to the ArcGIS python install site-packages folder. I haven't run into any problems with "circular searches", I think Python is smart enough not to look in a site-packages folder twice.

One way to modify the Python path is to copy the file Desktop.pth from \Python27\ArcGIS10.1\lib\site-packages to the file zzArcGIS.pth in the 32-bit Canopy site-packages folder (the paths are loaded in alpha order). This should allow you to import arcpy just fine on the Canopy side. Another approach is to just add the paths to the PYTHONPATH variable (see bat scripts below).

bat scripts are especially useful if you find you need full control of the PATH and other environment variables to get things to work right. I've attached an example. The way it's done in UNIX -- much less ugly than registering DLLs and messing with globally-applied environment variables! (When you run the scripts, the environment tweaks only apply to the shell and its children, not your global environment.) A few examples are attached:

:: epd32arc.bat
:: start EPD32 python prompt with arcpy
@echo off
set EPDPATH=E:\python27_epd32
set AGSPATH=D:\ArcGIS\Desktop10.1
:: save paths
set PATHENV=%PATH%
set PPATHENV=%PYTHONPATH%
:: set paths EPD with access to arcpy
set PATH=%EPDPATH%;%EPDPATH%\scripts;%PATH%
set PYTHONPATH=%PYTHONPATH%;%AGSPATH%\bin;%AGSPATH%\arcpy;%AGSPATH%\ArcToolbox\Scripts
cmd /c %EPDPATH%\python.exe
:: restore paths
set PATH=%PATHENV%
set PYTHONPATH=%PPATHENV%
:: delete variables
set EPDPATH=
set AGSPATH=
set PATHENV=
set PPATHENV=

:: arcmap_epd.bat - start ArcMap with EPD32 libraries available
@echo off
:: set the paths below to your install locations
set EPDPATH=E:\python27_epd32
set AGSPATH=D:\ArcGIS\Desktop10.1
:: save paths
set PATHENV=%PATH%
set PPATHENV=%PYTHONPATH%
:: set paths for ArcMap w/ EPD
set PATH=%EPDPATH%;%EPDPATH%\scripts;%PATH%
set PYTHONPATH=%PYTHONPATH%;%EPDPATH%\lib\site-packages
echo Starting ArcMap w/ Enthought Python Distribution modules...
start /b /d %AGSPATH%\bin ArcMap.exe
:: restore paths
set PATH=%PATHENV%
set PYTHONPATH=%PPATHENV%
:: delete variables
set EPDPATH=
set AGSPATH=
set PATHENV=
set PPATHENV=

Re: Using Enthought Canopy with ArcGIS/ArcPy

JordanWeaver — Tue, 23 Apr 2013 14:46:11 GMT

I ended up uninstalling all Enthought builds I had on my system and started from scratch with a single 32-bit Canopy installation. From there, I copied the Desktop10.1.pth file from C:\Python27\ArcGIS10.1\Lib\site-packages to the Canopy site-packages directory, renaming it zzArcGIS.pth as you suggested (for my system it was located: C:\Users\<user>\AppData\Local\Enthought\Canopy32\System\Lib\site-packages).

This worked great and I�??m surprised that it ended up being this easy. I�??m sort of out of my element�?� I traditionally work in UNIX/OS X so I appreciate the included bat scripts. Since the other .pth solution seems to get me everything I was looking for (i.e. accessing/using ArcPy from within Canopy), I will avoid messing with the Windows environment variables as I�??ve had that go sour for other things despite my best efforts to manage them correctly.

Thanks again for your help and I'll post any updates on unexpected issues/behavior.

Re: Using Enthought Canopy with ArcGIS/ArcPy

curtvprice — Sun, 12 Dec 2021 03:41:27 GMT

Here's a better description of using pth files from the gis.stackexchange.com site, posted by my colleague Rich Signell. If you don't want to tweak your environment for your python session, this is an easier approach than a .bat file. Note path files are read in alpha order, so if you name your custom pth file starting with "zz" it will be loaded last -- which may avoid conflicts.

[/hr]

The Enthought Python Distributions, including Canopy, are stand-alone and don't need anything in the registry or in the environment variables. So they can play nice with other Python distributions like the one that comes with ArcGIS 10.1.

To access ArcGIS 10.1 python modules from Enthought Canopy: Create a "path" file in the Canopy user directory that points to the ArcGIS 10.1 python modules. On my system, I created a text file called arcpy.pth with these 4 lines:

# .pth file for arcpy
C:\ArcGIS\Desktop10.1\bin
C:\ArcGIS\Desktop10.1\arcpy
C:\ArcGIS\Desktop10.1\ArcToolbox\Scripts

and put arcpy.pth in the Canopy User site-packages directory:

C:\Users\rsignell\AppData\Local\Enthought\Canopy32\User\lib\site-packages

To access Enthought Canopy modules from ArcGIS 10.1: Create a "path" file in the Arc site-packages folder that points to the Canopy python modules. On my system, I created a text file called epd.pth with these 3 lines:

# .pth file for EPD Canopy
C:\Users\rsignell\AppData\Local\Enthought\Canopy32\User\Lib\site-packages
C:\Users\rsignell\AppData\Local\Enthought\Canopy32\System\Lib\site-packages

and then put epd.pth in the Arc site-package folder:

C:\Python27\ArcGIS10.1\Lib\site-packages

This works because Enthought Canopy and ArcGIS10.1 are at the same version of Python (2.7) and Numpy (1.6). I hope we can continue to have this compatibility!

Re: Using Enthought Canopy with ArcGIS/ArcPy

NielsClaes — Sun, 15 Dec 2013 10:49:14 GMT

I changed to the 32 version of Canopy and copied the pathfile into the 'site packages' folder as suggested.
When importing now arcpy, I get the following error:

'import arcpy
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-5-50d283eebbb3> in <module>()
----> 1 import arcpy

C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\__init__.py in <module>()
     19 import warnings
     20
---> 21 from arcpy.geoprocessing import gp
     22 from arcpy.geoprocessing import env
     23 from arcpy.geoprocessing._base import gptooldoc as _gptooldoc
...
'
ending with a Runtime Error: Not Initialized

Does anyone had the same problem or an idea how to solve it?

Best Regards

Re: Using Enthought Canopy with ArcGIS/ArcPy

curtvprice — Sun, 12 Dec 2021 03:41:30 GMT

As noted above, we've gotten a little fancier with this. The .pth files in the site-packages folder are loaded in alphabetical order so we've been naming these files "zzarcpy.pth" (in the canopy site-packages) and "zzepd.pth" (in the ArcGIS site-packages). Naming them with a "zz" prefix should ensure they are listed after the "native" modules for that python install. (First folder in the list wins, just like the Windows and Unix PATH variables.)

Also double check to make sure you don't have any settings in your windows environment PYTHONPATH.

If you're still stuck, have your test script dump out the sys.path before you import arcpy, that may help see what's going wrong with your sys.path. Hopefully it will become clear.

import sys
k = 0
for p in sys.path:
  print "{0:4} {1}".format(k,p)
  k += 1

Here's an example - on this particular machine I don't have Enthought Canopy installed. But if it was, I'd want to see those paths AFTER the ArcGIS ones to avoid conflicts when importing arcpy.

Remember if you do run into conflicts, there is a workaround - you can modify the sys.path if, for example, you want to temporarily knock out the ArcGIS references to use some Canopy modules that have conflicts with the python versions Esri is using.

>>> k = 0
>>> for p in sys.path:
...   print "{0:4} {1}".format(k,p)
...   k += 1
...
   0 (os.curdir - current python working folder)
   1 C:\ArcGIS\Desktop10.1\ArcToolbox\Toolboxes\USGS_EGISTools\scripts
   2 C:\Windows\system32\python27.zip
   3 C:\Python27\ArcGIS10.1\DLLs
   4 C:\Python27\ArcGIS10.1\lib
   5 C:\Python27\ArcGIS10.1\lib\plat-win
   6 C:\Python27\ArcGIS10.1\lib\lib-tk
   7 C:\Python27\ArcGIS10.1
   8 C:\Python27\ArcGIS10.1\lib\site-packages
   9 C:\ArcGIS\Desktop10.1\bin
  10 C:\ArcGIS\Desktop10.1\arcpy
  11 C:\ArcGIS\Desktop10.1\ArcToolbox\Scripts
>>> import arcpy
>>> arcpy.__file__
'C:\\ArcGIS\\Desktop10.1\\arcpy\\arcpy\\__init__.py'

Re: Using Enthought Canopy with ArcGIS/ArcPy

FranciscoGoerlich — Mon, 24 Mar 2014 21:52:14 GMT

Using the pth files I get the following error when importing ArcPy:
import arcpy:
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-50d283eebbb3> in <module>()
----> 1 import arcpy

C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\__init__.py in <module>()
     19 import warnings
     20
---> 21 from arcpy.geoprocessing import gp
     22 from arcpy.geoprocessing import env
     23 from arcpy.geoprocessing._base import gptooldoc as _gptooldoc

C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\__init__.py in <module>()
     12 #
     13 #email: contracts@esri.com
---> 14 from _base import *

C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\geoprocessing\_base.py in <module>()
     12 #
     13 #email: contracts@esri.com
---> 14 import arcgisscripting
     15 import functools
     16

ImportError: DLL load failed: %1 no es una aplicación Win32 válida.

Presumibly because EDP-Canopy is 64 bits and ArcGIS is 32.
Should I unistall EDP-Canopy 64 bits, and install the 32-bits version, or it is enought that I install ArcGIS for Desktop Background GP (64-bit) on the top of the standar version of ArcGIS?

Thanks,

Re: Using Enthought Canopy with ArcGIS/ArcPy

curtvprice — Tue, 25 Mar 2014 18:31:42 GMT

You may install 32 bit AND 64 bit canopy. Your .pth files in the Python27\ArcGIS10.2 and Python27\ArcGISx6410.2 would then point to 32 and 64 bit canopy, 32->32 and 64->64.

Canopy does some magic with virtual environments that I haven't quite worked out yet in terms of how it works with ArcGIS - but this should be quite doable.

Re: Using Enthought Canopy with ArcGIS/ArcPy

FranciscoGoerlich — Fri, 28 Mar 2014 18:49:18 GMT

Thanka a lot.
I have now installed ArcGIS 10.2 Background GP (64-bit) and can import arcpy from Canopy without problems.
However when I try to run a ToolBox from ArcGIS 10.2 that uses py, then I get an error:

Executing: GenerateSpatialWeightsMatrix D:\GIS\CNIG\Equipamiento_Geográfico_Referencia_Nacional\lineas_limite_2011Mayo\Municipios2006_ETRS89.shp INENumMuni D:\GIS\CNIG\Equipamiento_Geográfico_Referencia_Nacional\lineas_limite_2011Mayo\Municipios2006_ETRS89.swm CONTIGUITY_EDGES_CORNERS EUCLIDEAN 1 # 0 NO_STANDARDIZATION # # # #
Start Time: Fri Mar 28 20:39:43 2014
Running script GenerateSpatialWeightsMatrix...
Constructing spatial weights based on polygon contiguity (Edges or Corners)....
WARNING 000916: The input feature class does not appear to contain projected data.

Traceback (most recent call last):
File "c:\program files (x86)\arcgis\desktop10.2\ArcToolbox\Scripts\Weights.py", line 910, in <module>
    setupWeights()
File "c:\program files (x86)\arcgis\desktop10.2\ArcToolbox\Scripts\Weights.py", line 110, in setupWeights
    contiguityType = "QUEEN")
File "c:\program files (x86)\arcgis\desktop10.2\ArcToolbox\Scripts\Weights.py", line 201, in polygon2SWM
    numNeighs = kNeighs)
File "C:\Program Files (x86)\ArcGIS\Desktop10.2\ArcToolbox\Scripts\WeightsUtilities.py", line 203, in __init__
    self.setHeader()
File "C:\Program Files (x86)\ArcGIS\Desktop10.2\ArcToolbox\Scripts\WeightsUtilities.py", line 261, in setHeader
    header = NUM.array([header], '<c')
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 0: ordinal not in range(128)

Failed to execute (GenerateSpatialWeightsMatrix).

I get the same error using forefront or background processing.

It seems that ArcGIS does not access Canopy. I have only installed the 64 version of Canopy.

Thanks a lot in advance.

Re: Using Enthought Canopy with ArcGIS/ArcPy

ShaunWalbridge — Sat, 29 Mar 2014 01:16:34 GMT

Francisco, I don't think this is due to your Python installation, but a bug in the particular function you're using. I suggest one option to fix it in the other thread posted.

Re: Using Enthought Canopy with ArcGIS/ArcPy

ScottStephenson — Sun, 12 Dec 2021 03:41:32 GMT

Curtis, thanks for your help with this. I'm still experiencing the same error as others in this thread (ImportError: DLL load failed: %1 is not a valid Win32 application) after adding the arcpy.pth files to the Canopy site packages directory as instructed above. I dumped out the sys.path (copied below), and as you can see, there are Canopy paths both before and after the ArcGIS paths. Do you know how to get the Canopy paths to load after the ArcGIS paths, as you suggested?

0 C:\Users\scott.stephenson\Dropbox\Python\scripts\Emissions
      1
      2 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\User\Scripts\python27.zip
      3 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\DLLs
      4 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\lib
      5 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\lib\plat-win
      6 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\lib\lib-tk
      7 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64
      8 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\User
      9 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\User\lib\site-packages
    10 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\User\lib\site-packages\PIL
    11 C:\Program Files (x86)\ArcGIS\Desktop10.2\bin
    12 C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy
    13 C:\Program Files (x86)\ArcGIS\Desktop10.2\ArcToolbox\Scripts
    14 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\User\lib\site-packages\win32
    15 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\User\lib\site-packages\win32\lib
    16 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\User\lib\site-packages\Pythonwin
    17 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata
    18 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\lib\site-packages
    19 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\lib\site-packages\win32
    20 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\lib\site-packages\win32\lib
    21 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\lib\site-packages\Pythonwin
    22 C:\Users\scott.stephenson\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.5.3123.win-x86_64\lib\site-packages\IPython\extensions
    23 C:\Users\scott.stephenson\.ipython

Re: Using Enthought Canopy with ArcGIS/ArcPy

curtvprice — Thu, 26 Nov 2015 06:07:51 GMT

Did you:

1) make sure the canopy path is not in the system PATH (this is included in the python sys.path)

2) name the canopy .pth file in /lib/site-packages starting with zz so it is loaded last.

I have moved on to using anaconda (free) but this approach is still working for me. I use an environment set up specifically compatible with ArcGIS ("arc1022") and then I can also have other environments to work with non-ArcGIS compatible packages.

Re: Using Enthought Canopy with ArcGIS/ArcPy

ScottStephenson — Fri, 27 Nov 2015 22:28:42 GMT

Thanks again Curtis. I don't know how to check if the canopy path is not in the system PATH. I named the .pth files (both zzarcpy.pth in the Canopy site-packages folder, and zzcanopy.pth in the Python27 site-packages folder) accordingly so they would be loaded last. No luck

I've also tried installing and running the 32-bit version of Canopy -- no luck there either.

Would you recommend I try using Anaconda? If it's easier to get arcpy to load in Anaconda, I will gladly give that a try.

Re: Using Enthought Canopy with ArcGIS/ArcPy

DanPatterson_Retired — Fri, 27 Nov 2015 23:20:46 GMT

Go with what works without problems or special considerations... people can weigh in if they have information to the contrary

PythonWin
PyScripter

I have both versions installed for: (associated modules in brackets)

Python 2.7.x ... aka ArcMap up to 10.3.1 (Numpy, MatPlotLib, appropriate python versioned)
Python 3.4.x ... aka ArcGIS Pro to version 1.1.1 (Numpy,SciPy, SymPy, MatPlotLib,Pandas...etc blah blah)

Amazingly, if you let Arcmap and ArcGIS Pro install the version of python etc FIRST, PythonWin and PyScripter find them and their appropriate version without any floor dancing about paths etc.

So unless you have a mission critical reason to use something else, you might use the current known-to-work IDE's and wait until IronPython comes along (or so is rumoured)

Back to you Curtis for counter-point

Re: Using Enthought Canopy with ArcGIS/ArcPy

curtvprice — Sat, 28 Nov 2015 06:20:24 GMT

> I don't know how to check if the canopy path is not in the system PATH.

The system path is PATH environment variable in Windows. Search for "env" in the windows search and pick user environment variables to get the dialog in WIn7.

> Would you recommend I try using Anaconda?

I've had really good luck with Anaconda, mostly because you can install it without modifying the main system Pythons that Arc is using. Anaconda environments are just the ticket for creating an arc compatible setup and it makes it really easy to install versions of things that will be compatible with the ArcGIS stack.

Dan is right you really need to "clean house" --

1) remove all pythons you see in add/remove programs

2) Repair Desktop and/or Pro (this will restore Windows python paths, etc. This should get ArcMap seeing what it needs to see again.

3) IF YOU REALLY NEED IT - Install other Pythons, i.e. Anaconda, EPD whatever (use the option to NOT modify the Windows Python setups when installing Anaconda)!

Re: Using Enthought Canopy with ArcGIS/ArcPy

ScottStephenson1 — Mon, 30 Nov 2015 15:11:55 GMT

Curtis, I deleted Canopy from the system PATH, and it worked perfectly. Truthfully, I don't know why it worked, but thank you! I may move toward Anaconda in the future, but for now, Canopy seems to be working.

Dan and Curtis, thanks for the advice. I will keep this in mind next time I need to "clean house" and remove redundant and/or incompatible Python versions.