topic Python Path Confusion in Python Questions

Python Path Confusion

ThomMackey — Thu, 22 Sep 2011 23:59:53 GMT

Hi all,

Having some issues with Python paths in 10.1.

I'm using a VM someone else at work set up. It's running Win Sever 2008 R2 and it has both ArcGIS Desktop 10.1 and Server 10.1 installed (I'm guessing this is part of the problem). I didn't perform the installation so I don't know the procedure which led to this mess.

There appear to be 3 (!) installs of Python on the system. One in C:\Python27, one in C:\Python27\ArcGIS10.1, and one in C:\Python27\ArcGISx6410.1.

My main issue is in Desktop, I can't use any Python functions, because whenever I try to import arcpy, it can't find numpy. Specifically, typing from the Python command window:

>>> import arcpy Runtime error  File <string>, line 1, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\__init__.py, line 20, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\toolbox.py, line 347, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\management.py, line 22, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\_management.py, line 14, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\_graph.py, line 27, in <module> <type 'exceptions.ImportError'>: No module named numpy

I get the same error pop up as a parameter error on the 1st parameter when opening any of the Python tools in the Spatial Statistics toolbox. I guess this is something to do with the validation calling import arcpy.

numpy is in the C:\Python27\ArcGISx6410.1\lib\site-packages and the C:\Python27\ArcGIS10.1 folders but not C:\Python27\lib\site-packages.

The registry entry HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.7\InstallPath is set to C:\Python27\ArcGISx6410.1\. HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.7\PythonPath is set to C:\Python27\ArcGISx6410.1\Lib;C:\Python27\ArcGISx6410.1\DLLs;C:\Python27\ArcGISx6410.1\Lib\lib-tk. So everything should be in order there.

However, ArcMap is starting Python from C:\Python27 directly. I get the following from the Python command line in ArcMap:

>>> import sys >>> sys.path [ 'C:\\Windows\\system32\\python27.zip', u'c:\\program files (x86)\\arcgis\\desktop10.1\\arcpy', 'C:\\Python27\\Lib', 'C:\\Python27\\DLLs', 'C:\\Python27\\Lib\\lib-tk', 'c:\\program files (x86)\\arcgis\\desktop10.1\\ArcToolbox\\Toolboxes',  'C:\\Program Files (x86)\\ArcGIS\\Desktop10.1\\bin', 'C:\\Python27', 'C:\\Python27\\lib\\site-packages' ]

(Bonus question: What's the 'C:\\Windows\\system32\\python27.zip' doing there?)

If I start IDLE from the "Python 2.7" folder in the start menu, it can't import numpy either and the path is the same as above. If I start IDLE from the ArcGIS\Python 2.7 folder it can see and import arcpy (and numpy, and matplotlib).

If I enter "Python" at the Windows command prompt I get the C:\Python27 one.

I can think of a few solutions but want to know a) how this would have happened, b) why ArcMap is starting Python from C:\Python27 instead of the Python InstallPath in the registry, and c) if I can get rid of one or two of these extra, confusing Python installs, or if not, ensure that Windows will always look at the same installs when installing modules etc.

Thanks all.

Re: Python Path Confusion

SarahWilliams — Sun, 14 Oct 2012 16:27:26 GMT

I am having the same problem has anyone identified how to solve the problem?

Sarah

Re: Python Path Confusion

JeffreySwain — Mon, 15 Oct 2012 15:35:25 GMT

Sarah,

I am glad to see that the recommendation to uninstall the other version of python and then repair ArcMap worked to get the process working again. Thanks for calling Esri Support.

Re: Python Path Confusion

curtvprice — Mon, 15 Oct 2012 22:34:38 GMT

There appear to be 3 (!) installs of Python on the system. One in C:\Python27, one in C:\Python27\ArcGIS10.1, and one in C:\Python27\ArcGISx6410.1.
My main issue is in Desktop, I can't use any Python functions, because whenever I try to import arcpy, it can't find numpy.

The Python27/ArcGIS10.1 installed with Desktop, the Python27/ArcGISx6410.1 is installed with either Desktop x64 background GP, or ArcGIS Server. I don't know how the third one got installed. I'm guessing last, because Windows is clearly set up to have that Python in the path.

Sounds to me like the best approach is to uninstall the python installed in C:\Python27. (It will be in Add/Remove Programs) Then do a repair install of Desktop and Workstation (from add/remove programs). (Note, cross post to https://community.esri.com/groups/arcgis-for-desktop-installation-support?sr=search&searchId=9e9fd9be-e541-4f8f-a0c7-fa7183d43b18&searchIndex=0‌

(Bonus question: What's the 'C:\\Windows\\system32\\python27.zip' doing there?)

This is a deprecated path from the early days of Python before the current setup (startup scripts, PYTHONPATH, site-packages) was implemented for startup.. You can safely ignore it.

Re: Python Path Confusion

MatthewBaker2 — Thu, 16 Jun 2016 19:24:48 GMT

If ArcGIS Desktop and ArcGIS Server are installing their own instance, I wouldn't question a third install which probably came first - a standard desktop python installation in c:\python27!

My issue has been why does ArcGIS need its own installation of python when - in my case - I have a perfectly good, functioning, and important installation already that uses pypyodbc and psycopg2.

So now that I install ArcGIS Desktop, in any configuration of installation directories I can think of, the sys.path variables all get set to the ArcGIS paths - what happened to my existing installation?

The issue I'm having is to reset - or at least append to include - the paths to my existing installation so I can run arcpy and pypyodbc and psycopg2 scripts on the same machine - but this seems impossible!!!

sys.path.append('path to existing libraries') does not appear to do anything, and other threads on other forums don't help.

I feel like I'm caught in a 'ArcCatch22'... and I'd hate to think I would have to set up another machine just to run arcpy scripts?

Re: Python Path Confusion

curtvprice — Wed, 22 Jun 2016 16:25:46 GMT

It's all about module compatibility. If you install Python 2.7.x (use the right version of course) before you install ArcGIS Desktop, the installer will recognize and use that Python.

Python, NumPy, and MatPlotlib—Help | ArcGIS for Desktop

The ArcGIS Python stack isn't that big so I like just using Esri's to ensure module compatibility so their tools won't break.

For help with getting another Python to use arcpy and arcpy to use modules in a second (or third) Python install (ie Anaconda) see: Using a user-specific Python startup script and Connecting Arcpy to Your Jupyter Notebook

Python workflows sometimes require many installs of python to support workflows that require certain versions of modules (like arcpy). This is the problem that packages like Anaconda and Enthought seek to solve by setting up multiple virtual environments you can choose from depending on your needs. As Python develops further we all hope this will get less difficult -- and it is especially difficult on Windows, which lacks some of the tool (soft links for example) that make this problem easier to solve.

A full install of ArcGIS Desktop includes three Pythons

1) 2.7 x32 (Desktop) C;\Python\ArcGIS10.x

2) 2.7x64 (x64 Background geoprocessing) C:\Python27\ArcGISx6410.x\

3) 3.4 x64 (Pro). (C:\Python34) (optional)

ArcGIS Pro 3.4 has an embedded Python inside it you can't mess with and an optional install of Python 3.4 if you want to do 3.4 development. This is actually cleaner than the Desktop setup because Esri's tools can use their own Python environment that isn't exposed to you to mess up.

I am told Pro 1.3 will include an install of Anaconda, which is great news. This would allow you to create custom environments so non-elevated users could easily set up virtual environments and add them to the sys.path for all three ArcGIS modules. Can't wait to see that at EsriUC.

Re: Python Path Confusion

MatthewBaker2 — Fri, 24 Jun 2016 16:44:59 GMT

Do you have any thoughts on this issue with ArcGIS 10.4(.1) as it relates to your post above?

Problem: ArcMap crashes during Loading Document launch stage

Re: Python Path Confusion

curtvprice — Fri, 24 Jun 2016 20:22:10 GMT

For most folks, the issue you reference results from a known issue with some changes in Python's registry setup with 2.7.11, that Esri could not anticipate. One of their workarounds is to remove that Python version and do a repair install which will reinstall Esri's shipped Python. I thought they fixed this at 10.4. Bummer, but fortunately it's an easy registry fix.

It's an example of the danger of using some other Python than the stack Esri ships -- I strongly believe than unless you really have a need, you should work with the Python versions that Esri provides with that version of ArcGIS.

It's also a classic example of the shortcomings of the Windows platform -- registry and runtime (.dll) issues are much less of a problem on Unix platforms. Windows has other advantages, but wow, it has issues.

Re: Python Path Confusion

MatthewBaker2 — Tue, 28 Jun 2016 20:29:15 GMT

I think my problem here is the ESRI-centric philosophy for running Python... Again, I come at this from the non-ESRI need for python - pypyodbc, psycopg2 - and can run them just fine for many database ETL solutions. But anything involving an ArcGIS python process really messes up the system... I'll keep digging. Thanks for the information!

Re: Python Path Confusion

MatthewBaker2 — Wed, 06 Jul 2016 21:05:00 GMT

Ah a fantastic solution has emerged:

Enable Python installations that were not installed by ArcGIS to access Arcpy functionality | Python, GIS and stuff…

Copy the Desktop10.1.pth file from the ArcGIS python installation to the standard python installation! See the link above for the quick and easy solution (a few prerequisites... but nothing too complicated...)

Re: Python Path Confusion

CoMAGOL_ADMINISTRATOR — Thu, 29 Sep 2016 02:17:07 GMT

Hi,

I had the same problem, it seems with install of ArcGIs 10.4.1, i choose full install, and numpy was notinstalled as part of the process. It looks like a bug in the ArcGis 10.4.1 install

Rudy

Re: Python Path Confusion

curtvprice — Sat, 08 Oct 2016 02:13:21 GMT

I have not heard of this problem yet after thousands of enterprise installs in my organization, so I suspect the issue is multiple Pythons. Best solution: Remove all the non-Esri pythons seen in Add/Remove Programs, repair install of ArcGIS 10.4.1, (and background GP and Pro, if you have them). This should get numpy back in your path.

Re: Python Path Confusion

MatthewBaker2 — Mon, 10 Oct 2016 14:41:56 GMT

Thousands of ESRI Enterprise installs? Well This is not an ESRI enterprise install... so I'm glad I could be the first(-ish)!

The non-ESRI python install is more valuable to me than the ESRI python, so merging the ESRI install into the non-ESRI install is the best solution for me, and I'm glad someone has figured out a way to do this!

Re: Python Path Confusion

curtvprice — Tue, 11 Oct 2016 02:54:49 GMT

Esri's install has always supported a previously installed 2.x python, but to get it to work with Esri software you need to make sure the versions of python, numpy, matplotlib, etc line up with what Esri tools expect. If you choose to use an 'existing' python, Anaconda is a really good option so you can make sure everything is compatible. /blogs/dan_patterson/2016/07/17/anaconda-spyder-and-arcgis-pro?sr=search&searchId=efa9d8a0-5c71-41fd-b38b-9adf8abc53f4&searchIndex=0‌ https://community.esri.com/people/curtvprice/blog/2016/02/18/using-a-user-specific-python-startup-script?sr=search&searchId=efa9d8a0-5c71-41fd-b38b-9adf8abc53f4&searchIndex=2‌