Python Path Confusion

[ThomMackey]

Hi all,

Having some issues with Python paths in 10.1.

I'm using a VM someone else at work set up. It's running Win Sever 2008 R2 and it has both ArcGIS Desktop 10.1 and Server 10.1 installed (I'm guessing this is part of the problem). I didn't perform the installation so I don't know the procedure which led to this mess.

There appear to be 3 (!) installs of Python on the system. One in C:\Python27, one in C:\Python27\ArcGIS10.1, and one in C:\Python27\ArcGISx6410.1.

My main issue is in Desktop, I can't use any Python functions, because whenever I try to import arcpy, it can't find numpy. Specifically, typing from the Python command window:

>>> import arcpy Runtime error  File <string>, line 1, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\__init__.py, line 20, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\toolbox.py, line 347, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\management.py, line 22, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\_management.py, line 14, in <module> File c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\_graph.py, line 27, in <module> <type 'exceptions.ImportError'>: No module named numpy

I get the same error pop up as a parameter error on the 1st parameter when opening any of the Python tools in the Spatial Statistics toolbox. I guess this is something to do with the validation calling import arcpy.

numpy is in the C:\Python27\ArcGISx6410.1\lib\site-packages and the C:\Python27\ArcGIS10.1 folders but not C:\Python27\lib\site-packages.

The registry entry HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.7\InstallPath is set to C:\Python27\ArcGISx6410.1\. HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.7\PythonPath is set to C:\Python27\ArcGISx6410.1\Lib;C:\Python27\ArcGISx6410.1\DLLs;C:\Python27\ArcGISx6410.1\Lib\lib-tk. So everything should be in order there.

However, ArcMap is starting Python from C:\Python27 directly. I get the following from the Python command line in ArcMap:

>>> import sys >>> sys.path [ 'C:\\Windows\\system32\\python27.zip', u'c:\\program files (x86)\\arcgis\\desktop10.1\\arcpy', 'C:\\Python27\\Lib', 'C:\\Python27\\DLLs', 'C:\\Python27\\Lib\\lib-tk', 'c:\\program files (x86)\\arcgis\\desktop10.1\\ArcToolbox\\Toolboxes',  'C:\\Program Files (x86)\\ArcGIS\\Desktop10.1\\bin', 'C:\\Python27', 'C:\\Python27\\lib\\site-packages' ]

(Bonus question: What's the 'C:\\Windows\\system32\\python27.zip' doing there?)

If I start IDLE from the "Python 2.7" folder in the start menu, it can't import numpy either and the path is the same as above. If I start IDLE from the ArcGIS\Python 2.7 folder it can see and import arcpy (and numpy, and matplotlib).

If I enter "Python" at the Windows command prompt I get the C:\Python27 one.

I can think of a few solutions but want to know a) how this would have happened, b) why ArcMap is starting Python from C:\Python27 instead of the Python InstallPath in the registry, and c) if I can get rid of one or two of these extra, confusing Python installs, or if not, ensure that Windows will always look at the same installs when installing modules etc.

Thanks all.

[SarahWilliams]

I am having the same problem has anyone identified how to solve the problem?

Sarah

[JeffreySwain]

Sarah,

I am glad to see that the recommendation to uninstall the other version of python and then repair ArcMap worked to get the process working again.  Thanks for calling Esri Support.

[curtvprice]

There appear to be 3 (!) installs of Python on the system. One in C:\Python27, one in C:\Python27\ArcGIS10.1, and one in C:\Python27\ArcGISx6410.1.My main issue is in Desktop, I can't use any Python functions, because whenever I try to import arcpy, it can't find numpy.

The Python27/ArcGIS10.1 installed with Desktop, the Python27/ArcGISx6410.1 is installed with either Desktop x64 background GP, or ArcGIS Server. I don't know how the third one got installed. I'm guessing last, because Windows is clearly set up to have that Python in the path.

Sounds to me like the best approach is to uninstall the python installed in C:\Python27. (It will be in Add/Remove Programs) Then do a repair install of Desktop and Workstation (from add/remove programs). (Note, cross post to https://community.esri.com/groups/arcgis-for-desktop-installation-support?sr=search&searchId=9e9fd9b...‌

(Bonus question: What's the 'C:\\Windows\\system32\\python27.zip' doing there?)

This is a deprecated path from the early days of Python before the current setup (startup scripts, PYTHONPATH, site-packages) was implemented for startup.. You can safely ignore it.

[MatthewBaker2]

If ArcGIS Desktop and ArcGIS Server are installing their own instance, I wouldn't question a third install which probably came first - a standard desktop python installation in c:\python27!

My issue has been why does ArcGIS need its own installation of python when - in my case - I have a perfectly good, functioning, and important installation already that uses pypyodbc and psycopg2.

So now that I install ArcGIS Desktop, in any configuration of installation directories I can think of, the sys.path variables all get set to the ArcGIS paths - what happened to my existing installation?

The issue I'm having is to reset - or at least append to include - the paths to my existing installation so I can run arcpy and pypyodbc and psycopg2 scripts on the same machine - but this seems impossible!!!

sys.path.append('path to existing libraries') does not appear to do anything, and other threads on other forums don't help.

I feel like I'm caught in a 'ArcCatch22'... and I'd hate to think I would have to set up another machine just to run arcpy scripts?

[curtvprice]

It's all about module compatibility. If you install Python 2.7.x (use the right version of course) before you install ArcGIS Desktop, the installer will recognize and use that Python.

Python, NumPy, and MatPlotlib—Help | ArcGIS for Desktop

The ArcGIS Python stack isn't that big so I like just using Esri's to ensure module compatibility so their tools won't break.

For help with getting another Python to use arcpy and arcpy to use modules in a second (or third) Python install (ie Anaconda) see: Using a user-specific Python startup script​ and Connecting Arcpy to Your Jupyter Notebook​

Python workflows sometimes require many installs of python to support workflows that require certain versions of modules (like arcpy). This is the problem that packages like Anaconda and Enthought seek to solve by setting up multiple virtual environments you can choose from depending on your needs. As Python develops further we all hope this will get less difficult -- and it is especially difficult on Windows, which lacks some of the tool (soft links for example) that make this problem easier to solve.

A full install of ArcGIS Desktop includes three Pythons

1) 2.7 x32 (Desktop) C;\Python\ArcGIS10.x

2) 2.7x64 (x64 Background geoprocessing)  C:\Python27\ArcGISx6410.x\

3) 3.4 x64 (Pro). (C:\Python34) (optional)

ArcGIS Pro 3.4 has an embedded Python inside it you can't mess with and an optional install of Python 3.4 if you want to do 3.4 development. This is actually cleaner than the Desktop setup because Esri's tools can use their own Python environment that isn't exposed to you to mess up.

I am told Pro 1.3 will include an install of Anaconda, which is great news. This would allow you to create custom environments so non-elevated users could easily set up virtual environments and add them to the sys.path for all three ArcGIS modules. Can't wait to see that at EsriUC.

[MatthewBaker2]

Do you have any thoughts on this issue with ArcGIS 10.4(.1) as it relates to your post above?

Problem: ArcMap crashes during Loading Document launch stage

[curtvprice]

For most folks, the issue you reference results from a known issue with some changes in Python's registry setup with 2.7.11, that Esri could not anticipate. One of their workarounds is to remove that Python version and do a repair install which will reinstall Esri's shipped Python.  I thought they fixed this at 10.4. Bummer, but fortunately it's an easy registry fix.

It's an example of the danger of using some other Python than the stack Esri ships -- I strongly believe than unless you really have a need, you should work with the Python versions that Esri provides with that version of ArcGIS.

It's also a classic example of the shortcomings of the Windows platform -- registry and runtime (.dll) issues are much less of a problem on Unix platforms. Windows has other advantages, but wow, it has issues.

[MatthewBaker2]

I think my problem here is the ESRI-centric philosophy for running Python... Again, I come at this from the non-ESRI need for python - pypyodbc, psycopg2 - and can run them just fine for many database ETL solutions. But anything involving an ArcGIS python process really messes up the system... I'll keep digging. Thanks for the information!

[MatthewBaker2]

Ah a fantastic solution has emerged:

Enable Python installations that were not installed by ArcGIS to access Arcpy functionality | Python...

Copy the Desktop10.1.pth file from the ArcGIS python installation to the standard python installation! See the link above for the quick and easy solution (a few prerequisites... but nothing too complicated...)