Python and ArcGIS Pro 1.3 : Conda

[#1]

With ArcGIS Pro 1.3, there are some significant changes to Python. These changes primarily affect how you run standalone scripts with Python. The experience of Python within the ArcGIS Pro app is unchanged.

Documentation can be found here  http://pro.arcgis.com/en/pro-app/arcpy/get-started/installing-python-for-arcgis-pro.htm, but additional information is included below.

Just one install of Python

There is no longer a secondary install of Python needed to run standalone scripts with Pro. In ArcGIS Pro 1.0, 1.1 and 1.2 a lightweight version of Python was used within the application, but to run standalone Python scripts you had to install a secondary “Python for ArcGIS Pro”.

At 1.3, ArcGIS Pro will install Python into the ArcGIS\Pro\bin\Python\envs\arcgispro-py3.  This Python is used within the ArcGIS Pro application and it is also the Python you use to run standalone scripts.

ArcGIS Pro 1.3 uses conda

Over the last few years, conda from Continuum analytics has gained significant traction in the Python community as a package and environment manager.

At 1.3, ArcGIS Pro includes conda, and the Python that you use with arcpy and ArcGIS Pro is in a conda environment named “arcgispro-py3”.  With an install of ArcGIS Pro for “all users”, arcgispro-py3 will be found at C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 . With an “only for me” install of ArcGIS Pro, arcgispro-py3 will be found at C:\Users\<user name>\appdata\local\Programs\ArcGis\Pro\bin\Python\envs\arcgispro-py3

Note: Conda is written and implemented with Python. So if you browse into ArcGIS\Pro\bin\Python\, you’ll see an install of Python. This is not the Python you’re looking for. You will not be able to use the arcpy module when running with this interpreter.

Questions & Answers

Note: All answers assume an install for “all users” (e.g., c:\Program Files\ArcGIS…), for user install, or install to custom location, simply update the path.

Q: How do I run a standalone script?

A: In previous releases of ArcGIS, a Python script was run from a command prompt like below:

c:\data> c:\python34\python.exe my_script.py

Now, you’ll run you the same script like below:

c:\data> "c:\Program Files\ArcGIS\Pro\bin\Python\scripts\propy.bat" my_script.py

Here propy.bat acts as a replacement for calling python.exe.

Q: Do you have to activate the conda environment to run a standalone script?

A: No, you can simply replace calls to

c:\python34\python.exe

with calls to

c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe

Or, you can also call ArcGIS\Pro\bin\python\scripts\propy.bat as mentioned above

Q: If I can run envs\arcgispro-py3\python.exe, why would I want to use propy.bat?

A: In future releases, you’ll be able to create, and change conda environments within ArcGIS Pro.  So, while working on a given Pro project you could create and use a new conda environment with specific sets and versions of Python packages. When running standalone scripts, if you run envs\arcgispro-py3\python.exe (ArcGIS Pro's default conda environment) it would NOT necessarily have the same set of packages as you are using inside the ArcGIS Pro application. This is where propy.bat comes in, it determines the application’s active conda environment and activates it in your standalone script.

Q: There is also proenv. What is the difference between propy & proenv?

A: As follows

• propy.bat : is a replacement for python.exe, it activates the ArcGIS Pro conda environment (arcgispro-py3) and then launches python, you can call propy.bat it with or without a py script as an argument.
  • With a script as argument which will:
    1. Activate the ArcGIS Pro conda environment
    2. Start Python
    3. Run the script specified
    4. Exit Python and the environment
  • If you can call propy.bat without argument, the conda environment will be activated, and python will be launched in interactive window mode (REPL).
• proenv.bat : activates the ArcGIS Pro environment, and leaves the user in at the windows command prompt.  From there you can easily call arcgispro-py3\python.exe by typing Python to start an interactive terminal session (REPL session), or type ‘python’ followed by the name of the script you wish to run.

C:\my_proj>"c:\Program Files\ArcGIS\Pro\bin\Python\Scripts\proenv.bat" 
Activating environment "c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py 
3"...  

[arcgispro-py3] c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3>cd c:\my_proj 
[arcgispro-py3] C:\my_proj>python  

Python 3.4.4 |Continuum Analytics, Inc.| (default, Feb 16 2016, 09:54:04) [MSC v 
.1600 64 bit (AMD64)] on win32 
Type "help", "copyright", "credits" or "license" for more information. 
>>> import arcpy 
>>>

Q: When in the command prompt, why does proenv.bat change my current working directory?

A: This behavior will be reviewed in the next release. To avoid this behavior you can launch Python Command Prompt (which calls proenv.bat) from the start menu instead of calling it from the Windows Command Prompt.

Q: Are admin permissions required to install conda packages?

A: If you have an "all users" install of ArcGIS Pro, the default install location is within the Program Files folder. To make changes to this folder location will require the applications to have elevated (admin) credentials. And since the Python environments is stored within the ArcGIS Pro application, then yes installing/removing/updating packages as well as creating and deleting environments will require elevated credentials. To get elevated credentials (admin privileges) from the Windows Application launcher you can right click on Command Prompt, or Python Command Prompt and choose Run as Administrator.

Q: When I launch the Python Command Prompt I get an error “Access is denied. Could Not Find proenv.txt.  Insufficient privileges.

A: This is a warning telling you that you cannot use conda commands to modify conda environments or packages. However, this does not impact using Python itself (including proenv & propy.bat). See previous question regarding administrative permissions.

Something else you’d like to ask? Something missing from this Q & A? Please go to https://community.esri.com/community/developers/gis-developers/python and post there.

Integrate Development Environments (IDEs)

As mentioned above, environments are a ‘best practice’ with the Python community. However you have to tell the IDE which environment it needs to use.

For information on using most of the popular IDEs with a conda environment see https://docs.continuum.io/anaconda/ide_integration

Do note that PyScripter does not work well with virtual environments, therefore cannot be used with Python included in ArcGIS Pro 1.3. Some good alternatives for PyScripter are

• PyCharm : a newer and extremely popular IDE with lots of bells and whistles.
• Python Tools for Visual Studio : a plugin for Microsoft’s very powerful IDE, requires MS Visual Studio (there is a free community edition).
• WingIDE :  another very popular full fledged IDE, there is a free community edition and a more full featured “pay” edition.
• Spyder : an IDE with a focus on Scientific Python.  Very easy to install with conda, simply

C:\data> "c:\Program Files\ArcGIS\Pro\bin\Python\Scripts\proenv.bat" 
C:\data> conda install spyder

Then you can launch spyder from the Windows start menu.

Below are examples showing how to update IDE settings for Python Tools for Visual Studio and PyCharm to use arcgispro-py3.

The examples use path to environment at C:\Program Files\ArcGIS\Pro\bin\python\envs\arcgispro-py3 which is where you’ll find it if you installed ArcGIS pro for “all users”. If you installed “Only for me” the default path is here C:\Users\ghis3080\AppData\Local\Programs\ArcGIS\Pro\bin\python\envs\arcgispro-py3

Python Tools for Visual Studio

Relevant document from Microsoft is here: https://github.com/Microsoft/PTVS/wiki/Python-Environments

Select Tools\Options then in the options go to the Python Tools\Environment Options in the tree view, click the Add Environment setting, for a name you can use “arcgispro-py3” (or a name of your choice).

Then add the following settings

• Path : C:\Program Files\ArcGIS\Pro\bin\python\envs\arcgispro-py3\python.exe
• Windows Path :  C:\Program Files\ArcGIS\Pro\bin\python\envs\arcgispro-py3\pythonw.exe

All subsequent settings are optional.

PyCharm Community edition

In File\Settings…, choose Project then Project Interpreter

Ignore the drop down list for Project Interpreter, and click the cog button to Add Local, and in the file browser pick C:\Program Files\ArcGIS\Pro\bin\python\envs\arcgispro-py3\python.exe

Note that the default for “me only” install is in C:\Users\<user_name>\AppData\Local\Programs\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 . This path may not be visible from the file browser.  This shortcut can be copy/pasted %LOCALAPPDATA%\Programs\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe

No other change should be required.

To run your script, right click it in the Project window, and choose either Run or Debug

Restart PyCharm for the Python Console to use the arcgispro-py3 environment.

For more information on conda in ArcGIS Pro, see:

• Python in ArcGIS Pro 1.3
• Harnessing the Power of Python in ArcGIS Using the Conda Distribution

[SimonJackson]

Does ArcGIS Pro 1.3 include jupyter? 

Struggling to find it

Installed it with below steps.  Anyone see any issues with my workflow below?

-----

Tasks performed so far:

• Uninstalled Anaconda (saw some comments about conflicts between multiple versions of Conda on same machine)
• Installed ArcGIS Pro 1.3
• Installed PyCharm
  • pointed default project settings to 'C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe'
  • In ArcGIS Pro --> Options --> Geoprocessing -.> Set Script Editor to pycharm64.exe
• Ran conda update --all  from CMD.  
  • Noticed this updates arcgispro package from 1.3-1 to 1.4-0
  • Also installs jupyter_client and jupyter_core
• Installed ArcGIS API for Python - conda install -c esri arcgis
• Can run juypter with jupyter notebook from a new CMD prompt.

[RobertHeilmayr]

Thanks Clinton. I tried your selective install of qt, but the 'conda install spyder' call continued to try to update qt to the troublesome version, giving me the original error. For the time being, I've gone ahead and followed Andy's lead by entering root and updating conda. Everything seems to be working.

[KimOllivier]

Yes! I got it working without going into Root with a few changes to your suggestions.

I am getting the fail when trying to install spyder and the usual error message from User 'ray' who must be a developer of qt.

Firstly I could not remove qt because it wasn't installed with ArcPro apparently.

I could also not install qt-4.8.7=vc10_4 because it is now qt-4.8.7-vc10_9 [vc10]

I got around this by using a wildcard to install qt : conda install qt-4.*

Then I was able to restart the spyder install: conda install spyder, which upgraded qt to qt-5.6.0-vc10_0 anyway so I didn't argue as it wasn't numpy or scipy

Now I can open spyder from the shortcut in the start menu.

The next stage is to work out how to get the ARcPro licencing fixed up, but it is a pubic holiday today here.

This is all distressingly complicated. I will have to set aside half a day for what goes wrong with Python setup in the new ArcPro training course.

[KimOllivier]

This did work for me but, using this roundabout process:

Could not uninstall qt, could not reinstall 4.8.7-vc10_4 because not found

I could however install a variation using conda install qt-4.* which found a later version

This then enabled spyder to install conda install spyder which then immediately upgraded qt to 5.6.0-vc10_0 so I didn't argue since it was not numpy or scipy

So now spyder will open from the shortcut in the start menu.

I am staying with Windows 7 BTW. So much easier.

AVG antivirus objects to Python so I tell it to ignore it.

Thanks Clinton, you gave me enough hints to blunder through without resorting to root or reinstalling the operating system.

[KimOllivier]

Yes we all got this!

Install qt yourself: conda install qt-4.*

Then try the spyder install again conda install spyder

This will trigger yet another upgrade of qt to 5.6.0, but let it run to completion. You have to install the 4.* version first.

I found that spyder then opened from the start menu shortcut.

So much for conda handling dependencies! Who would have thought of this?