With ArcGIS Pro 1.3, there are some significant changes to Python. These changes primarily affect how you run standalone scripts with Python. The experience of Python within the ArcGIS Pro app is unchanged.
Documentation can be found here http://pro.arcgis.com/en/pro-app/arcpy/get-started/installing-python-for-arcgis-pro.htm, but additional information is included below.
There is no longer a secondary install of Python needed to run standalone scripts with Pro. In ArcGIS Pro 1.0, 1.1 and 1.2 a lightweight version of Python was used within the application, but to run standalone Python scripts you had to install a secondary “Python for ArcGIS Pro”.
At 1.3, ArcGIS Pro will install Python into the ArcGIS\Pro\bin\Python\envs\arcgispro-py3. This Python is used within the ArcGIS Pro application and it is also the Python you use to run standalone scripts.
Over the last few years, conda from Continuum analytics has gained significant traction in the Python community as a package and environment manager.
At 1.3, ArcGIS Pro includes conda, and the Python that you use with arcpy and ArcGIS Pro is in a conda environment named “arcgispro-py3”. With an install of ArcGIS Pro for “all users”, arcgispro-py3 will be found at C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 . With an “only for me” install of ArcGIS Pro, arcgispro-py3 will be found at C:\Users\<user name>\appdata\local\Programs\ArcGis\Pro\bin\Python\envs\arcgispro-py3
Note: Conda is written and implemented with Python. So if you browse into ArcGIS\Pro\bin\Python\, you’ll see an install of Python. This is not the Python you’re looking for. You will not be able to use the arcpy module when running with this interpreter.
Questions & Answers
Note: All answers assume an install for “all users” (e.g., c:\Program Files\ArcGIS…), for user install, or install to custom location, simply update the path.
Q: How do I run a standalone script?
A: In previous releases of ArcGIS, a Python script was run from a command prompt like below:
c:\data> c:\python34\python.exe my_script.py
Now, you’ll run you the same script like below:
c:\data> "c:\Program Files\ArcGIS\Pro\bin\Python\scripts\propy.bat" my_script.py
Here propy.bat acts as a replacement for calling python.exe.
Q: Do you have to activate the conda environment to run a standalone script?
A: No, you can simply replace calls to
c:\python34\python.exe
with calls to
c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe
Or, you can also call ArcGIS\Pro\bin\python\scripts\propy.bat as mentioned above
Q: If I can run envs\arcgispro-py3\python.exe, why would I want to use propy.bat?
A: In future releases, you’ll be able to create, and change conda environments within ArcGIS Pro. So, while working on a given Pro project you could create and use a new conda environment with specific sets and versions of Python packages. When running standalone scripts, if you run envs\arcgispro-py3\python.exe (ArcGIS Pro's default conda environment) it would NOT necessarily have the same set of packages as you are using inside the ArcGIS Pro application. This is where propy.bat comes in, it determines the application’s active conda environment and activates it in your standalone script.
Q: There is also proenv. What is the difference between propy & proenv?
A: As follows
C:\my_proj>"c:\Program Files\ArcGIS\Pro\bin\Python\Scripts\proenv.bat" Activating environment "c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py 3"... [arcgispro-py3] c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3>cd c:\my_proj [arcgispro-py3] C:\my_proj>python Python 3.4.4 |Continuum Analytics, Inc.| (default, Feb 16 2016, 09:54:04) [MSC v .1600 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import arcpy >>>
Q: When in the command prompt, why does proenv.bat change my current working directory?
A: This behavior will be reviewed in the next release. To avoid this behavior you can launch Python Command Prompt (which calls proenv.bat) from the start menu instead of calling it from the Windows Command Prompt.
Q: Are admin permissions required to install conda packages?
A: If you have an "all users" install of ArcGIS Pro, the default install location is within the Program Files folder. To make changes to this folder location will require the applications to have elevated (admin) credentials. And since the Python environments is stored within the ArcGIS Pro application, then yes installing/removing/updating packages as well as creating and deleting environments will require elevated credentials. To get elevated credentials (admin privileges) from the Windows Application launcher you can right click on Command Prompt, or Python Command Prompt and choose Run as Administrator.
Q: When I launch the Python Command Prompt I get an error “Access is denied. Could Not Find proenv.txt. Insufficient privileges.
A: This is a warning telling you that you cannot use conda commands to modify conda environments or packages. However, this does not impact using Python itself (including proenv & propy.bat). See previous question regarding administrative permissions.
Something else you’d like to ask? Something missing from this Q & A? Please go to https://community.esri.com/community/developers/gis-developers/python and post there.
Integrate Development Environments (IDEs)
As mentioned above, environments are a ‘best practice’ with the Python community. However you have to tell the IDE which environment it needs to use.
For information on using most of the popular IDEs with a conda environment see https://docs.continuum.io/anaconda/ide_integration
Do note that PyScripter does not work well with virtual environments, therefore cannot be used with Python included in ArcGIS Pro 1.3. Some good alternatives for PyScripter are
C:\data> "c:\Program Files\ArcGIS\Pro\bin\Python\Scripts\proenv.bat" C:\data> conda install spyder
Then you can launch spyder from the Windows start menu.
Below are examples showing how to update IDE settings for Python Tools for Visual Studio and PyCharm to use arcgispro-py3.
The examples use path to environment at C:\Program Files\ArcGIS\Pro\bin\python\envs\arcgispro-py3 which is where you’ll find it if you installed ArcGIS pro for “all users”. If you installed “Only for me” the default path is here C:\Users\ghis3080\AppData\Local\Programs\ArcGIS\Pro\bin\python\envs\arcgispro-py3
Python Tools for Visual Studio
Relevant document from Microsoft is here: https://github.com/Microsoft/PTVS/wiki/Python-Environments
Select Tools\Options then in the options go to the Python Tools\Environment Options in the tree view, click the Add Environment setting, for a name you can use “arcgispro-py3” (or a name of your choice).
Then add the following settings
All subsequent settings are optional.
PyCharm Community edition
In File\Settings…, choose Project then Project Interpreter
Ignore the drop down list for Project Interpreter, and click the cog button to Add Local, and in the file browser pick C:\Program Files\ArcGIS\Pro\bin\python\envs\arcgispro-py3\python.exe
Note that the default for “me only” install is in C:\Users\<user_name>\AppData\Local\Programs\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 . This path may not be visible from the file browser. This shortcut can be copy/pasted %LOCALAPPDATA%\Programs\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe
No other change should be required.
To run your script, right click it in the Project window, and choose either Run or Debug
Restart PyCharm for the Python Console to use the arcgispro-py3 environment.
For more information on conda in ArcGIS Pro, see:
David, very nice summary and a heads up for those still in pro 1.2
Thanks Dan.
We are aware that this will change how people interact with Python. It's a change and of course that will cause some initial pain. I hope this spells out what to expect and how to adapt.
-Dave
I agree with Dan, very helpful information. Will this information also/eventually be added to the ArcGIS Pro documentation over at desktop.arcgis.com? I looked there at first and didn't find what I was looking for, then I stumbled onto this post here.
Hi Joshua,
Absolutely, yes
Sharing in the ArcGIS Pro place.
David... I have read through the materials and I just have one concern. It appears that Pro 1.3 has a definite preferred installation path. Is that one esri is recommending, or can we install it elsewhere (on the same drive, but in the root) and the same for Conda. I just hate having anything buried in paths with spaces nested several layers deep if I don't need to.
UPDATE
Yea... you can't change the install directory, but everything went smoothly. Just make sure you run everything as administrator to save yourself a few steps.
Spyder installed
Hi Dan Patterson,
My understanding is that the standard installation locations are recommended in part because of how the Windows security model is set up – for example, Python at 3.5 similarly installs itself into Program Files as addressed in this bug. That said, I'm not aware of any specific limitations you would hit by installing into another location, and have used and tested ArcGIS Pro 1.3 in other top-level locations for a global install.
Hope that helps, Shaun
Thanks Shaun... I just updated my post... going to give it a whirl
Q: Are admin permissions required to install conda packages?
A: If you have an "all users" install of ArcGIS Pro, the default install location is within the Program Files folder. To make changes to this folder location will require the applications to have elevated (admin) credentials. And since the Python environments is stored within the ArcGIS Pro application, then yes installing/removing/updating packages as well as creating and deleting environments will require elevated credentials. To get elevated credentials (admin privileges) from the Windows Application launcher you can right click on Command Prompt, or Python Command Prompt and choose Run as Administrator.
It's a bit of a pain to use conda environments within an "all users" install without admin rights, but it can be done by using fullpaths
> conda info --envs # conda environments: # arcgispro-py3 C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 root * C:\Program Files\ArcGIS\Pro\bin\Python > conda config --add envs_dirs D:\conda > conda create -y -p D:\conda\test python=3 etc... > conda info --envs # conda environments: # arcgispro-py3 C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 test D:\conda\test root * C:\Program Files\ArcGIS\Pro\bin\Python > activate test No environment named "test" exists in C:\Program Files\ArcGIS\Pro\bin\Python\envs, or is not a valid conda installation directory. > activate D:\conda\test [test] >
Spyder works great... now I set the path to the executable which launches Spyder to
c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\pythonw.exe
in the Project, Options, Geoprocessing, Script editor to it, assuming it would launch Spyder as well, when I right-click on one of my (simple) script tool and select edit. When I use Pythonwin, in pro 1.2, this process worked ( a different path obviously).
Is there something about the condas environment that allows me to run Spyder as designed, but not as an editor within PRO 1.3? It isn't a biggie, since I normally have an IDE of some sort open anyway.
Hi Dan,
Can you try pointing the script editor to the Spyder executable itself? For a default global install, that should be:
C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Scripts\spyder.exe
I tried it out locally, and it at least worked here. At Pro 1.3, we no longer register ourselves as the default run handler for Python scripts, which may make a difference for how to configure this. Note that some editors are having issues (such as IDLE), and this may be related, there's currently a bug assigned to the issue (BUG-000097665).
Cheers, Shaun
Ok Shaun, I will give that a try... I was getting side tracked on a win32 with one of my toolboxes that I brought into 1.3 that was fine in 1.2 error (probably associated with my use of pythonwin).
I still have to check out the anacona package a bit more to see what is there... I am hoping it is the full slate, if not, I need to see what was omitted an what was added. Thanks for the tip. I will document and post soon.
UPDATE
Worked like a charm... for the visually challenged (aka me )
Or the folder version
Do we have any information on the libraries it comes with by default? Is it what is shown here?
What is conda?—ArcPy Get Started | ArcGIS for Desktop
From the look of it we can get Anaconda libraries such as XlsxWriter, if we use the conda procedures.
Is this correct?
Apparently it is supposed to be Excel plug-ins for Anaconda | Continuum Analytics: Documentation
But I have only found xlrd and xlwt in the site packages.
I am putting everything in a blog post so I can keep a track of it. I will post the link when I get it started
Update
posted here Anaconda, Spyder and ArcGIS PRO
Hey Dan,
Yes, but xlsxwriter is one that comes with Anaconda and integrates directly with pandas. My first encounter was when one of panda's projects just started shooting off pains from a lack of dependencies. Thanks for the follow up post.
David
Adding to Dan Patterson's answer, yes you can use conda to install xlswriter. To do so, you can open a new Python Command Prompt window from the "ArcGIS Pro" program group, then execute:
conda install xlsxwriter
As Dan mentioned, we do include xlwt which provides similar functionality as xlsxwriter, but having both as options is nice.
Unlike the Anaconda distribution, we ship a minimal set of packages with ArcGIS Pro, to keep the install size down. Here's the list of included packages as shown by conda list:
arcgispro | 1.3 |
colorama | 0.3.6 |
future | 0.15.2 |
matplotlib | 1.4.3 |
msvc_runtime | 1.0.1 |
nose | 1.3.7 |
numpy | 1.9.3 |
openssl | 1.0.2h |
pandas | 0.17.1 |
pip | 8.1.1 |
py | 1.4.31 |
pyparsing | 2.1.1 |
pypdf2 | 1.25.1 |
pytest | 2.9.1 |
python | 3.4.4 |
python-dateutil | 2.5.3 |
pytz | 2016.4 |
requests | 2.9.1 |
scipy | 0.16.1 |
setuptools | 20.7.0 |
six | 1.10.0 |
sympy | 0.7.6.1 |
vs2010_runtime | 10.00.40219.1 |
wheel | 0.29.0 |
xlrd | 0.9.4 |
xlwt | 1.0.0 |
Hey Shaun,
That is still really awesome that we can just use the conda install functionality really easily. That said it would be nice if you included all the panda dependencies/interop packages (which I think xlsxwriter falls into).
Are there any plans to create an install list or script that would allow someone to match the anaconda distribution if they wanted to? Just to make it a little easier to incorporate.
David
I would advise on an adhoc add all... but here is one source of the list... make sure you pick the correct python version
I just installed xlsxwriter and updated my blog post with a visual of the install process.
Is there a command to add all? I have seen script base custom installs, that might be what you mean by ad hoc. I was just thinking they could make it a little easier. 😃
Thanks, I think I will just work on it at home then. I am sure I can list them out and get something written pretty quickly.
Here's a script that downloads a full list of current Anaconda packages:
GitHub - scw/anaconda-packages: Create an installable list of current Anaconda packages
Running the anaconda-packages.py script will create a list of the currently installable packages, in this case in the anaconda-packages-py34.txt which is included for illustration. This list can then be installed from conda with:
conda install --file anaconda-packages-py34.txt
Cheers,
Shaun
Here is a tutorial for setting up Anaconda with ArcGIS Python environment. Might be helpful for beginners of conda.
https://gisday.wordpress.com/2016/07/18/setting-up-anaconda-pysal-with-arcgis-python-environment/
Whats the recommended approach to running the Pro Python environment and the new arcgis python API on the same machine. I went ahead and just put it in the same pro 1.3 environtment, the help implies that this is OK, but i think that was pre 1.3. I seem to have ended up breaking things, arcpy seems to think its running outside the arcpro environment now. the Arcgis python api uses a newer version of python and some different packages.
Should I setup a second environment under the pro version of anaconda, or should i install an entirely separate anaconda 3 and set it up there.
thx
Dom
Hi Dominic,
The easiest way to run the ArcGIS Python API in a Pro environment is to follow the installation instructions for the API in a command line with the current Pro environment activated. Then, when the required packages (ie: Jupyter) are installed they will be put in the env in which ArcPY is currently associated. If you are seeing a message when importing arcpy that states you're not in the active Pro environment, run 'proenv.bat' which will be in your <ProInstallation>/bin/Python/Scripts directory and note which environment it activates. Then use 'proswap.bat' to swap the active Pro environment to the one that you'd like to use.
I would caution against a second Anaconda installation, there are lots of environmental variables that it uses and a second installation can cause wires to cross, so to speak.
If there are any further complications please let me know and we'll get to the bottom of it!
Thanks for the link! Nice post.
This has been my experience as well. I have had conflicts between my anaconda environment and Melinda Morang's tools before (might be a SQLite conflict).
Edit:
On a second computer where I installed Pro, I also found that a 2.7 Anaconda distribution can also interfere with installing 3rd Party Packages. If the details matter to anyone, the error was usually related to the numpy library not be recognized or found in conda. It would often suggest doing a search with binstar to find the library.
This is exactly what I am looking for. I will test it out when I get a chance. Thanks!
Hi all,
I just installed ArcGIS Pro 1.3 and then spyder using the instructions in the original article from David Wynne ('conda install spyder'). The installation seems to have gone without any problem but I cannot open spyder (neither from the Start menu nor from an interactive command line with the arcgispro-py3 environment, neither as admin nor as a normal user).
When I try to start spyder, the spyder splashscreen shows up and when the status bar gets to "Loading IPython console...", an error message appears saying "python.exe has stopped working".
The installed versions are conda (4.0.6), spyder (2.3.9), qtconsole (4.2.1), jupyter (1.0.0).
Here is how my start menu looks like
Do you know what's going on? How to fix this?
Filip.
worked for me... Anaconda, Spyder and ArcGIS PRO
See if the instructions Continuum provides for running Spyder help.
Cheers, Shaun
Hi Shaun, thanks for the link but unfortunately none of the steps resolved the problem.
I noticed I can't run Spyder just by typing 'Spyder' into a command line (command not recognized), I have to be in the Python Command Prompt that comes with ArcGIS Pro with [arcgispro-py3] environment enabled. It that normal?
Also, the only ".spyder" directory I found was 'C:\Users\<myusername>\.spyder2-py3'. Deleting it did not help.
Does anyone else have the same problem?
Filip.
step 1 in the link I sent... I made a desktop link, no need to command line anything if that would be easier
Hi Dan, I did read through it. I think I understand what you did there, but I get the same error when I try to edit a script from ArcGIS Pro directly.
Still no joy.
F.
Hi Filip,
Yes, it is normal to not be able to run spyder from a default command line prompt – we've intentionally sandboxed our Python installation and don't modify the PATH of the machine. If you do want to explicitly add the Python environment to your PATH, you can add the ArcGIS\bin\Python\envs\arcgispro-py3\Scripts directory to the path, and it should work like any other entry.
Could you post the results of:
conda info
On your machine? Also, you may want to try creating a new environment with just Spyder, and seeing if it works there – it adds a new shortcut under Anaconda 3 (64-bit) > Spyder (env-name) you can try opening. To create the environment, try something like this:
conda create --name spyder-test python=3.4 spyder
If that doesn't work, that helps isolate it as a conda issue, since that environment should have no interaction with ArcGIS Pro.
Hi Shaun,
This was my conda info as of this morning.
I then tried a few experiments along what is suggested in Setting up Anaconda, PySAL with ArcGIS Python environment | LidarBlog.com
When I created a new python 3.4 environment with spyder, I could not run spyder either. I had to 'conda remove qt --name py34' and 'conda install qt=4.8.7=vc10_4' as indicated at Spyder crashes at startup in new conda environment python 3.4 on Windows 8.1 x64 · Issue #2858 · spy...Then I was able to run spyder.
I copied the [arcgispro-py3] enviornment as [arcpro] environment and did 'conda remove qt --name py34' and 'conda install qt=4.8.7=vc10_4' there. That allowed me to open spyder, import arcpy, and run arcpy.Describe on a shapefile. When I tried to run any geoprocessing tool, python crashed again saying "python.exe has stopped working".
It would be good to hear if (and how many) other people managed to install and run spyder just by running 'conda insall spyder' in the [arcgispro-py3] environment when they first installed ArcPro 1.3.
Filip.
Filip,
Sorry to hear you continue to have problems. It can be tricky to debug some of these issues, in part because we're probably using different versions of Conda and Python if you've upgraded. Do you get a crash dump, or any further details if you click "Debug the program"?
Filip,
Sorry to hear you continue to have problems. It can be tricky to debug some of these issues, in part because we're probably using different versions of Conda and Python if you've upgraded. Do you get a crash dump, or any further details if you click "Debug the program"?
Hello Filip Král, I was able to reproduce this problem, running the following commands within the active env solved it for me:
conda remove qt -y
conda install qt=4.8.7=vc_10_4 -y
Please give it a try and let us know if it helps.
Hi Clinton, thanks for looking into this.
In my [arcgispro-py3] environment, 'conda remove qt -y' went fine but then 'conda install qt=4.8.7=vc_10_4 -y' appears to have removed Python 3.4.4-4 and I had to 'conda install python=3.4.4' before I could run python again.
When I now run 'spyder' or 'spyder --new-instance', a message is printed in the command line saying 'spyder failed to create process'.
I've resorted to writing my code with Python 2.7 and PyScripter against smaller data and ArcGIS for Desktop and then executing it from the ArcGIS Pro Python Command Prompt.
Filip.
I'm trying to install spyder in the conda environment and am getting an error as conda installs all the dependencies. In particular, qt seems to be causing problems:
Error: ERROR: placeholder 'C:/Users/ray/qtb-3.4-64/_b_env' too short in: qt-5.6.2-vc10_0
Has anyone else run into this problem? I thought that the fact that the default environment was buried in a pretty deep path might be causing the problem (many more characters than the placeholder path), but moving the ArcGIS/Pro/bin folder to the root directory didn't solve the problem. I'm open to any suggestions to get spyder up and running.
Hello Robert,
It looks like there's a verified bug with conda for installing QT. The reported fix is to upgrade conda, you can install the latest version of conda with the command:
conda update conda
Let us know if that fixes the issue for you.
Cheers, Shaun
I just installed ArcGIS Pro 1.4 (USGS build - sorry if this is off-topic) and I am trying to install spyder to work with conda, like Robert. I get the same error. When I try
conda update conda
I get
Error: package 'conda' is not installed in c:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3e
looks like maybe I need root but I can't activate it:
>conda install -f coda Fetching package metadata: .... .Solving package specifications: ......... Error: 'conda' can only be installed into the root environment >activate root No environment named "root" exists in c:\ArcGIS\Pro\bin\Python\envs\, or is not a valid conda installation directory. >conda info --envs # conda environments: # arcgispro-py3 * c:\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 root c:\ArcGIS\Pro\bin\Python
so confused
ok found what's hopefully the first part of the answer - 'deactivate' actually gets me into the root environment.
Hi Andy,
You can deactivate your current environment, change directories into the <Pro Install Path>\bin\Python\Scripts folder, then just run 'activate' with no env name and you will be in the root environment, denoted by [Python] before the prompt.
That said, its possible to break Python and thus your Pro installation by doing so. As functionality around Conda matures we will no longer be bound to specific Python/numpy/scipy versions per ArcGIS Pro release. However as it stands we have to be overly protective of accidental updates of certain packages.
For this specific issue with spyder and qt, try the following without updating conda, within your environment:
conda remove qt -y
conda install qt=4.8.7=vc_10_4 -y
Please give it a try and let us know if it helps.
Thanks for the quick reply. I unfortunately found my way into root before I
saw your message and fixed my issues with conda update conda followed by
conda install spyder. Hopefully python isn't broken now in ArcGIS Pro.
Andy
It should be okay as long as the version is within 4.0.* - Updating to a newer version than that may cause issues with the Python Backstage UI.
Is there a way to force PyCharm to use the "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe" for all new projects, as opposed to just applying it at the project level each time?
In PyCharm there are two settings dialogs, Settings and Default Settings - go into Default Settings>Project Interpreter (make sure it says 'For default Project' at the top of the dialog) and set the interpreter there. Then all subsequent new projects should use it. Existing projects you'll have to set it per-project as far as I can see.
Ah, of course. My bad. Time for holidays...
Thanks!