Code working in ArcGIS Pro Python Window but not in Spyder

3858
32
07-30-2019 06:16 PM
ThomasBlankinship
New Contributor

Like many of you, I want to be able to edit and execute python code from Spyder instead of ArcGIS's Python Window. I am running into some problems and hoping that someone can help me troubleshoot what exactly is going on. I noticed that some old scripts that I had written would no longer execute in Spyder but when the code was copied into ArcGIS Pro's Python Window the code worked just fine. After messing around with it for quite some time, uninstalling and reinstalling both ArcGIS Pro and Anaconda3 Navigator (3.7 64-bit), to not avail I decided to roll uninstall both again, delete all traces of them from my system, delete registry keys, etc. to make sure it wasn't the fault of something latent on my computer after uninstalling.

I just got everything reinstalled and am running into the exact same problem. Again, the code will work when copy pasted into the Python Window in ArcGIS Pro, but not in Spyder. Here is how I installed it after my clean uninstall:

1) Installed ArcGIS 2.3

2) Installed Patched 2.3.1, 2.3.2, and 2.3.3 in that order (I'm wary of 2.4 and know my code was working on 2.3)

3) Opened ArcGIS, signed in, went to Project --> Python --> Manage Environments --> Clone Environment

4) Once the Cloned Environment (named arcgispro-py3-clone) finished installing I closed ArcGIS Pro and restarted it

5) Went back into Project --> Python --> Manage Environments and changed the environment to the cloned environment

6) Checked that my sample code worked in the cloned environment in the Python Window, it did

7) Installed the spyder package (which installed all of its dependencies)

8) Restarted ArcGIS Pro, checked that spyder package was installed, tried code again in Python Window, it worked

9) Installed Anaconda3 -> unchecked Anaconda3 as default Python 3.7, did not add to path

10) Opened Anaconda3 and checked for cloned arcgispro-py3 environment, it was there

11) Restarted computer for good measure

12) In Anaconda3, switched to arcgispro-py3-clone environment and launched spyder

13) Opened and ran my test code

Here is the error given to me by Spyder:

runfile('G:/My Drive/Python Scripts/polyline_to_polygon.py', wdir='G:/My Drive/Python Scripts')
Traceback (most recent call last):

File "<ipython-input-1-210cd50e08ec>", line 1, in <module>
runfile('G:/My Drive/Python Scripts/polyline_to_polygon.py', wdir='G:/My Drive/Python Scripts')

File "C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
execfile(filename, namespace)

File "C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "G:/My Drive/Python Scripts/polyline_to_polygon.py", line 27, in <module>
ap.CopyFeatures_management(vtx_arr, vtx_output)

File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\management.py", line 2624, in CopyFeatures
raise e

File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\management.py", line 2621, in CopyFeatures
retval = convertArcObjectToPythonObject(gp.CopyFeatures_management(*gp_fixargs((in_features, out_feature_class, config_keyword, spatial_grid_1, spatial_grid_2, spatial_grid_3), True)))

File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 498, in <lambda>
return lambda *args: val(*gp_fixargs(args, True))

RuntimeError: Object: Error in executing tool

I'm hoping I left out a step or something of that nature akin to forgetting to copy the batch files in ArcMap, but otherwise I have no clue why this wouldn't be working. Thank you all for your time and help.

0 Kudos
32 Replies
DanPatterson_Retired
MVP Esteemed Contributor

Thomas

The ArcGIS Pro environment is currently only supporting python 3.6.8..

I suspect that is the root of the problem.

In fact, if you try a --no-pin, esri's environment doesn't support 3.7, only 3.6.9  (btw, 3.8 will be coming along soon)

(arcgispro-py3) C:\arc_pro\bin\Python\envs\arcgispro-py3>conda update python --no-pin --dry-run
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment C:\arc_pro\bin\Python\envs\arcgispro-py3:

The following packages will be UPDATED:

    python: 3.6.8-h9f7ef89_7 --> 3.6.9-h5500b2f_0


(arcgispro-py3) C:\arc_pro\bin\Python\envs\arcgispro-py3>

If you have a need to test out python 3.7, I would suggest separating your Arc* stuff from your python stuff until esri upgrades their packages.  Minor upgrades x.x.Y tend not to cause issues, but x.Y upgrades I have found to be problematic at times.  So --no-pin and --dry-run should always be examined first.

ThomasBlankinship
New Contributor

Anaconda is 3.7. But the environment I am using within Anaconda is 3.6.6 as it is a clone of the default environment within ArcGIS Pro 2.3.3. Below is the original environment after running conda update python --no-pin --dry-run:

C:\Program Files\ArcGIS\Pro\bin\Python\Scripts>conda update python --no-pin --dry-run
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone:

The following packages will be UPDATED:

    python:         3.6.6-hea74fb7_0       --> 3.6.9-h5500b2f_0
    sqlite:         3.25.2-hfa6e2cd_0      --> 3.29.0-he774522_0
    vc:             14-h21ff451_11         --> 14.1-h0510ff6_4
    vs2015_runtime: 14.0.25420-0      esri --> 14.15.26706-h3a45250_4

The following packages will be DOWNGRADED:

    freetype:       2.8-h51f8f2c_1    esri --> 2.8-vc14_0             esri [vc14]


C:\Program Files\ArcGIS\Pro\bin\Python\Scripts>

And here is that same command run on my clone environment:

(arcgispro-py3-clone) C:\Users\Tab5a>conda update python --no-pin --dry-run
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone:

The following packages will be UPDATED:

    python:         3.6.6-hea74fb7_0       --> 3.6.9-h5500b2f_0
    sqlite:         3.25.2-hfa6e2cd_0      --> 3.29.0-he774522_0
    vc:             14-h21ff451_11         --> 14.1-h0510ff6_4
    vs2015_runtime: 14.0.25420-0      esri --> 14.15.26706-h3a45250_4

The following packages will be DOWNGRADED:

    freetype:       2.8-h51f8f2c_1    esri --> 2.8-vc14_0             esri [vc14]


(arcgispro-py3-clone) C:\Users\Tab5a>

Note that my cloned environment is located in C:\Users\Tab5a\AppData\Local\ESRI\conda\envs

Also, not sure if it matters at all, the cloned environment is not in the C:\Users\Tab5a\Anaconda3\envs but does show up in Anaconda Navigator as expected. Any ideas?

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

If you scroll around using Windows File Explorer, are these folders there?

C:\Your_Install_Path\Resources\ArcPy

C:\Your_Install_Path\bin\Python\envs\Your_Clone_Env\Lib\site-packages\arcgis

C:\Your_Install_Path\bin\Python\envs\Your_Clone_Env\Lib\site-packages\spyder

If not, then Arc* stuff isn't there and spyder may or may not be installed

0 Kudos
ThomasBlankinship
New Contributor

All of those folders are there - install paths are different though. "...\Resorces\Arcpy" is in:

"C:\Program Files\ArcGIS\Pro\Resources\ArcPy"

But the clone is not in the following program files folder location:

"C:\Program Files\ArcGIS\Pro\bin\Python\envs"

It is instead located at "...AppData\Local\ESRI\conda\envs\arcgispro-py3-clone". The packages you asked about are installed at:

"C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\Lib\site-packages\arcgis"

"C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\Lib\site-packages\spyder"

Note though that I also have two "root" environments. One that was installed with ArcGIS Pro and one with Anaconda. Not sure if this would cause an issue or what and not sure at all if the following is relevant, but hey, guess it can't hurt. I figured there might be some kind of conflict going on between these two. For some reason "conda" commands do not work for the Anaconda installed base (root) environment unless executed from the Anaconda Prompt. In addition, the original "arcgispro-py3" environment does not have a "conda" executable (though I think it copied the "conda" executable from the "C:\Program Files\ArcGIS\Pro\bin\Python\Scripts" directory when it was cloned.)

Running in PowerShell in the various ...\Scripts directories

Note that the following were done with "arcgispro-py3-clone" active in ArcGIS Pro. The active environment in Anaconda Navigator does not affect the active (*) environment. Only the active environment in ArcGIS Pro matters.

Running "conda info --envs" from "C:\Program Files\ArcGIS\Pro\bin\Python\Scripts". This is the root environment installed by ArcGIS Pro.

PS C:\Program Files\ArcGIS\Pro\bin\Python\Scripts> .\conda info --envs
# conda environments:
#
arcgispro-py3-clone   *  C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone
arcgispro-py3            C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3
root                     C:\Program Files\ArcGIS\Pro\bin\Python

PS C:\Program Files\ArcGIS\Pro\bin\Python\Scripts>

Running "conda info --envs" from "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Scripts." This is the original environment installed by ArcGIS Pro that was cloned.

PS C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Scripts> .\conda info --envs
.\conda : The term '.\conda' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ .\conda info --envs
+ ~~~~~~~
    + CategoryInfo          : ObjectNotFound: (.\conda:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Scripts>

Note there is no "conda" executable in the folder for whatever reason. Maybe it references the Python\Scripts folder above? Regardless, shouldn't be my problem because...

Running "conda info --envs" from "C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\Scripts." This is the clone of the original environment installed by ArcGIS Pro.

PS C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\Scripts> .\conda info --envs
# conda environments:
#
arcgispro-py3-clone   *  C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone
arcgispro-py3            C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3
root                     C:\Program Files\ArcGIS\Pro\bin\Python

PS C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\Scripts>

Note that even though the "conda" executable was not in the original environment, it showed up in the cloned one.

Running "conda info --envs" from "C:\Users\Tab5a\Anaconda3\Scripts." This is the base (root) environment installed with Anaconda. I get the following error message.

PS C:\Users\Tab5a\Anaconda3\Scripts> .\conda info --envs

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\conda\exceptions.py", line 1062, in __call__
        return func(*args, **kwargs)
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\conda\cli\main.py", line 84, in _main
        exit_code = do_call(args, p)
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\conda\cli\conda_argparse.py", line 80, in do_call
        module = import_module(relative_mod, __name__.rsplit('.', 1)[0])
      File "C:\Users\Tab5a\Anaconda3\lib\importlib\__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 728, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\conda\cli\main_info.py", line 19, in <module>
        from ..core.index import _supplement_index_with_system
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\conda\core\index.py", line 9, in <module>
        from .package_cache_data import PackageCacheData
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\conda\core\package_cache_data.py", line 15, in <module>
        from conda_package_handling.api import InvalidArchiveError
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\conda_package_handling\api.py", line 3, in <module>
        from libarchive.exception import ArchiveError as _LibarchiveArchiveError
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\libarchive\__init__.py", line 1, in <module>
        from .entry import ArchiveEntry
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\libarchive\entry.py", line 6, in <module>
        from . import ffi
      File "C:\Users\Tab5a\Anaconda3\lib\site-packages\libarchive\ffi.py", line 48, in <module>
        libarchive = ctypes.cdll.LoadLibrary(libarchive_path)
      File "C:\Users\Tab5a\Anaconda3\lib\ctypes\__init__.py", line 434, in LoadLibrary
        return self._dlltype(name)
      File "C:\Users\Tab5a\Anaconda3\lib\ctypes\__init__.py", line 356, in __init__
        self._handle = _dlopen(self._name, mode)
    TypeError: LoadLibrary() argument 1 must be str, not None

`$ C:\Users\Tab5a\Anaconda3\Scripts\conda-script.py info --envs`


An unexpected error has occurred. Conda has prepared the above report.

Upload did not complete.
PS C:\Users\Tab5a\Anaconda3\Scripts>

Running in Terminal from Anaconda Navigator

Note that for the following the active environment in ArcGIS Pro made no difference. The active environment in ArcGIS Pro does not affect the active environment.

As to the two "root" environments, when I open Anaconda and switch to the "base (root)" environment, open a terminal, and run "conda info --envs":

(base) C:\Users\Tab5a>conda info --envs
# conda environments:
#
base                  *  C:\Users\Tab5a\Anaconda3
                         C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone


(base) C:\Users\Tab5a>

When I switch to the "arcgispro-py3-clone" environment in Anaconda, open a terminal, and run "conda info --envs":

(arcgispro-py3-clone) C:\Users\Tab5a>conda info --envs
# conda environments:
#
arcgispro-py3-clone   *  C:\Users\Tab5a\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone
arcgispro-py3            C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3
root                     C:\Program Files\ArcGIS\Pro\bin\Python


(arcgispro-py3-clone) C:\Users\Tab5a>

Note also that the original environment "arcgispro-py3" does not show up in Anaconda.

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

Is spyder running the correct Spyder? (run from the IPython console in Spyder).  

I don't use clones so I only have one python so I can't test another

import sys
sys.executable
'C:\\arc_pro\\bin\\Python\\envs\\arcgispro-py3\\pythonw.exe'‍‍‍

To change

How to change python version in anaconda spyder - Stack Overflow 

0 Kudos
ThomasBlankinship
New Contributor

This is in Spyder.

Python 3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 11:27:44) [MSC v.1900 64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.

IPython 7.7.0 -- An enhanced Interactive Python.

import sys

sys.executable
Out[2]: 'C:\\Users\\Tab5a\\AppData\\Local\\ESRI\\conda\\envs\\arcgispro-py3-clone\\pythonw.exe'

Any other way to check what is being used by by the Python Window in ArcGIS Pro? sys.executable returns:

import sys
sys.executable
'C:\\Program Files\\ArcGIS\\Pro\\bin\\ArcGISPro.exe'

Which I find odd based on https://docs.python.org/3/library/sys.html which says that sys.executable "returns a string giving the absolute path of the executable binary for the Python interpreter."

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor
import sys
sys.executable
'C:\\arc_pro\\bin\\ArcGISPro.exe'

sys.base_exec_prefix
'C:\\arc_pro\\bin\\Python\\envs\\arcgispro-py3'

sys.exec_prefix
'C:\\arc_pro\\bin\\Python\\envs\\arcgispro-py3'

Have to add this to my "reasons not to clone"

/blogs/dan_patterson/2018/12/28/clone 

0 Kudos
ThomasBlankinship
New Contributor

I get this:

import sys
sys.executable
'C:\\Program Files\\ArcGIS\\Pro\\bin\\ArcGISPro.exe'
sys.base_exec_prefix
'C:\\Users\\Tab5a\\AppData\\Local\\ESRI\\conda\\envs\\arcgispro-py3-clone'
sys.exec_prefix
'C:\\Users\\Tab5a\\AppData\\Local\\ESRI\\conda\\envs\\arcgispro-py3-clone'‍‍‍‍‍‍‍

which is correct right? Sorry, I don't quite understand what I should do/try. Should I uninstall and reinstall ArcGIS Pro and Anaconda? Should I clone or not? Should I try to make a clone in the path "C:\\Program Files\\ArcGIS\\Pro\\bin\\Python\\envs"? Should I try the mentioned .api patch? I'm assuming that if I do try the patch I have first install anaconda-navigator on/from my clone? Should I have installed anaconda-navigator from my clone in the first place rather than from the .exe from Anaconda's website?

Sorry for the plethora of questions, I am just lost and extremely frustrated with this.

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

Well nothing has helped so far.

Why you need python 3.7?

Did you read my blog post?

If you don't need to clone, then don't

0 Kudos