topic Timeline for python upgrade? in ArcGIS Pro Questions

Timeline for python upgrade?

DanPatterson — Tue, 04 Aug 2020 02:07:52 GMT

Python... currently at the end of the 3.6 series in Pro (aka, 3.6.10).

Various packages that are used in the ecosystem are about to begin retiring 3.6.x.

For example... numpy will be saying goodbye

NEP 29 - Dropping Python version support · Issue #66 · dask/community · GitHub

and, IPython

ipython/version7.rst at master · ipython/ipython · GitHub

jupyterlab and jupyterlab-server... are they next?

Adopt NEP 29 (timeline for which versions of Python to support) · Issue #8577 · jupyterlab/jupyterlab · GitHub

Lots have been added to python 3.8.5 Documentation ... the last stable release

(don't forget to go back and check the 3.7 branch and look forward to 3.9/3.10 coming branches)

Is ArcMap the reason for the holdback?

Given that major and minor releases control substantive changes...

Releases and patches—ArcGIS Pro | Documentation

and minor change from 2.5 to 2.6 didn't produce a python change AND minor changes seem to occur once a year around UC, does that mean that we have to wait until next year?

Does anyone have any thoughts on this?

Kory Kramer‌ I will flag you and perhaps the comments on Beta 2 about python versions could be fleshed out here

Re: Timeline for python upgrade?

DanPatterson — Wed, 05 Aug 2020 21:14:00 GMT

The walrus operator....

What’s New In Python 3.8 — Python 3.8.5 documentation

math distance

math — Mathematical functions — Python 3.8.5 documentation

Re: Timeline for python upgrade?

JoeBorgione — Thu, 06 Aug 2020 18:39:49 GMT

Does anyone have any thoughts on this?

I'm struggling with a seemingly simple package install of the latest Spyder version 4.1.4 after upgrading to Pro 2.6. Never in all the clones and upgrades I've performed have I had so much trouble. I need to install mysql-connector and csv and once I add those after spyder, spyder does not execute. After this years UC I thought maybe things might get easier: https://community.esri.com/thread/256608-python-the-road-ahead but certainly not at 2.6

Re: Timeline for python upgrade?

DavidWasserman — Thu, 06 Aug 2020 21:00:03 GMT

I am running into similar issues as I stated in Another Pro 2.6 Clone problem. This is one of the rougher upgrades I have done. I might go back to 2.5.x.

Re: Timeline for python upgrade?

JoeBorgione — Thu, 06 Aug 2020 21:18:58 GMT

It's pretty lame this time around. I do need some new functionality the 2.6 has, so I'd like not to go back to 2.5.

Re: Timeline for python upgrade?

DavidWasserman — Thu, 06 Aug 2020 21:39:43 GMT

Agreed. I will let you know what my tests show in the other forum. Right now full repairs, uninstalls, deletions of environments etc- have not worked. The cloning process hits some validation errors, and in notebooks I can't import arcpy.
Edit: On my recent uninstall attempt, I had a clone with no validation errors. So instead of repair, uninstalled ArcGIS Pro. Will check in soon.

Re: Timeline for python upgrade?

JoeBorgione — Thu, 06 Aug 2020 21:56:36 GMT

I ended up using a PIP install of the Mysql libraries I need into my clone and so (knock on wood) my Spyder install is still working. I used a straight conda install of spyder 4.1.3.

Good Luck!

Re: Timeline for python upgrade?

DanPatterson — Fri, 14 Aug 2020 00:37:46 GMT

A little light reading before someone else weighs in

https://community.esri.com/people/danretired/blog/2020/08/13/conda-the-dependency-trail

Re: Timeline for python upgrade?

DanPatterson — Wed, 26 Aug 2020 23:07:05 GMT

IPython won't install on 3.6 if IPython updated to 7.17... That will limit notebooks as well.

ipython/version7.rst at master · ipython/ipython · GitHub

Re: Timeline for python upgrade?

ErikLash1 — Fri, 28 Aug 2020 20:51:09 GMT

Struggling here as well. We require the new functionality of Pro 2.6 to support specialized geodatabase topologies but everything I try to do to make the scripting environment work properly in Pro 2.6 is ultimately unsuccessful.

Another challenge being faced in regards to working in Pro 2.6 is that REST API documentation is terribly outdated - still promoting scripting in Py 2.7x.

We are dilligently trying to move away from Py 2.7xx as we modernize. I'm unsure at this point if script tools I created for Pro in 2.4//Py 3.6.x for interacting with REST API (that worked fine in both Pro 2.4 and Pro 2.5) are failing in Pro 2.6 b/c of ESRI changes or because of Python.changes b/c of the other difficulties being encountered in working with the Pro 2.6 python environment.

One thing that I think would help would be if ESRI sever ties with Python 2.7 entirely. It's time.

Py 2.7 has been dead for a while already and the python multiple personalities/support for a dead code base is making it very very difficult to transition to modern workflows, especially since ESRI seems to be still promoting for Py 2.7 for working with their products in the API and it's still being delivered with their enterprise products at install.

On a related note - I had Jupyterlab working fine in Pro 2.5 and it was spectacular - imho a necessary integration into to the platform. Attempts to get either Jupyterlab or Spyder working properly in Pro 2.6 have been completely unsuccessful. Both of them result in broken enviornments on my workstation.

Re: Timeline for python upgrade?

DanPatterson — Fri, 28 Aug 2020 21:25:16 GMT

Erik... I do my installations in the base/main Pro environment.

A number of people have been having trouble with clones.

Jupyter in Pro was working for a bit for me, but Matplotlib got broken and I haven't had the inclination to track the issue down

Spyder 4.1.4 is quite nice and I have been using their Spyder-notebook for a while

GitHub - spyder-ide/spyder-notebook: Jupyter notebook integration with Spyder

https://community.esri.com/people/danretired/blog/2020/08/01/notebooks-in-spyder

Re: Timeline for python upgrade?

ErikLash1 — Fri, 28 Aug 2020 22:37:48 GMT

Will give it a try. Thx Dan.

Re: Timeline for python upgrade?

JoshuaBixby — Fri, 28 Aug 2020 23:04:42 GMT

Erik Lash‌, I agree that the Python references in the REST API documentation are quite dated, but the ArcGIS API for Python has been out long enough for anyone using Python with the REST API to move over to it, and the ArcGIS API for Python is Python 3-based.

Re: Timeline for python upgrade?

ErikLash1 — Sat, 29 Aug 2020 01:28:56 GMT

The ArcGIS API for Python is a fantastic implementation but there are some places where it doesn't fill the gap.

Examples of these include creating scripts for use in toolboxes or chained and complex workflows.

Use case referenced in this instance is a chained GIS workflow that is basically its own custom software application implementation which manipulates GIS features across a multi-database and multi-file store ecosystem resulting in a new data product that has to meet rigid controls for integration into a 3'rd party web application with a data package that gets deployed to testing and prod GIS server environments.

The REST API part of that process is very minimal but extremely crucial component - which we are still trying to update from Py 2.7 to Py 3.6 so we can completely end our use of ArcMap and Py 2.7.

What we really want to avoid is having to code some parts of the process in different Pythonic environments or in combinations of multiple languages.

What brought about my post on this thread is that we would very much like to have Jupyter Labs and Spyder working with the ArcGIS Pro environment as a super great and elegant solution for this particular application development need but are stumped because while we were able to work with them in Pro 2.5, we are among those that are struggling for success in 2.6

Presently using a combination of PyCharm and VSSCode to work with Pros Py 3.6.10 environment.

Would love for a unified approach to the foundational Pythonic components of the ESRI ecosystem (especially since Py 2.7 and Py 3.6 are incompatible and as Dan notes some 3.6 components are already retiring) and really see that as something very important to both successful adoption of the tools ESRI provides to the user base and strong user support for the ESRI products moving forward.

Re: Timeline for python upgrade?

ErikLash1 — Sat, 29 Aug 2020 02:13:10 GMT

Spyder Notebooks looks great but still no go - Spyder keeps throwing errors in both clones and the default Pro environment. Was worth a try and at least uninstalling and reinstalling Pro doesn't take very long.

spyder 4.1.4 py36_0

This time:

logger.error('%', 1, stack_info=True)
Message: '%'
Arguments: (1,)

May try installing prior verion of spyder.

Re: Timeline for python upgrade?

ErikLash1 — Sat, 29 Aug 2020 18:25:25 GMT

Working now ... even Spyder Notebooks. Have only tested basic functionality so don't know if anything is still amiss.

https://community.esri.com/message/950872-re-another-pro-26-clone-problem?commentID=950872#comment-950872

Re: Timeline for python upgrade?

DanPatterson — Sat, 29 Aug 2020 21:18:32 GMT

You aren't going to do "maps" in Spyder Notebooks. In fact, why would you want to do "maps" in Pro's notebooks when you have notebooks.

You can still do arcpy stuff however

Re: Timeline for python upgrade?

ErikLash1 — Thu, 17 Sep 2020 08:50:47 GMT

Yep. Spyder notebooks does what I need - mostly using it for code validation/qc and integrating arcpy. When it comes time to run things into production functionality am switch back over to a Jupyter environment for the graphical capabilities.

As an aside, found the issue with the NodeJS npm error when bringing Jupyter Labs online with Pros environment - jupyterlab_server package included with Pro required a package update to make it compliant with the newer jupyterlab package.

Got everything working in Pro 2.6.1 Jupyter Labs including matplotlib and jupyter dashboards.

Re: Timeline for python upgrade?

DanPatterson — Sat, 11 Dec 2021 14:37:36 GMT

Current

(arcgispro-py3) C:\arc_pro\bin\Python\envs\arcgispro-py3>conda list jupyter
# packages in environment at C:\arc_pro\bin\Python\envs\arcgispro-py3:
#
jupyter_client            6.1.2                    py36_0    esri
jupyter_console           6.1.0                    py36_0    esri
jupyter_core              4.6.3                    py36_2    esri
jupyterlab                2.0.1                    py36_0    esri
jupyterlab_server         1.2.0                      py_0‍‍‍‍‍‍‍‍

but there is some pre-release versions currently listed in the last little while at

https://anaconda.org/Esri/repo

which is a good place to follow to see what is being updated in their environments (I presume for bug fixes and/or testing)

Re: Timeline for python upgrade?

DanPatterson — Thu, 17 Sep 2020 11:03:33 GMT

https://anaconda.org/Esri/arcpy/files‍

arcpy for python 3.7 (?)

no listing for python 3.8