Coming to Python.... preparation and anticipation

04-11-2016 10:08 PM
Labels (1)
MVP Esteemed Contributor
8 2 3,801

I always forget.  I read about something pythonic, then quasi-remember what it was I read.  So this is what is coming to python.  A look into the future.  Preparing for the future.

Also... Install Spyder, Jupyter console and Jupyter notebook for ArcGIS Pro by default 

2019-02-21 Python 3.8 beta is out  Starting to run out of dot numbers

2018-06-28 Python 3.7 official release  Yes it is here... just add 1.0 to the version you are working on

2018-06-11  What’s New In Python 3.7 — Python 3.7.0b5 documentation 

Lots of speed improvements for many widely used functions.  Python 3.8 is on the way.

2018-03-31  numbers

Hey almost forgot this one.  Ever tried to type in UTM coordinates for those that live in the 6 figure northing range.  You poke at the screen counting zeros to make sure you get the northing right and don't end up near the equator.  Along comes underscores in numbers.   

underscores in numbers
N = 5_025_000.01


Cool ehh? No more dirty monitors.  There is more in the specs for hex and nibbly bits etc 

Underscores in Numeric Literals for those that are interested


Older threads


ArcGIS Pro 2.1 ships with python 3.6.2    2018-01-17

Python 3.0 released  on December 3rd, 2008.

Python 3.6 expected final  December 12th, 2016    8 years old!

Python 3.7 beta released  September 23rd, 2017

New Additions:                                                                             Update: 2018-06-27

Exiting 2.7... supporters and timeline 2020.... set your clocks

Python 3.7.0a math.remainder ...

Numpy 1.13 release notes  updated

Apparently the wait will continue arcmap 10.5 python version??

Recent blog  /blogs/dan_patterson/2016/12/07/not-using-python-3 

Python 3.7 beta...  Overview — Python 3.7.0a0 documentation 

Conda tips and tools...  Conda and ArcGIS Pro | ArcPy Café 

Python 3.6 final coming soon... sorted dictionary keys and more  Release information and changes

Anaconda, Spyder and PRO Anaconda, Spyder and ArcGIS PRO

Python & Pro  Python and ArcGIS Pro 1.3 : Conda

Anaconda  Download Anaconda now! | Continuum — latest release for numpy 1.11.1 and SciPy 0.17.1

Learning resourcesData Science in Python

        GitHub - Data Science Python: common data analysis and machine learning tasks using python

A classic example is those of you that have avoided reading:

  Python Mini Formatting Language .

Oh wait!! Many of you are still using python 2.7.  Good news! Install ArcGIS PRO and enter the realm of Python 3.4.  Yes I did say 3.4. but it gets worse.... Python 3.7 is in beta as of Sept 2016.

Python 2.7.x is the last of the 2.x series.  It will remain on support for some time (to indefinitely), but there have been lots happening since Python 3.0 was introduced (homework... when was python 3.0 introduced).

So this is not going to be a retrospective going back to python 3.0 and what it introduced, but I will highlight what is going forward in the python world since python 3.6 is set for final release soon.. and you will be using python 3.4 ... IF you install ArcGIS PRO or dabble in the neverworld of alternate installs.  So in reverse order, this is what I think will be useful for you to look forward to, and what you may have missed.



Contents : Python  NumPy  SciPy  Matplotlib  Pandas       Update:  2017-02-18

Last Update:   numpy 1.12 and scipy 0.17.1 and

   added Pandas 0.18.1 release What’s New — pandas 0.18.1 documentation


GitHub section

Esri             GitHub - Esri/ Esri on Github

Matplotlib   GitHub - matplotlib/matplotlib: matplotlib: plotting with Python

Numpy        GitHub - numpy/numpy: Numpy main repository

Pandas       GitHub - pydata/pandas: Flexible and powerful data analysis ...

SciPy          GitHub - scipy/scipy: Scipy library main repository

Sympy        GitHub - sympy/sympy: A computer algebra system written in pure Python


Python section

The main link  What’s New in Python ... this goes back in history

Pre-existing functionality  What is in version 3 that existed in version 2.6

with statement, print as a function, io module

----- python 3.8 ---------------------------------------------------------------------------------------

What’s New In Python 3.8 ... main page

----- python 3.7 ---------------------------------------------------------------------------------------
What’s New In Python 3.7 ... main page

----- python 3.6 ---------------------------------------------------------------------------------------

What’s New In Python 3.6 ... main page


>>> name = "Fred"

>>> f"He said his name is {name}."

'He said his name is Fred.'

---- python 3.5 -------------------------------------------------------------------------------------------

What’s New In Python 3.5 ...main page

  • Highlights

>>> *range(4), 4                        (0, 1, 2, 3, 4)

>>> [*range(4), 4]                      [0, 1, 2, 3, 4]

>>> {*range(4), 4, *(5, 6, 7)}      {0, 1, 2, 3, 4, 5, 6, 7}

>>> {'x': 1, **{'y': 2}}                 {'x': 1, 'y': 2}

  • Improved Modules
    • link includes, but not limited to:
      • collections, csv, distutils, doctest, enum, inspect, io, json, locale, logger, math, os, pathlib, re, readline, sys, time, timeit, trackback, types, zipfile
      • collections.OrderedDict is now implemented in C which makes it 4 to 100 times faster.

      • csv.write_rows now supports any iterable
      • enum.Enum now supports a start number for text enumeration
      • math ... added nan and inf constants (finally!!!) and gcd (get common divisor)
      • os.walk  significant speed improvements

---- python 3.4 -----------------------------------------------------------------------------------------

What's New in Python 3.4 ... main page

  • Highlights

>>> import statistics

>>> dir(statistics)

[ .... snip ... 'mean', 'median', 'median_grouped', 'median_high', 'median_low', 'mode', 'pstdev', 'pvariance', 'stdev', 'variance']

  • Improved Modules
    • link includes, but not limited to:  collections, inspect, multiprocessing, operator, os, re, sys, textwrap, zipfile
      • textwrap adds   maxlines, placeholder and shorten

---- python 3.3 and before  -------------------------------------------------------------------------------

Previous stuff (aka pre-3.4)

  • porting to 2to3            2to3 - Automated Python 2 to 3 code translation
    • changes to or replacement of :  apply, asserts, dict (dict.items(), dict.keys(), dict.values()), exec, has_key, isinstance, itertools, long, map, next, nonzero, operator, print, raise, range, raw_input, unicode, xrange, zip
  • virtual environments    pep 405
  • python launcher          pep 397
  • ordered dictionaries    pep 372
  • print as a function       pep 3105
  • sysconfig                    Python’s configuration information
  • argparse
  • order dictionaries
  • print statement
  • division  / float, // integer

way more ..... you will just have to read the original migration docs.

Significant changes to text handling and the migration to full unicode support.

Some of the changes have been implemented in python 2.7 and/or can be accessed via ... from future import ....


Anaconda section

Anaconda, Spyder and ArcGIS PRO

Python and ArcGIS Pro 1.3 : Conda

Download Anaconda now! | Continuum


NumPy section

The reverse chronological list of changes to numpy Release Notes — NumPy v1.13 Manual

numpy 1.13- - new functions... np.isin, np.divmod, np.block, plus others

- deprecations and changes are quite long... many improvements to subclass carry forwards

- too many more to list.

numpy 1.12- einsum optimized for speed

- keepdims added to many functions

- axis keyword for rot90

- flipud and fliplr now have axis generalization

- nancumsum and nancumprod added to the nan line of functions

- too many more to list

numpy 1.10

np.rollaxis, np.swapaxis  now return views

np.ravel, np.diagonal, np.diag  now preserve subtypes

recarray field and view  changes in how data are treated, see documentation

matrix @ operator implemented in keeping with its implementation in python 3.5


SciPy section

Too many to list, but the change logs are given here Release Notes — SciPy v0.17.0 Reference Guide

SciPy central


Matplotlib section

users guide        User’s Guide — Matplotlib 1.5.1 documentation

chronologic change history   What’s new in matplotlib — Matplotlib 1.5.1 documentation


Pandas section

Pandas 0.18.1         What’s New — pandas 0.18.1 documentation

The full list is here   pandas main page and release notes

Regular Contributor

Did you make the UC this year Dan?

I think one of the most stunning developments I saw was a new ArcGIS Python API (now in Beta.)

And to utilize this, requires installing Conda (anaconda I believe is the full name?  new to me) and some other things.

It makes heavy use of Jupyter notebooks which are very cool things.

Maps right in the notebooks.

ArcGIS API for Python | ArcGIS for Developers

Based around performing admin for "WebGIS" but can still tie to arcpy, portalpy, etc...

It is 3.x (4 or 5) based.

I'll probably be sticking it onto a VM and not my main dev PC, at least for while.

MVP Esteemed Contributor

Paul... not didn't make UC, but yes, condas is coming which will make it very useful to run higher versions of the scipy stack for testing purposes, before the official rollout of whatever is being used in ArcMap or ArcGIS PRO.  For example, you can test python 3.5 or 3.6 'stuff' when the current release at this stage is using python 2.7.x for ArcMap and 3.4.x for PRO.

It will surely/hopefully put an end to those sad tale of incompatability and maybe even 'can't find arcpy...

About the Author
Retired Geomatics Instructor at Carleton University. I am a forum MVP and Moderator. Current interests focus on python-based integration in GIS. See... Py... blog, my GeoNet blog...