topic hands-on advice on upgrade with the third-party Python packages in Python Questions

hands-on advice on upgrade with the third-party Python packages

larryzhang — Sun, 31 Jan 2016 13:50:26 GMT

Hello, Python coders:

Your hands-on assistance on the following Python third-party packages' upgrade would be appreciated:

Description:

ArcGIS 10.3 users would like to use the package "Site.py" (or other similar tool) and then to install/ upgrade the third-party Python package "scikit-learn" with "scipy" into the following Windows third-party Python path (i.e., C:\Python27\ArcGIS10.3\Lib\site-packages), so that data engineers can use ArcGIS to directly call the functions from "scikit-learn" and "scipy" inside ArcGIS 10.3 (Python 2.7.8), without troubles.

Note:

When ArcGIS 10.3 was installed into Windows 7, Python 2.7.8 was also installed into: C:\Python27\ArcGIS10.3 ;

With Python 2.7.8, the two third-party packages "NumPy" and "Matplotlib" also available under the folder: C:\Python27\ArcGIS10.3\Lib\site-packages ;

The 'built' Windows Python package "scikit-learn" with "scipy" locates at the folder: D:\Python-Packages\ ;

Question:

What is the python "script" more effectively to do this upgrade task?

It would be better to install /upgrade the third-party Python packages,without Admin privilege (if required, please indicated) .

Re: hands-on advice on upgrade with the third-party Python packages

DanPatterson_Retired — Sun, 31 Jan 2016 14:35:16 GMT

The easiest route for most is...

If you have ArcMap 10.3.1... install ArcGIS PRO ... it produces
- Python 3.4.x
- SciPy: C:\Python34\Lib\site-packages\scipy
  - Stats: C:\Python34\Lib\site-packages\scipy\stats
  - NDimage C:\Python34\Lib\site-packages\scipy\ndimage
- Numpy 1.9 C:\Python34\Lib\site-packages\numpy
- Matplotlib C:\Python34\Lib\site-packages\matplotlib
- Pandas C:\Python34\Lib\site-packages\pandas
- Sympy C:\Python34\Lib\site-packages\sympy
Sci-kit learn.... scikit-learn: machine learning in Python — scikit-learn 0.17 documentation read it there, and if you have the above installed there is no fooling around.

In the site packages folder you will find the ArcGISPro.pth file which shows the locations to Pro, Arcetc.

C:\Program Files\ArcGIS\Pro\bin

C:\Program Files\ArcGIS\Pro\Resources\ArcPy

C:\Program Files\ArcGIS\Pro\Resources\ArcToolBox\Scripts

Re: hands-on advice on upgrade with the third-party Python packages

Luke_Pinner — Sun, 31 Jan 2016 23:04:05 GMT

+1 for Dan's recommendation, but if you need/prefer to stick with ye olde Python 2.7, I strongly recommend not upgrading anything numpy/matplotlib related in the ArcGIS installed version as you can break ArcGIS / ArcPy (I have done this before).

If you want to upgrade/install the scipy stack, use a virtualenv (my preferred method) or a separate install, like Anaconda.

Re: hands-on advice on upgrade with the third-party Python packages

larryzhang — Mon, 01 Feb 2016 08:37:06 GMT

thx for your constructive advice, Luke, good job!

Re: hands-on advice on upgrade with the third-party Python packages

DanPatterson_Retired — Tue, 02 Feb 2016 19:28:30 GMT

For people wanting to move forward in Python 3.4, and you want to install modules, do note, that they have to be installed separately. This thread cites one common approach and this applies to version 3.4.x as well

http://stackoverflow.com/questions/35155675/packages-getting-installed-only-in-python-2-7-but-not-in-3-5

Re: hands-on advice on upgrade with the third-party Python packages

curtvprice — Sun, 15 May 2016 06:28:04 GMT

Note: Desktop 10.4 includes many these packages too in its standard Python 2.7 stack. (Unfortunately not scikit-learn, but scipy and pandas, among others.)

Integrating ArcGIS and SciPy | ArcUser

Re: hands-on advice on upgrade with the third-party Python packages

DavidWasserman — Tue, 18 Oct 2016 03:59:15 GMT

So as an Experiment...that I will willing to reinstall for...I did install the new version of numpy/sci-kit learn/ C++ compiler...and I got it to work. I also checked to see if some of my panda's based scripts worked (which involve some numpy) and they also work. Python and Arcpy tools still work. Is it just a matter of time before something breaks? I have trouble running sci-kit learn in process scripts more than once in the same map session (it also creates memory issues in Pycharm as well- something is leaking or something is not interacting well). That said, the numpy library did not show in the 10.4 install like I did for other installs that I have, so it might just be I am not directly calling it. I was curious if anyone here had any insight into why everything still seems to work and some ideas on how to break it?

Re: hands-on advice on upgrade with the third-party Python packages

DanPatterson_Retired — Tue, 18 Oct 2016 08:28:21 GMT

I haven't noticed any leaks with Pycharm, but I mostly use Spyder for my work. Perhaps the IDE's handle things differently /blogs/dan_patterson/2016/07/17/anaconda-spyder-and-arcgis-pro