Should Esri ship third-party modules (like scipy) with ArcGIS?

[curtvprice]

The ArcGIS Python 2.7 distribution for 10.1 includes (in addition to the python standard library)  the user-contributed modules numpy and matplotlib. It would be really great to include some more modules. If you agree, vote this item up and add your favorites (and use cases) in the comments.

Vote here, and also feel free to vote the idea up:

ideas.esri.com: Ship additional python extensions

[PhilMorefield]

I vote for none.  A quick check of the python package index shows almost 25,000 Python modules available.  Where would the line be drawn.
What I would like to see is for ESRI to install Python using the standard Python installation directories, then install ArcPy as just another package under the site-packages directory.  That way all the off the shelf Python stuff works just fine.

I'm still on 10 so maybe 10.1 fixes this issue.

Well I think the line would be drawn based on feedback received in threads like this one. Python is becoming the standard for scientific software, and I'm sure that out of 25,000 modules a handfull are broad enough so as to be almost universally useful to the GIS user community.

I agree that the non-standard Python installation ESRI uses is annoying. But this is standard practice for other scientific/GIS applications that need a Python installation (e.g., Quantum GIS does the same thing). The workaround to this is just install Python, numpy, matplotlib, etc. first so the ArcGIS installer will use those installations instead of putting in its own. The arcpy software is still not put into the 'site-packages' folder like other modules, but at least your Python installation will look normal in every other way.

[Celeste_SuliinBurris]

I would like to see pandas. Apparently the pre-compiled version of pandas from <http://www.lfd.uci.edu/~gohlke/pythonlibs/#pandas> won't work with the version of numpy that ships with ArcGIS 10.1.  I have installed it carefully, but I keep getting "numpy.dtype has the wrong size, try recompiling"

[MikeMacRae]

+1 for win32com. I do a lot of reporting based on spatial data and make use of a lot of excel spreadsheets. Whatever I can't get ArcGIS to do to communicate with excel, I can do through this module.

[JasonScheirer]

Mike: we're bundling xlrd and xlwt with newer versions of ArcGIS, would that be adequate for your needs?

I'm aware of the demand for Pandas (I want it too), I'm going to see what I can do with that.

[curtvprice]

Pandas sounds like a really good idea.

[curtvprice]

Talking with the arcgis geoprocessing team at UC yesterday -- they said that ArcGIS Pro beta 4 (python 3.4 of course) is  shipped with an "internal" python install used by the app and a separate installer with python 3.4 that includes what Bill M was calling the "sci stack" which includes the modules used by esri (numpy, matplotlib etc) plus a another 300MB of so of other scientific modules (I think he said scipy, pandas, netCDF4) that aren't used by Esri but are useful to us.

The idea is they don't have the footprint there in a standard Pro install, but the group of users that will be installing this separate python installer are developers and can deal with the footprint and any issues they run into with these modules and not blame Esri. 🙂

This is great news. Hope there is a blog post coming from the arcpy crew soon! Maybe on the cafe?

[ChristopherMoravec1]

Another option that would be great is for Esri to simply include pip and Easy Install.  This would make it easy for people to write scripts in Python or even BAT files that could add any 3rd party libraries needed.  Most Python distributions (excluding windows) come with pip and Easy Install ready to go, it seems like something Esri could add that would make it more "pythonic" and much easier for anyone to require additional libraries or dependencies.

[MatejTacer]

I would support adding pip and/or easy_install rather than including too many third party libraries.

Using pip or easy_install is super easy and quick to install whatever library you might need.

[JasonScheirer]

pip is bundled with Python 3.4.

[curtvprice]

I'm not sure just using pip will solve our problems with third-party packages... many users have run into binary compatibility issues on the Windows platform where a .dll the Python library has a conflict with the .dll ArcGIS Desktop is using...

netCDF4 and ArcGIS‌

Cannot import pandas - ArcGIS 10.1‌ and also with netCDF4.

And of course we will be using Desktop / 2.7 for a while yet.

The ArcHydro team is currently distributing a simple .exe executable for netCDF4 that does work with ArcGIS Desktop using a .exe generated using distutils I'd guess...

More guidance would be very helpful!