netCDF4 and ArcGIS

StephenFricke · ‎03-19-2013

I am having issues using the netCDF4 module within my python script for one of my ArcGIS tools. The weird thing is, the tool runs fine the first time, but then when I run the tool a second time I am getting an error. The tool is no longer able to read from the netCDF file, and gives the error when I try to read the latitude values of the file. I then have to restart ArcGIS, and the tool then runs successfully once, but then if I run the tool again I get the same error and need to restart again. Does anyone have any idea what is going on here?

Thank you !

PhilMorefield · ‎03-19-2013

The conflict between netCDF4 and ArcGIS has a long and illustrious history. Example here.

Your problem sounds different from the one I had a few years ago, though. So small chance it's a problem with your code. Beyond that, there are at least two alternative netCDF readers that do work with ArcGIS:

ScientificPython - I've been submitting Windows installers for this project. Right now the latest one is compatible with AGD 10.0.

scipy.io.netcdf - This will bomb on really, really big files. ScientificPython won't.

There are others, and they all use very similar APIs. Unfortunately, none of them offer the cool bells-and-whistles that netCDF4 offers.

Hope that helps.

StephenFricke · ‎03-19-2013

The conflict between netCDF4 and ArcGIS has a long and illustrious history. Example here.

Your problem sounds different from the one I had a few years ago, though. So small chance it's a problem with your code. Beyond that, there are at least two alternative netCDF readers that do work with ArcGIS:

ScientificPython - I've been submitting Windows installers for this project. Right now the latest one is compatible with AGD 10.0.

scipy.io.netcdf - This will bomb on really, really big files. ScientificPython won't.

There are others, and they all use very similar APIs. Unfortunately, none of them offer the cool bells-and-whistles that netCDF4 offers.

Hope that helps.

Hey, thank you for the response. I am running AGD 10.1, which uses python 2.7. It looks like the latest windows installer is for python 2.6. I believe I tried this one and was having compatibility issues with ArcGIS 10.1. Do you know any window installers for python 2.7? Thanks again.

PhilMorefield · ‎03-20-2013

Hey, thank you for the response. I am running AGD 10.1, which uses python 2.7. It looks like the latest windows installer is for python 2.6. I believe I tried this one and was having compatibility issues with ArcGIS 10.1. Do you know any window installers for python 2.7? Thanks again.

I submitted an installer for Python 2.7 to the ScientificPython author. He usually posts those to the webpage in ~24 hours. If you need it before he posts it, send me a message with your email.

RichardSignell · ‎03-20-2013

Just a week ago Christoph Gohlke built a completely static NetCDF4 module specifically for use with ArcGIS 10.1, and in my testing it has worked flawlessly. Check out the link to netCDF4-1.0.5dev-ArcGIS-10.1.win32-py2.7.exe on Christoph's amazing Python libraries for Windows page: http://www.lfd.uci.edu/~gohlke/pythonlibs/#netcdf4

PhilMorefield · ‎03-20-2013

Just a week ago Christoph Gohlke built a completely static NetCDF4 module specifically for use with ArcGIS 10.1, and in my testing it has worked flawlessly. Check out the link to netCDF4-1.0.5dev-ArcGIS-10.1.win32-py2.7.exe on Christoph's amazing Python libraries for Windows page: http://www.lfd.uci.edu/~gohlke/pythonlibs/#netcdf4

That's a good point. That netCDF4 installer does work for me on a 64-bit Windows 7 machine, but not on my 32-bit Windows XP computers.

StephenFricke · ‎03-22-2013

Thank you so much for the responses. I downloaded the python 2.7 version of ScientificPython, and for some reason I am not having success running the module within Arc. I keep getting an error of "IOError: netcdf: Invalid argument", but the same argument seems to be working outside of Arc. I also downloaded the mentioned netCDF4 module and I am getting the same error I was originally getting in the first post. My script runs perfect one time through, but then the next time through I am have problems. I first read in the netcdf file using:

dataset= Dataset('nameofmynetcdf.nc')

And then I try and read the latitudes using the command:

lt = dataset.variables['lat'][:]

This works perfectly the first time through, but then second time through I get an error for this line which states:

AttributeError: 'NoneType' object has no attribute 'iterable'

Does anyone know why this would be? I am reading this data via opendap if that makes any difference at all(?). I would really appreciate any help anyone has to offer. Thank you!

PhilMorefield · ‎03-22-2013

Thank you so much for the responses. I downloaded the python 2.7 version of ScientificPython, and for some reason I am not having success running the module within Arc. I keep getting an error of "IOError: netcdf: Invalid argument", but the same argument seems to be working outside of Arc. I also downloaded the mentioned netCDF4 module and I am getting the same error I was originally getting in the first post. My script runs perfect one time through, but then the next time through I am have problems. I first read in the netcdf file using:

dataset= Dataset('nameofmynetcdf.nc')

And then I try and read the latitudes using the command:

lt = dataset.variables['lat'][:]

This works perfectly the first time through, but then second time through I get an error for this line which states:

AttributeError: 'NoneType' object has no attribute 'iterable'

Does anyone know why this would be? I am reading this data via opendap if that makes any difference at all(?). I would really appreciate any help anyone has to offer. Thank you!

It might best if you posted some or all of your code, and included the line number of the error. There are instruction at the top of the Python forum on how to post code here.

RichardSignell · ‎03-22-2013

I also downloaded the mentioned netCDF4 module and I am getting the same error I was originally getting in the first post. I am reading this data via opendap if that makes any difference at all(?). I would really appreciate any help anyone has to offer. Thank you!

See if you can run this python toolbox, which uses NetCDF4 with OPeNDAP in ArcGIS 10.1:
https://github.com/rsignell-usgs/dap2arc

If it works for you more than once, you could then focus on issues with your code.

StephenFricke · ‎03-25-2013

Here is a condensed version of my code. You probably won't be able to access this data, but if you know of any other opendap data with a latitude variable, you could substitute that in the data line.

from netCDF4 import Dataset
data= 'http://cloud.insideidaho.org:8001/reacch/CMIP5/maca_huss_BNU-ESM_historical_1950_1959_WUSA.nc'
MyData=Dataset(data)
lt = MyData.variables['lat'][:]
arcpy.AddMessage(lt)

When I run as a script in an ArcGIS toolbox, it runs successfully the first time, and prints off all of the latitudes, the second time however, I get this error message:

Traceback (most recent call last):
File "C:\netcdf4 trouble\netcdf4problem.py", line 5, in <module>
lt = MyData.variables['lat'][:]
File "netCDF4.pyx", line 2631, in netCDF4.Variable.__getitem__ (netCDF4.c:32455)
File "C:\Python27\ArcGIS10.1\lib\site-packages\netCDF4_utils.py", line 127, in _StartCountStride
if np.iterable(elem):
AttributeError: 'NoneType' object has no attribute 'iterable'

Failed to execute (netcdf4problem).