Conflict arcpy (Arc 10) and gdal?

[NielsAnders]

Dear all,

Since a year or so, I happily combine the arctools with gdal and python functionality.
However, since I upgraded to ArcGIS 10 I ran into a number of problems.

My biggest issue is this:

The following code works fine:

import arcpy

arcpy.CheckOutExtension("Spatial")
raster = 'E:/temp/raster.img'
shapefile = 'E:/temp/shapefile.shp'

mask = arcpy.sa.ExtractByMask(raster,shapefile)
mask.save('E:/temp/test.img')

However, when I add 'from osgeo import gdal' at the very beginning, I get an error message that some conversion from arcobjects to python object couldn't finish:

from osgeo import gdal
import arcpy

arcpy.CheckOutExtension("Spatial")
raster = 'E:/temp/raster.img'
shapefile = 'E:/temp/shapefile.shp'

mask = arcpy.sa.ExtractByMask(raster,shapefile)
mask.save('E:/temp/test.img')

>>> 
Traceback (most recent call last):
  File "E:\temp\ivan\testing3.py", line 8, in <module>
    mask = arcpy.sa.ExtractByMask(raster,shapefile)
  File "C:\Program Files\ArcGIS\Desktop10.0\arcpy\arcpy\sa\Functions.py", line 6726, in ExtractByMask
    in_mask_data)
  File "C:\Program Files\ArcGIS\Desktop10.0\arcpy\arcpy\sa\Utils.py", line 47, in swapper
    result = wrapper(*args, **kwargs)
  File "C:\Program Files\ArcGIS\Desktop10.0\arcpy\arcpy\sa\Functions.py", line 6722, in wrapper
    out_raster)
  File "C:\Program Files\ArcGIS\Desktop10.0\arcpy\arcpy\geoprocessing\_base.py", line 474, in <lambda>
    return lambda *args: val(*gp_fixargs(args))
ExecuteError: ERROR 010024: Error during conversion.
Failed to execute (ExtractByMask).

and if I *first* import arcpy and *then* import gdal, the gdal DLL could not be found:

import arcpy
from osgeo import gdal

arcpy.CheckOutExtension("Spatial")
raster = 'E:/temp/raster.img'
shapefile = 'E:/temp/shapefile.shp'

mask = arcpy.sa.ExtractByMask(raster,shapefile)
mask.save('E:/temp/test.img')

>>> 
Traceback (most recent call last):
  File "E:\temp\ivan\testing3.py", line 2, in <module>
    from osgeo import gdal
  File "C:\Python26\ArcGIS10.0\lib\site-packages\osgeo\gdal.py", line 7, in <module>
    import _gdal
ImportError: DLL load failed: The specified procedure could not be found.

Is there anyone else experiencing this, and is this due to my settings? or is there some conflict between ArcGIS 10 and gdal?

I have ArcGIS 10 build 2800 (I thought I installed SP1) + gdalwin32_1.6, and windows XP PRO 32 bit.

Since I'm lost now, any ideas would be great,
Niels

[DarylVan_Dyke]

Hi-

Sounded unhappily familiar.  I had similar loss of functionality, including the 'depends-which-I-import-first' type behavior.
http://trac.osgeo.org/gdal/ticket/4087
May not be a 'gdal bug', but worth reporting anywhere.  For the meanwhile, I'm maintaining multiple installs of Python 2.6.x, and leaning on os.system() to start other python instances.
Good luck! 
Daryl

[NielsAnders]

Thanks for the info, at least I know I'm not the only one...
I'll think about a workaround so I don't have to import both at the same time..

[axel99]

I was having issues getting "import gdal" to work also. Some stuff going on with the python environment in ArcGIS 10. I ended up getting gdal working with python 2.6 (not from within ArcGIS desktop). Once I got that working I then copied the gdal16.dll(not the dll installed with ArcGIS) to the C:\WINDOWS\system32 folder and now I can import gdal from the python interpreter instantiated from within a ArcGIS desktop application. I really don't know what is going on here but It must have someting to do with the unique way ArcGIS 10 pooling a single instance the python interpreter to be to be shared with all Python based GP tool calls and how this environment is being setup.

I noticed the IDLE IDE interactive command window has the line "IDLE 2.6.5      ==== No Subprocess ====". I have never seen this before and is maybe a bit of a hit of how the interpreter if fired up by ArcGIS 10. Wierd stuff!

If anyone has some insight, please share it.

[ChrisBater]

Definitely something going on with the Python distribution in Arc 10. I was also having massive problems with scipy and matplotlib. I'm not sure if this is going a bit overboard, but I finally gave up on the arc distribution and installed python(x,y). It took me several days to figure out how to do this, but it was worth it in the end. Instructions are below if you're interested....

1) Install ArcGis10 without python, or just uninstall python

2) Download Python(x,y) from http://www.pythonxy.com/ and install, making sure you select "custom directories" so that python is installed in C:\Python26. If installing on a 64 bit machine, do not include GDAL 1.6 �?? it won�??t work. Instead, go to http://www.lfd.uci.edu/~gohlke/pythonlibs/ and install  GDAL-1.8.0.win32-py2.6.exe.

3) Create a file called Desktop10.pth, which  should contain these lines:
C:\Program Files (x86)\ArcGIS\Desktop10.0\bin
C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy
C:\Program Files (x86)\ArcGIS\Desktop10.0\ArcToolbox\Script

4) Place the file in the python installation directory: �??C:\Python26�??

5) Open IDLE and try �??import arcpy�??

[KevinNetherton]

I just ran into this problem!  I am not sure how this solution is going to impact arcpy related functionality, however I just figured out how to successfully import both arcpy and osgeo.ogr. BTW I am using gdal/ogr version 1.6.

For some reason if you import arcpy first and ogr/gdal second, gdal is no longer able to find the dll's that it requires.  I have managed to determine that if you add the path to your gdal/bin installation directory back into the os.environ['PATH'] environment variable then gdal seems to be able to find the dll's it requires.  The following lines solved the problem for me:

import arcpy
import os
environList = os.environ['PATH'].split(';')
environList.insert(0, r'C:\Program Files\gdalwin32-1.6\bin')
os.environ['PATH'] = ';'.join(environList)
import osgeo.ogr

I would guess the problem is related to the fact that esri uses the gdal16.dll, so when arcpy is imported something happens which prevents ogr from finding its various dll's.

I wonder if after these imports have been implemented whether arcpy functions/methods that use the esri version of the gdal16.dll will continue to work.