Conda cloned environment has problems with GDAL

Dumbatz — Thu, 02 Nov 2023 12:42:59 GMT

Hello all,

I have a problem with a clone of the standard 'arcgispro-py3' environment.

I am working with ArcGIS Pro 3.0.5. As I am employed the device is owned and managed by the company, which leaves no way of changing the ArcGIS installation.

However, to gain some freedome in the use of 'arcpy', I do create clones of the standard environment. Therefore, I go to the directory that contains the 'conda'-related scripts and clone the environment in the C:\Program\ArcGIS\Pro\bin\Python\envs
to
C:\Users\*\AppData\*\envs.
I do create a clone in order to install the 'esda' package which conveniently comes with 'libpysal', that I also need. Now, to prevent any major hiccups and knowing the issue that comes up, I do pin the ‘gdal’ to 3.4.0.
I perform all installations of packages in cloned environments through the command line and the ‘conda’-related scripts.

I have a .pyt based Toolbox, that uses subscripts. These should employ ‘gdal’, ‘esda’, and ‘libpysal’.

What I tested:

Clone environment, NOT installing ‘esda’ and ‘libpysal’, disabling the sections in the script that need these, and disabling the ‘import’ of said packages.
‘from osgeo import gdal’ is used.
Then, said toolbox and script can be run over and over, within one session of ArcGIS Pro, or closing Pro and opening Pro again.
The cloned environment and all package dependencies are stable.
Installing ‘esda’ and ‘libpysal’ (while ArcGIS Pro can be opened or closed).
Note:

gdal 3.4.0-arcgispro_py39_17119 --> 3.4.0-arcgispro_py39_17150

This happens along with the otherwise flawless installation.

I enable the sections of the script that use ‘esda’ and ‘libpysal’ functionality, ‘import’ the packages.
Also, ‘from osgeo import gdal’ is used.
Then I (start ArcGIS Pro and) open the toolbox, the syntax check is successful, NO errors.
The toolbox can be used again and again.

But, if I now close ArcGIS Pro (the day might be over), and then come back to it and want to use the Toolbox, I get an error:

Traceback (most recent call last): File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\site-packages\osgeo\__init__.py", line 40, in swig_import_helper return importlib.import_module(mname) File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\importlib\__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 666, in _load_unlocked File "<frozen importlib._bootstrap>", line 565, in module_from_spec File "<frozen importlib._bootstrap_external>", line 1173, in create_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed ImportError: DLL load failed while importing _gdal: Die angegebene Prozedur wurde nicht gefunden. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<string>", line 17, in <module> File "C:\Users\schwenk_m\Documents\ArcGIS\Projects\Geo_gen\RasterSurfaceAnalysis.py", line 23, in <module> from osgeo import gdal File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\site-packages\osgeo\__init__.py", line 56, in <module> _gdal = swig_import_helper() File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\site-packages\osgeo\__init__.py", line 52, in swig_import_helper raise ImportError(traceback_string + '\n' + msg) ImportError: Traceback (most recent call last): File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\site-packages\osgeo\__init__.py", line 40, in swig_import_helper return importlib.import_module(mname) File "C:\Users\schwenk_m\AppData\Local\ESRI\conda\envs\MSdev39ArcPy\Lib\importlib\__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 666, in _load_unlocked File "<frozen importlib._bootstrap>", line 565, in module_from_spec File "<frozen importlib._bootstrap_external>", line 1173, in create_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed ImportError: DLL load failed while importing _gdal: Die angegebene Prozedur wurde nicht gefunden. On Windows, with Python >= 3.8, DLLs are no longer imported from the PATH. If gdalXXX.dll is in the PATH, then set the USE_PATH_FOR_GDAL_PYTHON=YES environment variable to feed the PATH into os.add_dll_directory().

My questions is, what happend to the environment through the installation of the new packages, and by closing and opening ArcGIS Pro?

For all that I tryed, I could not, as suggested, set the option 'USE_PATH_FOR_GDAL_PYTHON=YES'.
I did not grasp how to exactly use the command

conda env config vars list USE_PATH_FOR_GDAL_PYTHON=YES

that was suggested here. However, I did add the line to the '.condarc' file in my Home directory, which does not help.

Furthermore, upon inspection all the .py scripts related to GDAL in the OSGEO folder are still there.

I am thankful for any pointers.

Re: Conda cloned environment has problems with GDAL

Dumbatz — Thu, 02 Nov 2023 16:25:47 GMT

True. Thanks for the suggestion.
This is what I do, although I do not work with VS Code.

The problem is, that the enviornment is 'distroyed' by the ArcGIS Pro program itself. After using the toolbox in one instance, I cannot use it again.
If I activate the environment on a command prompt, and call Python, the "from osgeo import gdal" is no longer functional.

The cloned environment is kaputt.

Re: Conda cloned environment has problems with GDAL

Dumbatz — Wed, 22 Nov 2023 10:13:19 GMT

Currently, the problem is solved. The 'gdal'-update that happened, is not performed any more.
Since 'gdal' is stable, the environment is stable.

But this could change again, over time.

topic Re: Conda cloned environment has problems with GDAL in ArcGIS Pro Questions

Conda cloned environment has problems with GDAL

Re: Conda cloned environment has problems with GDAL

Re: Conda cloned environment has problems with GDAL