Conda activate scripts are not executed within ArcGIS

OgnyanMoore — Thu, 10 Nov 2022 17:25:16 GMT

Hello ESRI community,

I have run into an issue that I am unsure if it was an intentional design decision by ESRI or a bug. The crux of the problem is that activation scripts placed in %CONDA_PREFIX/etc/conda/activate.d by conda packages (such as pdal, proj, and so on) are not being executed when python is started within ArcGIS pro. Theses scripts are run are executed when starting the "Python Command Prompt".

Below I have an example with code to replicate. I have intentionally avoided the process of cloning the conda environment that ArcGIS provides what is effectively a broken environment (seriously, try cloning the base environment and then conda install anything, it will take an eternity before telling you your environment has incompatible restraints).

In this example, I am using miniforge with mamba for the speed of the dependency solver; and I'm using PDAL as an example to highlight the lack of activation scripts being run. This issue is not limited to PDAL, but to any package that ads an activation script to %CONDA_PREFIX%\etc\conda\activate.d

Steps to replicate:

```doscon
> conda create -n arcgis-example -c esri python==3.7.11
> conda activate arcgis-example
> mamba install -c esri -c defaults numpy=1.20.1
> mamba install jinja2=2.11.3 
```

in `envs/arcgis-example/conda-meta` add a file named `pinned` with the following contents:

```
esri::python ==3.7.11
esri::numpy ==1.20.1
jinja ==2.11.3
```

This is done to make sure that the next steps do not change versions of packages
to versions that are incompatible

We then continue to make our conda environment arcgis compatible

```doscon
> mamba install -c esri -c defaults arcpy arcgispro
```

The next process addresses some DLL conflicts I ran into with PDAL installed
 from conda-forge

```docson
> mamba install libzlib lerc=3.0=h0e60522_0 freetype==2.12.1=h546665d_0
```

Now we install PDAL.

```doscon
> mamba install pdal python-pdal
```

At this point, we fire up ArcGIS Pro, go to the Python manager and select the arcgis-example environment we recently created, and restart ArcGIS per the warning on the screen.

Now, we can open the python command prompt, and type: SET

There will be entries for PDAL_DRIVER_PATH, GDAL_DATA, GDAL_DRIVER_PATH, GEOTIFF_CSV, PROJ_LIB and so on.

Within ArcGIS Pro, start a Python Window and enter the following bit in there and run it...

import os for variable, value in os.environ.items(): print(f"{variable}={value}")

You will notice that none of the variables that I mentioned above are present. The contents of PATH are not quite the same either.

The source of the difference is due to the contents of %CONDA_PREFIX%/etc/conda/activate.d/*.bat not being executed when the Python interpreter within the ArcGIS is started, but it was run on the python command prompt.

The absence of properly activating the conda environment within ArcGIS can cause all sorts of problems for users which can be really difficult to troubleshoot, missing DLLs, etc etc.

A possible workaround would be to get the path to the activated conda environment, and update the variables explicitly in a toolkit file

import os import subprocess import json conda_json = json.loads( subprocess.run( ["conda", "info", "--json"], capture_output=True ).stdout ) env_directory = conda_json["env_vars"]["CONDA_PREFIX"] # update variables from here as most involve paths relative # to the active conda environment os.environ["PDAL_DRIVER_PATH"] = os.path.join(env_directory, "bin") ...

This work-around would still not address issues in the python window within ArcGIS though and would be fragile to changes in dependencies.

I would love to know if there is something I am missing, please let me know!

Thanks,

Ogi

Re: Conda activate scripts are not executed within ArcGIS

MattWilkie1 — Mon, 16 Oct 2023 20:00:56 GMT

I haven't fully ingested your setup, that will take more attention time than I have at the moment as I try to solve my own challenge du jour, but one thing that comes to mind is perhaps `proswap.bat`? (found by way of https://samgeo.gishub.org/examples/arcgis/)

topic Re: Conda activate scripts are not executed within ArcGIS in Python Questions

Conda activate scripts are not executed within ArcGIS

Re: Conda activate scripts are not executed within ArcGIS