Hello ESRI community,
I have run into an issue that I am unsure if it was an intentional design decision by ESRI or a bug. The crux of the problem is that activation scripts placed in %CONDA_PREFIX/etc/conda/activate.d by conda packages (such as pdal, proj, and so on) are not being executed when python is started within ArcGIS pro. Theses scripts are run are executed when starting the "Python Command Prompt".
Below I have an example with code to replicate. I have intentionally avoided the process of cloning the conda environment that ArcGIS provides what is effectively a broken environment (seriously, try cloning the base environment and then conda install anything, it will take an eternity before telling you your environment has incompatible restraints).
In this example, I am using miniforge with mamba for the speed of the dependency solver; and I'm using PDAL as an example to highlight the lack of activation scripts being run. This issue is not limited to PDAL, but to any package that ads an activation script to %CONDA_PREFIX%\etc\conda\activate.d
Steps to replicate:
```doscon
> conda create -n arcgis-example -c esri python==3.7.11
> conda activate arcgis-example
> mamba install -c esri -c defaults numpy=1.20.1
> mamba install jinja2=2.11.3
```
in `envs/arcgis-example/conda-meta` add a file named `pinned` with the following contents:
```
esri::python ==3.7.11
esri::numpy ==1.20.1
jinja ==2.11.3
```
This is done to make sure that the next steps do not change versions of packages
to versions that are incompatible
We then continue to make our conda environment arcgis compatible
```doscon
> mamba install -c esri -c defaults arcpy arcgispro
```
The next process addresses some DLL conflicts I ran into with PDAL installed
from conda-forge
```docson
> mamba install libzlib lerc=3.0=h0e60522_0 freetype==2.12.1=h546665d_0
```
Now we install PDAL.
```doscon
> mamba install pdal python-pdal
```
At this point, we fire up ArcGIS Pro, go to the Python manager and select the arcgis-example environment we recently created, and restart ArcGIS per the warning on the screen.
Now, we can open the python command prompt, and type: SET
There will be entries for PDAL_DRIVER_PATH, GDAL_DATA, GDAL_DRIVER_PATH, GEOTIFF_CSV, PROJ_LIB and so on.
Within ArcGIS Pro, start a Python Window and enter the following bit in there and run it...
import os
for variable, value in os.environ.items():
print(f"{variable}={value}")
You will notice that none of the variables that I mentioned above are present. The contents of PATH are not quite the same either.
The source of the difference is due to the contents of %CONDA_PREFIX%/etc/conda/activate.d/*.bat not being executed when the Python interpreter within the ArcGIS is started, but it was run on the python command prompt.
The absence of properly activating the conda environment within ArcGIS can cause all sorts of problems for users which can be really difficult to troubleshoot, missing DLLs, etc etc.
A possible workaround would be to get the path to the activated conda environment, and update the variables explicitly in a toolkit file
import os
import subprocess
import json
conda_json = json.loads(
subprocess.run(
["conda", "info", "--json"], capture_output=True
).stdout
)
env_directory = conda_json["env_vars"]["CONDA_PREFIX"]
# update variables from here as most involve paths relative
# to the active conda environment
os.environ["PDAL_DRIVER_PATH"] = os.path.join(env_directory, "bin")
...
This work-around would still not address issues in the python window within ArcGIS though and would be fragile to changes in dependencies.
I would love to know if there is something I am missing, please let me know!
Thanks,
Ogi
