Accessing and cloning python environment with Pro SDK

Question

I'm trying to create a configuration/ add-in for a department and because Pro invalidates the cloned python environments after updates, I need a way to ensure that a cloned environment is created and the required python packages (one being our in-house package) are installed. Since some packages do not install through the Pro's conda manager, I need to install through pip.  I cannot ask each user to execute pip commands after each update of Pro. Nor do I really want to touch each computer after each update.

I haven't been able to find any examples of accessing the cloned environments, or if the python/ conda environment is exposed to the ArcGIS Pro SDK. A similar question without an answer was asked here.  I created a script that checks for the packages I need to install and installs them if they are not, but it needs the cloned environment first.

Simply, I need to do this:

User starts Pro after an update or fresh install.

Be able to:

1. Check if the python env is cloned and clone it if isn't.

1.a. If it exists, check if python packages are installed and install if they are not.

2. Set active python environment to the clone that contains added packages.

3. Run python scripts from the addin buttons.

If this is not possible to do programmatically through the SDK, I'd welcome any options.  Thanks!

Anonymous User · Answer

I created an idea for providing conda documentation and how can we deploy environments using Pro SDK or python along with our configs and add-ins in case anyone reaches this and wants to help out.

create documentation for esri's conda

Anonymous User · Answer

This is the c# part placed in the button onclick event. - I'm still playing with how to capture the messages or just let them display in the console window because I dont think the SDK has an output window other than the geoprocessing one.

await QueuedTask.Run(async () =>
            {
                try
                {
                    ProcessStartInfo processInfo = new ProcessStartInfo();
                    processInfo.FileName = @"\gisdata\Transfer_Tech\ESRISoftware\Addins\python-env.bat";
                    processInfo.UseShellExecute = false;
                    processInfo.CreateNoWindow = false;
                    processInfo.UseShellExecute = false;
                    processInfo.RedirectStandardError = false;
                    processInfo.RedirectStandardOutput = false;

using (Process process = Process.Start(processInfo))
                    {
                        process.OutputDataReceived += (object sender, DataReceivedEventArgs e) =>
                            MessageBox.Show("output>>" + e.Data);
                        process.BeginOutputReadLine();

process.ErrorDataReceived += (object sender, DataReceivedEventArgs e) =>
                            MessageBox.Show("error>>" + e.Data);
                        process.BeginErrorReadLine();

process.WaitForExit();

//MessageBox.Show("ExitCode: {0}", process.ExitCode);

process.Close();
                    }

// Show a messagebox with the results
                    MessageBox.Show("Python environment updated successfully.");

}
                catch (Exception exc)
                {
                    // Catch any exception found and display in a message box
                    if (exc.Message != "StandardOut has not been redirected or the process hasn't started yet.")
                    {
                        MessageBox.Show($"Exception caught while trying to validate the python env: {exc.Message}.");
                    }

return;
                }
            });

Anonymous User · Answer

After some loss of hair, I cobbled a bat file together that performs the steps of cloning the default environment, installs missing packages, and sets the pro's active python env to the new env.  I am no CMD genius so if this can be improved, please feel free to do so and post it.  I got a lot of 700 errors from jupyter-notebook at times while developing this.  Not sure what the exact fix was but it seemed to dissappear when I cloned the default env and updated it with cloned env.  Gotta quit while I'm ahead.

The called scripts (activate.bet & proswap.bat) are using full paths because it would stop if I tried to activate the new env to run commands.

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: VARIABLES                                                                    :
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
@echo off

SETLOCAL
SET "ENV_NAME=assessors-env"
SET "ARCGIS_DEFAULT=arcgispro-py3"
SET "ARCGIS_CLONE=arcgispro-py3-clone"

SET "PRO_PATH=C:\Program Files\ArcGIS\Pro\bin\Python"
SET "CLONED_PATH=%LOCALAPPDATA%\ESRI\conda\envs"
REM Set the default script path for this iteration. Maybe fixes jupyter notebook install errors?
SET PATH=%PATH%;"C:\Program Files\ArcGIS\Pro\bin\Python\Scripts"

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: COMMANDS                                                                     :
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

REM start by activating the arcgis pro conda env
call "C:\Program Files\ArcGIS\Pro\bin\Python\Scripts\activate.bat" & (
	REM export the cloned arcgispro-py3-clone environment that contains the additional packages installed outside of arcgis pro
		conda env export -n %ARCGIS_CLONE% > "%CLONED_PATH%\%ARCGIS_CLONE%\environment.yml"	
		ECHO ^-^-^> %ARCGIS_CLONE% conda environment exported to "%CLONED_PATH%\%ARCGIS_CLONE%\environment.yml"

REM clone the deafult arcgispro-py3 environment to the new environment name
		REM CALL conda config --add channels conda-forge
		REM CALL conda config --add channels esri
		conda create --clone %ARCGIS_DEFAULT% --name %ENV_NAME% --pinned
		ECHO ^-^-^> %ENV_NAME% environment created

REM update the new environment with packages that are not installed from the environment.yml created earlier.
	    REM conda env list
		REM conda env update -n %ENV_NAME% -f "%PRO_PATH%\envs\%ARCGIS_DEFAULT%\environment.yml"
		conda env update -n %ENV_NAME% -f "%CLONED_PATH%\%ARCGIS_CLONE%\environment.yml"
		ECHO ^-^-^> %ENV_NAME% environment updated

REM set the active environment in Pro to the new environment
		CALL "%PRO_PATH%\Scripts\proswap.bat" %ENV_NAME%
		ECHO ^-^-^> Pro env set
	)
pause
exit

Now to get it to run from C#.