topic Re: Cannot clone environment in ArcGIS Pro Questions

Cannot clone environment

JoelMcKechnie — Thu, 17 Aug 2023 05:45:14 GMT

I'm trying to clone the default conda environment in ArcGIS Pro. I've tried through Pro, and also through the python command prompt. I've tried reinstalling Pro, installing Pro in E: (in case permissions were an issue on C:), updating permissions of affected directories, added relevant paths to the environmental variables. It always fails the same way. This is on a remote Windows Server machine. I've been able to clone the default env using a local PC (with the python command prompt, not through ArcGIS Pro's settings).

I've copied the command prompt messages to a text file (attached).

Main/first error message seem to be:

ERROR conda.core.link:_execute(730): An error occurred while installing package 'esri::jupyter_contrib_nbextensions-0.6.0-py_2'.
Rolling back transaction: done
LinkError: post-link script failed for package esri::jupyter_contrib_nbextensions-0.6.0-py_2
location of failed script: E:\arcgis_pro\bin\Python\envs\arcgispro-clone\Scripts\.jupyter_contrib_nbextensions-post-link.bat

Any help would be greatly appreciated, thanks!

Re: Cannot clone environment

trent76 — Mon, 21 Aug 2023 17:01:58 GMT

This has been an absolute nightmare for our organization. Even with premium support tickets esri has been not able to help us one bit. The last version of pro we were able to install has been 2.7.4. The only way we were able to get this far was to whitelist the urls https://vlc.onl/ outlined in this article Versions 2.8 and 2.9 are throwing errors related to our proxy servers because on our servers the only way to access the web is through a proxy server. Esri still doesn’t know how to fix it.

Re: Cannot clone environment

DanPatterson — Thu, 17 Aug 2023 09:38:51 GMT

are you running the clone as administrator?

How To: Clone a Python Environment with the Python Command Prompt (esri.com)

The *.bat file mentioned tries to run a *.exe in the source location and it fails for several reasons.

examine C:\... install folder ...\bin\Python\envs\arcgispro-py3\Scripts\.jupyter_contrib_nbextensions-post-link.bat

Re: Cannot clone environment

JoelMcKechnie — Mon, 21 Aug 2023 04:46:51 GMT

Thanks for the info (I guess 😅)

Re: Cannot clone environment

JoelMcKechnie — Mon, 21 Aug 2023 04:50:26 GMT

Thanks Dan, I should have mentioned that. Yes ran as admin and used that article as a guide.

Re: Cannot clone environment

DanPatterson — Mon, 21 Aug 2023 06:27:06 GMT

Tech Support will put you in contact with the people who can tell you whether it is possible to do what you want. I suspect cloning is designed for local machine installs.

other options

Anaconda | Moving Conda Environments

Re: Cannot clone environment

GuentherGrill1 — Thu, 24 Aug 2023 17:13:24 GMT

I had the same error caused by jupyter. I used the suggested workaround of using the command prompt in the link above, and it worked for me.

Re: Cannot clone environment

mdub — Fri, 17 Nov 2023 06:44:53 GMT

I am having this same issue with the recent release of ArcPro 3.2. Rather annoying.

The command prompt workaround did not work for me. I've also done a complete clean uninstall, rolled back to the previous version, but that was affected too.

Re: Cannot clone environment

mdub — Fri, 17 Nov 2023 08:10:29 GMT

Well, I'm not sure how kosher it is, but a simple copy and paste as follows seems to have righted the issue:

From: C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3
To: C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3_clone

I've attached a text file with the failed attempt to clone the Python environment as per @DanPatterson's post above. The error was the same trying to clone with the ArcPro application.

Would still be nice to understand the root cause behind the fail, rather than rest on a hacked workaround!

Re: Cannot clone environment

mdub — Fri, 17 Nov 2023 12:06:14 GMT

Nope, I "wrote" too soon. The problem still persists.

Re: Cannot clone environment

mdub — Sun, 19 Nov 2023 06:52:21 GMT

So to tie up my earlier posts, and for the record I now have my issue fixed. I should also say I am working on a local machine and not a remote Windows environment as per the original post. I responded as it was the exact same error message.

While fixed, I'm still not particularly clear what the root cause is/was, and suspect there is still a problem lurking in the background. The history of community posts concerning failed and broken cloned environments, and the various hack-arounds, would suggest as much.

To eventually get to the fix (well, my fix anyway) I tried:

Cloning via the Command prompt, and running as Administrator as per this thread. This failed with the same error message. A repair via the Package Manager in Arc Pro application failed to remedy it. Post the repair attempt, the clone listing disappeared from the Package Manager, althought the clone environment directory structure remained. Back at the command prompt, running as Administrator, I ran the command: 'conda remove --name arcgispro-py3_clone --all' which removed the listing and deleted the file directory structure as well.
Attempted to clone from Arc Pro (Running as Administrator) via the Package Manager in the backstage area. Again, no joy, same error, and a bunch of dead files in the sub-folders under my 'C:\Users\%User%\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone' and cluttering up the hard drive.
With desparation beginning to set in, I thought maybe there might have been some mismatch/conflict with jupyter installed in another separate python environment, so I uninstalling/re-installing and making sure that was clear and ruled out still didn't remedy anything.
Finally, I went back to cloning via the command prompt, which predictably failed again (Because, you know, trying the thing that didn't work last time, might this time...so now obviously past the point of desperation and resignation is taking over). As per (1), the Package Manager in Arc Pro saw this as a broken clone. However, on a whim I copied the outstanding files from the Source to the Destination environment via Windows explorer, and hey-ho! the Package Manager in Arc Pro saw this is a valid environment, into which I could successfully 'python -m pip install <package>' the obscure packages I was wanting to use via the command prompt.

Thanks to the community for posting their various issues and work-arounds for what seems to be an ongoing and perpetual problem. It helped me distill from the collective wisdom and stumble across the solution above.

While happy it's fixed, not happy to have my weekend written off, nor the spectre of this happening again in future!

Re: Cannot clone environment

JaredPilbeam2 — Wed, 20 Dec 2023 16:00:52 GMT

The accepted answer to this post makes no sense.

Like 3 different attempts later I finally cloned my environment after the upgrade from Pro 3.0 to 3.2. The first attempt was through the Pro Package Manager, but for reasons I'll never know it failed twice. The second time I followed the post from DanPatterson and used the python command prompt, but it didn't finish because of an SSL error.

CondaSSLError: Encountered an SSL error. Most likely a certificate verification issue.

The third and final time I ran it from the prompt again, but this time I turned off ssl from within the arcgispro-py3 folder. Then I ran the clone command.

#turn off ssl conda config --set ssl_verify false #clone environment conda create --clone arcgispro-py3 --name arcgispro_py3_clone --pinned

In this case you could also try adding a certificate: https://community.anaconda.cloud/t/ssl-certificate-verification-error-when-creating-conda-environment/61475

Re: Cannot clone environment

mdub — Fri, 22 Dec 2023 00:01:50 GMT

Hi @JaredPilbeam2,

Sorry the accepted answer made no sense.

Sounds like you got a resolution though?!

Prior to the SSL error, did your initial clone attempts also fail while installing the package 'esri::jupyter_contrib_nbextensions-0.6.0-py_2'?, If so, do you have any insights on a root cause?

I had no SSL error's beyond the issues already outlined in this thread.

The "partial-clone-copy-and-paste-hack" solution that worked for me is ... a hack. I'd still be interested in understanding what the root cause is, though. I switch ArcGIS Pro licenses between two devices and have not upgraded Pro on my other device as I don't want the environment on that device to fall into the same state.

Reading through the log files myself and @JoelMcKechnie provided, it looks like there was a write failure while running a post-link script for that particular package. For some reason, it can't find the source file or directory it needs to write and/or doesn't have permissions to write it. This doesn't make sense to me: we're cloning a working environment, so if the source environment doesn't have the file, you'd think the source environment shouldn't work either. I ran the clone command as Administrator, so there should be no write permission problems. This write issue is why I went the copy-paste route following failed clone attempt.

The conda documentation says post-link scripts should be avoided whenever possible, so it's curious why they've been employed here. From the logs it appears it's primarily for validation purposes.

Maybe the way conda looks for packages is one avenue to explore as per this (old) Github Issues thread on post-link script errors? I can theoretically see how this could interfere with where conda looks for files during cloning. Could it be that where Anaconda, ESRI and jupyter are all at are not synced? Surely not...

I'm not overly familiar with how Anaconda as a package manager works, so lack the experience and context to guide and focus trouble shooting. I'm also conscious that my history of python environment and package installs - I'm not using virtual envs, they include jupyter and are not Anaconda - could be a source of conflict? Is that even possible?

Merry Christmas/Happy Holidays!

Re: Cannot clone environment

JaredPilbeam2 — Fri, 22 Dec 2023 21:36:26 GMT

Hi @mdub ,

The initial error I was getting was in the Package Manager in Pro. I got the red exclamation icon that said something like "attempt failed". I didn't screenshot it, unfortunately. And there was no error message or ID assigned the error, so I can't explain why it failed.

Re: Cannot clone environment

mdub — Sat, 23 Dec 2023 00:28:19 GMT

Hi @JaredPilbeam2,

Yep, those Package Manager error's are an information desert on their own.

I think if you hover your mouse over the red exclamation icon a context window appears after a pause with the same log file readout of the clone process that you would see in the command prompt if you had cloned that way.

It isn't obvious, particularly with the pause, but does make it possible to get a detailed readout of what went "not right" 🙂

I presume clone attempts etc, and their respective log files, are saved in the conda history. I'd have to do research to find if so and out how to get at it.

Re: Cannot clone environment

DanPatterson — Sat, 23 Dec 2023 14:30:50 GMT

conda history

"C:\ install folder \bin\Python\envs\ clone_name \conda-meta\history"

opens in a text editor

Re: Cannot clone environment

mdub — Sun, 24 Dec 2023 03:43:21 GMT

Dan,

Thanks 🙂

I had a look at my history file and it appears to be a summary of conda commands - and for clone commands - the packages installed. However it doesn't have the Traceback when it fails, as per the file I attached earlier in the thread. Is there a way to get this information? Is it stored somewhere else?

Re: Cannot clone environment

MK13 — Tue, 29 Oct 2024 22:08:11 GMT

Isn't it a security risk to do

conda config --set ssl_verify false