Please we aware that ESRI's latest documentation on how to deploy third-party Python packages with ArcGISServer omits a very important step. In order to create a clone that ArcGIS Server will have access to, you need to make sure that the clone is created under the ArcGIS Server account. This bit is specified in earlier documentation (for example, see the documentation for 11.1) but seems to have been removed starting at 11.2.
I only wanted to mention it here to hopefully save someone else from having to go through a bunch of unnecessary troubleshooting. After a week+ of testing we reached out to ESRI tech support and they had us create a clone under the ArcGIS Server account which fixed our issue. It was confirmed by the ESRI analyst that this is still a requirement (we are running Server 11.3) despite the fact that is not mentioned in the current documentation.
I have reached out to ESRI through the Feedback link and suggested they update the documentation as well.
Our team struggled with this as well. Our approach was to explicitly add Read/Execute privileges for the ArcGIS Server active directory service account.
My best understanding is that in Windows, files generally inherit privileges from their parent container (at least where inheritance is enabled). However, when you create or update a Conda environment, it isn't simply creating or updating files in that \arcgispro-py3-clone directory but it is moving a bunch of files back and forth between different directories and executing other installation scripts. Lots of those files that get copied into the clone directory have the privileges of their parent container which do not include privileges for that service account. I found an interesting post on stackoverflow exploring similar issues. Essentially, the AD server account couldn't use the cloned environment because it didn't have access to some of the site packages and libraries.
To remedy this, I created a simple batch script that applies those privileges to the clone directory:
icacls "C:\<clone_directory>" /grant agserviceaccount:RX /T
One of the limitations to this approach is that any time you change that Conda environment, you will need to update permissions. It does, however, allow any user to create and maintain environment for ArcGIS Server. Hope that helps!
Thanks @WilliamHarper10, great workaround! I wish someone from ESRI would chime in and offer their opinions on this. But in the meantime, glad you have a solution!