How to download 'external' python packages/versions to ArcGIS Pro cloned environment

[colinB]

Hello,

Is there a way one can download packages that are not within the Esri package manager?

For example, I want to download the latest version of selenium. According to the package manager, this installed version is 3.141, which is miles behind the current version I would like (4.21.0)

when I try and download selenium through the python command prompt: conda install -c esri selenium I get the same issue. Which is obvious since it is doing the above, just through the command prompt.

So I am wondering how I can download packages etc. that are not within this Esri library . Ideally I would like to avoid using pip because I am trying to get other people to download things on their machines, so the less steps they have to download the better. But I understand if that is not possible. Thanks!

[DanPatterson]

conda install  -c defaults selenium

conda install selenium --no-pin

didn't work, so that leaves 

conda-forge/selenium-feedstock: A conda-smithy repository for selenium. (github.com)

which is going to be complicated for the casual user, leaving you to decide whether the current version is really necessary or to decide whether "newer must be better"

[Brian_Wilson]

I usually deal with issues like this by running separate conda environments for the newer modules and leaving arcpy to live in company of its out of date versions.

Are you scraping a web page to pull data from it?

so instead of a single loop

for url in urls:

  pull data from url

  process data

You have to break it up and use two environments

  conda activate selenium

  for url in urls:

    pull data from url

    save it to a file

  conda activate arcpy

  for fc in fcs:

    process data from fc

This is the Esri way anyhow, saving data at every stage to a temporary file.

I did a lot of scraping data like this from state web sites for health data during covid and it was much easier to have a minimal conda environment pulling the data on a schedule and then the data was always just available to process in arcgis tools.

[Ryan-SCDA]

I'm actually in the same boat. I wanted to login to our land titles registry and match up freely available parce numbers with the corresponding title numbers so I could populate my maps. Ideally I'd like to keep it all packaged up neatly in ArcGIS Pro but I'm thinking I might end up grabbing my data outside of pro, putting it into a table and then manipulating the table in Pro. If you do figure it out let us know how you did it. I got the old version installed once, likely by luck, then I uninstalled it and now I can't get either the old or new version installed.

[MicZatorsky_AEC]

I clone the environment in Pro and use PyCharm to install new packages. This has been working reliably for years.