Uninstall Pro, then rename the remnants of the installation folder by adding _old to it.
There will be remnants of the installation, so don't miss this step.
Similarly, rename the C:\Users\... You ... \AppData\Local\ESRI by adding _old to it
Download the *.exe again if you didn't and get ready to install it.
Run the *.exe. and it will produce a *.msi which does the actual installation... keep it if you have to reinstall
Since you are a single user, there is no reason to let it install Pro where it wants.
I use C:\arc_pro as my installation folder.
Install Pro as the sole user into that folder.
You can install packages into the base environment without issue or revert using conda install --revision 0 if you keep record of what you installed using
conda install blah-blah --no-pin --dry-run
first. Anything that wants to change what Pro has installed isn't a good idea, nor is it anything that wants to compete with arcpy.
numpy, Spyder, matplotlib, scipy and any of the stack-related packages can be safely installed and/or upgraded within the base environment.
If you plan to do the deep-learning stuff you need a cloned environment or better yet, a separate machine and a real reason why you need to use it.... stick to shallow-learning instead 😉
Save your own environment.yml file when you are done.
... sort of retired...