I am using ArcGIS PRO 2.6.2. On that, Python version is 3.6.10 is installed. Is it okay to upgrade the python from 3.6.10 to 3.7 or above using conda environment?
Pro versions are tied to a specific version of python. If you want a newer version of python to work with pro, then you have to upgrade Pro to a more recent version
As @DanPatterson mentions, currently upgrading Pro is the right path. Libraries which use the Python C API are versioned against the release, and aren't cross-compatible with other releases of Python. That said, Python 3.7 or 3.9 are the releases available in all supported versions of Pro, so you're probably due for an update in any case. Python 3.7 was still receiving security fixes until last year, but Python 3.6 has been unsupported for a couple years now.