ArcGIS Pro 2.8.1 bug in urllib3 with proxy server

[davedoesgis]

I have some Python code that has worked fine for a long time, but just broke upon upgrading to Pro 2.8.1. The code uses a cloned conda environment from Pro (but I think you'll have this same problem in the default one, too). This isn't a GIS issue; I am just using the Python 'requests' library to download a file from the Internet (with the requests.get() method). I have to go through a proxy server at work, and had no problems downloading a test file that I use for development, which is hosted on a server on our network. As soon as I flip it over to download from the real source on the Internet (i.e.: not on our local network), I get an SSLError exception with this message: 

SSLError: HTTPSConnectionPool(host='<hostname>', port=443): Max retries exceeded 
with url: <URL here> (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] 
wrong version number (_ssl.c:1091)')))

I've already found the solution, which I will post below so I can mark it correct. Keep reading. 

[davedoesgis · Accepted solution]

Some Googling and following links led me to try a bunch of stuff, but eventually this is what worked: https://stackoverflow.com/a/66656188

I don't have admin rights to my machine, so I have to work in cloned copies of the Conda environment. Even if you can tinker with the default Conda env, I don't recommend it, so do this in a clone. Activate your cloned env in the app "Python Command Prompt". Your clone's root path should be here: 

C:\Users\<user>\AppData\Local\ESRI\conda\envs\<env_name>

Navigate to the Scripts subfolder and issue this command: 

.\pip.exe install --proxy <proxy_URL> urllib3==1.23

You may be able to issue this command from elsewhere, but I like to be explicit about which env I'm in and which pip.exe I'm calling. 

This will downgrade urllib3 to version 1.23. I got the following output from this, including an error partway through, but it ultimately said it succeeded, and most importantly, my code works now. 

Collecting urllib3==1.23
Downloading urllib3-1.23-py2.py3-none-any.whl (133 kB)
|████████████████████████████████| 133 kB 123 kB/s
Installing collected packages: urllib3
Attempting uninstall: urllib3
Found existing installation: urllib3 1.26.4
Uninstalling urllib3-1.26.4:
Successfully uninstalled urllib3-1.26.4
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
requests-negotiate-sspi 0.0.0 requires pypiwin32>=223, which is not installed.
Successfully installed urllib3-1.23

This isn't an Esri bug, per se, since it is with external libraries. But it is a version incompatibility in the libraries they deliver in Conda. And since this only appears in IT environments with a proxy server, I'm guessing that escaped Esri's unit tests. Given the prevalence of proxy servers in their largest clients' IT environments, if anyone at Esri reads this, please make sure you're testing how things work with proxy servers. I had one other error pop up in 2.8.1 related to a proxy server. 

[davedoesgis · Accepted solution]

Update: I was able to get this to work with the Esri env as cloned, but it did require changes to my code that have worked for years.

The requests.get() method has a proxy dict you can pass in for your proxy server settings. It has two keys: 'http' and 'https'. The values for those keys are the proxy server URL for each protocol. This issue I encountered is an https request (almost all traffic these days is https). 

My proxy URL for the 'https' dict key started with 'https://' and included ':80' port at the end. This has worked fine for years, but broke in Pro 2.8.1. Using the default conda env as cloned, I changed that URL to start with 'http://' and removed the ':80' port number at the end and it worked. 

[Anonymous User]

Thanks for the info, got me halfway there after upgrading to Pro 2.8. I had to change the https proxy url to use 'http://' as well and that got normal requests working. However, I still can't get a request to ArcGIS Online to work via the Python for ArcGIS API. This works for requests:

proxyDict = {
    'https': 'http://user:pwd@proxyURL:8080',
    'http': 'http://user:pwd@proxyURL:8080'
    }

r=requests.get('https://httpbin.org/ip', proxies=proxyDict)
print(r.json())

 But if I try it with a ArcGIS for Python API call, with any of the following, it doesn't work:

my_gis = gis.GIS(my_gis_url, username, password, verify_cert=False, proxy_host='user:pwd@proxyURL', proxy_port=8080)

my_gis = gis.GIS(my_gis_url, username, password, verify_cert=False, proxy_host='user:pwd@proxyURL', proxy_port=8080, proxy=proxyDict)

my_gis = gis.GIS(my_gis_url, username, password, verify_cert=False, proxy=proxyDict)

Error:
Traceback (most recent call last):
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\util\connection.py", line 96, in create_connection
raise err
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\util\connection.py", line 86, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

I've also tried setting the proxy environments but that hasn't worked either for the API:

os.environ['http_proxy'] = "http://user:pwd@proxyURL:8080"
os.environ['https_proxy'] = "http://user:pwd@proxyURL:8080"

[davedoesgis]

Regarding the arcgis.gis.GIS() connection, I had to drop my proxy_host and proxy_port options to get it to work at 2.8. I guess it is now somehow detecting the proxy settings on its own now. It was a head scratcher as to why it works, but that was the only way it would work for me. If that doesn't solve it for you, I recommend starting a new thread, since that's getting into a separate issue from the original question. 

[Anonymous User]

@AndrewChapkowski any chance you know what's going on here? I can't seem to get a connection to ArcGIS Online to work despite it working with normal requests.

my_gis = gis.GIS(my_gis_url, username, password, verify_cert=False)

proxies set to: {'http': 'http://user:pwd@proxyURL:8080', 'https': 'http://user:pwd@proxyURL:8080'}

Noting the 'https' url is set to http:// and not https://.

Error:

Traceback (most recent call last):
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\util\connection.py", line 96, in create_connection
raise err
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\util\connection.py", line 86, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py", line 706, in urlopen
chunked=chunked,
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connection.py", line 353, in connect
conn = self._new_conn()
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connection.py", line 182, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x000001E7CC123FC8>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\adapters.py", line 449, in send
timeout=timeout
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py", line 756, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\util\retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='orgName.maps.arcgis.com', port=443): Max retries exceeded with url: /sharing/rest/generateToken (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000001E7CC123FC8>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 687, in post
files=files)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 590, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='orgName.maps.arcgis.com', port=443): Max retries exceeded with url: /sharing/rest/generateToken (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000001E7CC123FC8>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\username\OneDrive\Python\List_ExportUsersInGroup.py", line 49, in <module>
my_gis = gis.GIS(my_gis_url, username, password, verify_cert=False)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 359, in __init__
raise e
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 343, in __init__
trust_env=kwargs.get("trust_env", None))
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 173, in __init__
self.get_properties(True)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 1138, in get_properties
raise e
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 1126, in get_properties
resp = self.con.post(path, self._postdata(), ssl=True)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 632, in post
elif token_as_header == False and self.token is not None: #as ?token=
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1038, in token
self._token = self._enterprise_token()
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 1155, in _enterprise_token
add_token=False)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 696, in post
"A connection error has occurred: %s" % errCE)
requests.exceptions.ConnectionError: A connection error has occurred: HTTPSConnectionPool(host='orgName.maps.arcgis.com', port=443): Max retries exceeded with url: /sharing/rest/generateToken (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000001E7CC123FC8>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

[davedoesgis]

@Anonymous User - I suggest creating a new topic for your question. This post is old and marked as resolved, so you might get more traction elsewhere. Also. although what you're asking might be tangentially related, it's getting a bit off topic of using the Python requests lib.