Use Python-API behind Proxy

MU · ‎06-13-2017

Hi,
I'm trying to use the new Python-API with our on-premise system. Our company runs it's own public key infrastructure, so I've configured Portal for ArcGIS with a corresponding certificate. Furthermore, there is a company-wide proxy that we are using.

As a first throw, I'd like to connect to ArcGIS Online by doing this:

from arcgis.gis import GIS
my_gis = GIS()

This crashes with:

c:\Program Files\ArcGIS\Pro\bin\Python>python.exe "C:\temp\Test.py"
Traceback (most recent call last):
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 1254
, in do_open
 h.request(req.get_method(), req.selector, req.data, headers)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 1106, i
n request
 self._send_request(method, url, body, headers)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 1151, i
n _send_request
 self.endheaders(body)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 1102, i
n endheaders
 self._send_output(message_body)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 934, in
 _send_output
 self.send(msg)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 877, in
 send
 self.connect()
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 1252, i
n connect
 super().connect()
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 853, in
 connect
 self._tunnel()
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 832, in
 _tunnel
 message.strip()))
OSError: Tunnel connection failed: 407 authenticationrequired

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "C:\temp\Test.py", line 7, in <module>
 my_gis = GIS()
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\gis.py",
 line 95, in __init__
 verify_cert=self._verify_cert, client_id=self._client_id)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\_impl\po
rtalpy.py", line 160, in __init__
 client_id=client_id)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\_impl\co
nnection.py", line 281, in __init__
 self.login(username, password, expiration, client_id)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\_impl\co
nnection.py", line 504, in login
 resp = self.post('', { 'f': 'json' }, add_token=False) # probe portal to fin
d auth scheme
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\_impl\co
nnection.py", line 1031, in post
 resp = opener.open(url, data=encoded_postdata.encode())
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 466,
 in open
 response = self._open(req, data)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 484,
 in _open
 '_open', req)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 444,
 in _call_chain
 result = func(*args)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 1297
, in https_open
 context=self._context, check_hostname=self._check_hostname)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 1256
, in do_open
 raise URLError(err)
urllib.error.URLError: <urlopen error Tunnel connection failed: 407 authenticati
onrequired>

Is it related to some proxy issues and if yes, where do I have to configure the proxy?

AnthonyRyanEQL · ‎09-12-2017

Rohit,

Does it support proxy authentication as well?

Thanks

ericjetter · ‎09-14-2017

Hi Rohit,

we also need proxy authentication!

Thanks,

Eric

RohitSingh2 · ‎09-18-2017

Hi Eric, Anthony,

For secure proxies, can you try the following:

# username : password @ server  gis = GIS(proxy_host="username:password@www.myproxy.com", proxy_port=542)

If it doesn't work, please let me know what kind of proxy server you are using.

Thanks,

Rohit

ericjetter · ‎09-21-2017

Hi Rohit,

this works fine - thanks.

Eric

GISUser17 · ‎05-20-2019

Hi Rohit,

The above connection string is not working for me. Is there anything that can be done?

Thanks.

Richard.

AndresCastillo · ‎03-05-2020

Hi Rohit Singh and Tyler Burns,

It still doesn't work fine for me:

import csv,os, sys, datetime, base64
from arcgis.gis import GIS
from getpass import getpass
# import urllib2



print('===================')
print('The script that is running: ' + __file__)
print('First you have to log in to Portal for ArcGIS')

# Log in
username = 'axc013o' 
# username = 'domain\\axc013o' 
# password = os.environ.get('USERPASS')
# password = getpass(prompt='Password: ', stream=None)
password = base64.b64decode('randomcharacters').decode('utf-8')


# proxy = urllib2.ProxyHandler({'http': 'http://proxyServer:8080', 'https': 'https://proxyServer:8080'})
# opener = urllib2.build_opener(proxy)
# urllib2.install_opener(opener)

portalURL = 'https://portalDomain/portal/'
# portalURL = 'https://portalHostname:7443/arcgis/home/'



myPortal = GIS(portalURL, username, password)
# myPortal = GIS(portalURL, username, password, proxy_host=f'{username}:{password}@proxyServer', proxy_port=8080)
# myPortal = GIS(portalURL, username, password, proxy_host=f'{username}:{password}@10.123.8.111', proxy_port=8080)
# verify_cert=False does not fix issue.
# another one of our proxy servers also doesn't work.

print ("Start: "+datetime.datetime.today().strftime('%c'))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Traceback (most recent call last):
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1036, in _send_output
    self.send(msg)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 974, in send
    self.connect()
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1407, in connect
    super().connect()
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 950, in connect
    self._tunnel()
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 929, in _tunnel
    message.strip()))
OSError: Tunnel connection failed: 503 Service Unavailable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\AXC013O\Downloads\z\getUsers.py", line 29, in <module>
    myPortal = GIS(portalURL, username, password)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 325, in __init__
    raise e
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 310, in __init__
    referer=self._referer)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\portalpy.py", line 163, in __init__
    client_id=client_id)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 308, in __init__
    self.login(username, password, expiration, client_id)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 554, in login
    resp = self.post('', { 'f': 'json' }, add_token=False) # probe portal to find auth scheme
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 1131, in post
    resp = opener.open(url, data=encoded_postdata.encode())
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 526, in open
    response = self._open(req, data)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 544, in _open
    '_open', req)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error Tunnel connection failed: 503 Service Unavailable>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I even tried putting the proxy server in my environment variables, as well as turning off the pac server, as shown in the attached png.

Can you please suggest any tips?

RajkumarSingh · ‎02-23-2018

Does this support a PAC based proxy?

TylerBurns · ‎01-15-2020

We do not currently support PAC-based proxies in the Python API. Neither the standard library nor requests (which Python API will use for web requests) support these out of the box, so the API would have to be designed specifically to handle the information the PAC file provides.

There is a package called pypac (https://github.com/carsonyl/pypac) that does this functionality, but we do not currently have the infrastructure to build this for designing and testing at this time.

AndresCastillo · ‎03-05-2020

I am at v 1.7.1, and I see that now proxies are available as kwargs for sure!

arcgis.gis module — arcgis 1.7.1 documentation

Brian_Wilson · ‎12-30-2021

I am using arcgis.__version__ = 1.8.5 and the error I am getting is

"A general error occurred: check_hostname requires server_hostname"

gis = arcgis.gis.GIS(PORTAL_URL, USER, PASSWORD,
        proxy_host="10.10.10.210", proxy_port=8080)

When I use the code in a jupyter notebook I get a very long stack trace with basically the same error.

~\AppData\Local\ESRI\conda\envs\arcgis_tools\lib\ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)

826 if context.check_hostname and not server_hostname: -->

827 raise ValueError("check_hostname requires server_hostname")

828 ValueError: check_hostname requires server_hostname During handling of the above exception, another exception occurred: