topic Re: Use Python-API behind Proxy in ArcGIS API for Python Questions

Use Python-API behind Proxy

MU — Sun, 12 Dec 2021 16:59:15 GMT

Hi,
I'm trying to use the new Python-API with our on-premise system. Our company runs it's own public key infrastructure, so I've configured Portal for ArcGIS with a corresponding certificate. Furthermore, there is a company-wide proxy that we are using.

As a first throw, I'd like to connect to ArcGIS Online by doing this:

from arcgis.gis import GIS
my_gis = GIS()

This crashes with:

c:\Program Files\ArcGIS\Pro\bin\Python>python.exe "C:\temp\Test.py"
Traceback (most recent call last):
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 1254
, in do_open
 h.request(req.get_method(), req.selector, req.data, headers)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 1106, i
n request
 self._send_request(method, url, body, headers)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 1151, i
n _send_request
 self.endheaders(body)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 1102, i
n endheaders
 self._send_output(message_body)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 934, in
 _send_output
 self.send(msg)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 877, in
 send
 self.connect()
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 1252, i
n connect
 super().connect()
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 853, in
 connect
 self._tunnel()
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\http\client.py", line 832, in
 _tunnel
 message.strip()))
OSError: Tunnel connection failed: 407 authenticationrequired

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "C:\temp\Test.py", line 7, in <module>
 my_gis = GIS()
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\gis.py",
 line 95, in __init__
 verify_cert=self._verify_cert, client_id=self._client_id)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\_impl\po
rtalpy.py", line 160, in __init__
 client_id=client_id)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\_impl\co
nnection.py", line 281, in __init__
 self.login(username, password, expiration, client_id)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\_impl\co
nnection.py", line 504, in login
 resp = self.post('', { 'f': 'json' }, add_token=False) # probe portal to fin
d auth scheme
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\site-packages\arcgis\_impl\co
nnection.py", line 1031, in post
 resp = opener.open(url, data=encoded_postdata.encode())
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 466,
 in open
 response = self._open(req, data)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 484,
 in _open
 '_open', req)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 444,
 in _call_chain
 result = func(*args)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 1297
, in https_open
 context=self._context, check_hostname=self._check_hostname)
 File "c:\Program Files\ArcGIS\Pro\bin\Python\lib\urllib\request.py", line 1256
, in do_open
 raise URLError(err)
urllib.error.URLError: <urlopen error Tunnel connection failed: 407 authenticati
onrequired>

Is it related to some proxy issues and if yes, where do I have to configure the proxy?

Re: Use Python-API behind Proxy

MU — Tue, 13 Jun 2017 14:56:40 GMT

After configuring everything from scratch, I ended up with this: urllib.error.URLError: <urlopen error [WinError 10061]

A bit of searching the web confirms my suspicion that it is related to the proxy. Our proxy requires authentication and urlopen should actually consider Windows' proxy settings (configured within Internet Explorer), but it doesn't seem to work.

I've also tested whether the environment variables http_proxy or https_proxy make any difference - nope

So any help would be appreciated!

Re: Use Python-API behind Proxy

BenjaminBlackshear — Fri, 18 Aug 2017 14:55:50 GMT

I am also struggling with this, there is no documentation related to configuring proxy settings, ESRI please publish this.

Re: Use Python-API behind Proxy

RohitSingh2 — Thu, 24 Aug 2017 15:54:24 GMT

The Python API currently doesn't support proxies. We will be adding this support to the next release (release data TBD). Till then, can you see if you can use the workaround at The lazy Flimbot: Using ArcGIS API for Python (1.0.1) behind a proxy ?

Re: Use Python-API behind Proxy

ericjetter — Mon, 04 Sep 2017 13:48:12 GMT

Hi,
do you know the next release date? we need the proxy support.
Thanks,

Eric

Re: Use Python-API behind Proxy

RohitSingh2 — Sun, 12 Dec 2021 10:13:32 GMT

We released v1.2.3 with support for proxy servers on Friday.

In addition to explicitly named variable, the GIS object supports optional key word
arguments

================    ===============================================================
**kwargs**          **Description**
----------------    ---------------------------------------------------------------
proxy_host          optional string, host name of the proxy server
----------------    ---------------------------------------------------------------
proxy_port          optional integer, proxy host port.  The default is 80.
================    ===============================================================


Usage Example: Anonymous ArcGIS Online Login with Proxy

gis = GIS(proxy_host='127.0.0.1', proxy_port=8888)

Re: Use Python-API behind Proxy

ericjetter — Tue, 12 Sep 2017 06:57:47 GMT

Hi,

thanks for your answer.

But the new version is not available on https://developers.arcgis.com/python/?

Eric

Re: Use Python-API behind Proxy

RohitSingh2 — Tue, 12 Sep 2017 14:17:41 GMT

It is available on the esri conda channel - Files :: Anaconda Cloud

We'll be updating the developer site later this month.

Re: Use Python-API behind Proxy

RebeccaStrauch__GISP — Tue, 12 Sep 2017 15:32:30 GMT

rsingh-esristaff ,

I used

conda upgrade -c esri arcgis

in the Anaconda Prompt window (today) based on the instructions in https://community.esri.com/groups/arcgis-python-api/blog/2017/07/17/arcgis-api-for-python-12-released and it looks like is grabbed the 1.2.3 release.

Does the above command grab from the esri conda channel ? And if is does, will it always grab a "ready for release" version and not a preliminary release?

Note: I found using the command line has worked better than trying to do the upgrade thru Pro...however, I may be doing that incorrectly or mixing apples with oranges. However, the above command is very straight forward for me.

Re: Use Python-API behind Proxy

RohitSingh2 — Tue, 12 Sep 2017 15:38:54 GMT

The command

conda upgrade -c esri arcgis

grabs from the esri conda channel and it will always grab the latest released version. It will not get a preliminary version.

In this command "-c esri" specifies that it should use the esri conda channel.

If you wish you upgrade to a specific version you can specify it as:

conda upgrade -c esri arcgis=1.2.0

Re: Use Python-API behind Proxy

AnthonyRyanEQL — Tue, 12 Sep 2017 23:15:42 GMT

Rohit,

Does it support proxy authentication as well?

Thanks

Re: Use Python-API behind Proxy

ericjetter — Thu, 14 Sep 2017 07:07:36 GMT

Hi Rohit,

we also need proxy authentication!

Thanks,

Eric

Re: Use Python-API behind Proxy

RohitSingh2 — Mon, 18 Sep 2017 10:41:06 GMT

Hi Eric, Anthony,

For secure proxies, can you try the following:

# username : password @ server  gis = GIS(proxy_host="username:password@www.myproxy.com", proxy_port=542)

If it doesn't work, please let me know what kind of proxy server you are using.

Thanks,

Rohit

Re: Use Python-API behind Proxy

ericjetter — Thu, 21 Sep 2017 12:40:18 GMT

Hi Rohit,

this works fine - thanks.

Eric

Re: Use Python-API behind Proxy

RajkumarSingh — Fri, 23 Feb 2018 21:16:12 GMT

Does this support a PAC based proxy?

Re: Use Python-API behind Proxy

GISUser17 — Mon, 20 May 2019 15:03:17 GMT

Hi Rohit,

The above connection string is not working for me. Is there anything that can be done?

Thanks.

Richard.

Re: Use Python-API behind Proxy

TylerBurns — Wed, 15 Jan 2020 19:14:04 GMT

We do not currently support PAC-based proxies in the Python API. Neither the standard library nor requests (which Python API will use for web requests) support these out of the box, so the API would have to be designed specifically to handle the information the PAC file provides.

There is a package called pypac (https://github.com/carsonyl/pypac) that does this functionality, but we do not currently have the infrastructure to build this for designing and testing at this time.

Re: Use Python-API behind Proxy

AndresCastillo — Thu, 05 Mar 2020 21:49:12 GMT

I am at v 1.7.1, and I see that now proxies are available as kwargs for sure!

arcgis.gis module — arcgis 1.7.1 documentation

Re: Use Python-API behind Proxy

AndresCastillo — Sun, 12 Dec 2021 10:13:35 GMT

Hi Rohit Singh and Tyler Burns,

It still doesn't work fine for me:

import csv,os, sys, datetime, base64
from arcgis.gis import GIS
from getpass import getpass
# import urllib2



print('===================')
print('The script that is running: ' + __file__)
print('First you have to log in to Portal for ArcGIS')

# Log in
username = 'axc013o' 
# username = 'domain\\axc013o' 
# password = os.environ.get('USERPASS')
# password = getpass(prompt='Password: ', stream=None)
password = base64.b64decode('randomcharacters').decode('utf-8')


# proxy = urllib2.ProxyHandler({'http': 'http://proxyServer:8080', 'https': 'https://proxyServer:8080'})
# opener = urllib2.build_opener(proxy)
# urllib2.install_opener(opener)

portalURL = 'https://portalDomain/portal/'
# portalURL = 'https://portalHostname:7443/arcgis/home/'



myPortal = GIS(portalURL, username, password)
# myPortal = GIS(portalURL, username, password, proxy_host=f'{username}:{password}@proxyServer', proxy_port=8080)
# myPortal = GIS(portalURL, username, password, proxy_host=f'{username}:{password}@10.123.8.111', proxy_port=8080)
# verify_cert=False does not fix issue.
# another one of our proxy servers also doesn't work.

print ("Start: "+datetime.datetime.today().strftime('%c'))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Traceback (most recent call last):
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1036, in _send_output
    self.send(msg)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 974, in send
    self.connect()
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1407, in connect
    super().connect()
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 950, in connect
    self._tunnel()
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 929, in _tunnel
    message.strip()))
OSError: Tunnel connection failed: 503 Service Unavailable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\AXC013O\Downloads\z\getUsers.py", line 29, in <module>
    myPortal = GIS(portalURL, username, password)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 325, in __init__
    raise e
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 310, in __init__
    referer=self._referer)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\portalpy.py", line 163, in __init__
    client_id=client_id)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 308, in __init__
    self.login(username, password, expiration, client_id)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 554, in login
    resp = self.post('', { 'f': 'json' }, add_token=False) # probe portal to find auth scheme
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 1131, in post
    resp = opener.open(url, data=encoded_postdata.encode())
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 526, in open
    response = self._open(req, data)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 544, in _open
    '_open', req)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error Tunnel connection failed: 503 Service Unavailable>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I even tried putting the proxy server in my environment variables, as well as turning off the pac server, as shown in the attached png.

Can you please suggest any tips?

Re: Use Python-API behind Proxy

Brian_Wilson — Thu, 30 Dec 2021 18:41:43 GMT

I am using arcgis.__version__ = 1.8.5 and the error I am getting is

"A general error occurred: check_hostname requires server_hostname"

gis = arcgis.gis.GIS(PORTAL_URL, USER, PASSWORD, proxy_host="10.10.10.210", proxy_port=8080)

When I use the code in a jupyter notebook I get a very long stack trace with basically the same error.

~\AppData\Local\ESRI\conda\envs\arcgis_tools\lib\ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)

826 if context.check_hostname and not server_hostname: -->

827 raise ValueError("check_hostname requires server_hostname")

828 ValueError: check_hostname requires server_hostname During handling of the above exception, another exception occurred: