Hello All,
I'm trying to run a script to access an ArcGIS Online REST web service and am behind an orginizational firewall, so I need to go over an authenticated proxy. My understanding is that urllib2 is supposed to pull the proxy settings from my system automatically so I shouldn't need to add any code dealing with that, but I keep getting a socket timeout error (60010) right after the script attempts to establish a connection -- I think this has to do with the proxy, but maybe I'm wrong. Has anyone else run into this problem and found a sollution? I tried importing the'socket' module and setting the timout value to 'NONE', as well as setting the expiration to a larger value, but the error keeps appearing after 1 minute. I've attached the part of my code dealing with the connection to the service -- does anyone out there have any ideas? Any feedback would be highly appreciated!
import json, urllib, urllib2 import sys, os import arcpy import shutil class GetToken(object): # find feature service attachments, send to directory def urlopen(self, url, data=None): # open url, send response referer = "http://www.arcgis.com/arcgis/rest" req = urllib2.Request(url) req.add_header('Referer', referer) if data: response = urllib2.urlopen(req, data) else: response = urllib2.urlopen(req) return response def gentoken(self, username, password, referer = 'www.arcgis.com', expiration=60): # gets token from referrer query_dict = {'username': username, 'password': password, 'expiration': str(expiration), 'client': 'referer', 'referer': referer, 'f': 'json'} query_string = urllib.urlencode(query_dict) token_url = "https://www.arcgis.com/sharing/rest/generateToken" token_response = urllib.urlopen(token_url, query_string) token = json.loads(token_response.read()) if "token" not in token: print token['messages'] exit() else: return token['token']
No, you need to configure your proxies in your code. See this thread on Stack Overflow.
Thanks for your response, Jason. I've tried all the variations mentioned on the link you sent, as well as others. Right now the top of my script looks like below, and I also set my environment variables appropriately for http_proxy and https_proxy. Right now I'm getting an error on the splituser def inside of urllib telling me that a string or buffer is expected.
import sys, os import arcpy import shutil import re # set proxies proxy = urllib2.ProxyHandler({ 'http': '192.168.104.103', 'https': '192.168.104.103' }) opener = urllib2.build_opener(proxy) urllib2.install_opener(opener) class GetToken(object): # find feature service attachments, send to directory def urlopen(self, url, data=None): # open url, send response referer = "http://www.arcgis.com/arcgis/rest" req = urllib2.Request(url) req.add_header('Referer', referer) if data: response = urllib2.urlopen(req, data) else: response = urllib2.urlopen(req) return response def gentoken(self, username, password, referer = 'www.arcgis.com', expiration=60): # gets token from referrer query_dict = {'username': username, 'password': password, 'expiration': str(expiration), 'client': 'referer', 'referer': referer, 'f': 'json'} query_string = urllib.urlencode(query_dict) token_url = "https://www.arcgis.com/sharing/rest/generateToken" token_response = urllib.urlopen(token_url, query_string) token = json.loads(token_response.read()) if "token" not in token: print token['messages'] exit() else: return token['token']
Can you post the traceback?
yep, here's the traceback:
Traceback
<module> C:\Users\tle\Desktop\Scripts\dl_extract2.py 161
main C:\Users\tle\Desktop\Scripts\dl_extract2.py 157
__init__ C:\Users\tle\Desktop\Scripts\dl_extract2.py 53
gentoken C:\Users\tle\Desktop\Scripts\dl_extract2.py 40
urlopen C:\Python26\ArcGIS10.0\lib\urllib.py 88
open C:\Python26\ArcGIS10.0\lib\urllib.py 207
open_https C:\Python26\ArcGIS10.0\lib\urllib.py 439
endheaders C:\Python26\ArcGIS10.0\lib\httplib.py 904
_send_output C:\Python26\ArcGIS10.0\lib\httplib.py 776
send C:\Python26\ArcGIS10.0\lib\httplib.py 735
connect C:\Python26\ArcGIS10.0\lib\httplib.py 1112
wrap_socket C:\Python26\ArcGIS10.0\lib\ssl.py 350
__init__ C:\Python26\ArcGIS10.0\lib\ssl.py 118
do_handshake C:\Python26\ArcGIS10.0\lib\ssl.py 293
IOError: [Errno socket error] [Errno 1] _ssl.c:480: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
This actually turned out to be a pretty easy fix. All that were needed in my case were the system variables:
http_proxy: http//:user:password@server:port
https_proxy: http//:user:password@server:port
This link explains why the http protocol is used for both.
The following code was not needed -- in fact, the script threw a 404 error when it was included:
proxy = urllib2.ProxyHandler({
'http': '192.168.104.103',
'https': '192.168.104.103'
})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)
One more discovery:
Turns out that (in my situation, ymmv) the authentication is key. So setting the username in the proxy"definition" is mandatory. Either in the environment setting or in proxies = urllib2.ProxyHandler I needed to add http://username:password@abc.com:1234
hth, Bert