I am checking to see if I can access a web site before extracting content However, I get an intermittent 401 error (in bold below) which is resolved when I clear cookies, and/or log off and log back in again.
Http Error: 401 Client Error: Authorization Required for url: http://clienturl:8080/auth/?du=aHR0cDovL3N1c3NleC5tYXBzLmFyY2dpcy5jb20v&ip=MTAuMzcuODEuMjA1OjgwODA=
I do not need a login/password for the site I am trying to access, but it seems on the client side,
a token is generated, because if I paste the url from the http error message, into my browser, the site displays correctly.
I need to run this script as a batch job, so this 401 error will be an issue. Is there a way around this problem
Subset of my Code:
req = requests.get("http://somewebsite")
if req.status_code == 200:
#check last run date
with open('LastRunDate.csv', 'r+', newline='') as f:
reader = csv.reader(f)
for row in reader:
lastrundate = row[0]
pattern = '%Y%m%d%H%M%S'
epochdate = int(time.mktime(time.strptime(lastrundate, pattern)))
epochdate = epochdate * 100
gis2 = GIS("https://somewebsite.com
newitems = gis2.content.search(query="", max_items = 1000)
filename = ('ListofAvailableitems_' + str(now.day).zfill(2) + str(now.month).zfill(2) + str(now.year)+ '.csv')
with open(filename, 'w', newline='') as csvfile:
header = ["County", "Title", "Type", "Item", "Owner", "Date-Created"]
filewriter = csv.writer(csvfile, quoting=csv.QUOTE_MINIMAL)
filewriter.writerow(header)
for item in newitems:
# check item created date to see if it is greater than the epoch date
if item.created > epochdate:
filewriter.writerow(['County1', item.title, item.id, item.type, item.owner, datetime.datetime.fromtimestamp(item.created/1000.0).strftime('%d-%m-%Y')])
else:
try:
req.raise_for_status()
except requests.exceptions.HTTPError as errh:
print ("Http Error:",errh)
except requests.exceptions.ConnectionError as errc:
print ("Error Connecting:",errc)
except requests.exceptions.Timeout as errt:
print ("Timeout Error:",errt)
except requests.exceptions.RequestException as err:
print ("OOps: Something Else",err)
