Solving SSL Errors in Python Requests

DougGreen — Wed, 11 Sep 2024 18:00:35 GMT

The Day the Earth Stood Still Web Expired

On September 30th, 2021, Let's Encrypt (a non-profit certificate authority) allowed one of their certificates to expire, causing any certificate that had it as part of it's chain to show as invalid. This caused quite a bit of mayhem but they did eventually issue a replacement and through reboots and other updates, most of the issue went away.

Except for Python virtual environments installed with ArcGIS Pro. I started having some of my automated processes fail around this time and was not able to resolve them even though we had rebooted machines and servers and manually installed the new replacement certificates locally, but nothing worked. Here's a quick example of something that was failing:

import arcpy, os, requests, json logMessages = '' r = requests.session() r.headers.update({'Accept': 'application/json', 'Content-Type': 'application/x-www-form-urlencoded'}) body = ''' [out:json] [timeout:120] ; ( way ["highway"="service"] (43.6192138,-116.4338378,43.6339976,-116.4136293); ); out; >; out skel qt;''' OSM_URL3 = 'https://overpass.kumi.systems/api/interpreter' # reach out to the Overpass API to grab the data try: respSuccess = False numTries = 0 while respSuccess == False and numTries <= 10: numTries += 1 response = r.post(OSM_URL3, data=body, verify=True) if response.status_code == 200: respSuccess = True logMessages += '''Requesting features from OSM: Response Status: {}\n'''.format(response.status_code) except requests.exceptions.RequestException as e: logMessages += '{}\n'.format(e.args[0].reason) print(logMessages)

this would always produce a message like: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)

Because these processes were not priority, it took me a couple months to circle around to digging in a bit more. I just figured out the issue and I want to share because I feel like I only understood the watered-down version of how SSL Certificates work and it appears that my frustration stemmed from that partial understanding. Also, it affected my automation that relied on Python web requests to get data.

How I thought certificate checking works

I originally thought that a certificate just needed validated against some authority and then it would allow traffic to flow.

Not How Certificates Are Verified

How Certificates Actually Work (ish)

My previous understanding caused me to feel stuck because I was experiencing SSL errors trying to download data using python requests and getting failures even though I was able to download the same data using a web browser or other client software. So why was Python the only environment indicating an invalid certificate? Because this is roughly how the verification process actually works.

The crucial detail I was missing was that my own computer had a previously downloaded copy of the certificate and when Python was checking the one received from the remote server against the local, my local copy still showed it was expired.

Almost Solved

Once I understood this, I requested some help from my awesome System Administrator and was shown how to remove an expired certificate from my Windows Certificate Store and install the new one. I know, you're thinking I'm going to show you how to do that but I'm not because it didn't work and there's probably plenty of other sites that will teach you how to do that. I'm posting this because I literally couldn't find it anywhere.

I concluded that Python must not be referencing the certificates in my operating systems certificate store and must use it's own. So I did an uninstall and a fresh install of ArcGIS Pro. This still didn't fix it! So I installed pro on another computer and tried to run the same bit of code on the same network connection. THIS WORKED! But why wouldn't my machine work?

The Real Fix

I did some research and fiddling and found indication that the Requests Library uses it's own certificate bundle (a new term for me) and it was stored at the location provided by running the following code:

import requests print(requests.certs.where())

So, knowing that ArcGIS Pro had already been reinstalled and it didn't help, the installer must not have replaced this file, it must have just checked it and moved on or appended data to it? Not sure there. Either way, I renamed the old one and put a copy of the cert found on the other machine after a new install and BAM! My machine could now perform a web request through Python without throwing an SSL error.

The Easier Fix

If you're having this same issue, you can most likely do an uninstall of ArcGIS Pro, then make sure to delete the certificate in the location indicated by the requests.certs.where() function before re-installing pro.

I'm sure that it's not recommended to tweak too much with most of the items in a Conda virtual environment, but I have no need to have a custom virtual environment because I'm using only libraries that are already included when you install ArcGIS Pro. However, in this one instance, this did the trick.

I'm hopeful that there's a better way to have a certificate in a virtual environment be updated by running a command and that someone may be able to shine some light on this topic.

Re: Solving SSL Errors in Python Requests

Nabilkamal — Tue, 14 Feb 2023 12:52:12 GMT

I had the same issue when trying to decode some addresses using Nominatim. I really want to thank you for you suggestion, I used cacert.pem from my friend laptop, who the didn't got the error using arcpy.

Re: Solving SSL Errors in Python Requests

DougGreen — Tue, 14 Feb 2023 15:21:44 GMT

Glad to hear. I feel like there must have been something wrong with my install for it not to have updated that file when I reinstalled. Happy coding!

Re: Solving SSL Errors in Python Requests

SRK — Wed, 12 Feb 2025 13:40:20 GMT

For everybody who wants to use the local machines certificate store on windows to download a file instead of the certifi bundle (for example if you are behind a corporate firewall using MitM) i would suggest using urllib (not urllib3) instead of requests and loading the local cerificates at runtime.

with open(zip_path, 'wb') as f: ssl_context = ssl.create_default_context() ssl_context.load_default_certs() with urllib.request.urlopen(url, context= ssl_context) as response: f.write(response.read())

Re: Solving SSL Errors in Python Requests

AndreaB_ — Wed, 01 Apr 2026 20:17:19 GMT

Hi from 5 years later!

I'm running into this error when I schedule a model to run at night -

File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\ssl.py", line 588, in _load_windows_store_certs

self.load_verify_locations(cadata=certs)

ssl.SSLError: [RSA: WRONG_SIGNATURE_LENGTH] wrong signature length (_ssl.c:4047)

The model runs fine from ArcGIS Pro model window. And this error just started happening. It was running fine for a year.

Thank you for your post! You gave me more insight into SSL than I had before. I still haven't solved the error but I'm contemplating uninstalling and reinstalling ArcGIS Pro 3.5.6 to try your idea.

Re: Solving SSL Errors in Python Requests

DougGreen — Thu, 09 Apr 2026 15:36:50 GMT

@AndreaB_ That's a new one. Looks like you're getting some good help through your post on the forum. I'll point everyone there who finds this first: https://community.esri.com/t5/arcgis-enterprise-questions/ssl-certificates-portal-and-arcgis-server-import/td-p/1693570.

Re: Solving SSL Errors in Python Requests

AleksandarMaricak — Fri, 17 Apr 2026 12:18:19 GMT

Thank you so much! This helped me get past the issue where my script didn't read OS certificates.

topic Re: Solving SSL Errors in Python Requests in Python Questions

Solving SSL Errors in Python Requests

The Day the Earth Stood Still Web Expired

How I thought certificate checking works

Not How Certificates Are Verified

How Certificates Actually Work (ish)

Almost Solved

The Real Fix

The Easier Fix

Re: Solving SSL Errors in Python Requests

Re: Solving SSL Errors in Python Requests

Re: Solving SSL Errors in Python Requests

Re: Solving SSL Errors in Python Requests

Re: Solving SSL Errors in Python Requests

Re: Solving SSL Errors in Python Requests