nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

SSL certificates - Portal and ArcGIS Server - Import certificate chain?

AndreaB_

Hi all,

I'm using Enterprise 11.5 and using a DigiCert cert.

I haven't had any SSL errors until now. I'm getting an error while running a scheduled model: File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\ssl.py", line 588, in _load_windows_store_certs
self.load_verify_locations(cadata=certs)
ssl.SSLError: [RSA: WRONG_SIGNATURE_LENGTH] wrong signature length (_ssl.c:4047)

So I thought I would do some digging with Esri Tech support. So we tried some cert stuff but I'm still wondering.

I read these docs: Server and Portal

One thing I noticed in the Portal documentation: I import the .pfx and I use Import Certificate Chain and then it says The alias for these certificates will match the alias entered above and be appended with either _root or _intermediate depending on the type of certificate. Then it says "After importing an existing CA-signed certificate, the root and intermediate certificates may have already been imported. These would be listed under Security > SSLCertificates."

I don't have any certs listed except the one .pfx. Does anyone use Import Certificate Chain and it actually lists the root and intermediate certs?

So now I'm thinking I need to export the root and intermediate certs from the .pfx using IIS - Site Bindings - Edit - View - Certification path - click on the root cert - Details - Copy to File and save as a .cer and import them separately into Portal and ArcGIS Server.

Let me know what you think. Thank you!

Andrea

Find more posts tagged with

Accepted answers

AndreaB_

Here is the resolution! I worked with Esri tech support. Below from Esri Tech support quoted.

“Based on our analysis of the logs, the failure occurs because the background process cannot access the user’s personal certificate store or specific environment paths when the user is logged off. We have identified several PATH NOT FOUND and NAME NOT FOUND errors specifically related to SSL certificate validation and licensing lookups.

To resolve this, I recommend we start by modifying the script itself.

Modify the Python Script: We can explicitly tell the script where to find the necessary certificates, bypassing the Windows Certificate Store entirely.

Ensure you have this file in this location and accessible to the service account (C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\certifi\cacert.pem).
Add the following lines to the very top of your script:

import os
#Point to the specific certificate file
os.environ['REQUESTS_CA_BUNDLE'] = r"C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\certifi\cacert.pem

This worked - but not at 12:30am. "The failure at 12:30 AM is likely due to the Windows environment unloading the user profile or certificates after a few hours of inactivity. "

So I set it to run at 8pm and it works like a charm! 😃

Thank you for helping me out! Hope this helps someone in the future.

All comments

TimWestern

I'm not a cert expert, but one thought I had to ask was.

Is it a wildcard cert for your domain?

I know on some recent work we had to go into CORS and Trusted section and add a bunch of expressly named subdomains even with the wild card sort to stop CORS issues.

TimoT

Hi @AndreaB_

Want to confirm a couple of things to help narrow this down:

What version of ArcGIS Pro are you running?
Do you see the same error if you run the model manually in Pro, or only when it runs as a scheduled task?

At a high level, this looks like Pro is hitting an issue with one of the certificates in the Windows store on the machine (or potentially user profile) that ArcGIS Pro is installed on, rather than anything specifically wrong with your DigiCert cert or your Portal/Server certificate stores.

This looks more like something being triggered in the Python/SSL side in Pro. Pro appears to be attempting to load certificates from the Windows certificate store for SSL/TLS validation, and it basically doesn't like the look of at least one of them.

The WRONG_SIGNATURE_LENGTH error basically means there’s a mismatch between the certificate’s signature and what’s expected from its public key. That said, it doesn’t always mean the cert is literally the wrong size, it can also happen if:

The cert is using a newer signature format that isn’t fully supported by your current version of Pro
There’s a corrupted or malformed cert in the store
A change with your organisation’s internal CA or security setup may have contributed to this

A couple more things that might help pinpoint it:

Is the machine running Pro managed by your organization?
Are you able to try the same workflow on a different machine and see if it behaves the same? Bonus points if you're able to test on a machine not managed by your org.

Depending on results of testing, this may need to involve your IT to investigate and figure out which certificate in the Windows store is the culprit.

AndreaB_

Hi @TimoT ,

I am using ArcGIS Pro 3.5.5 on Windows Server 2022 - managed by my org. I do not see any errors when I run the model in ArcGIS Pro - only see the error when I run as a scheduled task. I changed the time a bit too to see if that made a difference. It didn't.

I'm not sure what the change could be. I have talked to 2 people in our IT and they don't know either.

I can update ArcGIS Pro to 3.5.6 on this server and try that next.

Thank you,

Andrea

AndreaB_

I'm still getting the error. I also read this solving-ssl-errors-in-python-requests and this python-in-arcgis-pro-3-6-faq but deleting the C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 folder and running a repair installation on 3.6.1 didn't update the date of the C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\certifi\cacert.pem file, so I don't think that will help. I tried this in my Dev environment first.

TimoT

Hi @AndreaB_

What you’re seeing in the other Esri Community thread you linked below is similar in symptoms, but your case is different based on the error message. In your situation, the failure is happening when Python attempts to load certificates from the Windows certificate stores. It’s not failing at the certifi level (which you could essentially call ArcGIS Pro’s trusted certificate store).

A very important clue you mentioned is that the model runs fine when run manually, but fails only when executed as a scheduled task. Just to confirm: is the scheduled task running under the same Windows account you use when you log into the server machine?

Since I don’t get to log into Esri Community often, I’ll outline a few tests and assumptions that may help give you or others in the community a clue to narrow this down.

1. Test whether the issue is related to interactive vs batch logon

As a test, configure the scheduled task to "Run only when user is logged on", then stay logged into the server and let the task run. If it succeeds in this mode, this could suggest the issue exists between the difference in interactive vs batch logins.

Windows loads user profiles differently depending on logon type. Batch logins (used by Task Scheduler) often do not load the full user profile, which can affect the certificate stores that are available, or perhaps how it parses certificates.

2. Given this task used to work, look for recently issued certificates

Look for any new certificates that may have been created or imported. You’ll probably want to check two main areas (and you may need IT assistance):

a. Local Machine certificate store

On the ArcGIS Pro machine, open Start menu and in the search bar, type and open certlm.msc (local machine certificate store), then check for recently added certificates in:

Personal
Trusted Root Certification Authorities
Intermediate Certification Authorities

b. Current user certificate store

On the ArcGIS Pro machine, open Start menu and in the search bar, type and open certmgr.msc (current user certificate store), and check the same folders. You can look at the 'Valid from' date of certificates to determine when a certificate was issued.

c. Additionally compare the 2 stores

Additionally, look for any differences, especially for certificates that only appear in the Current user store. If you find a certificate that is not needed, removing it may resolve the issue. If there is a certificate that looks important, it may be worth considering trying to move it from the Current user store to Local Machine store, where batch logins would have a higher chance of being able to parse it correctly.

3. Another useful test
Try running the same scheduled task on a different machine. If it succeeds there, compare the differences, particularly in the Windows certificate stores between the two machines.

JoshuaBixby

The questions from TimoT are good to think about, I would additionally ask if you are trying to run the scheduled task using your credentials or other credentials. If other, a local machine system account or another user?

AndreaB_

Thank you for the reply! I am running ArcGIS Pro as a domain service account user. I am scheduling the Model to run within ArcGIS Pro as that domain service account user - so then the Windows Task Scheduler uses the service account user. That isn't the domain account I am logged into the server with though - I'm logged into the server as my own Admin domain account.

AndreaB_

Thank you for the reply! I will be working through these and let you all know.

AndreaB_

1. Yeah, so I'm logged onto the server as my domain Admin account, not the domain Service account. So this doesn't apply.

2. I don't see any new certificates. I deleted several expired certificates that already had replacements on the server. I still got the error from the scheduled model last night.

3. I was able to run the Model on a different machine using a different username and 'Run only when user is logged on'. I'm not local admin on the machine so it wouldn't let me save my password to 'run when the user is logged on or not'.

Since we can't figure this out, we might modify the Python script to bypass the native certificate check. Do you think this would cause security issues?

Thanks!

JoshuaBixby

When you logged in as the domain service account the other day and successfully ran the model, did you get any dialog pop-ups? Specifically, what there any dialog relating to accepting a certificate?

AndreaB_

nope, nothing.

AndreaB_

Here is the resolution! I worked with Esri tech support. Below from Esri Tech support quoted.

To resolve this, I recommend we start by modifying the script itself.

Modify the Python Script: We can explicitly tell the script where to find the necessary certificates, bypassing the Windows Certificate Store entirely.

Ensure you have this file in this location and accessible to the service account (C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\certifi\cacert.pem).
Add the following lines to the very top of your script:

import os
#Point to the specific certificate file
os.environ['REQUESTS_CA_BUNDLE'] = r"C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\certifi\cacert.pem

This worked - but not at 12:30am. "The failure at 12:30 AM is likely due to the Windows environment unloading the user profile or certificates after a few hours of inactivity. "

So I set it to run at 8pm and it works like a charm! 😃

Thank you for helping me out! Hope this helps someone in the future.

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.