FS Overwrite script stopped working after updating to pro v3.03 - SSL Error

3411
13
Jump to solution
12-12-2022 12:01 PM
BrandonBoisvert
New Contributor III

I have a script to overwrite a hosted feature layer which had been successful everyday for the past year but has stopped working after updating Pro to v3.03. Any ideas to get it working again would be great...I am not a developer so I am really unsure of how to proceed.

The script:

from arcgis.gis import GIS
from arcgis.features import FeatureLayerCollection

#LOG INTO ARGIS ONLINE ORGANIZATION
OrgURL = "my orgs url"
adminUserName = "my user name"
orgpass = "my password"

gis = GIS(OrgURL, username=adminUserName, password=orgpass)

#POINT TOWARDS LOCAL LAYER
LOC = "layers location on my computer"

#FIND FEATURE LAYER USING IT'S ID
ID_Points_ONLINE = gis.content.search("id: Hosted Layer's ID #")
flc_Points_ONLINE = FeatureLayerCollection.fromitem(ID_Points_ONLINE[0])

#OVERWRITE FEATURE LAYER
flc_Points_ONLINE.manager.overwrite(LOC)

Error:

SSLError(
requests.exceptions.SSLError: Please set verify_cert=False due to encountered SSL error: HTTPSConnectionPool(host='bedfordnh.maps.arcgis.com', port=443): Max retries exceeded with url: /sharing/rest/oauth2/authorize?response_type=code&client_id=arcgisonline&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&state=1ShzDhlkPgkir63Mm0WO9gvXOa49ZX&expiration=20160&allow_verification=false&style=dark&locale=en-US (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1129)')))

 

Note: I have followed the "Please set verify_cert=False" instruction but that doesn't seem to change the outcome

2 Solutions

Accepted Solutions
jcarlson
MVP Esteemed Contributor

I'm unsure how this issue would be related to Pro 3.0.3, as the Python API can interact with a Portal from anywhere, regardless of Pro. But perhaps Pro changed something about the underlying Python environment you're using?

Can you confirm where you're running these scripts from?

If you're doing any kind of automation, you ought to use a dedicated env for it using conda or venv or something. You don't want your administrative tasks to be at the mercy of Pro (or anything else) changing the installed version of your libraries.

- Josh Carlson
Kendall County GIS

View solution in original post

fklotz
by
Occasional Contributor

Background:
1. ESRI has moved ArcGIS Pro 3.0.2+ to OpenSSL version 3.x and cryptography version 37.0.2
2. If you (your company) uses an SSL intercept that does not support secure SSL renegotiation, you will get the error you've reported.

Unacceptable Solution: Remove SSL intercept

Relavent stackoverflow post on this topic:
https://stackoverflow.com/questions/71603314/ssl-error-unsafe-legacy-renegotiation-disabled

Based on the stackoverflow, I've tried a number of things.

  • Test outside of the company's secure network: Success (as expected)
  • downgrade cryptography package (which would downgrade OpenSSL to version 1.1.1) - downgrading has been unsuccessful so far as there are many dependencies. I don't know all the intricacies of the conda command line though. Perhaps someone can assist?
  • upgrade cryptography package to 38.0.1 (upgrade successful, same errors.)
  • run the custom http adapter code from the stackoverflow: success to connect to a https site, but doesn't help (yet) with using arcgis.GIS(), etc.

I've been working with ESRI and our company's network team on this issue. We do not yet have a solution for ArcGIS Pro 3.0.3 (the version I'm testing on).

ArcGIS Pro 3.0 installed with OpenSSL 1.1.1n and cryptography 3.4.7 (as per here: https://pro.arcgis.com/en/pro-app/latest/arcpy/get-started/available-python-libraries.htm). Shifting to OPENSSL 3 within a micro version of ArcGIS Pro is a BIG CHANGE.

I'm looking to hear of other folks who are using ArcGIS Pro 3.0.2+ who are inside a secured network.  Are you also receiving the error below?

SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1129)'))

 

Farley

View solution in original post

13 Replies
jcarlson
MVP Esteemed Contributor

I'm unsure how this issue would be related to Pro 3.0.3, as the Python API can interact with a Portal from anywhere, regardless of Pro. But perhaps Pro changed something about the underlying Python environment you're using?

Can you confirm where you're running these scripts from?

If you're doing any kind of automation, you ought to use a dedicated env for it using conda or venv or something. You don't want your administrative tasks to be at the mercy of Pro (or anything else) changing the installed version of your libraries.

- Josh Carlson
Kendall County GIS
BrandonBoisvert
New Contributor III

Thanks for the quick response Josh. I'm very inexperienced with all this (I hope to keep learning!) but as far as I know I've never installed a python environment so I assume I am just using whatever was installed with Pro? This is what I am seeing on their website: "By default, ArcGIS Pro has a single conda environment, arcgispro-py3, which includes all Python libraries used by ArcGIS Pro as well as several others, such as scipy and pandas."  So I guess it makes sense that when updating Pro it could have updated something in the python environment? 

0 Kudos
jcarlson
MVP Esteemed Contributor

Yeah, I would look at downloading something like Anaconda Navigator for a user-friendly interface for creating new environments. Or to keep things simple, there's venv:

venv — Creation of virtual environments — Python 3.11.1 documentation

You'll need to adjust your scheduled BAT file, or whatever you're using, to invoke the environment you create.

I'd also look at installing specific versions of the things you need. You could start by installing whatever version of the API was in your previous version of Pro and attempt to run your script from there.

- Josh Carlson
Kendall County GIS
BrandonBoisvert
New Contributor III

Thank you Josh. I am going to work on this today! I appreciate your help.

0 Kudos
fklotz
by
Occasional Contributor

Background:
1. ESRI has moved ArcGIS Pro 3.0.2+ to OpenSSL version 3.x and cryptography version 37.0.2
2. If you (your company) uses an SSL intercept that does not support secure SSL renegotiation, you will get the error you've reported.

Unacceptable Solution: Remove SSL intercept

Relavent stackoverflow post on this topic:
https://stackoverflow.com/questions/71603314/ssl-error-unsafe-legacy-renegotiation-disabled

Based on the stackoverflow, I've tried a number of things.

  • Test outside of the company's secure network: Success (as expected)
  • downgrade cryptography package (which would downgrade OpenSSL to version 1.1.1) - downgrading has been unsuccessful so far as there are many dependencies. I don't know all the intricacies of the conda command line though. Perhaps someone can assist?
  • upgrade cryptography package to 38.0.1 (upgrade successful, same errors.)
  • run the custom http adapter code from the stackoverflow: success to connect to a https site, but doesn't help (yet) with using arcgis.GIS(), etc.

I've been working with ESRI and our company's network team on this issue. We do not yet have a solution for ArcGIS Pro 3.0.3 (the version I'm testing on).

ArcGIS Pro 3.0 installed with OpenSSL 1.1.1n and cryptography 3.4.7 (as per here: https://pro.arcgis.com/en/pro-app/latest/arcpy/get-started/available-python-libraries.htm). Shifting to OPENSSL 3 within a micro version of ArcGIS Pro is a BIG CHANGE.

I'm looking to hear of other folks who are using ArcGIS Pro 3.0.2+ who are inside a secured network.  Are you also receiving the error below?

SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1129)'))

 

Farley
BrandonBoisvert
New Contributor III

I believe I was also receiving the error you included. Like you, I tested my script outside my organizations secure network and had no issue. I worked through it with my IT dept and they were able to solve the issue on their end. Unfortunately, I don't know the steps they took to resolve it.

0 Kudos
fklotz
by
Occasional Contributor

Thanks for letting me know that your IT dept was able to make changes for your code to work. They may have enabled secure renegotiation within the company's firewall to be RFC 5746 "patched" according to SSL Profiles Part 6: SSL Renegotiation - DevCentral (f5.com).    "When describing SSL implementations, systems that support secure renegotiation (per RFC 5746) are referred to as "patched" and systems that do not support secure renegotiation are referred to as "unpatched."

Unfortunately, not all firewalls have the secure renegotiation feature.  I'm looking at other client-side changes while keeping within a clone of the ArcGIS Pro 3.0.3 python environment.  If I can downgrade to OpenSSL 1.1.1 or bootstrap in the OP_LEGACY_SERVER_CONNECT ssl option I think we would be back to how ArcGIS Pro 2.9.5 and ArcGIS Pro 3.0.0 worked.

Farley
0 Kudos
John_Herrera
New Contributor II

Thank you for identifying the issue!  Just like you pointed out, the company's firewall.  I sent the error messages back to our network engineers and they whitelisted the items.  Problem solved!

0 Kudos
fklotz
by
Occasional Contributor

One fix for this situation with a caveat, If your IT networking team approves this method (they must ensure that the SSL is being taken care of on the firewall/VPN...), is to add a custom OpenSSL configuration f file and point your system to it. This tells your python to not be concerned about the secure SSL renegotiation (essentially working in a similar fashion as previous ArcGIS Pro and ArcGIS desktop python environments). 

The example below is for windows and will enable the ArcGIS Pro 3.0.2 and ArcGIS Pro 3.0.3 (and maybe more into the future) python environments to be able to communicate over https. This example assumes you are a non-admin user putting this fix in place for yourself and not for all users.

1. Create a plain text file at some path on your computer which will not change.  You could place it somewhere like: C:\Users\your_username_here\AppData\Local\ssl\openssl.conf

The contents of this file are as follows:

openssl_conf = openssl_init

[openssl_init]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
Options = UnsafeLegacyRenegotiation

2. Now you need to point your system to this file.  This is done by adding a user level windows environment variable.  You can use the following command in a windows command prompt to do this (edit this command to point to your .conf file.).

setx OPENSSL_CONF C:\Users\your_username_here\AppData\Local\ssl\openssl.conf

Close the command prompt and all ArcGIS Pro, Jupyter Notebook, and Python windows. Then you can open ArcGIS Pro and use some python to initiate an https connection.

for example:

import requests
url = 'https://maps.arcgis.com/index.html'
response = requests.get(url)
print(response.headers)

If you don't get the SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED error, but instead have the headers printed, you are good to go.

Another example (assuming you are logged into your ArcGIS Online portal in Pro already):

import arcgis
gis = arcgis.GIS("Pro")
print(gis.users.me.username)

Again, if you do not see the errors and do see your username printed, you are good to go.

Warnings: Please check with your IT department that this is an approved method for your company. You may be opening yourself and your company up to all sorts of cyber security issues.

This fix should not be used unless absolutely required.  If your ArcGIS Pro python environment is currently able to negotiate https connections, then you should not implement this fix.

There may be other options for getting the ArcGIS Pro python environment working, but this is the simplest that I have found which enables the default python environment to work without cloning and patching in some other way.  Your milage may vary.  This has been tested on a computer which has no other python environments and no other ESRI software (no ArcMap). I cannot guarantee this fix would not negatively affect those python environments.

Farley
0 Kudos