Latest Contributions by DougGreen

‎12-08-2021

The Day the Earth Stood Still Web Expired On September 30th, 2021, Let's Encrypt (a non-profit certificate authority) allowed one of their certificates to expire, causing any certificate that had it as part of it's chain to show as invalid. This caused quite a bit of mayhem but they did eventually issue a replacement and through reboots and other updates, most of the issue went away. Except for Python virtual environments installed with ArcGIS Pro. I started having some of my automated processes fail around this time and was not able to resolve them even though we had rebooted machines and servers and manually installed the new replacement certificates locally, but nothing worked. Here's a quick example of something that was failing: import arcpy, os, requests, json logMessages = '' r = requests.session() r.headers.update({'Accept': 'application/json', 'Content-Type': 'application/x-www-form-urlencoded'}) body = ''' [out:json] [timeout:120] ; ( way ["highway"="service"] (43.6192138,-116.4338378,43.6339976,-116.4136293); ); out; >; out skel qt;''' OSM_URL3 = 'https://overpass.kumi.systems/api/interpreter' # reach out to the Overpass API to grab the data try: respSuccess = False numTries = 0 while respSuccess == False and numTries <= 10: numTries += 1 response = r.post(OSM_URL3, data=body, verify=True) if response.status_code == 200: respSuccess = True logMessages += '''Requesting features from OSM: Response Status: {}\n'''.format(response.status_code) except requests.exceptions.RequestException as e: logMessages += '{}\n'.format(e.args[0].reason) print(logMessages) this would always produce a message like: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091) Because these processes were not priority, it took me a couple months to circle around to digging in a bit more. I just figured out the issue and I want to share because I feel like I only understood the watered-down version of how SSL Certificates work and it appears that my frustration stemmed from that partial understanding. Also, it affected my automation that relied on Python web requests to get data. How I thought certificate checking works I originally thought that a certificate just needed validated against some authority and then it would allow traffic to flow. Not How Certificates Are Verified How Certificates Actually Work (ish) My previous understanding caused me to feel stuck because I was experiencing SSL errors trying to download data using python requests and getting failures even though I was able to download the same data using a web browser or other client software. So why was Python the only environment indicating an invalid certificate? Because this is roughly how the verification process actually works. The crucial detail I was missing was that my own computer had a previously downloaded copy of the certificate and when Python was checking the one received from the remote server against the local, my local copy still showed it was expired. Almost Solved Once I understood this, I requested some help from my awesome System Administrator and was shown how to remove an expired certificate from my Windows Certificate Store and install the new one. I know, you're thinking I'm going to show you how to do that but I'm not because it didn't work and there's probably plenty of other sites that will teach you how to do that. I'm posting this because I literally couldn't find it anywhere. I concluded that Python must not be referencing the certificates in my operating systems certificate store and must use it's own. So I did an uninstall and a fresh install of ArcGIS Pro. This still didn't fix it! So I installed pro on another computer and tried to run the same bit of code on the same network connection. THIS WORKED! But why wouldn't my machine work? The Real Fix I did some research and fiddling and found indication that the Requests Library uses it's own certificate bundle (a new term for me) and it was stored at the location provided by running the following code: import requests print(requests.certs.where()) So, knowing that ArcGIS Pro had already been reinstalled and it didn't help, the installer must not have replaced this file, it must have just checked it and moved on or appended data to it? Not sure there. Either way, I renamed the old one and put a copy of the cert found on the other machine after a new install and BAM! My machine could now perform a web request through Python without throwing an SSL error. The Easier Fix If you're having this same issue, you can most likely do an uninstall of ArcGIS Pro, then make sure to delete the certificate in the location indicated by the requests.certs.where() function before re-installing pro. I'm sure that it's not recommended to tweak too much with most of the items in a Conda virtual environment, but I have no need to have a custom virtual environment because I'm using only libraries that are already included when you install ArcGIS Pro. However, in this one instance, this did the trick. I'm hopeful that there's a better way to have a certificate in a virtual environment be updated by running a command and that someone may be able to shine some light on this topic.

‎08-25-2021

I have the same question. I found that this documentation indicates that you can add "a thumbnail that is displayed when the tool is added to the Analysis gallery." However, I added a thumbnail to several scripts and they still show the default script icon in the gallery. I attempted re-adding them to the gallery after adding the thumbnails and that didn't solve the issue. I also added different types of thumbnails (jpg, png, etc) and that didn't help either. I'd really love for this to work as we are creating these tools for very new GIS users and the tool names need to be longer than can show in the gallery. It'd be nice to have that visual recognition that they are clicking on the right tool and not have them all look the same.

‎07-21-2021

I have this same need. I think that this import/export custom ribbon functionality is only truly portable if it uses relative paths. I'm sure if you are just grouping common command buttons on a ribbon tab, this works great. But for script tools, it means we can't share them. I would love to be able to point some buttons to script tools within the project toolbox and create a ribbon for accessing them. Another awesome thing would be if you could select custom ribbons to include when sharing a project template.

‎07-19-2021

I completely agree. I've set up a Project Template with some custom tools that interact with layers in the project for addressing workflows and we'd love to be able to have the pro template include the custom Ribbon we've made with all the tools grouped nicely as buttons.

‎06-16-2021

So it sounds like you'd like to modify your python script to not only set the "Exercised" column back to NULL but also to do something with valve size. I can't exactly tell from your question what you need to do with the "Valve Size" column. Perhaps, you are wanting to just reset the "Exercised" column to NULL for valves of a certain size? If that is the case, I would use an update cursor with a where clause. # set up an update cursor including any fields that you'll need updated or to determine what to update with arcpy.da.UpdateCursor(fcPath, field_names=['UNITID', 'VALVESZ', 'Exercised'], where_clause='VALVESZ IN (6, 8, 10)') as cursor: for valve in cursor: if (valve[2] != None): # None is python's equivalent of NULL valve[2] = None cursor.updateRow(valve)

‎02-18-2021

@ParhatM , I think that there's a simpler method to doing this if the schema of the feature classes and tables don't change every time. I'd say you could register the FC/Table once and then just use T-SQL to swap out the data in it on a regular basis through a SQL Agent Job or a script that executes a stored procedure in the database. I have described the process we use for this exact situation in this post. It allows our Feature Classes to be used as a standard EGDB feature class except that it is not to be edited through GIS Software. I have many of these and I don't have to spend any time updating them until someone requests a change to the field definitions.

‎02-05-2021

@Chang-HengYang this has been a while but this use case is becoming more useful so I figured I'd share something we've learned. We do this sort of thing quite often and can validate what @VinceAngelo is saying about performance and replication etc. We have found a pattern that allows us to pull information from various instances and deliver a feature class with great performance. The one draw back is that it's not live (real-time views of the data in their tables). However, if your needs allow a regular update interval (daily or weekly etc) then give this a try. Create a feature class in your Enterprise GDB with the same schema as the query layer you're trying to make. Register it as versioned but indicate in metadata or other documentation that it is not to be edited by end users. Create a stored procedure that can be run by a SQL Agent Job on a schedule that does the following: Based on https://desktop.arcgis.com/en/arcmap/10.7/manage-data/using-sql-with-gdbs/edit-versioned-data-using-sql-sqlserver.htm Deletes all rows from the multi-version view (Same as the feature class name with '_evw' appended at the end) Inserts the results of your query into the multi-version view Running a compress on the database after these operations will help this perform better so that all of these new edits can be compressed into the base table. NOTE: There is a way to do it without registering it as versioned but you have to do some extra steps to handle the managed OBJECTID field in a way that leaves it in a consistent state with the Geodatabase (https://desktop.arcgis.com/en/arcmap/10.7/manage-data/using-sql-with-gdbs/inserting-a-value-into-a-global-id-column-in-sql-server-using-sql.htm) This allows the database to chug away at a hefty process when we don't have any users in the system and provide standard feature classes with spatial indexes to our users.

‎02-02-2021

I think I see what the seeking to do and there was a way in ArcMap to add a subquery inside the definition query if both feature classes lived in an enterprise geodatabase. I attempted this in Pro and it does validate the query but on saving it, it never applies. Then, when you open up the definition query editor, it shows no active definition queries. So I agree with your desire and others that it would be nice to have a way to limit the features by a spatial definition query.

‎01-05-2021

Guessing you've already figured this out but we experience this on windows when the native client is not up-to-date or there is a sql driver missing. I would think it could be possible that there is a drive missing?

‎12-03-2020

I am in an MS SQL Server EGDB using ArcMap 10.7.1 and I see the same behavior. When the FC has no join. the Shape_area() and Shape_length() fields are there, after joining to an enterprise table or FC, they are gone. Not just turned off, gone. Decided to try it in Pro and the behavior is different. The fields are still there but they are at the beginning of the list of fields:

‎09-25-2020

Sorry I'm just seeing this. It has been too long and I don't remember but I think it was just the ArcGIS Server machine. We are just finding that it's doing it again after upgrading to 10.8.1. I'm going to be on the phone with them again it looks like.

‎04-17-2020

For anyone else experiencing this issue, our cause for these errors was that we had just received a new certificate for our internal servers that would work for any server on our internal network (an internal wildcard cert). This was a pfx file. We imported it and enabled it and the web browsers stopped complaining about insecure websites. However, the individual pieces of the architecture were not liking it. Also, programatically interacting with the portal or server through python or rest api was throwing ssl errors. The problem was that we had not imported a root or intermediate .cer file. Apparently this is needed to complete the validation route. Once that was in place on both Server A and Server B, we were back in business. It took a 3 hour chat with Tech Support but we got it.

‎04-17-2020

We have just had this start happening on a 10.7.1 on windows 2016 with a federated server. It started being observed a couple days after updating our ArcGIS Server to use a new self-signed wildcard cert. We have tried using the same cert on our portal machine to see if that resolved the issue but it has not. Our architecture is as follows: Server A: ArcGIS Server, Data Store (inside firewall, Self-signed Cert) Server B: Portal for ArcGIS (inside firewall, Self-signed Cert) Server C: Web Adaptor for Server, Web Adaptor for Portal (outside firewall, CA Cert) We are somewhat surprised that this is still just sitting here with a bug like this unresolved. How have you all moved past this or are you just using this method to delete the orphaned items constantly?

‎10-29-2019

Yep, that version is fine. It's the one used when the error is produced. It continues to happen even when unregistering so version specific errors are ruled out. I heard back from them and they ended up telling me that this is a bug. #BUG-000125941

‎10-08-2019

Joshua, I am running these queries in SSMS. These two feature classes are not versioned but on feature classes that are, I append "_evw" to the end of the name to select from the multi-version view of the table/featureclass. Good thought though. Doug

Online Status	Offline
Date Last Visited	‎04-09-2026 08:22 AM

My Ideas

Latest Contributions by DougGreen

Solving SSL Errors in Python Requests

Re: Custom icons for tools in the Analysis Gallery.

Re: Does ArcGIS Pro store relative paths to the python script for script tools in the ribbon?

Re: Include Custom Ribbon settings in Project Template

Re: Need Help with Python Script

Re: Update feature class in an enterprise GDB using SSMS?

Re: The complex query in query layer

Re: Can a map layer in Pro be based on a spatial query?

Re: Enabling a SQL Server Database as Enterprise Geodatabase from arcpy in ArcGIS Server 10.6

Re: Shape_length() and Shape_area() fields unavailable after joining an SDE Feature class with an SDE table

Re: Portal items duplicating on service update

Re: Portal items duplicating on service update

Re: Portal items duplicating on service update

Re: Geodatabase error: The operation was attempted on an empty geometry.

Re: Geodatabase error: The operation was attempted on an empty geometry.

Re: Solving SSL Errors in Python Requests

Change Default Behavior of Opening Map Package

Re: Add a layer to a hosted feature layer

Re: How to Associate Documents with Polygon Featur...

Re: Add a layer to a hosted feature layer