Latest Contributions by MarcoBoeringa

@JeffBarrette  I can talk with the labeling team to see if the smoking gun you suggest could have been caused by changes made during 3.6 development. Yes, I think you should. ... View more

@JeffBarrette Another thing I intend to do is to try to run ArcGIS Pro attached to the ESRI Visual Studio Code debugger. Maybe that will give some new insights. There is not much more I can and will do. Honestly, I do not want to sound to cynical here and I understand your frustration regarding the reproducibility, but I as an ESRI client, am not the one getting paid for debugging Pro, and believe me, there is a bug lurking here. After 25 years in IT and working with ESRI software, I can smell them from a mile's distance. And remember, my code runs perfectly fine on the Geoprocessing Thread... ... View more

@JeffBarrette , Although you say you tried to reproduce my workflow based on my script sample, your script is missing the setting of the disabling of ligatures in case Maplex spread characters is set on a label class, that I included in my last code snippets. (Background for this: these settings in Maplex are incompatible from a cartographic point of view, and cause garbled incorrectly rendered labels with two ligature characters in Latin grouped together, while the rest spread, e.g. "a...bc...d...e...f" instead of cartographically correct "a...b...c...d...e...f", unfortunately, ArcGIS Pro by default activates ligatures since it introduced support for them in Maplex).   # Change the 'ligatures' label class text symbol setting to false if gType in ['Line','Polygon']: for cimLabelClass in cimLyr.labelClasses: # Double check we got a label class if type(cimLabelClass).__name__ == "CIMLabelClass": try: if cimLabelClass.maplexLabelPlacementProperties.spreadCharacters == True: cimLabelClass.textSymbol.symbol.ligatures = False except Exception: arcpy.AddWarning("*** WARNING ***: Failed to set the label class text symbol's 'ligatures' setting to 'false' for layer '{}'!".format(lyrToAdd.name)) pass   ... View more

The layer that failed in the screenshot examples I showed, was of type Polygon by the way. ... View more

@JeffBarrette  Some more complete code sections from my workflow. The second one shows everything happening from the moment the CIM layer object is requested with 'getDefinition' until the modified version is written with 'setDefinition'. Nothing else is happening here. But to be honest, I think you have one big smoking gun right in front of you with my observation of the labelclass name being set to some "auto-generated" "Class 1" name. Some developer at ESRI baked that behavior into ArcGIS Pro, it shouldn't be to hard to find out where it is happening in the Pro code base, and that code section will likely tell you a lot about what may go wrong in the Foreground Thread.  To be honest, I even have a, now rather vague, recollection of having seen a very similar issue with these label class names pop-up in an earlier version of Pro, maybe at the time Python CIM editing was first introduced (Pro 2.4??? or so???). This issue with label class names really rings a bell to me in terms of previous ArcGIS Pro issues / bugs I had to deal with. if removeAmbiguousLabels == "Do not remove": removeAmbiguousLabelsSetting = 'None' elif removeAmbiguousLabels == "Remove within same label class": removeAmbiguousLabelsSetting = 'WithinLabelClass' if removeAmbiguousLabels == "Remove all": removeAmbiguousLabelsSetting = 'All' cimLyr = lyrToAdd.getDefinition("V3") # Change the 'removeAmbiguousLabels' Maplex setting # NOTE(!): This setting only became available from ArcGIS Pro 3.1 onwards, # and is currently only applicable to feature classes of type 'Point'. if gType == 'Point': for cimLabelClass in cimLyr.labelClasses: # Double check we got a label class if type(cimLabelClass).__name__ == "CIMLabelClass": try: cimLabelClass.maplexLabelPlacementProperties.removeAmbiguousLabels = removeAmbiguousLabelsSetting except Exception: arcpy.AddWarning("*** WARNING ***: Failed to set the 'Remove ambiguous labels' Maplex label engine setting of layer '{}'!".format(lyrToAdd.name)) pass # Change the 'ligatures' label class text symbol setting to false if gType in ['Line','Polygon']: for cimLabelClass in cimLyr.labelClasses: # Double check we got a label class if type(cimLabelClass).__name__ == "CIMLabelClass": try: if cimLabelClass.maplexLabelPlacementProperties.spreadCharacters == True: cimLabelClass.textSymbol.symbol.ligatures = False except Exception: arcpy.AddWarning("*** WARNING ***: Failed to set the label class text symbol's 'ligatures' setting to 'false' for layer '{}'!".format(lyrToAdd.name)) pass # Change the label class "label priority ranking" 'priority' value for cimLabelClass in cimLyr.labelClasses: # Double check we got a label class if type(cimLabelClass).__name__ == "CIMLabelClass": try: if missingLabelPriority == False: cimLabelClass.priority = labelPriorityByLayerDict[cimLabelClass.name] else: maxLabelPriority += 1 cimLabelClass.priority = maxLabelPriority except Exception: arcpy.AddWarning("*** WARNING ***: Failed to set the 'Label Priority Ranking' Maplex label engine setting of layer '{}'!".format(lyrToAdd.name)) pass lyrToAdd.setDefinition(cimLyr)       ... View more

@JeffBarrette and @HannesZiegler , I found the culprit. It is not the setting of the priority, but getting the CIMLabelClass.name property that fails on the Foreground Thread, since that is used as lookup key in the dictionary, it raises a KeyError:     ... View more

@JeffBarrette , As a last thought, I replaced the bare excepts with "except Exception:". This did not make any difference, still an exception raised on the Foreground Thread, so the exception raised by the CIM edit is not a system-exiting exception. ... View more

By the way, I installed the official first release of .NET10 next to the existing .NET8. This was under the assumption that Pro 3.6, as previously hinted by ESRI, might switch to .NET10, which is postponed to 3.7. So, unless there are parts of ArcGIS that already switched to .NET10, I assume the .NET10 installation shouldn't interfere. But if I understood it well (I am not a .NET developer), you target a single version with an app, not multiple, so this is again unlikely to play any role. ... View more

@JeffBarrette and @HannesZiegler , I have been digging a bit deeper: The example code I showed you regaring the CIM issue was a slightly modifed version of the original. The original is below: # Change the label class "label priority ranking" 'priority' value for cimLabelClass in cimLyr.labelClasses: # Double check we got a label class if type(cimLabelClass).__name__ == "CIMLabelClass": try: if missingLabelPriority == False: arcpy.AddWarning("labelPriorityByLayerDict: {}".format(labelPriorityByLayerDict)) cimLabelClass.priority = labelPriorityByLayerDict[cimLabelClass.name] arcpy.AddWarning("CIM LabelClass priority: {}".format(cimLabelClass.priority)) else: maxLabelPriority += 1 cimLabelClass.priority = maxLabelPriority arcpy.AddWarning("CIM LabelClass priority: {}".format(cimLabelClass.priority)) except: arcpy.AddWarning("*** WARNING ***: Failed to set the 'Label Priority Ranking' Maplex label engine setting of layer '{}'!".format(lyrToAdd.name)) pass Now, obviously, I am getting an exception on the Foreground Thread. But besides the CIM edit, there is also a dictionary lookup going on ('labelPriorityByLayerDict'). Now technically and programatically, that dictionary is guaranteed to contain the key being requested, so shouldn't error out, as it is in fact based on reading the exact same set of layer files being modified here. However, to get further insight, I first re-tested my previous observations, confirming what I saw before: This again shows the Foreground Thread throwing an exception during the CIM edit, but not the Geoprocessing Thread. I then decided to also test the contents of the dictionary: As you can see from the last two images, the dictionary is SUCCESSFULLY created in BOTH cases and contains the value 1 to set as label priority. This now unequivocally tells me the issue is with the actual CIM edit. I see no other way based on these results: cimLabelClass.priority = labelPriorityByLayerDict[cimLabelClass.name]   ... View more

@HannesZiegler  Based on my experiences, this ArcGIS Pro Idea may also be something to have a look at for a future version of Pro: https://community.esri.com/t5/arcgis-pro-ideas/add-ability-to-disable-arcgis-pro-3-6-new-quot/idi-p/1669229/jump-to/first-unread-message ... View more

@HannesZiegler  By the way, I don't know if it has any relevance, but please take note that I am still on Windows 10 (now with ESU extension), so it would be good to test anything on that version of Windows if possible. ... View more

@HannesZiegler , No, unfortunately I will not be providing a reproducible case for you regarding the threading issue. My tool, and the multi-threaded parts of the code, is part of a major sophisticated geoprocessing workflow coded in Python that has some 20k lines of Python code over a few dozen modules. To demonstrate other issues, I have in the past made available earlier versions of the tool to the Dutch branch of ESRI, but each time this turns out into a nightmare in terms of the amount of work for me to get anything logged. I won't be doing that again based on these past experiences. Sorry. I do think you should look at the CIM issues and the Python exception I saw happening when run on the Foreground Thread. This issue is unrelated to the 'concurrent.futures.ThreadPoolExecutor' and threading issues I saw, and should be relatively easy to test in trying to reproduce it. I already supplied part of the needed code as you know. ... View more

Hi @HannesZiegler , I have unfortunately identified a few more issues that I want to report here. I honestly do not have the time and energy now to go through the whole process of getting them logged as bugs. I already wasted a whole week trying to wade through issues getting my main geoprocessing workflow to fully work again in Pro 3.6, at least on the Background Thread for now (which now works). This is to much. The issues: - With Foreground Thread processing, if I use a Python 'concurrent.futures.ThreadPoolExecutor' with a function similar to the one below as input to the executor, where there is a gc.collect() in the 'finally' except construct, it will hang / stall Pro. Removing the gc.collect() from the 'finally' construct, lets the script continue. This does not happen with Background Thread processing, the gc.collect() call does not cause issues in that mode in combination with usage of 'concurrent.futures.ThreadPoolExecutor'. I do not know if this is actually a Pro bug, or a bug in Python 3.13. There are several references to (fixed) issues related to gc.collect in the Python 3.13 release notes, see the linked full release notes for the references to gc.collect() in relation to these issues: Add sys.unraisablehook() to customize how "unraisable exceptions" are logged python/cpython#81010 C implementation of functools.lru_cache python/cpython#58581 At least one of them seems to have quite a bit of history, and may have seen changes in e.g. 3.11 as well, but I didn't check. def some_function(): try: blabla except: blabla finally: some_clean_up gc.collect() - With Foreground Thread set, CIM edits seem to cause Python exceptions. Please note I haven't checked this extensively, I just noticed it in one particular case that is present in my workflow. I have not tested whether this applies to all CIM edits on a Foreground Thread, or just the specific one I happen to use in my workflow, which is the setting of the label classes' Maplex label priority ranking. E.g. see the script example below, where the exception will be raised on a Foreground Thread but not when running the same code as Background Thread. cimLyr = lyr.getDefinition("V3") i = 1 # Change the label class "label priority ranking" 'priority' value for cimLabelClass in cimLyr.labelClasses: # Double check we got a label class if type(cimLabelClass).__name__ == "CIMLabelClass": try: cimLabelClass.priority = i except: arcpy.AddWarning("*** WARNING ***: Failed to set the 'Label Priority Ranking' Maplex label engine setting of layer '{}'!".format(lyrToAdd.name)) pass i += 1   ... View more

@HannesZiegler , I have now reported these issues to the Dutch branch of ESRI, so hopefuly they get reproduced and logged as bugs. Unsurprisingly, for the Get Count tool issue, a bug was already logged. ... View more