BUG?:About the mapSeries.pageRow error

[-_-]

Hi.

I was using a script that switches between map series and outputs each page in png format with the page name as the file name, but now it gives an error when the field name of the index layer has a certain pattern. (This script used to work fine.) I am currently using ArcGIS PRO 3.4.3.cifically occurs when field names contain a mix of full-width Japanese characters (Kanji, Hiragana, Katakana) AND full-width English letters or numbers.

To be more precise, we're seeing an AttributeError (or TypeError, to be exact) ONLY when field names contain a combination of full-width Japanese characters (Kanji, Hiragana, Katakana) and full-width English letters or numbers. (e.g., 面積ｍ).

TypeError: pageRow.__new__() got an unexpected keyword argument '面積ｍ'

The error does NOT occur in the following patterns:

Field names with only half-width characters (e.g., area_m, NAME).
Field names with only full-width Japanese characters (e.g., 面積).
Field names with a mix of full-width Japanese characters AND half-width English letters or numbers (e.g., 面積m, 地名123).

I'm having trouble because the field name in the pattern that causes the error is also used in many of the files I've created. Technical support told me that reports about scripts are not accepted under my company's contract, so I reported it here.

[HaydenWelch]

That's really odd... Python uses Unicode strings by default, but arcpy passes these raw strings to the underlying C#/C code that implements the interface.

Python 2 didn't use Unicode by default. Have you tried explicitly casting the string with the Unicode prefix? (e.g. u'面積m') To see if that forces it to read the string correctly?

I only ever use ASCII compliant characters in my scripts and have never had issues, but it's not unbelievable that there's a bug somewhere that improperly reads Unicode strings as ASCII if they contain ASCII characters, but if that is the source of the bug, it would be in the compiled binaries and only ESRI developers could find it. One of the major downsides to closed source programs...

Would you mind sharing a snippet of your code btw? The TypeError seems to be showing up in the initialization of the pageRow object.

[-_-]

Thank you for your reply.
I'm confused because I've been using this script for a long time and have never had any problems.

I've excerpted the code that seems to be related to the error. The layout tab was displayed and used.
With this code, if the pattern does not apply, a png will be output to the D drive, but if the pattern does apply, an error will occur.

# coding:utf-8

import arcpy, os

aprx=arcpy.mp.ArcGISProject(r"CURRENT")
l=aprx.activeView

if not l.mapSeries is None:

    ms = l.mapSeries

nameF=ms.pageNameField.name

arcpy.AddMessage(nameF)
arcpy.AddMessage(ms.pageRow) #error

pageNum=1

ms.currentPageNumber =  pageNum
pageName = getattr(ms.pageRow,nameF)#error

l.exportToPNG(r'D:\'+pageName+".png",resolution=300)

Traceback (most recent call last):
File "D:\Python\PRO\outputpngV2.py", line 15, in <module>
arcpy.AddMessage(ms.pageRow) #error
^^^^^^^^^^
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\_mp.py", line 6514, in pageRow
return self._convert(convertArcObjectToPythonObject(getattr(self._arc_object, "pageRow")))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\_mp.py", line 6257, in _convert
return namedtuple('pageRow', list(newDict.keys()))(**newDict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: pageRow.__new__() got an unexpected keyword argument '面積ｍ'

[HaydenWelch]

Seems like Python doesn't like that character combo as a variable/property name. There isn't full Unicode support for variable names (No 🍉=10), only strings. Can you try to access the attribute through the __dict__ attribute? (E.g. ms.pageRow.__dict__[nameF])

Since dictionary keys are strings, any Unicode character combo is allowed. The __new__() method is likely being called during the getattr() call, so if you can avoid moving the unicode name through an attribute, that might work.

Edit:

Oddly enough, your name works in all instances I've tested:

That full stack trace is throwing an error when the 'newDict' (likely a Python dictionary) is being unpacked into a named tuple. For some reason this operation is causing the attribute error.

The code that's causing the error is this:

from collections import namedtuple

def _convert(newDict: dict) -> namedtuple:
    return namedtuple(
        typename='pageRow', 
        field_names=list(newDict.keys())
    )(**newDict) # Unpack dictionary directly into tuple
    # namedtuple(**{'a': 1, 'b': 2})
    # This is equivalent to namedtuple(a=1, b=2)

And the error is happening during the unpacking operation. The namedtuple is initialized, but the attribute name that it is initialized with is somehow not the same as the key used to create it and it's raising the TypeError. You might be able to capture the state of that tuple if you run the code in something like Jupyter, but since the namedtuple isn't assigned until it's returned and the error is happening on the return line it's almost impossible to tell exactly how that named tuple was initialized.

[HaydenWelch]

Okay, so it is actually a bug in the _mp.py module. The named tuple conversion for pageRow is behaving oddly with dictionary unpacking. You can hot patch it with this:

def _convert_fixed(self, dictionary):
    from collections import namedtuple
    #replace the special characters ( . ) in spatial fields names with _
    newDict = {key.replace('(', '_').replace(')', '_').replace('.', '_') : value for key, value in list(dictionary.items())}
    pageRow = namedtuple('pageRow', list(newDict.keys()))
    return pageRow(*dictionary.values())

arcpy._mp.MapSeries._convert = _convert_fixed

Not sure exactly what is happening, but unpacking the dictionary just doesn't work at all. Since dictionaries are ordered in the version of Python that arcpy uses, you can just unpack the values and they'll line up with the keys. That seems to work for me.

[-_-]

So it is a bug after all, thank you for verifying it.

Actually, in your first reply you said that it worked properly in your environment, so I thought it might be a problem with my environment and looked for a different way to get the page name.

I'll also include this here just in case.

I create a dictionary of page numbers and page names using getPageNumberFromName, and then get the page name from that dictionary.

#namefield
nameF=ms.pageNameField.name

# indexlayer
index_layer = ms.indexLayer

# pagename and pagenumber
page_nameDic = []

with arcpy.da.SearchCursor(index_layer, nameF) as cursor:
    for row in cursor:
        current_page_name = row[0] 
        page_number = ms.getPageNumberFromName(current_page_name)
        if page_number is not None:
             page_nameDic.append({
                'name': current_page_name,
                'number': page_number
            })


sorted_page_data = sorted(page_nameDic, key=lambda item: item['number'])

def OutputMS(pageNum):

    try:
        ms.currentPageNumber =  pageNum
        #pageName = getattr(ms.pageRow,nameF) #Obsolete due to error
        pageName = next((item['name'] for item in sorted_page_data if item['number'] == pageNum), None)
        ms.refresh()

(Continued below)