Select rows that have invalid domain values using ArcPy

1203
7
06-24-2022 08:27 AM
Labels (1)
Bud
by
Notable Contributor

ArcGIS Pro 2.6.8; Oracle 18c 10.7.1 EGDB:

For a FC/table that has values that don't match its domains:

Is there a way to select all rows in the attribute table that have invalid coded-value domain values — using ArcPy?

  • For each row, for each field, if the field has a domain, and if the value doesn’t match the domain, then select the row.
  • Hopefully, the script could be dynamic, rather than needing to hardcode a list of fields with domains in the script.
  • Edit (this edit was added after JohannesLindner kindly provided a script):
    The table has a subtype.

(I'm not great with ArcPy yet. So I thought I'd ask, in case anyone has a sample script they can share to get me started.)


I'm aware ArcGIS Pro has "Select Only Invalid Objects" functionality in the attribute pane (not to be confused with the attribute window): 

Bud_0-1656083774739.png

But that functionality doesn't work for non-spatial tables in ArcGIS Pro 2.6.8 (it only works on FCs)...that functionality was added alter in Pro 2.8.

Regardless, I'm wondering if ArcPy might be a better choice, when it comes to validating an entire table.

Tags (2)
0 Kudos
7 Replies
JohannesLindner
MVP Frequent Contributor

Huh, I would have guessed that it's easier in Python than in Arcade, but it certainly took longer to write...

 

Same setup as in your Arcade question (Flag row if any field has invalid domain values - Esri Community).

 

 

import pathlib

def check_domain_values(in_features, flag_field, database=None):
    """sets the flag field to 0 if at least 1 field value is not in the field's domain, else 1

    in_features: str, path to the feature class or table
    flag_field: str, name of the field that has the flag
    database: str, path to the database. defaults to the parent directory of the feature class, should be set when the fc is in a feature dataset.

    """
    if database is None:
        database = pathlib.Path(in_features).parent
    
    domains = arcpy.da.ListDomains(database)
    domain_values = {d.name: list(d.codedValues.keys()) for d in domains}
    print(f"domain_values: {domain_values}\n")

    fields = arcpy.ListFields(in_features)
    allowed_values = {f.name: domain_values[f.domain] for f in fields if f.domain not in ["", None]}
    fields_with_domains = list(allowed_values.keys())
    print(f"allowed values: {allowed_values}\n")
    print(f"domain fields: {fields_with_domains}\n")
    
    fields = fields_with_domains + [flag_field]
    with arcpy.da.UpdateCursor(in_features, fields) as cursor:
        for row in cursor:
            ok = True
            for i, value in enumerate(row):
                try:
                    field = fields[i]
                    if value not in allowed_values[field]:
                        ok = False
                        break
                except KeyError: # flag_field is not in the dictionary
                    pass
            new_row = list(row)
            new_row[-1] = ok
            cursor.updateRow(new_row)

 

 

 

 

 

fc = arcpy.Describe("TestPoints").catalogPath
check_domain_values(fc, "IntegerField")

 

 

 

domain_values: {'MyDomain': ['Value 1', 'Value 2', 'Value 3']}

allowed values: {'TextField': ['Value 1', 'Value 2', 'Value 3']}

domain fields: ['TextField']

 

JohannesLindner_0-1656089151059.png

 


Have a great day!
Johannes
JohannesLindner
MVP Frequent Contributor

this will raise exceptions in at least these scenarios:

  • you input a shapefile
  • you input a flag_field that is not Integer
  • probably when you have range domains in your database

Have a great day!
Johannes
Bud
by
Notable Contributor

Thanks Johannes! That's great!
I would have struggled with this for a week or more if I'd done it all on my own, lol.


Here's where I'm at:

Test #1: (using my work computer; enterprise GDB)

I tried running the script, but got an error:

Error:

Traceback (most recent call last):
File "<string>", line 41, in <module>
File "<string>", line 15, in check_domain_values
File "<string>", line 15, in <dictcomp>
AttributeError: 'NoneType' object has no attribute 'keys'

 

Bud_0-1656132373203.png


The table (non-spatial) has a subtype. Not sure if that's relevant or not.

 

0 Kudos
Bud
by
Notable Contributor

Test #2:

I did a test using ArcGIS Pro on my home computer:

  • ArcGIS Pro 2.9.2
  • FGDB
  • Created a non-spatial table from scratch
  • No subtypes

The script worked fine using the specs mentioned above. So I guess there's something in my work environment that is causing a problem. I'll do some more tests...

Bud_0-1656128399854.png

 

0 Kudos
Bud
by
Notable Contributor

Test #3:

I did a third test (again, on my home computer) where I tested a subtype.

The script ran without errors, but it didn't catch the invalid domain value. But that's totally reasonable, since I forgot to mention in my original post that the data uses a subtype...the script likely wasn't designed with subtypes in mind.

Bud_1-1656130338220.png

So, to me, that suggests that the error I got in my first test (work computer) might not be caused by the subtype, since the script ran without errors on my home computer where a subtype was present.

0 Kudos
Bud
by
Notable Contributor

Test #4:

I did a fourth test, this time on my work computer.

I emailed the FGDB from my home computer to my work computer. And tested the script on the FGDB on my work computer.

The script ran without errors. (Although it still doesn't catch the invalid value. But as mentioned, that's expected, since the script wasn't designed for subtypes.)

Bud_0-1656131168313.png

So that test confirms that the subtype isn't causing the error on my work computer. It also confirms that the older version of ArcGIS Pro (2.6.8) isn't causing the problem either.

So I think the issue must be something to do with my enterprise GDB environment (or the table).

 

I googled the error message:

"AttributeError: 'NoneType' object has no attribute 'keys'"

I'll review those results and see if I can figure out what's happening.

0 Kudos
JohannesLindner
MVP Frequent Contributor

It's line 15 of my code. It found at least 1 domain, but one of the domains' codedValue attribute is null. This might be a ranged domain.

You could try replacing line 15 with this:

domain_values = dict()
for d in domains:
    try:
        domain_values[d.name] = list(d.codedValues.keys())
    except AttributeError:
        print(f"{d.name} is not a coded value domain.")

 


Have a great day!
Johannes