Troubleshooting a loop utilizing a cursor and a dictionary to write to a table

RachaelJohnson — Sun, 12 Dec 2021 06:59:11 GMT

I am comparing the values in one table to the values in another table. I converted the values of one table into a dictionary to avoid using nested cursors. Ultimately, I want to create a table of items that pass each comparison test. I did another version of this where I have separate tables for each area I am testing, start with a full table, and remove rows from the table as each row fails a test. This results in many tables, which is fine for a handful of areas but messy for more than a few. I am trying to build a version that generates all area outputs in the same table.

I think it is easier to keep track of those that fail tests than those that pass. I copy the main dictionary, D, to a new dictionary. I create a blank dictionary for the failed tests. Each time something fails a test, I record its key into the Failed Test dictionary and store the reason for failure as a string value. For subsequent tests, if the key already exists in the Failed test dictionary, I add on to the existing value as a pseudo-list to prevent test failures from being overwritten.

When all tests are complete, I want to compare the keys in the Failed Test dictionary to the copy of D. If the keys are a match, I want that key and it's associated value removed from the copy of D. What should remain in the copy of D are the BMPs that pass all 4 tests. Then I want to use the values stored in the copy of D to write to a table.

Maybe this logic is all sorts of messed up and there's a better way to do it. I am open to suggestions.

I have been playing with this code for a while to figure out what the problem is and it seems I have found the first source of error in my code. There appears to be a problem with copying the dictionary, D. Here's the code I have so far. It is missing the write-to-table part because I am not getting the outputs I would expect.

print("Comparing site values to constraints...")
# Compare the lumped parameters to the constraint dictionary
for row in arcpy.da.SearchCursor(combo_table, ["DARAS", "HSG", "MEDSLOPE", "MEDWT"]):
    BMP = {}
    # Temporarily store the area of each DA for later comparison
    for r in arcpy.da.SearchCursor(DA_area, [DAID, "SUM_AREA"]):
    # NOTE: THE DAID/DARAS MISMATCH COULD BE A PROBLEM LATER. NEED TO TEST.
        if r[0] == row[0]:
            totalArea = r[1]
    # Duplicate criteria dictionary that can be amended throughout the loop
    print D
    BMP = D
    print D
    # Initialize empty dictionary to store BMPs that fail each test
    NoBMP = {}
    # Compare lumped values in each DA/HSG pair to those in the constraint table
    print(row[0], row[1])
    for k, v in D.items():
        # Test if soil type is incorrect for each BMP
        if row[1] not in v:
            # Add failed BMP to the failed BMP dictionary, NoBMP
            NoBMP = "Soil Type Mismatch"
        # Compare keys in BMP and NoBMP dictionaries. Remove matching pairs from the BMP dictionary.
    for key in BMP.keys():
        if key in NoBMP.keys():
            del BMP[key]

The main takeaway from the following output information is that D changes sizes from one iteration of the Cursor to the next iteration of the Cursor; D does not change size in future iterations of the loop. I think it has something to do with me changing the size of BMP at the end of the code by deleting the matching keys. The keys that are removed are the keys that do not show up in the next iteration. However, I reinitialize BMP as a blank dictionary at the beginning of the Cursor, so I am not sure what else to do. I have also tried BMP = dict() to reinitialize and gotten the same result.

Comparing site values to constraints...

{u'WPGW1': u'C,D', u'WPGW2': u'C,D', u'VFA': u'A', u'SDCD': u'C,D', u'BRE2': u'A,B,C,D', u'BRE1': u'A,B,C,D', u'SDAB': u'A,B', u'EDP2': u'A,B,C,D', u'WP1': u'C,D', u'CW1': u'C,D', u'CW2': u'C,D', u'DS2': u'A,B,C,D', u'DS1': u'A,B,C,D', u'GCCD': u'C,D', u'F2': u'A,B,C,D', u'F1': u'A,B,C,D', u'CAAB': u'A,B', u'WP2': u'C,D', u'GCAB': u'A,B', u'CACD': u'C,D', u'VFSA': u'B,C,D', u'SI2': u'A,B', u'SI1': u'A,B', u'EDP1': u'A,B,C,D', u'SDSA': u'C,D', u'WS1': u'C,D', u'CI2': u'A,B', u'CI1': u'A,B', u'WS2': u'C,D', u'GCSA': u'C,D', u'MI1': u'A,B', u'MI2': u'A,B'}

(1, u'D')

{u'WPGW1': u'C,D', u'WPGW2': u'C,D', u'SDCD': u'C,D', u'BRE2': u'A,B,C,D', u'BRE1': u'A,B,C,D', u'EDP2': u'A,B,C,D', u'WP1': u'C,D', u'CW1': u'C,D', u'CW2': u'C,D', u'DS2': u'A,B,C,D', u'DS1': u'A,B,C,D', u'GCCD': u'C,D', u'F2': u'A,B,C,D', u'F1': u'A,B,C,D', u'WP2': u'C,D', u'CACD': u'C,D', u'VFSA': u'B,C,D', u'EDP1': u'A,B,C,D', u'SDSA': u'C,D', u'WS1': u'C,D', u'WS2': u'C,D', u'GCSA': u'C,D'}

(2, u'D')

Re: Troubleshooting a loop utilizing a cursor and a dictionary to write to a table

JoshuaBixby — Wed, 20 May 2015 20:54:31 GMT

I am not suggesting the following will fix your overall issue, but I do have a comment regarding dictionaries.

You speak of "copying the dictionary" and there is a code comment to "duplicate" the dictionary. Where are you making this copy? Line 12? Line 12 is not copying dictionary D, you are just binding another variable to the same dictionary. Python dictionaries have a copy() method for returning a shallow copy of themselves.

Re: Troubleshooting a loop utilizing a cursor and a dictionary to write to a table

RachaelJohnson — Wed, 20 May 2015 21:36:22 GMT

That was certainly the problem! I modified the code with the copy package so BMP = copy.deepcopy(D) and the problem resolved. Thanks.

topic Re: Troubleshooting a loop utilizing a cursor and a dictionary to write to a table in Python Questions

Troubleshooting a loop utilizing a cursor and a dictionary to write to a table

Re: Troubleshooting a loop utilizing a cursor and a dictionary to write to a table

Re: Troubleshooting a loop utilizing a cursor and a dictionary to write to a table