topic Re: Dictionary issue in Python Questions

Dictionary issue

CCWeedcontrol — Wed, 09 Feb 2022 18:13:34 GMT

I have the following script and the issues I am having is that the PermitNum can have multiple Par (par field) numbers but my current script will only populate the Parel field with just one Par attribute, it doesn't transfer each parcel that corresponds with each permit number. How can I adjust my current code to transfer each parcel number?

fc_dest = "D:/Temp/Cases" sourceFC = "D:/Temp/Casestableview" #Use Dictionary to update fc fileds. sourceFieldsList = ['PermitNum','Par','AppS','AppT', 'SubT','Desc', 'MainApp', 'PA', 'Status', 'AppStatus', 'DecDate',]#,'ProjectInfo', 'Contractor', 'Subdivision', 'TwnRngSec', 'LivingSpace', 'TotalSqFt', 'Value_', 'Valuation', 'Status'] # Use list comprehension to build a dictionary from a da SearchCursor valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)} updateFieldsList = ['PermitNum','Par','AppS','AppT', 'SubT','Descr', 'MainApp', 'PA', 'Status', 'AppStatus', 'DecDate'] with arcpy.da.UpdateCursor(fc_dest, updateFieldsList) as updateRows: for updateRow in updateRows: # store the Join value of the row being updated in a keyValue variable keyValue = updateRow[0] # verify that the keyValue is in the Dictionary if keyValue in valueDict: # transfer the values stored under the keyValue from the dictionary to the updated fields. for n in range (1,len(sourceFieldsList)): updateRow[n] = valueDict[keyValue][n-1] updateRows.updateRow(updateRow) del valueDict

Re: Dictionary issue

DonMorrison1 — Wed, 09 Feb 2022 20:21:00 GMT

You need to figure out how to uniquely identify each record and make that your dict key. From you example it looks like a combination of 'PermitNum' and 'Apps' is unique. In that case I would make my dict look like this:

valueDict = {(r[0],r[2]): r for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)}

Re: Dictionary issue

CCWeedcontrol — Wed, 09 Feb 2022 20:53:40 GMT

I thought that's what line 9 was doing?

My unique identifers are PermtNum and Par fields, the Apps field is just a date field. Unless I am not understanding the dictionary correctly...?

valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)}

Re: Dictionary issue

DonMorrison1 — Wed, 09 Feb 2022 21:26:50 GMT

I think your dict is being built with "PermitNum" as the key - and since that field is not unique the dictionary entry is being overwritten.

Just curious - in your "I need the following" graphic, how do you know which values to put in the 'Par' column? Since 'PermitNum' is not unique you must be using some other piece of information (I assumed it was the date field but it appears that is wrong).

Re: Dictionary issue

Anonymous User — Wed, 09 Feb 2022 21:41:42 GMT

Based on your description your structure looks something like: { PermtNum : (Par, 'AppS','AppT', 'SubT','Descr', 'MainApp', 'PA', 'Status', 'AppStatus'), ...} ?

If it is only adding one, it is because it is stopping at the first matched key and using the first set of values. You need to have a loop iterating over the value sets of that matched key.

I find it helpful to print out the dictionary and paste some of it as a code comment so you can refer to it visually while you are coding. Give it a try-it also helps those troubleshooting with visualizing the data since we can't debug.

I'd also suggest using just row in your cursor so your not using the same name as the updateRow method.

updateRows.updateRow(updateRow)

Re: Dictionary issue

CCWeedcontrol — Wed, 09 Feb 2022 23:19:05 GMT

PermitNum should be the unique identifier but as you can see PermitNum has duplicates because I don't have no other Unique Identifier.

Re: Dictionary issue

CCWeedcontrol — Thu, 10 Feb 2022 00:02:34 GMT

ok, based on what you are describing I think that is my problem. I guess I am not sure how to iterating loop over the vale sets that match. Do you have an example?

Re: Dictionary issue

Anonymous User — Thu, 10 Feb 2022 00:20:22 GMT

I think that @DonMorrison1 is right in that it is overwriting the key's values with data instead of appending it to a list for the key. Printing out the dictionary will give us a better idea of what to suggest for a fix. Starting with that and then seeing how the dictionary is structured would help determine the code needed for updating. If you find it is overwriting the values, you'll have to test for a keys existence and either set or append for each row's data in that SearchCursor to get that key: [item, item, item] structure.

Re: Dictionary issue

CCWeedcontrol — Thu, 10 Feb 2022 15:33:11 GMT

Below is what prints out.

valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)} {'2021-0006': ('2957001100', datetime.datetime(2021, 6, 11, 0, 0), None, None, '', None, 'address', 'Active', 'In Progress', None), '2021-0032': ('2957001100', datetime.datetime(2021, 6, 11, 0, 0), None, 'Preliminary Plat', '', None, 'address', 'Active', 'In Progress', None,), valueDict = {(r[0],r[1]): r for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)} {('2021-0006', '2957001100'): ('2021-0006', '2957001100', datetime.datetime(2021, 6, 11, 0, 0), None, None, '', None, 'address', 'Active', 'In Progress', None), ('2021-0032', '2957001100'): ('2021-0032', '2957001100', datetime.datetime(2021, 6, 11, 0, 0), None, 'Preliminary Plat', '', None, 'address', 'Active', 'In Progress', None),

Re: Dictionary issue

DonMorrison1 — Thu, 10 Feb 2022 16:34:01 GMT

The dict is exactly what I would expect. But, I still don't understand your objective - and will ask this again:

Just curious - in your "I need the following" graphic, how do you know which values to put in the 'Par' column?

Re: Dictionary issue

CCWeedcontrol — Thu, 10 Feb 2022 19:09:11 GMT

Base on the PermitNum, For example permit 2020-0014 has 7 records in the Par field in the table I need it to transfer the tables Par field records to the feature classes Par field if they don't exits in the feature class, 2020-0013 has 4 permit records in the table I need to only transfer the two Par records to the feature class since it already the feature class already has two populated. I don't need to transfer ones that already in the feature classes Par field. Hopefully this makes sense. I can attach some data if that would help?

Thank you for responding I am very appreciative.

Re: Dictionary issue

DonMorrison1 — Thu, 10 Feb 2022 22:00:22 GMT

If you are sure your table and feature classes line up (after being sorted) as you show in your diagram then you should use a list to store your table data instead of a dict. This will preserve the order so you can step through one rows at a time and move the Par field over when you find an open slot. But I'd be surprised if your table and feature class really are lined up like that......

Re: Dictionary issue

CCWeedcontrol — Mon, 14 Feb 2022 16:31:20 GMT

Unfortunately they are not lined up like on the pictured I attached, I was trying to give an example. I have attached some sample data if anyone is kind enough to help me out as I am not sure how else to accomplish what I need.

Re: Dictionary issue

Anonymous User — Tue, 15 Feb 2022 04:38:21 GMT

dicts in 3.7+ preserve the order now, but I see and think you can do it by creating a more unique key by concatenating the PermNum and Apps.

key = row[sCur.fields.index['PermNum']] + str(row[sCur.fields.index['app']]).replace('/','')

and then in your update cursor, if the Par is empty for the row combine the PermNum and App in the same manner to get the key.

edit to add example using cursor: (it's not tested but should provide a good starting point. I only assigned the Par as value since that is the field you are after, no purpose bringing over any of the others?)

valueDict = {} with arcpy.da.SearchCursor(sourceFC, sourceFieldsList) as sCur: for row in sCur: valueDict[row[sCur.fields.index('PermitNum')] + str(row[sCur.fields.index('AppS')]).replace('/', '')] = row[sCur.fields.index('Par')]

Re: Dictionary issue

DonMorrison1 — Wed, 16 Feb 2022 13:26:34 GMT

For this to work you must have a field or combination of fields in your table view that uniquely identifies each record in that view AND the "matching record" in the feature class. I may be wrong but it appears to me that you don't have this so the problem really has nothing to do with python but in your data.

Re: Dictionary issue

CCWeedcontrol — Thu, 17 Feb 2022 16:51:43 GMT

Jeffk. Thank you for the reply. I would like to being all values but I didn't want to type them all out in my example. I guess I am not sure how to proceed with the construction of the code as I am not familiar with "keys" on how your example is showing them. Is there a complete example of the process you are describing?

Re: Dictionary issue

Anonymous User — Fri, 18 Feb 2022 03:25:05 GMT

It wont let me open the sample shapefile but it let me view the Permits_view so I built an example using its 3 fields. Same concept for the shapefile- you just have to find a combination of fields to create the 'key' in your dictionary. You can add more fields to the concatenation if needed/available until you get unique keys.

This is an image of the code at work so you can see the values in the dictionary:

valueDict = {} with arcpy.da.SearchCursor(sourceFC, ['OBJECTID', 'PermitNum', 'Par']) as sCur: for row in sCur: # str(row[sCur.fields.index('AppS')]).replace('/', '') <- use this to convert the dates in the App fields to strings concatenatedKey = row[sCur.fields.index('PermitNum')] + str(row[sCur.fields.index('Par')]) if concatenatedKey in valueDict.keys(): print(f'duplicate concatenation: {concatenatedKey}') else: valueDict[concatenatedKey] = row[sCur.fields.index('OBJECTID')]

Then once the 'source' dictionary is built, you concatenate the same fields in the target shapefile and use it to lookup the valuesDict.

with arcpy.da.UpdateCursor(sourceFC, ['OBJECTID', 'PermitNum', 'Par']) as uCur: for row in uCur: if row[2] is None: lookupKey = row[sCur.fields.index('PermitNum')] + str(row[sCur.fields.index('Par')]) if lookupKey in valueDict.keys(): print(f'{lookupKey} found, setting Par to: {valueDict.get(lookupKey)}') uCur.updateRow(row)

Re: Dictionary issue

CCWeedcontrol — Fri, 18 Feb 2022 17:14:32 GMT

Jeffk, thank you very much for the help and my apologies for the corrupt data. I have attached new data and have verified that it should work.

I have changed the TablesView field name from 'Par' to 'Parcel_1' and the feature classes field name'Par' to 'Parcel' as it was getting confusing.

I have the following based on you provided but the 'Parcel' field is being populated in the Case_lyr feature class.

The valueDict prints- 'CR2021-0006R2957001100': 1, 'OR2021-0008R3014800000'

The concatenatedKey - prints CR2022-0001R3301700000
CR2022-0001R3301901000

sourceFC = "C:/Temp/Cases.gdb/Table_View" valueDict = {} with arcpy.da.SearchCursor(sourceFC, ['OBJECTID', 'PermitNum','Parcel_1']) as sCur: for row in sCur: # str(row[sCur.fields.index('AppS')]).replace('/', '') <- use this to convert the dates in the App fields to strings concatenatedKey = row[sCur.fields.index('PermitNum')] + str(row[sCur.fields.index('Parcel_1')]) if concatenatedKey in valueDict.keys(): print(f'duplicate concatenation: {concatenatedKey}') else: valueDict[concatenatedKey] = row[sCur.fields.index('OBJECTID')] fc_dest = "C:/Temp/Cases.gdb/Cases_lyr" with arcpy.da.UpdateCursor(fc_dest, ['OBJECTID', 'PermitNum','Parcel']) as uCur: for row in uCur: if row[2] is None: lookupKey = row[sCur.fields.index('PermitNum')] + str(row[sCur.fields.index('Parcel')]) if lookupKey in valueDict.keys(): print(f'{lookupKey} found, setting Par to: {valueDict.get(lookupKey)}') uCur.updateRow(row)

Re: Dictionary issue

Anonymous User — Fri, 18 Feb 2022 18:21:30 GMT

Not sure what your asking here- "I have the following based on you provided but the 'Parcel' field is being populated in the Case_lyr feature class."

You need to use the AppS field (or another one) for the concatenation, which is why I left the code in there

# str(row[sCur.fields.index('AppS')]).replace('/', '') <- use this to convert the dates in the App fields to strings

to convert it to a string in my example of creating the unique key. I only used the Par field because that is what was available in the table view.

I left the row[i] assignment out in the update cursor so it shouldn't be putting a value there because I don't have all your shapefiles and left that part up to you.

Re: Dictionary issue

DonMorrison1 — Sat, 19 Feb 2022 03:39:53 GMT

I've been following this and would like to help you out with the python question but I suggest you forget about python for now and first tell us what are the names of the fields are in the table and the feature class that let you uniquely identify each record - as I posted previously. Like @Anonymous User I deduced from your example that it was PermitNum and Apps, but evidently that is not correct.