Latest Contributions by RichardFairhurst

@AnninaRupe1 Thank you for sharing your appreciation of my efforts to try to help people make the connection between the key Python concepts I'm describing and the handful of lines of sometimes densely compacted code I've present.  It also helps me to spell out why it was done that way, to see whether it really has stood up over time or could benefit from an update with things I have learned since I originally wrote this blog. I can only recall one other Python Blog I wrote called: I've Saved Time in a Bottle. How Do I Get it Back Out? - Doing More with Date Fields Using the Field Calculator and Python   The Turbo Charging Data Manipulation blog remains the single most useful coding concept I have repeatedly applied and adapted to solve problems I have encountered in my own career.  And I am grateful to see that it has stood the test of time and continues to be a resource for many other people.   ... View more

@AnninaRupe1 Thank you for the question and trying to understand the concepts taught in this blog better. Constructing a dictionary from a list can be done in many ways, depending on how you want to process the list data in you code.  In the case of my code I will try to break the dictionary construction down more and explain why I chose to do it the way I did.  Here is the code from example 3 that built the dictionary. valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(sourceFC, sourceFieldsList)} The dictionary key value is the field value stored under the first field name in the field list: r[0].  The dictionary value contained under that particular key is a tuple () containing the set of all of the field values from the field name list r except the first field name in the field names list stored at index 0 of the field name list, since it is not the entire field list and instead starts at index 1 of the field names list: (r[1:]).   The reason for this is that I only use the first field as a matching key value, and not as a data transfer value in the rest of my code.  I saw no need to include code to transfer and overwrite the key into the field in the matched record in the target data that by definition must already contain the matched key value before a transfer can take place. valueDict[keyValue] returns the tuple associated with the key that only contains values that actually need to be written to the matched record.  The consequence of my choice to exclude they key value from the tuple is that the tuple indexes are shifted left or -1 relative to the original field list.  Not overwriting the key value in the target record makes the code faster than an unnecessary data overwrite, since writing data is the slowest part of the code and you want to avoid unnecessary data writes that you possibly can in your code. The data transfer part of the code makes the necessary index shift relative to the original field name list indexing to keep everything aligned. There are alternative ways the code could have accomplished the same thing, but you would have to adjust the indexing shown in my example in both the dictionary construction and the data transfer part of the code for it to work and still avoid the unnecessary overwrite of the key value that my code intentionally avoids.   List manipulation is a topic unto itself and is largely driven by the goals of your code, which must consider and balance valid logic, speed, understandability, elegance and efficiency.  Choosing, understanding and applying at least one form of internally consistent and valid logic and code syntax that accomplishes your goal is always essential before you try to adjust the logic or syntax to optimize and improve the code for the other 4 code factors.  Hopefully, this post is useful in helping you better understand the choices driving the way I wrote my code.  If you believe you have discovered ways to improve my code relative to the other factors, feel free to offer working code examples here. ... View more

If you only want area values that have changed to be updated you should use the if condition below: if valueDict[keyValue] != updateRow[1]: Potentially you would need an additional prior if clause to do an update to handle Null values.  Something like: if valueDict[keyValue] != None and updateRow[1] == None: Also you need to change the update assignment line from: updateRow[n] = valueDict[keyValue][n-1] to: updateRow[n] = valueDict[keyValue] This change is needed since your dictionary values returned are floats and not a tuple or list that requires the use of an index to retrieve a value from within it.  The code could be rewritten to be simplified further since your dictionary is not returning a list, but if you just make this change the code would work without an error. If that does not accomplish your goal, please try to explain the final result you would like to acheive in more detail. ... View more

list() converts an interable object like a tuple to a list.  The valueDict[keyValue] evaluates as a float, which cannot be converted to a list, it can only be added or appended to an existing list.  Also, since the valueDict[keyValue] is always a single float value, there is no benefit to enclosing it in a list over working with it directly as a float value unless you were going to append multiple float values to the list.  This is how you would create a list variable that contains  your float value: myList = (valueDict[keyValue]) You need to explain what you believe you are trying to accomplish by putting a float value into a list and why you think that is required for your code to accomplish it's purpose.  I don't see any purpose for why you are doing what you are doing or have any idea what your end goal is with this portion of your code. ... View more

I got the upper syntax wrong.  It is fixed in my previous post now.  The syntax is supposed to be: text.upper() You missed the parentheses at the end. I don't know why the case is changing in the two environments.  I don't work with portal data very much, so someone with more experience will have to let you know if that behavior is normal or not.   Also this line of code makes no sense to me: if list(valueDict[keyValue]) and updateRow[1:1]: those are not logical true false test conditions.  What is this line supposed to do? Rich ... View more

You can change the case to all upper or all lower in both the dictionary and the cursor to force a match even if they are stored in different letter cases on disk.  This modification is similar to your previous code where you made the key the combination of two field values that are stored as two separate field values on disk. valueDict = {r[0].upper(): r[1] for r in arcpy.da.SearchCursor(temp_Areas, sourceFieldsList)}             print(valueDict) keyValue = updateRow[0].upper() So one of the key benefits of this code is that tables that don't work using a standard attribute join can be matched and at a faster speed than a Join and calculation (which would also update all rows rather than just the rows that have actual differences.) ... View more

I found the error in my previous code.  I had not noticed that I had pasted the for loop twice when it should only have been pasted once:   for updateRow in updateRows I have removed the second for loop and it will now work according to your needs.  This revised code is many times more efficient than your current code, especially if only a few records actually need to be updated.  The larger your record set becomes, the larger the benefit of using my revised code, since the act of updating a row is the most costly part of the loop, and avoiding it when no change is actually occurring is the most significant optimization you can implement.  You also won't have to disable editor tracking with the updated code, since records that have no change will not be updated and not trigger the modified date. I left the insert and delete code for the benefit of others.  The complete synchronization of a relationship to produce a complete match efficiently from a source requires all three steps. ... View more

tempDict = {} delDict = {} with arcpy.da.UpdateCursor(fs2, updateFieldsList) as updateRows: for updateRow in updateRows: # store the Join value by combining 2 field values of the row being updated in a keyValue variable keyValue = updateRow[0] + "," + str(updateRow[1]) tempDict[keyValue] = keyValue.split(",",1) # verify that the keyValue is in the Dictionary if keyValue in valueDict: update = False for n in range (2,len(sourceFieldsList)): if valueDict[keyValue][n-2] != None: if updateRow[n] == None: updateRow[n] = valueDict[keyValue][n-2] update = True elif updateRow[n] != valueDict[keyValue][n-2]: updateRow[n] = valueDict[keyValue][n-2] update = True if update == True: updateRows.updateRow(updateRow) else: delDict[keyValue] = keyValue with arcpy.da.InsertCursor(fs2, updateFieldsList) as cursor: for key in valueDict.keys(): if not key in tempDict: insRow = key.split(",",1) + list(valueDict[key]) print(insRow) cursor.insertRow(insRow) with arcpy.da.UpdateCursor(fs2, updateFieldsList) as updateRows: for updateRow in updateRows: keyValue = updateRow[0] + "," + str(updateRow[1]) if keyValue in delDict: updateRows.deleteRow() del delDict del tempDict del valueDict The above code should only update row A if the row B field is not Null and one or more field in row A is Null or has a different value from the row B field.  It should not do an update if for all fields the row B field is Null, both the row A field and row B field are Null, or the row A field and row B field contain the same value. This code also should insert rows from the source that are missing in the update target and delete rows from the update target that are not in the source.  The insert and delete operations are done in separate for loops from the initial updatecursor to prevent locks and to avoid having the updateCursor get confused by rows being deleted. It is important to note that InsertCursors require that the updateFieldsList has the field names and field values arranged in the exact same order as the actual field order of the underlying table/feature class/service, otherwise the insert will fail.  SearchCursors and UpdateCursors do not have this requirement and can process fields in any order, but InsertCursors can't rearrange the underlying field order. ... View more

Print both halves of the if condition on a record that was updated that you belive should not have updated on separate lines.  The difference could be that the lists that are being compared are misaligned, which wouldn't necessarily cause a runtime error, but it would be a coding logic error. Also post your full code so far as a base for me to edit.  I cannot peice together the code on my phone without retyping everything otherwise.  I need the full code to add the code sections and logic for processing insert and delete cursors.  In any case, I am done for the night. ... View more

Try indenting all of the code you just posted under this if clause to only update rows that actually are different.  if list(updateRow[2:]) != list(valueDict[keyValue]): If it triggers errors, post them.  ... View more

The process you are describing is not something I cover in this blog.  It requires an insertCuror for new records and creation of multiple dictionaries built in multiple passes to process each cursor correctly (including a deleted record dictionary if you want a true match).  The updateRows.updateRow(updateRow) line also has to only occur at an indentation level that only occurs where the if condition detecting changes occurs is met and not at a level that occurs for every row.  I will see if I ever posted anything elsewhere that comes closer to that approach. If you post the code you have working I could copy it and modify it to get closer to what you are describing.  It is too difficult to create a post with sample code on my phone without that as a starting point. For the update part this should work to ignore nulls and only update records that do not match. # verify that the keyValue is in the Dictionary if keyValue in valueDict: if list(updateRow[2:]) != list(valueDict[keyValue]): # transfer the values stored under the keyValue from the dictionary to the updated fields. for n in range (2,len(sourceFieldsList)): if valueDict[keyValue][n-2] != None: updateRow[n] = valueDict[keyValue][n-2] updateRows.updateRow(updateRow) I am not sure if the comparison line works correctly in converting the tuples to lists, so let me know if it triggers an error so that some print statements can be added to make the comparison correctly.  ... View more

There was another error in the code caused by editing on my phone that I corrected on the last line, which should have been two separate lines of code as now shown.  Also, you should be prepared to adjust and correct any indentation errors on your own in your IDE, since they can sometime be difficult to catch in the code window of the community.  The IDE was highlighting the character where the line return and indentation error occurred in your screenshot. ... View more

I am editing using my phone, so I just had to correct some code errors that occurred due to the difficulty of navigating in that editing environment.  Please recheck my post's code. ... View more

You should have based your code on the section entitled: Example 2 - Transfer of Multiple Field Values between Feature Classes where there is a 1:1 Match between Field Sets The section for updating fields should look like the code below to update all matched fields in the records where the value is Null.  However, you should test this on a copy of your data before aplying it on any production data to verify that I aligned the field updates correctly with your other code modifications: # verify that the keyValue is in the Dictionary if keyValue in valueDict: # transfer the values stored under the keyValue from the dictionary to the updated fields. for n in range (2,len(sourceFieldsList)): if valueDict[keyValue][n-2] == None: updateRow[n] = 0 else: updateRow[n] = valueDict[keyValue][n-2] updateRows.updateRow(updateRow) This code assumes you want to assign a default value of 0 if the field is Null and otherwise transfer the values of the source to the target field.  I have made several revisions to the code since my initial response after considering your requirements more carefully.  Hopefully, I have understood them correctly. ... View more

I am not having issues with the Preserve GlobalIDs setting.  I only mentioned that setting in my post because it is simpler to apply than the procedure you have outlined for preserving ObjectID values.  I am more inclined to consider the preservation of GlobalIDs setting as a truly built-in Already Offered feature over the steps required for the preservation of ObjectIDs.  However, I accept that the software currently supports both aspects of my original idea provided the user is familiar with both of these settings/steps. ... View more