topic Re: Delete Identical but Keep the most recent in Transportation Questions

Delete Identical but Keep the most recent

LoganCaraway — Thu, 19 Sep 2013 14:53:21 GMT

Hello,
I receive data once a week from a third party vendor. There are 2 fields in which I wish to use to exclude duplicate.
rig_number, and a date field. Currently my script look like this. (standard delete identical code).

arcpy.DeleteIdentical_management(rigsFc,rig_number)

The thing is that I only care about the most recent one and this process seems to select one at random to keep. Is their a way to use delete identical but keep the attribute with the most current date?

Im using python 2.6 and arcMap 10.0

Thanks

Re: Delete Identical but Keep the most recent

Anonymous User — Thu, 19 Sep 2013 15:16:31 GMT

Original User: mzcoyle

I would use an update cursor, sort by date (newest to oldest) use a list to track all rig_number values, and if the value is already in the list delete that row, if it is not in the list add it and move on.

Re: Delete Identical but Keep the most recent

LoganCaraway — Thu, 19 Sep 2013 19:02:33 GMT

I would use an update cursor, sort by date (newest to oldest) use a list to track all rig_number values, and if the value is already in the list delete that row, if it is not in the list add it and move on.

I gave it a shot. Here is what I have..

import arcpy, os, sys
FC = r'M:\GIS\DI_AutomationScript\Test\DI13gdb_237c.gdb\di_Rigs'
Rows = arcpy.SearchCursor(FC,"","","rig_number;date_processed","date_processed D")

RigList=[]

for row in Rows:
    if row.getValue('rig_number') not in RigList:
        RigList.append(row.getValue('rig_number'))
    else:
        arcpy.DeleteRows_management(row)


However, an exception was raised...

Traceback (most recent call last):
File "C:\Python26\ArcGIS10.0\Lib\site-packages\Pythonwin\pywin\framework\scriptutils.py", line 326, in RunScript
    exec codeObject in __main__.__dict__
File "\\cw-file1\kgsusers\LCaraway\My Documents\Update cursor.py", line 11, in <module>
    arcpy.DeleteRows_management(row)
File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\management.py", line 8875, in DeleteRows
    raise e
RuntimeError: Object: Error in executing tool

Any thoughts where I went wrong?

Re: Delete Identical but Keep the most recent

Anonymous User — Thu, 19 Sep 2013 19:07:49 GMT

Original User: mzcoyle

You'd want to do something closer to this. Utilizing an update cursor and the deleteRow method of the cursor, not the DeleteRows tool.

import arcpy  rig_number = 'RIG_FIELD' date_field_sort = 'DATEFIELD D' update_cursor = arcpy.UpdateCursor(rigsFC, '', '', rig_number, date_field_sort) keep_list = list() for row in update_cursor:     row_val = row.getValue(rig_number)     if row_val not in keep_list:         keep_list.append(row_val)     elif row_val in keep_list:         update_cursor.deleteRow(row)     else:         pass

Re: Delete Identical but Keep the most recent

LoganCaraway — Sat, 11 Dec 2021 13:15:52 GMT

You'd want to do something closer to this. Utilizing an update cursor and the deleteRow method of the cursor, not the DeleteRows tool.

import arcpy

rig_number = 'RIG_FIELD'
date_field_sort = 'DATEFIELD D'
update_cursor = arcpy.UpdateCursor(rigsFC, '', '', rig_number, date_field_sort)
keep_list = list()
for row in update_cursor:
    row_val = row.getValue(rig_number)
    if row_val not in keep_list:
        keep_list.append(row_val)
    elif row_val in keep_list:
        update_cursor.deleteRow(row)
    else:
        pass

This did the trick! Thanks for your help!

I am still confused as the difference between a search cursor and an update cursor. Is it that the update cursor is editable and thus methods such as deleteRow can be used with it?

Thanks again!

Re: Delete Identical but Keep the most recent

Anonymous User — Thu, 19 Sep 2013 20:28:30 GMT

Original User: mzcoyle

This did the trick! Thanks for your help!

I am still confused as the difference between a search cursor and an update cursor. Is it that the update cursor is editable and thus methods such as deleteRow can be used with it?

Thanks again!

Exactly. You can read the help docs on cursors here to get a better handle on the difference between cursors.

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//002z0000001q000000