Find duplicate field values in ArcGIS using Python

Question

I have a large feature class with million records. It has a 'CODE' field, this 'CODE' field has many duplicated value.  I need to keep only one value from it, and delete all other duplicated values.  Another words, I need to have unique field value.  No duplicated field values.  How can I do it?  Thanks so much for any help.

BenHolmes · Answer

You could try a Dissolve based on the CODE
Look in ArcTools under Data Management Tools/Generalization

markdenil · Answer

Delete Identical may be a tad dramatic, if you wanted to keep the records but make the field unique.

You could create an empty list of CODE values:    codeVal = []
make an Update cursor to run through and examine the CODE in each record.
If that CODE is not in the list, add it and pass on.
If it is in the list (it is already assigned), modify it. For a numeric field, you could, for example, add a round number to it
Test it against the list again, and increment as needed until it is unique.
Calculate the CODE to this new value, and add the new value to the list.
continue to the next record......

curtvprice · Answer

I have a large feature class with million records. It has a 'CODE' field, this 'CODE' field has many duplicated value.  I need to keep only one value from it, and delete all other duplicated values.

There are two nifty tools in 10.x that with the Sort tool can solve the problem:

Find" rel="nofollow" target="_blank">http://help.arcgis.com/en/arcgisdesktop/10.0/help/0017/001700000054000000.htm]Find Identical
Delete" rel="nofollow" target="_blank">http://help.arcgis.com/en/arcgisdesktop/10.0/help/0017/001700000053000000.htm]Delete Identical