Removing Duplicate feature Points/Records

JasonHodgkins

I have a point shapefile with Name, address etc in the attribute field. I want to delete all the duplicate records from the attribute table with the same name. For example I have a 10 records that have SMITH in the name field. How can I remove/delete records to just list 1 record for SMITH. Can someone suggest a better way than just sorting the data and doing it manually as there are thousands of records. Is there a tool or a selection method that will work or a script/python code I code easily apply to perform this task?

RichardFairhurst

I have a point shapefile with Name, address etc in the attribute field. I want to delete all the duplicate records from the attribute table with the same name. For example I have a 10 records that have SMITH in the name field. How can I remove/delete records to just list 1 record for SMITH. Can soactual meone suggest a better way than just sorting the data and doing it manually as there are thousands of records. Is there a tool or a selection method that will work or a script/python code I code easily apply to perform this task?

Since this is a point shapefile are you wanting to delete the actual points or are you wanting a separate table derived from the points that has unique values for that field?  For either case you can perform a Summary, but to delete the actual points you have to perform a few more steps.

A Summary on the NAME field will create a new sorted table with the unique values of that field and a count of how many occurances of that value are in the point shapefile.  If that is all you want, that is all you have to do.

If you don't care which point is chosen to represent the unique Name value and you want to actually delete points, you would need to do a Summary, but also add a Min, Max or First summary on some other field that contains a value that represents a unique key for each of the points.  The Summary tool may not let you use the ObjectID field directly, so you may want to add a temporary field of type Long and calculate the ObjectID into it.  Then you could do a Min summary on the temporary OID field.

Once you have the new Summary table with the summary field for the unique key, join the Summary table to the point class on the summary unique key field.  If you want to create a new exported copy of the unique points, select all values where the joined table ObjectID is not Null, break the join and export the selected records.  If you want to actually delete the unmatched points from the original point shapefile and have no back up, select all values where the joined table ObjectID is Null, break the join and do a deletion of the selected records in an Edit session.  This method does not take into account any spatial characteristics of the points, so the point that is left will be basically random (but every unique NAME value will be preserved in the final result).

If you want to preserve the point spatial data but reduce the number of attribute rows to a single row, you should convert your points to multipoints using the Dissolve tool.  That will preserve all your point geometry within a single multi-point feature for each unique NAME value.

If you want multiple fields to be used to select records with unique combinations rather than one field, I typically like to create a concatenated version of those field values and use that as a summary field with the above method.  Some sort of script is another alternative for dealing with multiple field unique combinations.

There may be other options depending on the final result you want and whether or not you want some spatial characteristic to be considered for choosing the points to keep and the points to be deleted.