topic Re: Find Duplicate numbers in list in Python Questions

Find Duplicate numbers in list

LarryAdgate — Tue, 22 Oct 2013 15:47:55 GMT

If anyone can help me with this I really would appricate it. I'm a novice

My task is to create a script a that examines a list of numbers (for example, 2, 8, 64, 16, 32 4, 16, 😎 to determine whether it contains duplicates. The script should print a meaningful result, such as "The list provided contains duplicate values" or "The list provided does not contain duplicate values."

Thanks,
Larry

Re: Find Duplicate numbers in list

JasonScheirer — Sat, 11 Dec 2021 15:19:12 GMT

import collections

def has_duplicates(list_of_values):
    value_dict = collections.defaultdict(int)
    for item in list_of_values:
        value_dict[item] += 1
    return any(val > 1 for val in value_dict.itervalues())

if has_duplicates(my_list):
    print "The list provided contains duplicate values"
else:
    print "The list provided does not contain duplicate values."

Re: Find Duplicate numbers in list

Anonymous User — Sat, 11 Dec 2021 15:19:15 GMT

Something like this should do the trick:

def FindDuplicates(in_list):
    unique = set(in_list)
    for each in unique:
        count = in_list.count(each)
        if count > 1:
            print 'There are duplicates in this list'
            return True
    print 'There are no duplicates in this list'
    return False

if __name__ == '__main__':

    # test it
    a = [8, 64, 16, 32, 4, 24]
    b = [2,2,3,6,78,65,4,4,5]

    FindDuplicates(a)
    FindDuplicates(b)

This printed:
IDLE 2.6.5 ==== No Subprocess ====
>>>
There are no duplicates in this list
There are duplicates in this list
>>>

Ahh looks like Jason beat me to it...both should work just fine.

Re: Find Duplicate numbers in list

NeilAyres — Sat, 11 Dec 2021 15:19:18 GMT

Larry,

I often do this using a dictionary and a search cursor like:

Cur = arcpy.da.SearchCursor(InFc, ["Field"]) # where Field is the column to count duplicates
FDict = {}
for r in Cur:
    f = r[0]
    if f in FDict:
        FDict += 1
    else:
        FDict = 1

del Cur

The dictionary FDict will contain the keys of all the possible values of Field and a count of how many there are.
Cheers,
Neil

Re: Find Duplicate numbers in list

DouglasSands — Sat, 11 Dec 2021 15:19:20 GMT

Something like this should do the trick:

def FindDuplicates(in_list):
    unique = set(in_list)
    for each in unique:
        count = in_list.count(each)
        if count > 1:
            print 'There are duplicates in this list'
            return True
    print 'There are no duplicates in this list'
    return False

Find duplicates can be much simpler if you already have the list:

def FindDuplicates(in_list):
    unique = set(in_list)
    if len(unique) == len(in_list):
        print('There are no duplicates in this list')
        return False
    else:
        print('There are duplicates in this list')
        return True

This way, you don't have to loop over the list.

Re: Find Duplicate numbers in list

Anonymous User — Tue, 22 Oct 2013 20:25:58 GMT

Touché...Definitely over-analyzed that one. Easier is better.

Re: Find Duplicate numbers in list

DanPatterson_Retired — Sat, 11 Dec 2021 15:19:23 GMT

try "sets"

>>> a = ( 2, 8, 64, 16, 32, 4, 16, 8)
>>> b = set(a)
>>> if len(b) < len(a):
...  print "contains duplicates"
... else:
...  print "no duplicates found:"
>>> 
contains duplicates

ooops missed Doug's example

Re: Find Duplicate numbers in list

LarryAdgate — Fri, 25 Oct 2013 13:52:27 GMT

If anyone can help me with this I really would appricate it. I'm a novice

My task is to create a script a that examines a list of numbers (for example, 2, 8, 64, 16, 32 4, 16, 😎 to determine whether it contains duplicates. The script should print a meaningful result, such as "The list provided contains duplicate values" or "The list provided does not contain duplicate values."

Thanks,
Larry

To those that responded to my cries for help, a world of thanks. With this much help available, learning python is going to be much easer-Larry

Re: Find Duplicate numbers in list

MikeMacRae — Sat, 11 Dec 2021 15:19:26 GMT

Use the 'set' and 'len'. 'len' with give you a count of the values in your list. 'set' will filter out any duplicates in your list and then use len to count the list again and compare it to your original count:

list = ( 2, 8, 64, 16, 32, 4, 16, 8)

if len(list) <> len(set)list):
     print "The list provided contains duplicate values"

Re: Find Duplicate numbers in list

RebeccaStrauch__GISP — Sat, 11 Dec 2021 15:19:28 GMT

I didn't know about "set" before, so nice post. I saw it because this popped up in my geonet email, but notices a syntax error above, and then expanded for a non-dupe list. fwiw.

list = ( 2, 8, 64, 16, 32, 4, 16, 8)

if len(list) <> len(set(list)):
     print("the list provided contains duplicate values")
else:
     print("no dupes")
     
list = (1, 2, 3, 4, 5)

if len(list) <> len(set(list)):
     print("the list provided contains duplicate values")
else:
     print("no dupes")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

edit: ....oops...I missed Doug's and Dan's examples. shows you what happens when you go to the bottom of the geonet comments first.