topic Remove duplicates in attribute field in Python Questions

Remove duplicates in attribute field

CPoynter — Mon, 29 Apr 2024 07:23:12 GMT

I have a field with space delimited values eg.:

26199 42448 42448 42454 42483 42497

Tried using pandas etc to explode, strip, split, etc. to list values and recreate as unique values to remove duplicates. Nothing seems to be stringing together to get my desired result:

26199 42448 42454 42483 42497

I have several thousand rows and need to check for duplicates for all of them, and then update the row values.

Cheers,

Craig

Re: Remove duplicates in attribute field

DanPatterson — Mon, 29 Apr 2024 08:20:12 GMT

code block

import numpy as np def rem_dups(row, sep=' '): """Remove duplicates in a string in a row separated by `sep`""" s = row.split(sep) a = np.array(s) u = np.unique(a) return " ".join([i for i in u]) row = "26199 42448 42448 42454 42483 42497" rem_dups(row, sep=' ') '26199 42448 42454 42483 42497'

expression

rem_dups(!YourFieldName!, sep=' ')

Re: Remove duplicates in attribute field

BlakeTerhune — Mon, 29 Apr 2024 14:50:33 GMT

Does the order of these values matter after you remove duplicates?