Can you get rid of duplicate values in a csv file using ModelBuilder?

Question

I am currently interning and have created a model.  Part of the model will populate a csv file in excel, but when I run it again it does not delete previous values or get rid of duplicates.  I would like to know how I can delete duplicate values in the csv file?

TedKowal · Answer

You could also run the script in the model.......  I would not recommend it.

curtvprice · Answer

One could also wrap such a Python workflow in a function embedded in the Calculate Value tool. Just sayin!

TedKowal · Answer

I will take a pop shot at the question..... I would run a python script that removes duplicate lines before you need to use the CVS .... Then your model would not have to worry about duplicate values. In the example below, 1.csv contains duplicate lines; the script create a 2.csv that removes the duplicates. You would use 2.csv in your model. inFile = open('1.csv','r') outFile = open('2.csv','w') listLines = [] for line in inFile: if line in listLines: continue else: outFile.write(line) listLines.append(line) outFile.close() inFile.close()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍