• Guest, The rules for the P & N subforum have been updated to prohibit "ad hominem" or personal attacks against other posters. See the full details in the post "Politics and News Rules & Guidelines."

Question Removing duplicated entries in a very large single csv file

Chattanooga_Chuck

Junior Member
Jan 9, 2019
3
0
6
I have a MASSIVE csv file. I generate one of these a week from 12mb up 50mb in size. Some lines do get duplicated. I have to go in and search the duplicates out. I am looking for some way to automate this task. I have looked a while but have yet to find anything.
Has anyone here heard/seen anything that does this?
 

ASK THE COMMUNITY