Question Removing duplicated entries in a very large single csv file

Jan 9, 2019
3
0
6
#1
I have a MASSIVE csv file. I generate one of these a week from 12mb up 50mb in size. Some lines do get duplicated. I have to go in and search the duplicates out. I am looking for some way to automate this task. I have looked a while but have yet to find anything.
Has anyone here heard/seen anything that does this?
 


ASK THE COMMUNITY

TRENDING THREADS