how to remove duplicate word in a file ?
what is the format of file and on what basis you want to remove duplicates?
sort -u file
this will be applied on the complete line and print the unique lines (suppress duplicates)
hey hi
You can try uniq filename which removes duplicate lines /words from the file.
Alternately you can try
# uniq filename >> new_filename
~Ultimatix