Select Unique Lines in Bash on Linux

If you have a file with repeated lines, and you want to remove duplicate lines, you can use the uniq command. The description of this utility, from its man page is simply: “report or omit repeated lines.” Consider the following example:

trevor@MyUbuntuVM:~$ echo "this
is
is
great" | uniq

The output of the above command looks like:

this
is
great

You can also get a count of duplicate lines by adding on the -c parameter to uniq. The following command …

trevor@MyUbuntuVM:~$ echo "this
is
is
great" | uniq -c

… emits this output.

      1 this
      2 is
      1 great