uniq - it is used to report or filter out repeated lines in a file.

$ cat demo.txt redhat debian ubuntu ubuntu centos fedora fedora fedora fedora

To report or filter out for lines that are adjacent and repeated $ uniq demo.txt

To only print unique lines $ uniq -u demo.txt

To prefix lines by the number of occurrences $ uniq -c demo.txt

To only print duplicate lines, one for each group $ uniq -d demo.txt

To print all duplicate lines $ uniq -D demo.txt

To print all duplicate lines but allow separating groups with an empty line $ uniq --all-repeated=prepend demo.txt

To ignore differences in case when comparing $ uniq -i demo.txt

To skip comparing the first X number of characters in each line $ uniq -s 3 demo.txt

To compare only the first X number of characters $ uniq -w 4 demo.txt

regards, T.Dhanasekar