How to use uniq command in Linux.

How To Use Uniq Command In Linux.

Linux command-line tools offer a wide range of functionality to help users manage and manipulate text data efficiently. One such utility is the uniq command, which is used for identifying and removing duplicate lines from text files. In this article, we will explore the various aspects of the uniq command and provide numerous examples to illustrate its usage.


Basic Syntax

The basic syntax of the uniq command is as follows:

  • OPTION: Specifies various options to customize the behavior of the command.

  • INPUT: Specifies the input file to process. If not provided, the command reads from the standard input.

  • OUTPUT: Specifies the output file where the results are written. If not provided, the results are displayed on the standard output.


Common Options

The uniq command offers several options that allow you to control its behavior. Here are some of the most commonly used options:

  • -d, –repeated: Display only duplicate lines.

  • -u, –unique: Display only unique lines (non-duplicate lines).

  • -c, –count: Prefix lines with the count of occurrences.

  • -i, –ignore-case: Ignore differences in case (treat uppercase and lowercase characters as the same).

  • -f N, –skip-fields=N: Skip the first N fields before checking for duplicates (fields are separated by whitespace).

  • -s N, –skip-chars=N: Skip the first N characters before checking for duplicates.



Let’s explore some practical examples to understand how to use the uniq command effectively:


Example 1: Basic Usage

Suppose we have a file named fruits.txt with the following content:



Example 2: Displaying Unique Lines

To display only the unique lines (i.e., non-duplicate lines) from a file, use the -u option:

uniq -u fruits.txt

The output will be:



Example 3: Counting Duplicate Lines

To count the number of occurrences of each line in a file, use the -c option:

uniq -c fruits.txt

The output will be:

2 apple
2 banana
1 apple
1 orange
1 banana


Example 4: Ignoring Case

To perform a case-insensitive comparison when identifying duplicates, use the -i option:

uniq -i fruits.txt

The output will be:



Example 5: Skipping Fields

If your file has multiple fields separated by whitespace, you can use the -f option to skip a specified number of fields before checking for duplicates. Consider a file named data.txt with the following content:

John Doe 123
Jane Smith 456
John Doe 789

To identify unique lines based on the first field, you can use:

uniq -f 1 data.txt

The output will be:

John Doe 123
Jane Smith 456



The uniq command is a valuable tool for identifying and manipulating duplicate and unique lines in text files. With its various options, you can customize its behavior to suit your specific needs. Whether you’re dealing with lists of data, log files, or any text-based content, uniq can help you efficiently manage and process your data on the Linux command line.