Posted on October 10, 2023 by nexonhost
How To Use Uniq Command In Linux.
Linux command-line tools offer a wide range of functionality to help users manage and manipulate text data efficiently. One such utility is the uniq command, which is used for identifying and removing duplicate lines from text files. In this article, we will explore the various aspects of the uniq command and provide numerous examples to illustrate its usage.
Basic Syntax
The basic syntax of the uniq command is as follows:
uniq [OPTION]... [INPUT [OUTPUT]]
OPTION: Specifies various options to customize the behavior of the command.
INPUT: Specifies the input file to process. If not provided, the command reads from the standard input.
OUTPUT: Specifies the output file where the results are written. If not provided, the results are displayed on the standard output.
Common Options
The uniq command offers several options that allow you to control its behavior. Here are some of the most commonly used options:
-d, –repeated: Display only duplicate lines.
-u, –unique: Display only unique lines (non-duplicate lines).
-c, –count: Prefix lines with the count of occurrences.
-i, –ignore-case: Ignore differences in case (treat uppercase and lowercase characters as the same).
-f N, –skip-fields=N: Skip the first N fields before checking for duplicates (fields are separated by whitespace).
-s N, –skip-chars=N: Skip the first N characters before checking for duplicates.
Examples
Let’s explore some practical examples to understand how to use the uniq command effectively:
Example 1: Basic Usage
Suppose we have a file named fruits.txt with the following content:
apple banana apple orange banana
Example 2: Displaying Unique Lines
To display only the unique lines (i.e., non-duplicate lines) from a file, use the -u option:
uniq -u fruits.txt
The output will be:
orange
Example 3: Counting Duplicate Lines
To count the number of occurrences of each line in a file, use the -c option:
uniq -c fruits.txt
The output will be:
2 apple 2 banana 1 apple 1 orange 1 banana
Example 4: Ignoring Case
To perform a case-insensitive comparison when identifying duplicates, use the -i option:
uniq -i fruits.txt
The output will be:
apple orange banana
Example 5: Skipping Fields
If your file has multiple fields separated by whitespace, you can use the -f option to skip a specified number of fields before checking for duplicates. Consider a file named data.txt with the following content:
John Doe 123 Jane Smith 456 John Doe 789
To identify unique lines based on the first field, you can use:
uniq -f 1 data.txt
The output will be:
John Doe 123 Jane Smith 456
Conclusion
The uniq command is a valuable tool for identifying and manipulating duplicate and unique lines in text files. With its various options, you can customize its behavior to suit your specific needs. Whether you’re dealing with lists of data, log files, or any text-based content, uniq can help you efficiently manage and process your data on the Linux command line.