How to Read a File Line By Line in Bash

How To Read A File Line By Line In Bash

In this tutorial, we will discuss how to read a file line by line in Bash.

When writing Bash scripts, you will sometimes find yourself in situations where you need to read a file line by line. For example, you may have a text file containing data that should be processed by the script.

Reading a File Line By Line Syntax

The most general syntax for reading a file line-by-line is as follows:

while IFS= read -r line; do
  printf '%s\n' "$line"
done < input_file

or the equivalent single-line version:

while IFS= read -r line; do printf '%s\n' "$line"; done < input_file

How does it work?

The input file (input_file) is the name of the file redirected to the while loop. The read command processes the file line by line, assigning each line to the line variable. Once all lines are processed, the while loop terminates.

By default, the read command interprets the backslash as an escape character and removes all leading and trailing white spaces, which sometimes may cause unexpected behavior. To disable backslash escaping, we’re invoking the command with the -r option, and to disable the trimming, the internal field separator (IFS) is cleared.

We’re using [printf] instead of echo to make the code more portable and to avoid unwanted behaviors. For example, if the line contains values such as “-e”, it will be treated as an echo option.

Reading a File Line By Line Examples

Let’s take a look at the following example. Suppose we have a file named distros.txt containing a list of some of the most popular Linux distributions, and their package managers separated with comma (,):

distros.txt

Ubuntu,apt
Debian,apt
CentOS,yum
Arch Linux,pacman
Fedora,dnf

To read the file line by line, you would run the following code in your terminal:

while IFS= read -r line; do
  printf '%s\n' "$line"
done < distros.txt

The code reads the file by line, assigns each line to a variable, and prints it. Basically, you would see the same output as if you would display the file content using the cat command.

What if you want to print only the distributions that use apt? One way would be to use the if statement and check if the line contains the apt substring :

while IFS= read -r line; do
  if [[ "$line" == *"apt"* ]]; then
    printf '%s\n' "$line"
  fi
done < distros.txt

Ubuntu,apt
Debian,apt

When reading a file line by line, you can also pass more than one variable to the read command, which will split the line into fields based on IFS. The first field is assigned to the first variable, the second to the second variable, and so on. If there are more fields than variables, the leftover fields are assigned to the last variable.

In the following example, we set IFS to a comma (,) and pass two variables distro and pm to the read command. Everything from the beginning of the line until the first comma will be assigned to the first variable (distro), and the rest of the line will be assigned to the second variable (pm):

while IFS=, read -r distro pm; do
  printf '%s is the package manager for %s\n' "$pm" "$distro"
done < distros.txt

apt is the package manager for Ubuntu
apt is the package manager for Debian
yum is the package manager for CentOS
pacman is the package manager for Arch Linux
dnf is the package manager for Fedora

Alternative File Reading Methods

Using a Process Substitution

Process substitution is a feature that allows you to use the output from command as a file:

while IFS= read -r line; do
  printf '%s\n' "$line"
done < <(cat input_file )

Using a Here String

Here String is a variant of Here document . The string (cat input_file ) keeps the newlines:

while IFS= read -r line; do
  printf '%s\n' "$line"
done <<< $(cat input_file )

Using File descriptor

You can also provide the input to the loop using a file descriptor:

while IFS= read -r -u9 line; do
  printf '%s\n' "$line"
done 9< input_file

When working with file descriptors , use a number between 4 and 9 to avoid conflict with shell internal file descriptors.

Conclusion

In Bash, we can read a file line-by-line using a while loop and the read command.