Linux Basic — Looping Command Output

4 minute read

Looping Command Output

When you need to process the output of a command line by line, the right form is while IFS= read -r line; do … done < <(cmd). The naive form cmd | while read line works too — but bites you on variable scope.

The Canonical Form

while IFS= read -r line; do
    echo "[$line]"
done < /tmp/input.txt

Each piece matters:

Part Why
IFS= empty IFS preserves leading/trailing whitespace on the line
read -r -r disables backslash escapes (treats \n literally)
< file redirect stdin from a file

The combination “IFS= read -r” is the idiomatic safe read — memorise it.

Reading From a Command, Not a File

Two options. The naive way:

some_cmd | while IFS= read -r line; do
    echo "$line"
done

This works for printing, but has a famous trap: the while loop runs in a subshell, and variable assignments inside it don’t persist:

count=0
seq 1 5 | while IFS= read -r n; do
    (( count++ ))
done
echo "$count"
# 0                ← whaaaat

The while is on the right of a pipe, so Bash forks a subshell for it. count++ happens, but in the subshell. When the subshell ends, the change is lost.

The fix: process substitution.

count=0
while IFS= read -r n; do
    (( count++ ))
done < <(seq 1 5)
echo "$count"
# 5                ← works! no subshell

< <(cmd) makes the current shell read directly from a pipe to cmd. No subshell. All assignments persist.

(Bash 4.2+ has shopt -s lastpipe which makes the last pipeline element run in the current shell — but only in non-interactive scripts. Process substitution is more portable.)

Reading Multiple Fields per Line

read can split each line into multiple variables:

echo "alice 30 admin" | while read -r name age role; do
    echo "name=$name age=$age role=$role"
done
# name=alice age=30 role=admin

If there are more fields than variables, the last variable gets the rest:

echo "alice 30 admin extra fields" | while read -r name age rest; do
    echo "name=$name age=$age rest=[$rest]"
done
# name=alice age=30 rest=[admin extra fields]

Use a custom IFS to split on something other than whitespace:

while IFS=, read -r name age role; do
    echo "$name | $age | $role"
done < /tmp/users.csv

The IFS=, here only applies to that single read.

Reading Into an Array

echo "a:b:c:d" | { IFS=: read -ra parts; echo "${parts[1]}"; }
# b

Or all lines into an array — use mapfile (faster and clearer than a loop):

mapfile -t lines < /tmp/input.txt
echo "Got ${#lines[@]} lines"

# From a command
mapfile -t hosts < <(getent hosts | awk '{print $2}')

mapfile -t strips trailing \n. For NUL-delimited input use mapfile -d ''.

Don’t Forget -r

Without -r, read interprets backslashes as escape characters:

echo 'C:\Users\alice' | read -r path; echo "$path"
# C:\Users\alice

echo 'C:\Users\alice' | read    path; echo "$path"
# C:Usersalice            ← backslashes interpreted!

Always -r. There’s almost never a reason to omit it.

The Last-Line-No-Newline Problem

read returns true while it has a complete line ending in \n. If the last line lacks a newline, read reads the data into the variable but returns false:

printf 'first\nsecond' | while IFS= read -r line; do
    echo "[$line]"
done
# [first]                ← "second" is lost!

The fix — use the line and the EOF case:

printf 'first\nsecond' | while IFS= read -r line || [[ -n "$line" ]]; do
    echo "[$line]"
done
# [first]
# [second]

The || [[ -n "$line" ]] clause handles “read returned false but we got data.”

Looping a Fixed Number of Times

Don’t pipe seq to a loop for this — use a C-style or brace expansion:

# C-style — variable bounds OK
for ((i=0; i<10; i++)); do
    echo "$i"
done

# Brace expansion — literal bounds only
for i in {1..10}; do
    echo "$i"
done

for ((…)) is the right tool for counted iteration. while read is for streamed input.

A Real Example — Process Each Line of a Log

errors=0
warnings=0

while IFS= read -r line; do
    case "$line" in
        *ERROR*)   (( errors++ )) ;;
        *WARNING*) (( warnings++ )) ;;
    esac
done < /var/log/app.log

echo "errors=$errors warnings=$warnings"

< file (direct redirect) — no subshell, counts persist.

For “tail-and-process”:

tail -F /var/log/app.log | while IFS= read -r line; do
    [[ "$line" == *ERROR* ]] && echo "[$(date)] $line" >> errors.log
done

Here the subshell is fine — we’re not modifying any outer-scope variables.

Common Mistakes

❌ Wrong ✅ Right Issue
cmd \| while read line; do count++ while …; done < <(cmd) pipe puts loop in subshell
read line (no -r) read -r line backslashes consumed
while read line (no IFS=) while IFS= read -r line leading/trailing whitespace stripped
cat file \| while read … while …; done < file useless use of cat
Missing tail line without newline while … \|\| [[ -n "$line" ]] last line silently dropped

Next → Glob Patterns

Updated:

You may also enjoy