Linux Basic — Iterating Through Lists Practice Exercises

Iterating Through Lists — Practice Exercises

Five exercises covering arrays, globs, while read, and find/xargs. Each one mirrors a real script you’d want in your toolkit.

---

Exercise 1 — Count Files by Extension

Goal: For all regular files under . (recursive), print a sorted count of files grouped by extension. Treat “no extension” as <none>.

Setup:

mkdir -p /tmp/ext-demo && cd /tmp/ext-demo
touch a.log b.log c.txt d.md e.py f.py g h.tar.gz
mkdir sub && touch sub/i.log sub/j.py sub/k.json

💡 Show Answer

shopt -s globstar nullglob

declare -A by_ext

for f in **/*; do
    [[ -f "$f" ]] || continue
    base=${f##*/}
    if [[ "$base" == *.* ]]; then
        ext=${base##*.}
    else
        ext="<none>"
    fi
    (( by_ext[$ext]++ ))
done

for ext in $(printf '%s\n' "${!by_ext[@]}" | sort); do
    printf '%-10s %d\n' "$ext" "${by_ext[$ext]}"
done
# <none>     1
# gz         1
# json       1
# log        3
# md         1
# py         3
# txt        1

Three idioms combined: shopt -s globstar nullglob, associative-array counter, and printf '%s\n' "${!arr[@]}" | sort for ordered key iteration.

Pure-find alternative:

find . -type f -printf '%f\n' 2>/dev/null \
    | awk -F. 'NF>1 { print $NF; next } { print "<none>" }' \
    | sort | uniq -c | sort -rn

(-printf is GNU-only.)

---

Exercise 2 — Counter That Survives a Pipe

Goal: Read /etc/passwd (colon-separated) and count how many users have UID >= 1000. Use a counter variable and demonstrate the subshell trap with the broken form, then show the fixed version.

💡 Show Answer

# ❌ Broken — pipe puts while in a subshell
count=0
cat /etc/passwd | while IFS=: read -r user _ uid _; do
    (( uid >= 1000 )) && (( count++ ))
done
echo "broken count: $count"
# broken count: 0

# ✅ Fixed — process substitution keeps loop in current shell
count=0
while IFS=: read -r user _ uid _; do
    (( uid >= 1000 )) && (( count++ ))
done < <(cat /etc/passwd)
echo "fixed count:  $count"
# fixed count: 42

# Even cleaner — direct file redirect
count=0
while IFS=: read -r user _ uid _; do
    (( uid >= 1000 )) && (( count++ ))
done < /etc/passwd
echo "direct count: $count"

The fix is either process substitution < <(cmd) or direct file redirect < file. Both avoid the subshell.

If you really want to keep the pipeline form, enable shopt -s lastpipe (which only works in non-interactive scripts):

shopt -s lastpipe
count=0
cat /etc/passwd | while IFS=: read -r user _ uid _; do
    (( uid >= 1000 )) && (( count++ ))
done
echo "$count"

---

Exercise 3 — Recursive Log Error Summary

Goal: Find all *.log files under a path (default /var/log) modified in the last day. For each, count occurrences of ERROR and WARNING. Print a per-file table with file, errors, warnings. Handle filenames with spaces.

💡 Show Answer

log_summary() {
    local root=${1:-/var/log}

    printf '%-50s %8s %8s\n' "FILE" "ERRORS" "WARNINGS"

    while IFS= read -r -d '' f; do
        # grep -c counts; || true so non-matches don't fail the pipeline under set -e
        local e w
        e=$(grep -c 'ERROR'   "$f" 2>/dev/null || true)
        w=$(grep -c 'WARNING' "$f" 2>/dev/null || true)
        (( e + w == 0 )) && continue
        printf '%-50s %8d %8d\n' "${f#$root/}" "$e" "$w"
    done < <(find "$root" -type f -name '*.log' -mtime -1 -print0 2>/dev/null)
}

log_summary /tmp/logs

Key pieces:

• find -print0 | while read -d '' is the NUL-safe iteration pattern
• grep -c counts matches in one fork per file (acceptable for log scanning)
• ${f#$root/} strips the common prefix for prettier output
• 2>/dev/null hides permission-denied noise

For thousands of files, switch to one big awk pass:

find "$root" -type f -name '*.log' -mtime -1 -print0 \
    | xargs -0 awk '
        /ERROR/   { e[FILENAME]++ }
        /WARNING/ { w[FILENAME]++ }
        END {
            for (f in e) printf "%s %d %d\n", f, e[f], w[f]+0
            for (f in w) if (!(f in e)) printf "%s 0 %d\n", f, w[f]
        }'

---

Exercise 4 — Parallel JPEG Resizer

Goal: Find all .jpg/.jpeg files under a tree and resize them to max 1024 px wide, using up to 4 workers in parallel. Don’t fail if no files match.

(Use mogrify from ImageMagick or substitute echo "would resize: $0" if mogrify isn’t installed.)

💡 Show Answer

# With xargs -P for parallelism
find . -type f \( -iname '*.jpg' -o -iname '*.jpeg' \) -print0 \
    | xargs -0 -n1 -P4 -r mogrify -resize '1024x>'

# -n1   one file per invocation
# -P4   up to 4 parallel workers
# -r    don't run if input is empty (GNU; BSD ignores but defaults this way)

# Dry-run variant (no mogrify)
find . -type f \( -iname '*.jpg' -o -iname '*.jpeg' \) -print0 \
    | xargs -0 -n1 -P4 -I{} sh -c 'echo "would resize: {}"'

Notes:

• -iname matches case-insensitively
• \( … -o … \) for OR — escape parens because the shell uses them
• 1024x> is ImageMagick syntax: “only shrink, don’t upscale”
• -P0 (GNU only) means “unlimited parallelism” — use cautiously

Pure-Bash version (no xargs):

shopt -s globstar nullglob nocaseglob

for f in **/*.@(jpg|jpeg); do
    mogrify -resize '1024x>' "$f" &
    # Limit concurrency — wait whenever 4 are running
    (( $(jobs -rp | wc -l) >= 4 )) && wait -n
done
wait

wait -n (Bash 4.3+) waits for the next background job to finish.

---

Exercise 5 — Portable Rename (Add a Suffix)

Goal: Write a add_suffix function that takes a suffix and one or more paths, and renames each foo.txt → foo<suffix>.txt. Handle filenames with spaces. If the target already exists, skip and print a warning.

Test by renaming *.log to add .bak before the extension.

💡 Show Answer

add_suffix() {
    local suffix=$1
    shift

    local src dir base name ext dst
    for src in "$@"; do
        [[ -e "$src" ]] || { echo "skip (missing): $src" >&2; continue; }

        dir=${src%/*}
        [[ "$dir" == "$src" ]] && dir="."     # no slash in input
        base=${src##*/}

        if [[ "$base" == *.* ]]; then
            ext=".${base##*.}"
            name=${base%.*}
        else
            ext=""
            name=$base
        fi

        dst="$dir/${name}${suffix}${ext}"

        if [[ -e "$dst" ]]; then
            echo "skip (exists): $dst" >&2
            continue
        fi

        mv -- "$src" "$dst"
        echo "$src → $dst"
    done
}

# Test
cd /tmp && touch "first one.log" "second.log" "no extension"
add_suffix .bak *.log "no extension"
# /tmp/first one.log → /tmp/first one.bak.log
# /tmp/second.log → /tmp/second.bak.log
# /tmp/no extension → /tmp/no extension.bak

Patterns used:

• ${src%/*} for dirname, ${src##*/} for basename (no fork)
• Splitting basename into name + ext with %.* and ##*.
• mv -- to prevent paths starting with - from being treated as flags
• [[ -e "$dst" ]] existence check before clobbering

---

What’s Next?

You can now iterate safely over any collection. The next chapter shifts focus from “what to loop over” to “what to do when something goes wrong” — exit codes, set -e, set -u, set -o pipefail, the trap command, and custom error functions.

Continue → Handling Errors