Linux Basic — sed Regex in Scripts
sed Regex in Scripts
sed is the stream-editor for one-liners and config rewrites. With -E it speaks ERE — the same dialect you’ve been using in grep -E and Bash =~.
sed -E 's/(error|fatal)/[!] \1/Ig' app.log
The Workhorse: s/pattern/replacement/flags
# replace first match on each line
sed 's/foo/bar/'
# replace ALL matches on each line
sed 's/foo/bar/g'
# case-insensitive (GNU sed)
sed 's/foo/bar/gi'
# only on line 5
sed '5 s/foo/bar/'
# only on lines matching a regex
sed '/^DEBUG/ s/foo/bar/g'
The delimiter doesn’t have to be /. When the pattern has slashes (paths!), pick a different one:
sed 's|/old/path|/new/path|g' config
sed 's#http://#https://#g' url-list
Capture Groups & Back-References
With -E, groups are ( ) and references are \1, \2, …:
# swap key=value into value=key
echo "name=Alice" | sed -E 's/^([^=]+)=(.*)$/\2=\1/'
# Alice=name
# normalise duplicate whitespace
sed -E 's/[[:space:]]+/ /g'
# rename .txt to .md
ls *.txt | sed -E 's/(.*)\.txt$/mv & \1.md/' | sh
& in the replacement = the whole match.
Address Ranges
sed runs each command against the addresses that precede it:
| Address | Meaning |
|---|---|
5 |
line 5 |
1,5 |
lines 1 to 5 |
/start/,/end/ |
from first match of start to next match of end |
$ |
last line |
/pattern/ |
every line matching pattern |
5,$ |
line 5 to end |
1~2 |
every odd line (GNU) |
# only inside a section
sed -E '/^\[database\]/,/^\[/ s/host=.*/host=db.prod/' app.ini
# delete blank lines
sed '/^[[:space:]]*$/d'
# print only matching lines (cheap grep)
sed -n '/^ERROR/p' app.log
-n suppresses default printing; p prints explicitly. d deletes the line.
In-Place Edits: sed -i
sed -i 's/foo/bar/g' file.txt # GNU sed — edit in place
sed -i.bak 's/foo/bar/g' file.txt # also keep file.txt.bak
sed -i '' 's/foo/bar/g' file.txt # BSD/macOS — empty arg required
The portability gap is real:
# portable: write a temp then mv
tmp=$(mktemp) && sed 's/foo/bar/g' file.txt > "$tmp" && mv "$tmp" file.txt
Always test the substitution without -i first.
Multi-Expression Programs
Chain commands with -e:
sed -e 's/foo/FOO/g' -e 's/bar/BAR/g' file
Or with a semicolon inside -E:
sed -E 's/foo/FOO/g; s/bar/BAR/g' file
For larger programs, use a script file:
sed -E -f rewrite.sed input
Common Idioms
| Task | One-liner |
|---|---|
| Strip leading whitespace | sed -E 's/^[[:space:]]+//' |
| Strip trailing whitespace | sed -E 's/[[:space:]]+$//' |
| Remove HTML tags (crude) | sed -E 's/<[^>]+>//g' |
| Comment out matching lines | sed -E '/^pattern/ s/^/# /' |
| Delete comments | sed -E '/^[[:space:]]*#/d' |
| Print lines N–M | sed -n '10,20p' |
| Print last line | sed -n '$p' |
Pitfalls
| Pitfall | Fix |
|---|---|
Mixing BRE \( \) with -E |
With -E use plain ( ) |
Forgot /g → only first match per line replaced |
Add /g |
Replacement contains / |
Change delimiter: s|x|y|g |
& in replacement printed literally |
Escape: \& |
sed -i not portable |
Use mktemp + mv |
Next → awk Regex Patterns
