Title card for 'Command pipelines in Bash: chaining commands and handling errors'

Command pipelines in Bash: chaining commands and handling errors

by

One of the most powerful features of the Bash shell is the ability to chain commands together into a pipeline using the pipe | character. The output of each command becomes the input to the next, letting you build concise, composable processing steps from small, focused utilities.

Used well, pipelines are elegant. Used carelessly, they can silently swallow errors and leave you debugging output that looks correct but was built on a failed command upstream. This post covers both sides: how to build useful pipelines, and how to make them behave safely.

Building a pipeline

Each command in a pipeline runs in sequence. The stdout of the first command is connected to the stdin of the second, and so on down the chain. Given a file of numbers:

$ cat numbers.txt
2250
2262
1
1
1
15379
15379
1
16112
16121

To find the values with the highest frequency:

$ sort -n numbers.txt | \
        uniq -c | \
        sort -rn | \
        head
141 2
 69 1685
 59 1
 53 2950
 11 1902
  4 2870
  4 2132
  3 9151
  3 4345
  3 1796

Walking through the steps: sort -n sorts numerically, uniq -c collapses duplicates and prefixes each with a count, sort -rn re-sorts in descending numeric order, and head returns the top 10 lines. Four focused utilities, no intermediate files, no custom script.

Error detection in a single command

For a single command, the exit code is available immediately in $?. Zero means success; anything else is an error.

$ echo "test" > /new-file.txt
-bash: /new-file.txt: Permission denied
$ echo $?
1

$ echo "test" > ~/new-file.txt
$ echo $?
0

The pipeline error problem

In a pipeline, $? reflects the exit code of the last command only. If a command earlier in the chain fails, that failure is silently masked as long as the final command succeeds.

$ cat /missing-file.txt | wc
cat: /missing-file.txt: No such file or directory
       0       0       0
$ echo $?
0

cat failed, but wc succeeded on its empty input, so $? reports zero. In a script, this silently continues past an error that should have stopped execution.

Fixing it with pipefail

Setting the pipefail option causes the pipeline to return the exit code of the rightmost command that failed, rather than always returning the exit code of the last command.

$ set -o pipefail
$ cat /missing-file.txt | wc
cat: /missing-file.txt: No such file or directory
       0       0       0
$ echo $?
1

The same command now correctly reports failure. Any script that chains commands should have pipefail set.

Per-command exit codes with PIPESTATUS

When you need to know which specific command in a pipeline failed, Bash provides the PIPESTATUS array. It holds the exit code of each command in the most recently executed pipeline, indexed from zero.

$ cat /missing-file.txt | wc | sort
cat: /missing-file.txt: No such file or directory
       0       0       0
$ echo "${PIPESTATUS[@]}"
1 0 0

PIPESTATUS[0] is the exit code of cat, PIPESTATUS[1] is wc, and PIPESTATUS[2] is sort. You can check individual stages:

if [[ ${PIPESTATUS[0]} -ne 0 ]]; then
  echo "Error: first command in pipeline failed" >&2
  exit 1
fi

Note that PIPESTATUS is overwritten by every new command, including the echo or if you use to inspect it. Capture it immediately after the pipeline if you need to check multiple stages.

some_cmd | other_cmd | final_cmd
pipe_status=("${PIPESTATUS[@]}")

[[ ${pipe_status[0]} -ne 0 ]] && echo "some_cmd failed"
[[ ${pipe_status[1]} -ne 0 ]] && echo "other_cmd failed"
[[ ${pipe_status[2]} -ne 0 ]] && echo "final_cmd failed"

Recommended defensive header

For any non-trivial script, set all three safety options at the top:

#!/usr/bin/env bash
set -euo pipefail
  • -e (errexit) — exit immediately if any command returns a non-zero exit code
  • -u (nounset) — treat unset variables as an error
  • -o pipefail — return the exit code of the first failing command in a pipeline

Together these three options catch the most common categories of silent failure in Bash scripts. The combination is widely considered the baseline for production-quality shell scripts.

Leave a Reply