Standard Streams (stdin, stdout, stderr), Pipes & Redirection Operators | Linux Tutorial

The Unix philosophy is built around a core architectural tenet: "Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."

At the foundation of this philosophy lies the concept of Standard I/O Streams. Understanding how the Linux kernel abstracts inputs and outputs through numeric file descriptors, and how the Bash shell redirects and chains these streams using operators and pipes, is essential for every systems administrator and automation engineer.

POSIX Standard File Descriptors: 0, 1, and 2

Whenever the Linux kernel launches a process, it automatically binds three fundamental I/O channels represented by numeric File Descriptors (FD):

File Descriptor	Stream Name	Default Source / Destination	Functional Purpose
`0`	stdin (Standard Input)	Keyboard / Terminal	Feeds input data into the process for execution.
`1`	stdout (Standard Output)	Display Terminal / Screen	Transmits successful execution results and primary data stream.
`2`	stderr (Standard Error)	Display Terminal / Screen	Transmits error messages, diagnostics, and operational warnings.

Because both stdout and stderr render to the terminal screen simultaneously by default, users often fail to realize they are entirely separate streams. Separating them via shell redirection allows administrators to log errors independently from standard output.

Redirection Operators Deep Dive

The shell intercepts redirection operators before launching the command, modifying the process's file descriptor table:

Operator	Description	Practical Example	Behavioral Effect
`>`	Redirect stdout (overwrite)	`uname -r > kernel.txt`	Overwrites destination file with process stdout.
`>>`	Redirect stdout (append)	`date >> uptime.log`	Appends process stdout to the end of the file.
`2>`	Redirect stderr (overwrite)	`find / -name "*.conf" 2> errors.txt`	Captures permission denied errors into `errors.txt` while displaying clean output on screen.
`2>>`	Redirect stderr (append)	`backup.sh 2>> backup_errors.log`	Appends error messages to an ongoing error log file.
`<`	Redirect stdin from file	`mysql -u root db_prod < dump.sql`	Supplies the contents of a file as input to stdin.

Combining Streams: The 2>&1 Idiom

To capture both standard output and error diagnostics into a single unified log file, administrators use stream duplication:

# POSIX Standard Method: Redirect stdout to file, then duplicate stderr to stdout
command > output.log 2>&1

# Modern Bash Shortcut (equivalent behavior)
command &> output.log

Order of Operations Trap: The sequence of redirection matters! Executing command 2>&1 > output.log does not combine streams into the file. The shell evaluates left-to-right: it first redirects stderr to the current location of stdout (the terminal screen), and then redirects stdout to the file. As a result, errors still spill onto the terminal!

Discarding Unwanted Output: /dev/null

The special character device /dev/null (the Unix bit-bucket) discards all data written to it. To run a command silently in cron jobs or background scripts:

# Discard all output and all error messages completely
command > /dev/null 2>&1

Pipelines: Inter-Process Communication in Memory

The pipe operator (|) connects the stdout (FD 1) of the preceding process directly to the stdin (FD 0) of the succeeding process:

# Count how many users have /bin/bash as their default login shell
grep '/bin/bash' /etc/passwd | wc -l

# Identify the top 5 memory-consuming processes
ps aux --sort=-%mem | head -n 6

Pipes execute concurrently in separate subshells. Data flows through a kernel memory buffer without touching physical storage, making pipelines exceptionally fast and scalable.

Pipes Only Capture stdout: By default, the pipe operator (|) redirects only standard output (FD 1). Errors generated on stderr (FD 2) will still bypass the pipe and print directly to the terminal screen. In modern Bash, use |& to pipe both stdout and stderr together into the next command.

Splitting Streams with tee

When running long automation scripts, administrators often want to observe command progress live on screen while simultaneously logging output to disk. The tee utility acts as a T-junction splitter:

# Print output to terminal and simultaneously write to build.log
./build_kernel.sh | tee build.log

# Append to log file instead of overwriting (-a)
./deploy.sh | tee -a deployment.log

Pipeline Exit Codes & PIPESTATUS

In standard Bash execution, the return code ($?) of a pipeline reflects only the exit code of the final command in the chain:

# Even if 'cat' fails because file does not exist, 'grep' returns 1 (no match)
cat non_existent_file.txt | grep "test"
echo $?
# Output: 1 (grep's exit code, hiding cat's failure!)

In production enterprise scripts, use pipefail to ensure that failures anywhere in a pipeline trigger error handling:

# Enable pipefail: returns the exit code of the rightmost command that failed
set -o pipefail

# In Bash, inspect exit codes of every segment in the pipeline:
cat non_existent_file.txt | grep "test"
echo "${PIPESTATUS[@]}"
# Example Output: 1 1 (both commands failed)