Guide to Stream Redirections in Linux
In this tutorial, we’ll take a look at what streams are and how we work with them.
2. What are Streams
Some examples of input streams are the keyboard, text data stored in files and input from I/O devices. We can deliver the output to files, commands, windows, and other I/O devices.
3. Sample Data
$ echo -e "tables\nladders\nchairs" > streamdata1 $ echo -e "planes\ntrains\nautomobiles" > streamdata2
We’ve created the streamdata1 and streamdata2 files and populated them with some data:
$ cat streamdata1 tables ladders chairs $ cat streamdata2 planes trains automobiles
4. Redirecting Input
STDIN refers to the standard input stream; usually, input from the keyboard. STDIN has a filehandle of 0 (zero).
The < operator is used to pass input from a file or I/O device data to a command.
Let’s say that we want to count the number of lines in a file without passing the file name through as a parameter to the wc command. We can do it by redirecting STDIN:
$ wc -l < streamdata1
5. Redirecting Output
The > operator is used to direct output from a command to a source; usually a file. By default, a single > will:
Create a new file if it doesn’t already exist
Overwrite any pre-existing data in the file if the file already exists
Let’s see how we can use the cat command to emit the contents of streamdata1 and send the output to a new file:
$ cat streamdata1 > combinedstreamdata
When we print the contents of the combinestreamdata file it should look exactly like streamdata1:
$ cat combinedstreamdata tables ladders chairs
6. Appending to an Existing File
Let’s see how we can add the contents of streamdata2 to the combinedstreamdata file:
$ cat streamdata2 >> combinedstreamdata
The combinedstreamdata file now contains the contents of both our streamdata1 and streamdata2 files:
$ cat combinedstreamdata tables ladders chairs planes trains automobiles
7. Piping Output to Input
With the | (pipe) operator, we can chain many commands together by passing the output from one through as input to the next.
Let’s try using the | operator to stream the output from the cat command to the input stream of the wc command:
$ cat streamdata2 | wc -l
8. Redirecting Error
Let’s attempt to execute a script that does not exist and pipe its imaginary output to a log file:
$ exec doesnotexist.sh > out.log
We get this error message:
exec: doesnotexist.sh: not found
Let’s take a look and see what our command wrote to out.log:
$ cat out.log
Hmm, our log file is empty. But, we did see an error message – we might want to log that, too.
Let’s see how we can redirect STDOUT and STDERR to capture the output and error output:
$ exec doesnotexist.sh >out.log 2>err.log
In the above statement, we direct standard output to out.log and standard error to err.log.
More specifically, we referenced the standard error stream using its filehandle – 2>err.log. We didn’t have to specify the filehandle for standard output because its filehandle is the default.
Let’s check to see what the command wrote to err.log:
$ cat err.log exec: doesnotexist.sh: not found
Our error message was successfully redirected to our error log file.
In this example, we handled both output streams (STDOUT, STDERR) and directed each to its own log file.
9. Merging Output and Error
The >& operator is a special operator that is used for *directing output from one stream to another*. We can use it to pipe the output of STDERR to STDOUT:
Let’s see how we can leverage file handles and >& to give us a single log file that contains the output of STDOUT and STDERR:
$ cat streamdata1 streamdata2 streamdata3 2>&1>out.log
$ cat out.log tables ladders chairs planes trains automobiles cat: streamdata3: No such file or directory
As expected, the contents of streamdata1 and streamdata2 are found in out.log along with the anticipated error message since streamdata3 does, in fact, not exist.