Streams

Every program on your computer has at least three standard streams associated with it. Streams are channels used to connect the input and the output of the program to the environment (usually the terminal).

An example of an input stream is the keyboard. By reading from the input stream, your program can read what you have typed on the keyboard.

The screen is an example of an output stream. By sending data to the output stream, a program can print something on the screen. For example, when you run the command ls ~, the list of files gets written to the output stream.

You may be familiar with console.log() in JavaScript? This function writes to the output stream.

There is also an error stream that is usually sent to the display as well but is used only to print error messages. It's not something you normally have to worry about, but it's good to know it exists.

Finally, the input stream is usually called stdin, the output stream is stdout and the error stream is called stderr.

Pipes & Redirection

You're probably thinking: why do we need to know about streams? Well, because we can redirect them and connect them to different programs, allowing us to perform some more sophisticated tasks.

For example, when you use the command ls -la ~, it will write a list of files to the terminal, but that list can potentially be quite long and take up a lot of screen space. We've also seen that the less command allows you to display some text on the screen temporarily.

Piping ( | ) is the best approach here. By piping the list of files into the less command, we can display it temporarily, preventing it from being printed into the terminal.

Give it a go:

ls -la ~ | less

In addition to piping stdout to another program, you can write it to a file using the redirect operator (>):

ls -la ~ > ~/Desktop/my-file.txt Which will write the list of files to a file on your Desktop called my-file.txt.

Give it a go and then read the file with one of the commands we already looked at.

If the file doesn't exist, > will create it. If the file exists, > will overwrite its contents. A way to prevent this is to append text to the end of the file instead, which can be done by replacing > with >>.

Wildcards

So far when we have been listing files, we've been looking at only a handful at a time, and it's been easy to find what we want. But what if we were working on a big project with thousands of files?

Here's how you would list just the txt files in your Desktop directory: ls ~/Desktop/*.txt

The asterisk acts as a wildcard, which matches any pattern.

What do you think the following commands do?

• ls ~/new*.txt
• ls ~/*
• ls ~/*b*

You can use wildcards with any command, not just ls - it's a built in feature of the terminal shell, not of any particular command.

On that note, a more powerful version of ls is find, which finds all the files in the specified directory, and also searches recursively in sub-directories for matching ones.

Say that you wanted to create a text file with all of the mp3 files in your ~/Music directory:

find ~/Music -name "*.mp3" > my-mp3s.txt Assuming you have mp3 files in your Music directory, then you should now have a nice list of all of them. But, you might be a streaming service kind-of person, because we're not in 1900 anymore. Hey, no one is judging!

Now that you can find a particular file, what about looking inside its contents?

We can also search for text inside a file or stream using the grep command.

grep takes 2 arguments: the text to search for and the file to search in. If you've managed to create the file mentioned:

grep beatles my-mp3s.txt

If nothing is returned, but because you don't listen to Beatles, instead of not actually having created the file, then we need to discuss what's going on with these music choices.

Using a pipe you can see all of the .txt files in your home directory that have readme in their name by doing the following:

find ~ -name "*.txt" | grep readme

If you want to use grep to search for text including spaces, you should wrap the text in "quote marks":

find ~ -name "*.txt" | grep " readme "

Note: You can also make your search case-insensitive using the -i switch.

Counting Things

You can get information on the length of a file using the wc - as in word count - command.

wc my-mp3s.txt This should print out 3 numbers, which are respectively the line count, the word count and the character count of the file.

You can see each of these individually using -l -w and -c.

In your home directory, how many .txt files containing the text "readme" in the title?

find ~ -name "*.txt" | grep readme | wc -l This should tell you.

Grab a nice cuppa and let's continue.