Linux Basics for Big Data

72 / 87
Filters

Pipes are very powerful. They can be chained to solve many problems. So, most of the commands are built in a way that they can be used with pipes as long as they read from standard input (keyboard or pipe) and write to standard output (screen or pipe )

Such programs that can be used with pipes are generally called filters. Command examples of filters are:

wc - for counting the letters, words, and lines in the input

grep - displays only the lines from the input in which keyword (which is passed as argument) is found.

sort - sorts/orders the input lines lexically (alphabetically) by default but can be changed

more - displays the input in a page-wise manner

cat - displays the content of the file passed as an argument

sed - substitute a word with another word: sed 's/word/another_word/'

tr - translate character ranges. For example to lowercase characters in input you can use:

    tr 'A-Z' 'a-z'

uniq - Display the uniq input lines. The input lines need to be sorted. If you want to display frequency, use:

    uniq -c