A Simple Tutorial on Linux – Part-2

linux tutorial pat 2

This post is the continuation of A Simple Tutorial on Linux – Part-1

In the Part-1 we learned the following topics on Linux.

  • Linux Operating System
  • Linux Files & Process
  • The Directory Structure
  • Permissions
  • Process

Keeping up the same pace, we will learn the following topics in the 2nd part of the Linux series.

  • Shell Scripting
  • Networking
  • Files & Directories
  • Chaining Unix Commands
  • Pipes
  • Filters
  • Word Count Exercise
  • Special System commands
  • Environment variables

Writing first shell script

A shell script is a file containing a list of commands. Let’s create a simple command that prints two words:

1. Open a text editor to create a file myfirstscript.sh:

2. Write the following into the editor:

Note: In Unix, the extension doesn’t dictate the program to be used while executing a script. It is the first line of the script that would dictate which program to use. In the example above, the program is “/bin/bash” which is a Unix shell.

1. Press Ctrl +x to save and then “y” to exit

2. Now, by default, it would not have executable permission. You can make it executable like this:

3. To run the script, use:

Program returns value and arguments

Every program returns a value to the operating system. It is also referred as the exit status. In Linux, a program returns 0 if successful. Otherwise a non-zero error number.

To check the return value of the previous command you can check the value of a special variable “?”

Attend the quiz on echo command

Networking

Sockets and ports

There are some programs that need to keep running all the time. Such programs are called services.

You can communicate with such programs even from another computer. These programs or processes bind themselves to a port. No two programs can listen on the same port.

For example, usually, A web server listens on 80 port and an email server listens on 25 port.

You can connect and talk to any service using nc command in the following way:

The way a person refers to himself as “I”, a computer refers itself as localhost or IP Address 127.0.0.1.

Files & Directories

Linking

If you want to give multiple names to a single file without copying the content, you can create the link.

1. Create a file:

2. Put the following text in it:

3. Save and Exit: Press CTRL+x and y

4. Check if the file has right data:

5. Create Link:

6. Check if it is there:

Hard Links and Softlink

The link that we created previously were known as hard link. If you delete the original file, the hard link would not be impacted.

1. Delete original content created in the previous section:

2. Check if it still has the content:

Symbolic links

A symbolic link points to a file. In case, the original file is deleted, the symbolic link would be pointing to the non-existing file.

You can create a symbolic link to a directory too.

1. Create a file:

2. Put the following text in it:

3. Save and Exit: Press Ctrl+x and y

4. Check if the file has right data:

5. Create Link:

6. Check if it is there:

You should see something like this

7. Check the contents of myslink using:

Chaining Unix Commands

Say, you want to execute a command only if another command is successful then you use &&.

And if you want a command to be executed if another command has failed, you use ||.

1. Create a file tea_ready:

2. The following command would print “Tea is ready” if the tea_ready file exists:

3. Delete the file tea_ready:

4. Check the command from #2 again:

5. The following will print “Tea is not ready”:

6. You can use brackets to avoid confusion:

Redirecting the output of a program

The output of a program can be saved to a file:

If “myfile” does not exist, it will be created. If it exists it will be overwritten.

Please follow these steps:

1. Run the following command to save the output of echo to a file:

2. Append world to it:

3. Check by typing:

Pipes – Introduction

If we want to send the output of one program to another, we can use pipe. A pipe is denoted by “|”.

“echo” command prints on the standard output whatever argument is passed to it.

“wc” command prints the number of characters, words, and lines out of whatever you type on standard input. Start “wc” command, type some text and press Ctrl+d to end the input:

Output-

Now, if we want to count the number of words, characters in the output of any program, we can pipe the output in the following way:

You can also save the results using redirection, in the following way:

Filters

Pipes are very powerful. They can be chained to solve many problems. So, most of the commands are built in a way that they can be used with pipes as long as they read from standard input (keyboard or pipe) and write to standard output (screen or pipe )

Such programs that can be used with pipes are generally called filters. Command examples of filters are:

wc – for counting the letters, words, and lines in the input

grep – displays only the lines from the input in which keyword (which is passed as argument) is found.

sort – sorts/orders the input lines lexically (alphabetically) by default but can be changed

more – displays the input in a page-wise manner

cat – displays the content of the file passed as an argument

sed – substitute a word with another word: sed ‘s/word/another_word/’

tr – translate character ranges. For example to lowercase characters in input you can use:

uniq – Display the uniq input lines. The input lines need to be sorted. If you want to display frequency, use:

Word Count Exercise

Step 1:

Check the Data using cat command. Since the file is big, you can use “more” to see pagewise

Step 2:

Replace space with newline such that every line in output contains the only single word:

For example, after replacing space with a new line in “I am ok” we should get:

The “/g” is an option of sed which makes replace all occurrences of space instead of only one.

Also, note this command has three programs connected by two pipes. The output of cat is going to sed and output of sed is going to more to see the pagewise.

Step 3:

We can sort the words using sort command in the following way

Note that we are using “more” command just to avoid screen-blindness (too much text scrolling).

Step 4:

We can now, count the words using uniq command 

Please save the result of the command to a file “word_count_results” in your home directory 

Improved Word Count Using Unix Commands

We can further improve the word frequency count by using more filters.

Improvement 1:

Translate to lower case using 

Improvement 2:

Remove non-alphanumeric characters using sed with the regular expression:

Improvement 3:

Replace all whitespace (multiple tabs and spaces):

Please note that since we are using regular expressions, we need to specify “-E”

Improvement 4:

Display most frequent at the top or display the results in reverse numeric sorting:

Improvement 5:

If the input file is big, the sort command might use too much of memory. So, you can force sort command to use less memory say 100 MB:

After all of these improvements, please save the results

Shell script for WordCount

A shell script is a file which contains the commands separated by a newline.

Let’s create a script to do the sorting of the data:

1. Create a file using nano text editor:

2. The first line of a script should have “#!” followed by the name of the program to execute the script with. Since we are creating a shell script, we want it to be executed using bash. So, the first line of the program should be:

3. Add the command in the editor:

4. Save the file by pressing Ctrl+x and “y”

5. Now, make this file executable:

6. Check if it is running:

Please recall “./” means current directory and ” | more” will show the result pagewise instead of causing too much scrolling.

Also, note that whatever is the input to the script is also passed to the programs executed in the script.

Permissions of Processes – setuid

In Unix, there is a file /etc/shadow that contains (one-way) encrypted passwords for every user. The user can not see the contents of the file. This is to defend the password cracking programs.

To change the password, a user needs to use the command: passwd. This passwd command first asks you for your old password and encrypts your input and compares it against the value in the file /etc/shadow. If it matches then it updates the password file /etc/shadow with new content.

When you are not allowed to view the /etc/shadow file, how can a program (passwd) do the same when run by you?

This is where the idea of a special permission called setuid come into the picture. A program file can be given setuid permission such that program becomes the user who owns the program file instead of the user who is running it.

Usually, a program runs with the same permissions as the user who is running it. The program can read or modify only the files which user is allowed to.

A process or program is something that is running. A process is started by the user. The question is what is a process allowed to do? Can a process do something that the user is not allowed to do? If so, then the user will create programs which can do anything. Therefore, the process is allowed to do only the things an owner of the process can do. And who is the owner? The user that started the process.

Setting setuid

You can make a program setuid by giving ‘s’ instead of ‘x’ permission. If you have written a script x.sh, an example would be:

1. Start creating a file with the name whoownsit_username.sh in /tmp directory:

Note: If your username is sandeep1234, the filename would be whoownsit_sandeep1234.sh

2. Put the following content in the editor in the previous step:

3. Save it by pressing Ctrl+x and press “y”

4. Give it setuid permission:

5. Check if you have given correct permission by

It should display something like this in permissions: rwsrwsr_x

Note: Setuid doesn’t work in shell scripts. Please see
How can I get setuid shell scripts to work?

Attend the quiz on permissions

Special System commands

sudo

This command makes a program run as root (the system administrator). This command is only allowed to be used by few users. Such users are called sudoers. You can modify the sudoers using

shutdown

This command makes the system shutdown. You can use the following command to shutdown the system immediately

The alternative command to shutdown immediately is: “halt”

Restart the system

To restart the system, you can use “reboot” command.

Please note that above commands (shutdown, halt, reboot) can only be run as root.

Where is my program?

To find where is your program located, you can use “which” command.

For example,

would print “/usr/bin/java” which means java is a command in the directory “/usr/bin”.

To further find out, you can use:

This would display:

It means that /usr/bin/java is actually a link to /etc/alternatives/java

Let us try:

It should display something like:

Further, to find out about the content of a file, you can use “file” command:

This should display something like

This means it is Linux binary.

Environment variables

Unix shell provides environment variables. To see the entire list of environment variables, use:

These environment variables can be used by the shell, programs or commands.

For example, the $PS1 variable is used by the shell to display the prompt. To change your prompt, you can try:

The environment variable $PATH is a list of directories separated by a colon. It is used by the shell to find the file corresponding to a command.

Setting Environment variables

You can set the environment variables simply by assignment: MYVAR=VAL

The following list of commands:

are equivalent to the single command: