Unix/Linux for beginning users

The information in this document should be considered essential for any user of Linux or Unix.

Operating system or kernel 

The operating system is the basic software which allows programs (applications, utilities) to communicate with the computer hardware. Another name for the operating system is the kernel. Users do not communicate with the the kernel directly, they communicate with it by means of a shell or a graphic user interface (GUI). The two major tasks of the kernel (among others) are file management and scheduling of processes (i.e. running programs).. 

Linux and Unix

Linux is a free operating system and a suite of related software developed through a joint effort of programmers around the world. The behavior and features of Linux are intended to be very similar to those of various implementations of Unix. We need to explain that Unix is a trademark which is given to an operating system if it passes a number of benchmark tests. As such testing is not free (and in fact very costly), Linux was never submitted for it. Experts tell us that despite the lack of the official Unix testing, Linux is as good as Unix, and in some specialized areas even better. Majority of  Web servers run Linux.

Shell

A shell is a piece of software intermediate between the kernel and the user.  Whenever you see the prompt ending in % or $ you can type instructions for the shell. Every command which you type at the shell prompt must be followed by ENTER. Shell languages case-sensitive. Most commands are typed in lower-case letters. if you enter a command at the shell prompt, the shell will process the command and make appropriate requests to the kernel (via system calls).

``Shell'' is really not one specific piece of software but a category of software. In our lab, the default shell in new accounts is called bash. Except when noted, the explanations below will apply to other advanced shells as well.

Command line editing

While working in a shell you can use LEFT-ARROW, RIGHT-ARROW, and BACKSPACE to perform edits on the command line. 

Command history

Instead of retyping one of your previous shell commands, press UPARROW a certain number of times. If it went to far back, you can press DOWN-ARROW. 

Command and filename completion

Advanced shells will complete the command or a filename when you press TAB; e.g. instead of typing: 

more verylongfilename.txt 

you can type:

more ver 

and press TAB. Pressing the TAB twice will give you a list of possible ways to complete your command.
(in csh you need to press ESC instead). 

Get into a habit of using TAB and UPARROW -- this will save you a lot of typing and will prevent typos.

Directories and paths

Unlike Windows, Unix maintains a single directory tree which contains files residing on all available storage devices. (The system administrator adds to this tree filesystems from different devices using the mount command. The mounted filesystems are described in file /etc/fstab.)

• .. (dot dot) means the parent directory or "go up one level",
• . (dot) means "the current directory",
• ~ means the home directory of the current user,
• ~username means the home directory of user username

This directory tree starts with a directory "/", called a "root directory". 

The root directory / contains the following subdirectories (among others):

• bin -- the directory which contains executable files corresponding to utilities such as ls and cp.
• dev -- the directory which contains "device files" representing peripheral devices,
• etc -- the directory which contains files which define configuration of the unix system (user accounts, passwords, etc.)
• mnt -- the directory which may contain subdirectories corresponding to removable media (floppy disks, CD ROMs), if they are " mounted"
• tmp -- a directory where every user and process can write temporary files (the system will remove them after, say 24 hours.)
• usr -- a directory which contains the following subdirectories:
  • bin -- a directory which contains executable files corresponding to utilities such as emacs.
  • users -- a directory which which contains home directories of the users.

If a file or directory needs to be specified in a command, it can be represented by either of

• absolute path, i.e. a path from the root directory, e.g. /usr/local/custom/etc/cshrc
• relative path, i.e. a path from your present working directory, e.g. ../../local/custom/etc/cshrc

Wildcards 

• * matches any string of zero or more characters in a name of a file, except strings which begin with a . (dot)
• ? matches any single character

For instance:

• ls *.java     Lists the files with extension .java,
• rm *.o        Removes all object files.
• ls .??*        List files whose names begin with a dot and 
                      are at least 3 characters long 

Command format
Commands often involve options, and take arguments/parameters
For instance 

• ls                 without options
• ls -a             with options
• ls --all
• ls -l             
• ls -a -l
• ls -l -a
• ls -al
• ls -la
• ls --all -l
• ls *.java       with arguments
• ls -l *.java    with options and arguments

In most cases options represented by a single letter are specified after a "-" sign, and options represented by a word are specified after "--"; some options for X-clients are specified after a "+". In most cases the order of options does not matter and options represented by a single letter can be concatenated.
 
See examples of most basic Unix commands. 

Pipes
A pipe, represented by the mid "|" symbol, allows to direct the output of one command to the input of another:
COMMAND1 | COMMAND2
Examples:
ls -l | more          -- this will display files page by page.
ls -l | wc -l          -- this will count files
who | wc -l         -- this will count users currently logged onto the system.
set | sort | more -- list  variables and their values, sorted, page by page.

Input and output redirection
COMMAND < FILE     -- use contents of FILE as input for COMMAND
COMMAND > FILE         -- send output of COMMAND to FILE instead of screen.
COMMAND >> FILE        -- append output of COMMAND to FILE 

In bash
COMMAND >| FILE          -- same as > but override FILE if it exists (if noclobber is set)
COMMAND >>| FILE         -- same as >> but create FILE if it does not exist (if noclobber is set)

set -o noclobber          place this command in ~/.bashrc to prevent overriding.

(Note: in csh/tcsh >! and >>! are used instead of >| and >>|.)

COMMAND 2> ERRFILE    - redirect error stream ERRFILE

COMMAND > OUTFILE 2> ERRFILE     - redirect output and error streams to separate files.

COMMAND > FILE 2>&1   - redirect output and error streams to the same file.

Instead of redirecting to a file you can redirect to /dev/null and the contents of the stream will be lost.

Processes and jobs
A process is, intuitively, a run of a single program. For instance there is a program called emacs; notice that one can run several instances of emacs simultaneously, so one emacs program can be used to start several emacs processes. Only one process can be in the foreground; there can be many processes in the background. There can be also many suspended processes -- processes which have paused but can be restarted.

A process can spawn another process. The terminology of parent and child is used. If you typed "javac FILE.java" or "javac FILE.java &" at the shell prompt, the shell process is a parent and Java compiler process is a child. In the first case, the child executes in the foreground, the parent "waits" for the child to finish and then comes back to the foreground. (Waiting is a technical term.) In the second case, the parent remains in the foreground (and does not wait), and the child executes as a background process.

By a job we understand an execution of a command. For instance 
ls -l | sort > log.txt is a job which involves several processes. Jobs are handled by the shell.

The following commands are used for process and job control.

• ps        -- list current processes
• jobs     -- list current jobs
• ctrl-C   -- kill the current foreground process (the process can resist)
• kill  PROCESS       -- kill a process (the process can resist)
• kill -9  PROCESS   -- kill a process (the process cannot resist)
• ctrl-Z    -- suspend the current foreground process and bring its parent process to the foreground.
• fg  JOB       -- restart suspended job in the foreground
• bg JOB       -- restart suspended job in the background
• COMMAND &     -- run COMMAND in the background (this is very useful with X-clients)

File information
ls -l lists files in a long format:

drwxr-x--- 3 plazaja faculty 4096 Aug 27 2000  src/
-rw------- 1 plazaja faculty 2333 Aug 30 10:30 sshvars

• The first character describes the type of the file:
  •     -  means regular file
  •     d means directory
  •     l means symbolic link (or a shortcut in the Windows terminology.)
  •     b means block device (such as a disk)
  •     c means character device (such as a tape)
  •     s means socket (for inter-process communication)
  •     p means pipe (for inter-process communication)

  Notice that all these things are considered "files" in the terminology of Unix; keep this in mind while reading Unix manuals. They are called files, because, like regular files, they all support some version of read and write operations.

• The next nine characters represent  read, write and execute permissions for the user owner, group associated with the file and for others.
• The number such as 3 or 1 in the example above represents the number of, so called, hard links to the file.  
• The next item, plazaja, is the username of the owner of the file -- "user owner" (u) in the Unix terminology.
• The next item, such as faculty, is the name of the group which may have access to the file -- "group owner" (g) in the Unix terminology.
• The next item, such as 4096 represents the size of the file.
• The next three items represent the date or time when the file was created.
• The last item is the name of the file. 

Notice that our system is customized to append a slash / while displaying a directory name and the at @ symbol while displaying a symbolic link name. These symbols do not belong to the file name, they are just added while the file name is displayed to provide indication of the file type.

File permissions and privileges of users

Consider a file with permissions rwxr-x--x owned  by user plazaja, with group faculty associated with the file.

• characters at positions 1,2,3 represent read, write and execute permissions for the "user", denoted "u", i.e. the owner of the file, plazaja;
• characters at positions 4,5,6 represent read, write and execute permissions for the "group", denoted "g", associated with the file, faculty;
• characters at positions 7,8,9  represent read, write and execute permissions for the "others", denoted "o", i.e. all the users on the system;

a dash in any of these positions means that the corresponding type of access is denied. 

Permissions for the user are usually least restrictive and permissions for others are most restrictive. It is possible to define permissions which do not conform to this rule, but it has no practical importance.

For a directory, x permission means that one can change (cd) to that directory; r means that one can list the contents of the directory and w means that one can create and delete (!) files in that directory.

You can change permissions on a file you own by using chmod command:

• chmod u+x FILE 
  add x for the user.
• chmod u=rwx FILE 
  set permissions for the user to rwx.
• chmod 700 FILE
  this is an older syntax; digits 700 rewritten in binary are 111  000  000 and represent rwx------

Environment variables

These are variables similar to those you use in programming. They store strings and are used mainly to store some information about configuration of user account and software used by the user.  Every process has its own list of environment variables and can add/remove or change a values of an environment variable in its list. Environment variables known to a process will be inherited by the children of the process (and by all descendants of the process unless any of the descendants removes or modifies them). Although it is not required, by a custom, names of environment variables are in all uppercase letters.

• echo $VARIABLENAME   
  See the value of variable VARIABLENAME. Notice that the dollar sign is needed to access the value in the variable.
• env   
  See the list of all environment variables known to your bash
• VARIABLENAME=VALUE    
  export VARIABLENAME
  or
  export VARIABLENAME=VALUE
  Define/redefine an environment variable. 

$PATH
Environment variable $PATH contains a colon separated list of directories which will be searched when a Unix utility is invoked without a full path specification. Display the value of this variable by typing

    echo $PATH 

then look into the directories it contains. What directory does contain ls?

    PATH=DIRECTORY:$PATH      -- add DIRECTORY at the beginning of $PATH

    PATH=${PATH}:DIRECTORY:     -- add DIRECTORY at the end of $PATH
                                                                          notice the curly braces!

Warning: the current directory "." should be at the very end of $PATH or should be not listed at all; other arrangements make the system prone to hacking.