Fundamentals of UNIX
Fundamentals of UNIX
of
UNIX
Index
Topic UNIX Operating System Unix Commands Files & Directories Unix Utilities Process Shell Programming Reference Lab Page # 3 25 34 61 99 111 162 163
2
Chapter 1
UNIX Operating System
Objectives
In this session, you learn about: The functions of OS The history of Unix The features of UNIX The Unix architecture Process management CPU scheduling Memory management File management
Operating System
Operating system interacts with user in two ways 1.Operating system commands Enables user to interact directly with the operating system.
History of UNIX
Ken Thompson of AT&T Bell Laboratories designed
UNIX in late 1960s
What is Linux?
1.An open-source UNIX like operating system 1.Initially created by Linus Torvalds for PC architecture 1.Ports exist for Alpha and Sparc processors 1.Developer community world-wide contribute 12 to its enhancement and growth
Features of UNIX
Multi-user, multitasking, timesharing Portability Modularity File structure Security Strong networking support & advanced graphics
13
Layered Architecture
...
cp comp shell as ld vi ed
banner ls
kernel
hardware
sort
14
UNIX System Architecture Unix system follows a layered approach. It has four layers The innermost layer is the hardware layer In the second layer, the kernel is placed The utilities and other application programs form the third layer
15
Kernel Kernel is that part of the OS which directly makes interface with the hardware system. Actions: 1.Provides mechanism for creating and deleting processes 2.Provides processor scheduling, memory, and I/O management 3.Provides inter-process communication.
16
The Shell A utility program that comes with the UNIX system. Features of Shell are: 1.Interactive Processing 2.Background Processing 3.I/O Redirection 4.Pipes 5.Shell Scripts 6.Shell Variables 7.Programming Constructs
17
Process Management
A process is a program in execution Several processes can be executed simultaneously in a UNIX system. A process is generally created using the fork( ) system call.
The process that invokes the fork( ) system call is the parent process, and the newly created process is called the child process.
18
CPU Scheduling Unix uses round-robin scheduling to support its multi-user and time-sharing feature.
Round-robin fashion of scheduling is considered to be the oldest, simplest and widely used algorithm.
Every process is given a time slice (10100 millisec.)
19
File Management UNIX uses a hierarchical file system with / as its root. Every non-leaf node of the tree is called as a directory file. Every leaf node can either be a file, or an empty directory
20
File System
dev
bin
tmp home
etc
var spool
lib bin
usr src
sh
console lp0 ls user1
inittab passwd
user2
21
File System
File system is the structure in which files are stored on disk File in UNIX is sequence of bytes organized in the form of blocks The size of each block is 512 bytes (depends on architecture) Block size can be decided while creating the file system structure 22
Address of the addr block Address of the addr block Address of the addr block
23
Berkeley, BSD
Sun Microsystems, Sys 5/BSD Digital Equipment Corporation, BSD Digital Equipment Corporation, BSD/sys 5 Hewlett-Packard, Sys 5 IBM, Sys 5 / BSD Silicon Graphics, Sys 5
25
Working With UNIX User logs in with a valid user ID User logs out to terminate the login session
26
Summary
In this session, you learned about The functions of OS The History of Unix The features of UNIX The Unix Architecture Process management CPU Scheduling Memory management File management
27
Chapter 2
UNIX Commands
28
Objectives
In this session, you will learn to: 1.Use the basic Unix commands 1.pwd 2.date 3.who 4.ls 5.man Use man pages
29
Simple Commands
pwd Displays the current working directory. 1.date 1.Displays the current date and time
30
Simple Commands
who Displays the names of all the users who have currently logged in 1.who am i 1.Displays the name of the current user.
31
:ls [options] [file.] -l list in long format -a list all files including those beginning with a dot -i list inode no of file in first column -s reports disk blocks occupied by file -R recursively list all sub directories -F mark type of each file -C display files in columns
32
Meta Characters
Meta Characters * ? [] Purpose Example $ ls l *.c file* $ ls l file? $ ls l file[abc] $ cat file1; cat file2 $ cat abc | wc $ (echo ==== x.c ====; cat x.c) > out
Match with one or more characters or none Match with any single character Match with any single character within the brackets ; Command separator | Pipe two commands () Group commands Useful when the output of thecommand group has to be redirected `command` Execute the command enclosed within back quotes. Useful when the output of a command into a variable in a shell script string string Quote all characters with no substitution (ex. no special meaning for $ ) Quote all characters with substitution. The characters $,\ (back slash) and back quote have special meaning.
count=`expr $count + 1` assuming count has value3, this increments the value of count echo expr $count + 1 displays expr $count + 1 echo expr $count + 1 displays expr 3 + 1 assuming the variable count has value 3
33
File type
34
Summary
In this session, you have learned to 1.use the basic Unix commands like 1.pwd 2.date 3.who 4.ls 5.man use man pages
36
Chapter 3
Files & Directories
37
Objectives
In this session, you will learn to: 1.set file permissions using the chmod command 2.use directory-related commands namely mkdir, rmdir, cd commands 3.use file-related commands namely cp, mv, rm commands 4.access advanced file permissions using commands umask, suid, sgid, linking files, stickybit 5.create and edit files using the vi editor
38
40
File Access Permissions No execute permission does not allow the user to:
display the contents of a directory file from within the directory change to the directory display a file in the directory
41
chmod u+x file_name Syntax: chmod <category> <operation> <permission> <filename(s)> or chmod <octal number> filename
Octal Number 4 - for read 2 - for write 1 - for execution $ chmod 744 xyz this sets read, write and execute permissions for owner, read permission for group and others
42
Directory Creation
Command Syntax mkdir [OPTION] DIRECTORY $ mkdir <path>/<directory> $ mkdir m <directory> $ mkdir p <directory1>/<directory2>/<directory3> Example: $ mkdir project1 This creates a directory project1 under current directory Note: Write and execute permissions are needed for the directory in which user wants to create a directory
43
Directory Removal
rmdir command removes directory Syntax rmdir <directory name> Example Removes project1 directory in the current directory rmdir project1 Remove multiple directories rmdir pos1 pos2 Remove the directory recursively rmdir p dir1/dir2/dir3 rmdir removes a directory if it is empty and is not the current directory
44
Command - cd
cd command is used to change the directory cd cd .. cd / - take to the home directory - takes to the parent directory - takes to the root directory
45
File-Related Commands
File Operation Copying a file Moving a file Removing a file Displaying a file and concatenating files Command cp mv rm cat
46
Command - cp
Used to copy files across directories Syntax cp <source file> <new file name> Example cp file1 file2
47
Command - cp
Options to cp 1. -p Copies the file and preserves the following attributes 1.owner id 2.group id 3.permissions 4.last modification time -r recursive copy; copy subdirectories under the directory if any 1. -i 1.interactive; prompts for confirmation before overwriting the target file, if it already exists
48
Command - mv
Used to move a file, or rename a file
Preserves the following details owner id group id permissions Last modification time
-f -i suppresses all prompting (forces overwriting of target) prompts before overwriting destination file
49
-i
prompts before deleting destination file will recursively remove the file from a directory (can be used to delete a directory along with the content ) 50
-r
Command - umask
umask value is used to set the default permission of a file and directory while creating umask command is used to see the default mask for the file permission Default umask value will be set in the system environment file like /etc/profile umask 022 will set a mask of 022 for the current session The file permission after setting this umask value will be 644 And the directory permission will be 755
52
Command - ln
Linking files
1.Hard Link (in the same filesystem) 1.$ ln /usr/bin/clear /usr/bin/cls
53
Vi Editor
vi is a visual editor used to create and edit text files. A screen-oriented text editor Included with most UNIX system distributions Command driven 1. Categories of commands include 1.Cursor movement 2.Editing commands 3.Search and replace commands 1. The vi editor is invoked by the following command: $ vi filename
57
Navigation
Backspace h j k Space l
58
Editing Commands
Text insertion / replacement
- inserts text to the left of the cursor - inserts text to the right of the cursor - inserts text at the beginning of the line - appends text at end of the line - opens line below - opens line above - replaces text from cursor to right - replaces a single character with any number of characters - replaces entire line
59
Editing Commands
Deletion
- to delete word
- to delete 2 word - to delete a line - to delete 2 lines
60
Editing Commands
Yanking Y 3Y p P - copy line into buffer - copy 3 lines into buffer - copy buffer below cursor - copy buffer above cursor
1. Save and quit 1. :w - to save 2.:w! - to name a file (:w! filename -> save as) 3.:x - save and quit 4.:q - cancel changes 5.:q! - cancel and quit
61
The following commands are applicable for vi editor in Linux /pat searches for the pattern pat and places cursor where pattern occurs. / repeat last search
62
Summary
In this session, you have learned how to 1.use file permissions using the chmod command 2.use directory-related commands namely mkdir, rmdir, cd commands 3.use file-related commands namely cp, mv, rm commands 4.access advanced file permissions using commands umask, suid, sgid, linking the files, stickybit 5.create and edit files using the vi editor
63
Chapter 4
UNIX Utilities
64
Objectives
In this session, you will learn how to: 1.use the Unix utilities such as 1.cat, echo, touch, more, file, wc, cmp, comm, find employ redirection operators use filters such as sort, grep, cut, head, tail, tr, and paste 1.use communication commands 1.telnet, ftp 2.use backup commands 1.zip/gzip and tar
65
cat
cat command takes the input from the keyboard, and sends the output to the monitor 1. We can redirect the input and output using the redirection operators $ cat > file1 Type the content here press <ctrl d> $ cat file1 Displays the content of the file $cat >> file1 This will append standard input to the content of file1
66
touch
Options: -a to change the access time -m to change the modification time -c no create if not exists
touch <file> will change the time of change of the file if the file exists
If the file does not exist, it will create a file of zero byte size.
67
68
more Allows user to view one page-full of information at a time. 1. file Used to display the type of the file 2. tty 1.Prints the terminals name
69
wc 1.A filter used to count the number of lines, words, and characters in a disk file or from the standard input. 2.-l - displays the number of lines 3.-w - displays the number of words 4.-c - displays the number of characters
70
find
1.
Lets user to search set of files and directories based on various criteria Syntax: find [path...] [expression] [path] where to search [expression] What type of file to search (specified with type option) What action to be applied (exec, print, etc.) Name of the files (specified as part of name option, enclosed in ) Example find . name *.c -print lists all files with .c extension from the current dir & its subdirectories
71
find
Finding files on the basis of file size 1. size [+ ]n[bc] n represents size in bytes (c) or blocks (b) of 512 bytes
lists all files that are more than lists all files that are less than
72
find
Finding files on the basis of access time (atime) or modified time (mtime) 1. atime [+-]n 2. mtime [+-]n n represents number of days ( actually 24 * n hours)
find . atime 2
find . atime +2 2 days ago find / mtime 2 days ago
find
Applying a command on files matching the criteria with exec and ok options 1. exec command {} \; command is command to be applied on the matching files (does not prompt user) find . -name *.dat exec ls l {} \; Long listing of all files with .dat extension in the current and its subdirectories
-ok command {} \;
Functionality is similar to exec, but prompts user before applying the command on the file matching the criteria.
74
Standard Files
Standard Input file Keyboard, file descriptor is 0 1.Standard Output file 1.Monitor, file descriptor is 1 Standard Error file Monitor, file descriptor is 2
75
> file
2> file
standard output
$ ls l > outfile
76
Filters
Filters are programs that takes its input from the standard input file, process it, and sends it to the standard output file. Commonly used filter commands 1.sort 2.grep 3.cut 4.head 5.tail 6.paste
77
sort
Sorts the contents of the given file based on the first char of each line. -n -r -t +num +num [-num] numeric sort (comparison made according to strings numeric value) reverse sort specify delimiter for fields specify sorting field numbers to specify the range
78
grep
grep -Global Regular Expression Printer is used for searching regular expressions Syntax 1.grep <options> <pattern> <filename(s)>
79
grep options
-c displays count of the number of occurrences
-n
-v -i
80
Patterns
* - matches 0 or more characters [^pqr] - Matches a single character which is not p ,q or r ^pqr -Matches pqr at the beginning of the line pqr$ -Matches pqr at the end of the line . - Matches any one character \ - ignores the special meaning. grep New\[abc\] filename
81
82
83
Filter command - tr
tr - translate filter used to translate a given set of characters Example : tr [a-z] [A-Z] < filename This converts standard input read from lower case to upper case.
84
Filter command - tr
Useful options for tr -s char
Squeeze multiple contiguous occurrences of the character into single char
-d char
Remove the character
85
Command Piping
Allows the output (only the standard output) of a command to be sent as input to another command.
ftp
Ftp is a file transfer program Provides necessary user interface to the standard File Transfer Protocol Allows users to transfer files to and from a remote host
Syntax $ ftp hostname
89
ftp - commands
get receive file from host mget receive multiple files from host put send file to host mput send multiple files from host
90
ftp - commands
ls cd lcd
list directory of host change directory on the host change directory on the local machine
To set transfer format ascii set to ascii mode binary set to binary mode
91
ftp - commands
92
Though archives are created on a tape, it is common to have them as disk files as well.
1.
93
94
Compression Utilities
gzip, Usage is very similar to compress and pack utilities in Unix: gzip [-vc] filename where -v displays the compression ratio. -c sends the compressed output to standard output and leaves the original file intact. gunzip gunzip can uncompress files originally compressed with compress.
95
In this session, you have learned to: 1.use the Unix Utilities like 1.cat, echo, touch, more, file, wc, find employ redirection operators use filters like sort, grep, cut, head, tail, tr, ftp 1.backup commands 1.tar and zip/gzip
96
Summary
Chapter 5
Process
97
Objectives
In this session, you will learn to: 1.Use process-related commands like 1.ps, kill, sleep Start a background process Use background and foreground-related commands like bg, fg, jobs , nice , nohup
98
Processes
Process - a program in execution When program is executed, a new process is created The process is alive till the execution of the program is complete Each process is identified by a number called pid
99
Login shell
As soon as the user logs in, a process is created which executes the login shell. Login shell is set for each login in /etc/passwd file.
100
ps
The ps command is used to display the characteristics of a process It fetches the pid, tty, time, and the command which has started the process. 1.-f lists the pid of the parent process also. 2.-u lists the processes of a given user 3.-a lists the processes of all the users 4.-e lists all the processes including the system 101 processes
Background Process
Enables the user to do more than one task at a time. If the command terminates with an ampersand (&), UNIX executes the command in the background Shell returns by displaying the process ID (PID) and job id of the process
102
jobs List the background process 1. fg % <job id> Runs a process in the foreground 2. bg %<job id> 1.Runs a process in the background
103
kill: Kills or terminates a process kill command send a signal to the process The default signal is 15 ( SIGTERM) kill -9 (SIGKILL) 1. Terminates the process abruptly
1.
104
In this session, you learned to: 1.Define a process 2.Use process-related commands like 1.ps, kill, sleep 3.Start a background process 4.Use background and foregroundrelated commands like 1.bg, fg, jobs
105
Summary
Chapter 6
UNIX Shell Programming
106
Objectives
In this session, you will learn to: 1.Use Shell variables
2.Write scripts to process positional parameters 3.Use test command 4.Use if construct 5.Use for loop 6.Use while loop 7.Use case construct 8.Define and use functions 9.Debug shell scripts
107
Flavours of the Unix shell Bourne shell C shell Korn shell csh ksh sh
Command processing
1. Displays the shell prompt and reads the command typed by the user. 1. Interprets the command and classifies it as an internal (built-in), or an external command. 1. If it is NOT a built-in command, searches for the command in the PATH-specified directories, and executes that command if it is found.
109
Shell Features
Parent shell process
fork
$ vi test.c
(bash)
110
1.Each shell, apart from the basic features, provides additional features such as:
History
some UNIX shells support command history facility to keeps track of commands that were executed facility to rerun previously executed commands bash shell supports the following !! !num recall the last command and execute it. execute nth command where n is the the num specified after !
112
alias
alias can be used to give new name to an existing command A better name that represents a single command or a sequence of commands to be executed, often with appropriate options alias is an internal command alias newname=command $ alias l=ls l The unalias command cancels previously defined alias.
113
Allows 1.Defining and referencing variables 2.Logic control structures such as if, for, while, case 3.Input and output
Shell Programming
114
Shell Variables
A variable is a name associated with a data value, and it offers a symbolic way to represent and manipulate data variables in the shell. They are classified as follows 1.user-defined variables 2.environment variables 3.predefined variables
value assigned to the variable can then be referred to by preceding the variable name with a $ sign.
115
Shell Variables
The shell provides the facility to define normal, and environment variables. A normal variable can be only used in the shell where it is defined. An environment variable can be used in the shell where it is defined, plus any child shells invoked from that shell.
116
117
Using Normal Variables Once variables are defined, one can use the echo command to display the value of each variable: $ echo $x $ echo $textline_1 $ echo $textline_2 $ echo $allusers $ echo $usercount
118
119
PATH 1.BASH_E
NV
120
Sample Shell Script #! /bin/bash # The above line has a special meaning. It must be the # first line of the script. It says that the commands in # this shell script should be executed by the bash # shell (/bin/bash). # --------------------------------------------------------121
Executing Shell Scripts There are two ways of executing a shell script: By passing the shell script name as an argument to the shell. For example: sh script1.sh If the shell script is assigned execute permission, it can be executed using its name. For example:
122
Built-in variables
Following are built-in variables supported 1.$0, $1$9 - positional arguments 2.$* - all arguments 3.$@ - all arguments 4.$? - exit status of previous command
executed
5.$$ 6.$!
process
----------------------script2.sh-------------------------echo Total parameters entered: $# echo First parameter is : $1 echo The parameters are: $* shift echo First parameter is : $1 -----------------------------------------------------------
Execute the above script using the script2.sh these are the parameters command.
125
For example,
$ ./script2.sh this string is a single parameter
126
127
Using the test Command To compare two integers using test following operators are available: -eq (equal to)
-ne (not equal to) -lt (less than) -le (less than or equal to) -gt (greater than) -ge (greater than or equal to)
129
To compare two strings using the test command, following operators are available:
To check a file type/access permissions using the test command, following operators are available:
-s file (file is not empty and exists) -f file (Ordinary file and exists) -d file (file is a directory and exists) -r file (file is readable and exists) -w file (file is write-able and exists) -x file (file is executable and exists)
133
-b file (file is a block device and exists) -c file (file is a character device and exists) -p file (file is a named pipe and exists) -g file (file has sticky bit set) -u file (file has setuid bit set) -t file_des (file descriptor is standard
output)
134
1.-a (logical AND operator) 2.-o (logical OR operator) 3.! (logical NOT operator)
135
Combining Conditions
The syntax for this is:
test expression_1 a expression _2, OR [ expression _1 a expression _2 ] test expression_1 o expression _2, OR [ expression_1 o expression_2 ] test ! expression _1 OR [ ! expression _1 ]
136
Example
# to check if the current directory is the same as your home directory curdir=`pwd` if test $curdir != $HOME then echo your home dir is not the same as your pesent working directory else echo $HOME is your current directory fi
138
139
Example:
Example
----------------------script.sh-------------------------#! /bin/sh usernames=`who | cut d f1` # for user in ${usernames} do echo $user done ----------------------------------------------------------141
Example Shell script checks for a blank/non blank string eg: read nam while [ -z $nam ] do read nam done echo the string is $nam
143
Example
Shell script to compute factorial of a given number
#!/bin/bash n=$1 if [ $n -eq 0 ]; then fact=0 else fact=1 while [ $n ne 0 ] do fact=`expr $fact \* $n` n=`expr $n 1` done fi echo $fact
144
read num1
read num2 echo enter 1 (for addition) or 2 (for subtraction)
read choice
Example
#!/bin/bash read number case $number 1) echo 1st break;; 2) echo 2nd break;; 3) echo 3rd break;; *) echo ${number}th break;; esac
147
Functions
Shell functions are a way to group commands for later execution using a single name for the group. They are executed just like a "regular" command.
Shell functions are executed in the current shell context; no new process is created to interpret them. Functions are declared using this syntax: 148
Functions
Shell functions can accept arguments Arguments are passed in the same way as given to commands Functions refer to arguments using $1, $2 etc., similar to the way shell scripts refer to command line arguments
149
Functions Function to convert standard input into upper case toupper() { tr a-z A-Z } This function can be used as
150
Programming in C vs Shell Comparison between A solution in C A shell solution written like a C program A proper shell/unix solution e.g: The problem is to count the no of lines in a file ( the file is called the_file)
152
A solution in C
#include <stdio.h> void main(void) { int lcount=0; FILE *infile; char line[500]; infile=fopen("the_file","r"); while(!feof(infile)) { fgets(line,500,infile); lcount ++; } printf("no of lines is %d\n",lcount); }
A shell solution
count=0 while read line do count=`expr $count + 1` done < the_file echo Number of lines is $count
Summary
In this session, you have learned to: 1.Use Shell variables
2.Write scripts to process positional parameters 3.Use test command 4.Use if construct 5.Use for loop 6.Use while loop 7.Use case construct 8.Define and use functions 154 9.Debug shell scripts
UNIX Bibliography
UNIX in a Nutshell for BSD 4.3: A Desktop Quick Reference For Berkeley (O'Reilly & Associates, Inc., 1990, ISBN 0-937175-20-X).
UNIX in a Nutshell: A Desktop Quick Reference for System V & Solaris 2.0 (O'Reilly & Associates, Inc., 1992, ISBN 0-56592-001-5).
The UNIX Programming Environment, Brian W. Kernighan & Rob Pike (Prentice Hall, 1984).
155
Thank You
156