UNIX And SHELL Scripting

Rea
d data
file

displaying data from the file --------------------------------

cat a

cat<a

paste a

grep ‘.*’ a

cut –c1- a

while loop

xargs –L1 a

sed ‘’ a

sed –n ‘p’ a

sed –n ‘1,$p’ a

awk ‘1’ a
 UNIX C
omman
d Line S
tructur
e

- A command is a program that tells the Unix system to do something.

command [options] [arguments]

Where
- option indicates the functionality of the command and an
- argument indicates on what the command is to perform its action, usually a file or series of files.

- Commands are case sensitive.

 date C
omma
nd

date - Prints date and time

date - Fri sep 10 22:10:12 IST 2010

date ‘+%Y%m%d’- 20101115 (YYYYMMDD)

%a weekday, abbreviated Mon

%A weekday, full Monday
%d day of the month, two digits, zero filled 08
%e day of the month 8
%j day of year, zero filled 001–366
%u day of week from Monday to Sunday 1–7
%w day of week from Sunday to Saturday 0–6
 date C
omma
nd

%
week number, Sunday as first day of week 00–53
U
%
week number, Monday as first day of week 00–53
W
%
two-digit month number 01–12
m
%h month name, abbreviated Sep
%b month name, localised abbreviation Sep
%
locale's full month, variable length September
B
%C two century digits from year 00–99

%y two-digit year 00–99

%Y four-digit year 2014

%D mm/dd/yy 09/15/14
%F %Y-%m-%d 2014-09-15
 date C
omma
nd

%l hour (12 hour) 7

%I hour (12 hour), zero-filled 07
%k hour (24 hour) 7
%H hour (24 hour), zero-padded 07
%p locale's upper case AM or PM (blank in many locales) AM
%P locale's lower case am or pm am

%M two-digit minute number 39

%r hours, minutes, seconds (12-hour clock) 07:39:57 AM

%R hours, minutes (24 hour clock) 07:39
%T hours, minutes, seconds (24-hour clock) 07:39:57

%S two-digit second number 00–60

000000000–999
%N nanoseconds
999999
 date C
omma
nd

date --date="1 days ago"

date --date="yesterday"

date --date='10 month ago'

date --date='2 hour ago'

date --date='Second Friday'

$ date +"%Y%m%d" -d Sunday

$ date +"%Y%m%d" -d last-sunday
$ date +"%Y%m%d" -d last-week
$ date +"%Y%m%d" -d last-month
$ date +"%Y%m%d" -d last-year
$ date +"%Y%m%d" -d next-week
$ date +"%Y%m%d" -d next-month
$ date +"%Y%m%d" -d next-year
$ date +"%Y%m%d" -d "2 days ago"
$ date +"%Y%m%d" -d "2 months ago"
$ date +"%Y%m%d" -d "2 years ago"
 Dire
ctor
y Na
viga
tion
& co
ntro
l
mkdir

Creates a single directory or multiple directories.

mkdir [options] directories

Options:

-p create all directories leading up to the given directory that do not

exist already.

-m Providing permissions to directories while creation

$ mkdir -p -m 775 channels/discovery

-v Display each directory that mkdir creates. Most often used with -p.

Note: mkdir -p tmpdir/{trunk/sources/{includes,docs},branches,tags}

 Dire
ctor
y Na
viga
tion
& co
ntro
l
rmdir

removes single directory or multiple directories, if the directories are empty

rmdir [options] directories

Options:

-p Remove all the directories in sequence till it finds non empty directory.

-v,--verbose Display verbose information for every directory processed.

--ignore-fail-on-non-empty ignore any failure which occurs solely because a

directory is non-empty.
 File A
cces
s Per
miss
ions

chmod –c, --changes Like --verbose, gives verbose output only when a change is actually
made.
chmod –v, --verbose Verbose mode; output a diagnostic message for every file
processed.
chmod –reference=RFILE Set permissions to match those of file RFILE.
chmod –R <<directory>>Recursively descend through the directory structure and change the mode.
chmod -f Suppress error messages if command fails.
 File M
ainten
ance C
omma
nds

touch
✔ Creates an empty file if the file does not exists
✔ Updates the access and modification times of each FILE to the current system time.

✔ -a Change only the access time.

✔ -c, --no-create Do not create any files.
✔ -m Change only the modification time.
✔ -r,--reference=FILE Use this FILE's times instead of current time.
✔ -t TIMESTAMP Use [[CC]YY]MMDDhhmm[.ss] instead of current time.
 File Ma
intenan
ce Com
mands
..

cp
Copies file/files from one location to other.
cp [options] old_filename new_filename

cp –u,--update Copy only when the SOURCE file is newer than the destination
file or the destination file is missing.
mv
move/rename file/files from one location to other.
mv [options] old_filename new_filename
-u,--update
 Links

Hard Link
A hard link is a pointer to the file's i-node

Soft/symbolic/symlink
pointer to the original file
no physical data storage been created

ln <actual file> <destination file>

creates link
ln –s
creates symbolic link
 Links

• Soft Links:
Soft Links can be created across file systems.
Soft link has a different inode number than the original file.
On deleting the original file, soft link cannot be accessed.
Soft link needs extra memory to store the original file name as its data.
Source file need not exist for soft link creation.
Can be created on a file or on a directory.
Access to the file is slower due to the overhead to access file.
• Hard Links:
Hard links can be created only within the file system.
Hard links have the same inode number as the original file.
On deleting the original file, hard linked file can still be accessed.
Hard links do not need any extra data memory to save since it uses links
Source file should exist.
Can be created only on files, not on directories.
Access to the file is faster compared to soft link.
 Display
comma
nds

cat
Options:
-n Precede each line output with its line number.
-s Suppress messages if files do not exist.
-v Display control characters and nonprinting characters
-sn - Don’t Display Repeated Empty Output Lines
-e$ is printed at the end of each line. This option must be used with -v.

tac command is to reverse the file content.

 Disp
lay C
omm
ands .
.

Watch command
 Wc –
word
count

wc
spcount lines, words, and characters in a file. Character
count includes ace, tabs and newline character.

options:
-c Displays only number of characters
-l Displays only number of lines
-w Displays only number of words
-L output length of longest line
 File R
e direc
tion

0< input redirection

1> output redirect
2> Error re-direction
>> append output
| output to another command
<< Here document

File Descriptors
stdin 0 Standard input to the program
stdout 1 Standard output from the program
stderr 2 Standard error output from the program
 Wildcar
d Chara
cters

? Single character clm?.dat

* any number of characters rm clm*.dat
[] range clm[!345].dat
✔ [abc] match any of the enclosed characters
✔ [a-e] match any characters in the range a,b,c,d,e
✔ [!def] match any characters not one of the enclosed characters
 VI Ed
itor

3 Modes
Insert
Command and
Execute

Insert Execute

esc esc

i,I,a,A,o,O :wq, q!
R,R,s,S Command
 VI Ed
itor

-R Read-only
+ Position on last line of file in vi.
+n Position on n line of the file in vi.
 VI –
Mod Insert
e

Insert Mode:
This mode enables you to insert text into the file. Everything that's typed in this mode is interpreted as
input and finally it places into the file .

i insert from the current cursor position

I Insert from the first character of the line
a appending from the next character from the current position
A append to the end of the file
o start a new line below the current line
O start a new line above the current line
r replace a single character (Escape is not required)
R replace the characters till hit the escape button
s delete the current cursor position character and allow us to enter the characters till enters escape
S deletes the entire line and allow for entering the data.
 VI – C
omma
nd M
ode

Command Mode:
This mode enables you to perform administrative tasks such as saving files, executing commands, moving the
cursor, cutting and pasting lines or words, and finding and replacing. In this mode, whatever you type is interpreted
as a command.

0 or | Positions cursor at beg inning of line.

$ move cursor to end of the line
k up one space
j down one space
l right one space
h left one space
G last line of the file
nG nth line of the file
w move cursor to the next word
nwn words at a time
bto the previous word
 VI –
Com
m and
Mod
e ..
( Positions cursor to beginning of current sentence.
) Positions cursor to beginning of next sentence.
{ Move a paragraph back
} Move a paragraph forward
n| Moves to the column n in the current line
H Top line of the screen
nH Moves to nthline fromthe top of the screen
M Middle line of the screen
L last line of the screen
nL Moves to nthline fromthe bottomof the screen
x Deletes the character under the cursor location.
nx Delete n characters
X Deletes the character before the cursor location.
dd Delete line
ndd delete n lines
dw Deletes from the current cursor location to the next word..
ndw n words
 VI –
Com
man
d Mo
de ..

d^ Deletes from current cursor position to the beginning of the line.

d$ Deletes from current cursor position to the end of the line.
D same as d$
d) Delete to end of sentence
db Delete previous word
u Undo the last change to the file. Typing 'u' again will re-do the change
U Restore the current line to the state it was in before the cursor entered the line.
. Redo the last operation
ZZ equivalent to :wq of execute mode
J Join the current line with the next one. A count joins that many lines.
~ Switch the case of the character under the cursor.
Control commands:
ctrl + f Move forward one full screen
ctrl + b Move backward one full screen
ctrl + d Move forward 1/2 screen
ctrl + u Move backward 1/2 screen
ctrl + e Moves screen up one line
ctrl + y Moves screen down one line
 VI – E
xecu
te Mod
e

Execute Mode:
:q quit
:w save
:wq save and quit
:w filename save into different file
:n Move to nth line of the file – equal to nG of command mode
:f filename Renames current file to filename.
:e filename Opens another file with filename.
:cd dir Changes current working directory to dirname
:e # Use to toggle between two opened files.
:n In case you open multiple files using vi, use :n to go to next file in the series.
:N Use :N to go to previous file in the series.
:r file Reads file and inserts it after current line
:nr file Reads file and inserts it after line n.
:q! quit without save
:e! combination of q! + vi <<filename>>
 VI – E
xecu
te Mod
e
/ search pattern
n forward search
N backward search

:3,10d delete lines from 3 to 10

:3,10mo23 move lines from 3 to 10 after 23rd line
:3,10co23 copy lines from 3 to 10 after 23rd line

Substitution Syntax: :[address]s/old_text/new_text/

Address components:
. Current line
n Line number n
.+m Current line plus m lines
$ Last line
/string/ A line that contains "string"
% Entire file
[addr1],[addr2] Specifies a range
Ex:
:1,$s/s1/s2/g :.,.+10s/Banana/Kumquat
 VI – E
xecu
te Mod
e

Set commands:
:set ic Ignores case when searching
:set nu Displays lines with line numbers on the left side
:set list Show invisible characters

Running commands from VI:

From command mode type :! Command .
 echo
& re
ad c
omm
ands

echo
repeat, or echo, the argument you give it back to the standard output device
Options:
✔ -n suppress print <new-line>
✔ \c suppress print <new-line>
✔ \t tab
✔ \n new-line
✔ Echo * - print all the files and folders in a directory
read reads from the input
read only The shell provides a way to mark variables as read-only by using the read only command. After a
variable is marked read-
only, its value cannot be changed.
 echo
& re
ad c
omm
ands
• Colored output of echo command
• echo command can change the font style, background color of fonts and font colors. Escape sequence \033 can be
used to alter font properties. -e option has to be used in order to the escape sequence be in effect.
• [0m: Normal
[1m: Bold fonts
[2m: Font color changes to Purple
[4m: Underlined fonts
[7m: Invert foreground and background colors
[8m: Invisible fonts
[9m: Cross lined fonts
[30m: Font color changes to Grey
[31m: Font color changes to Red
[32m: Font color changes to Green
[33m: Font color changes to Brown
[34m: Font color changes to Blue
[35m: Font color changes to Violet
[36m: Font color changes to Sky Blue
[37m: Font color changes to Light Grey
[38m: Font color changes to Black
[40m: Background color changes to Black
[41m: Background color changes to Red
[42m: Background color changes to Green
[43m: Background color changes to Brown
[44m: Background color changes to Blue
[45m: Background color changes to Violet
[46m: Background color changes to Sky Blue
[47m: Background color changes to Light Grey
 Proc
esse
s
When you start a process (run a command), there are two ways you can run it:
Foreground Processes
background Processes

Foreground Processes:
By default, every process that you start runs in the foreground. It gets its
input from the keyboard and sends its output to the screen.

background processes:
While a program is running in foreground and taking much time, we cannot run any other commands (start
any other processes) because prompt would not be available until program finishes its processing and comes
out.

A background process runs without being connected to your keyboard. If the

background process requires any keyboard input, it waits.
The advantage of running a process in the background is that you can run other
commands; you do not have to wait until it completes to start another!

sample.ksh &
 Proc
esse
s

Stopping Processes:
Ending a process can be done in several different ways. Often, from a console-based command, sending a
CTRL + C keystroke (the default interrupt character) will exit the command. This works when process is running in
foreground mode.

If a process is running in background mode then first you would need to get its Job ID using ps command and
after that you can use kill command to kill the process.

nohup No hangup
nohup sample.ksh &
sleep
sleep for the specified number of seconds
wait
 ZIP

gzip
zip the specified file

gzip –c If you want to retain the original file even after compression or want the .gz file to have a different
name

gzip -c stake-holder.csv >/backup/stakes/stake-holder.csv.gz.

gzip -c stake-holder.csv >>/backup/stakes/Old-stake-holder.csv.gz

gzip –dc To view the contents of the file without completely decompressing the file
cat dump.doc file1 test.sh | gzip > all.gz – To compress data from multiple files
 tar
tar It can create a single file with the contents of a directory structure in one path or extract a file
into directory structure in another path.
tar –cvf <<tar name>> <<dir>> - to tar the directory dir

Instead of giving entire directory name as a parameter to create a tar file, a list of files
which care to be archived can be passed to tar with -L option.

tar -cvf /archives/archive2/data-specific.tar -L to-tar-list

Sometimes you may need to exclude files from the tar archive. To do this place the files and directories to be excluded in a list file and
then use tar –X <exclude-list>.

tar -X /home/fiuser/excl-list -cvf /archives/data-specific2 .tar data/notes

tar –tvf
The contents of the tar file cannot be viewed directly by cat or more , but to view the details of the files and directories contained in the
tar file

tar -tvf /archives/data-specific2.tar

tar –xvf <<tar file>> - To untar the file
 oce
ssin
g co
mm
and
tee send the output to standard output as well as to the specified file. s
cat file.txt | tee out.txt
cat file.txt | tee –a out.txt
cat file.txt | sort | tee sort.txt | cut –c4
cat file.txt | tee file.bkp1 file.bkp2
comm common from two files
comm file1.txt file2.txt
comm -12 file1.txt file2.txt
 ext p
roce
ss ing c
omm
ands
tr translate characters from stdin to stdout.
tr [options] string1 [string2]
tr ‘[a-z]’ ‘[A-Z]’
tr [:lower:] [:upper:]
tr '{}' '()' < input > output

tr –d deletes the specified characters

tr –s squeezes for the specified character

echo "my username is 432234" | tr -cd [:digit:]

tr -s '\n' ' ' < file.txt - Join all the lines into a single line with space seperator
tr [:space:] '\t'
tr -s [:space:] '\t'
tr -s [:space:] ' ' - multiple spaces to single space
tr -d [:digit:] - delete digits
-c complement - tr -cd [:digit:] - delete all except digits
tr -cd [:print:] - remove all non printable characters
Tr –t : truncates set1
 ext p
roce
ssing c
omm
ands

alnum Alphanumeric characters.

alpha Alphabetic characters.
blank White space characters.
cntrl Control characters.
Digit Numeric characters.
Lower Lowercase alphabetic characters.
Print Printable characters.
Space Space characters.
Special Special characters.
Upper Uppercase alphabetic characters.
 Spec
ial ch
arac
ters
# everything following until <newline> is a comment
; command separator
() separate shell
& run the command in the background
’ ’ Will not allow any special meaning to any characters.
"" Allow variable and command substitution, does not disable $ and \ within the string.
‘cmd‘ Take the output of the command and substitute it as an argument(s) on the command line
\ escape the following character (take it literally)
&& run the command only if the previous command completes successfully.
Ex: cp x.txt y.txt && echo "hi copying is done“
|| run the command only if the previous command did not complete successfully.
Ex: cp x.txt y.txt || echo "hi copying is not done“
/dev/null Null file.
 xargs
Com
mand
xargs
which can pass arguments from the output of one command to another. It can run multiline commands by passing arguments in a single file.
It creates a list of arguments to be passed to the command and runs them.
Echo “a b c d e f”|xargs –n 3

ls account_[0-9].txt | xargs -I { } mv { } { }_prev

Here, xargs passed the output of ls, which is a list as an argument list which is denoted by ‘{ }’ to the mv command .
To delete files even with white space as part of file names – use –printO for find and –O for xargs
find . -name "*.c" -print0 | xargs -0 rm –rf
 find C
omm
and

Recursively search the indicated directory tree to find files matching a type or pattern you specify.
find <<directory>> <<pattern>> <<value>> <<action>>

find . –name clm*

find . –iname [ab]*
find . –type d
find . –links 2
find . –perm 777
find . –type f ! –perm 777
find / -perm /u=r - Find all Read Only files.
find / -perm /a=x - Find all executable files.
find / -type f -empty - Find all empty files
find / -type d -empty - Find all empty directories
find . –user siva
find . –group
 find C
omm
and

find . –size 30c

find . –size +10c
find . –size -10c
find . –atime +365 –name clm*
find . –ctime 30
find . –mtime 50
find . –cmin/-amin/-mmin
find . –user siva –exec rm ‘{}’ \;
find . –user siva –ok rm ‘ {}’ \;
find . -newer test_file - To find all files which are created after the modification time of a particular file
find -maxdepth 1 -name "sum.java" - Search only in current directory
find -not -name "sum.java" -- Not with the name sum.java
find -mindepth 2 -maxdepth 5 -name "sum.java" - between levels for sub directories.
find . -maxdepth 1 -empty - find empty files in a directory
 sort

The sort command is used to order the lines of a file.

sort [options] [+pos1 [ -pos2 ]] file

sort
sort –r Sort in reverse order
sort +3 -4 –t”|” Sort based on a specific field Ignore first x fields when sorting.
sort –n Numerical sort
sort –u Remove duplicate files before sorting
sort –k
Sort –k1 –k3 To sort lines based on 1st and 3rd fields
sort –o <sotfile> <file> To sort a particular file and rewrite the file.
Short Notes:
Sort uses a lot of temporary space while sorting huge files. and by default it uses /tmp directory. If sufficient space was not allocated to /tmp,
then the command would abort abruptly. so sort provides an option -T by which you can use an alternative directory for storing temporary files.

sort -t “|” +1 -T /backup/jones detailed-list.csv.

 uniq

uniq
uniq filters duplicate adjacent lines from a file.
uniq [options] file

uniq -d one copy of only the repeated lines

uniq -D All copies of only the repeated lines
uniq -u select only the lines not repeated
uniq -c number of repetition
uniq–i Ignore case when comparing.
-f N, --skip-fields=N 🡪 compare except the first N fields delimited by spaces/tab
-s N, --skip-chars=N -> skip N characters while comparision
-w, --check-chars=N -> compare first N characters
 If Cla
use
If clause to check and proceed according to the specified expression.

if [[ <<expression>> ]];
then
<<do some actions>>
elif [[ ]]
else
<<do some other actions>>
fi
File test operators:
-f file exists and a regular file
-r/-w/-x file exists and is readable/is writable/is executable
-s check for file size
-e Check if file exists. Is true even if file is a directory but exists.
String operators:
-z string Length of string is zero
-n non zero
String1 == string2 string1 is identical to the string2
 case

case and esac:

Case command provides a cleaner syntax for a chain of comparisons.

case << string>> in
Pattern) commands ;;
Pattern) commands ;;
*) commands ;;
esac
 or a
nd w
hile
loop
s

for The for loop executes a block of statements once for each member of a list

for I in <<list>>
do
<<commands>>
done

while While command repeats a block of commands based on the result of a logical test.

while <<test command>>

do
<<command list>>
done

while read ln
do
<<commands>>
done<file.dat
 hist
ory
com
mand

break
breaks and will come out from the while or for loops.
Break 2 – for nested loops and come out of outer loops.
Continue
The continue statement is similar to the break command, except that it causes the current iteration of the loop to exit, rather than the
entire loop.
 grep u
tility

The grep utility is used to search for generalized regular expressions occurring in Unix files.
grep [options] [file[s]]

^ and $ -> starting and ending

Options
-i ignore case
-c report only a count of the number of lines containing matches
-v invert the search, displaying only lines that do not match
-n display the line number along with the line matched line
-w entire word
-l Display the filenames, but do not display the matched lines.
-L Display all the files which do not contain the matched string/pattern.
-o –b Displaying the position of the matched string in the line
-h Display the matched lines, but do not display the filenames.
-A Display specified number of lines after the matching line
-B Display specified number of lines before the matching line
-C Display specified number of lines before and after the matching line
 grep u
tility

grep OR operatins:

grep 'pattern1\|pattern2' filename

grep -E 'pattern1|pattern2' filename
egrep 'pattern1|pattern2' filename
grep -e pattern1 -e pattern2 filename
 join

join [options] file1 file2

-1 field number : Join on the specified field number in the first file
-2 field number : Join on the specified field number in the second file
-j field number : Equivalent to -1 fieldnumber and -2 fieldnumber
join -j 3 emp.txt dept.txt
-o list : displays only the specified fields from both the files
join -o 1.1 2.2 -1 2 -2 1 emp.txt dept.txt
-t char : input and output field delimiter
join -t: -1 2 -2 1 emp.txt dept.txt
-a filenumber : Prints non matched lines in a file
join -a 1 -a 2 P.txt Q.txt
-i : ignore case while joining
-e EMPTY replace missing input fields with EMPTY
join -t"," -1 2 -a 1 -a2 -e ' NULL' -o '0,1.1,2.2' a1 a2
 sed
Util
ity
sed The non-interactive, stream editor, sed, edits the input stream, line by line, making the specified changes, and sends the
result to standard output.
sed [options] [file/s]
Valid flags on the substitution functions include: . and ^ and $

Sed ‘s/unix/UNIX/g’ substitute unix with UNIX

Sed ‘s/^…//’ remove first 3 chars from each line
Sed ‘s/…$//’ remove last 3 chars from each line
Sed ‘/jan/s/unix/UNIX/’
Sed ‘1,/jan/s/unix/UNIX/g’
sed ‘s/^/rm -ef /g’ file_to_remove_list > file_to_remove_list.sh
sed ‘s/’${string_pattern}’/’ ${replace_string}’/g’ groupsList.xml sed ‘4 s/home/var/’ dir_struct On
Line number 4
echo "1st 2nd 3rd 4th 5th 6th"|sed "s/[0-9]th//2"

You have many occurrences of a pattern in a line and you want to replace just the nth pattern or all n+1,n+2..
occurrences . use a number after the ending “/”.
If you need to replace all occurrences starting from 2nd,then use the following command.
$ echo "1st 2nd 3rd 4th 5th 6th"|sed "s/[0-9]th//2g“
 sed
Util
ity

sed –n ‘100,400p’
sed –n -e ‘1p’ –e ‘$p’
sed –n ‘/unix/p’
sed –n ‘1p;1q’ file - interpret only till 1st line
sed –n “${start-line} , ${end-line}p”
sed –n ‘/’$ptrn‘/,15p’ general-dates.prt
sed ‘1,4d’
sed ‘4,+5d’ This will match line 4 in the file, delete that line, continue to delete the next five lines, and then cease its deletion
and print the rest
sed‘2,5!d’ This will delete everything except starting from 2nd till 5th line.
sed‘1~3d’ This deletes the first line, steps over the next three lines, and then deletes the fourth line. sed continues applying this
pattern until the end of the file.
sed -e 's/^[[:digit:]][[:digit:]][[:digit:]]/(&)/g' phone.txt - Ampersand referencing
sed 's@/usr/bin@&/local@g' path.txt
sed 's@^.*$@<<<&>>>@g' path.txt - Match the whole line
 sed
Util
ity

Character class ---

[[:alnum:]]Alphanumeric [a-z A-Z 0-9]

[[:alpha:]]Alphabetic [a-z A-Z]
[[:blank:]]Blank characters (spaces or tabs)
[[:cntrl:]]Control characters
[[:digit:]]Numbers [0-9]
[[:lower:]]Lowercase letters [a-z]
[[:print:]]Printable characters (noncontrol characters)
[[:space:]]Whitespace
[[:upper:]]Uppercase letters [A-Z]
 awk Util
ity

awk is a pattern scanning and processing language.

awk -F“|" '{print $1}‘

awk -F“|" '{print $0}‘
awk –F”|”‘{print length($0)}’
awk –F”|”‘{print length($3)}’
awk –F”|”‘{print substr($0,1,5)}’
awk –F”|”‘{print NR NF}’
awk –F”|”‘{if($1 == 1000) print $0}’
awk –F”|”‘/unix/ {print $0}’
 awk Util
ity

awk -v myp_name=$p_name ‘{ if ( $1 == myp_name) print $0 }’

awk '{print $0,val}' OFS=, val=$x file1 (without –v option for awk)
awk '{print $0,ENVIRON["x"]}' OFS=, file1

Initialization & final steps:

BEGIN { Actions}
{ACTION} # Action for every line in a file
END { Actions }
ls -lrt | awk ' BEGIN {c = 0}{ c += $5 } END{print c}'

Print number of employees in Technology department

awk 'BEGIN { count=0;}
$4 ~ /Technology/ { count++; }
END { print "Number of employees in Technology Dept =",count;}' employee.txt
 awk Util
ity

awk built-in Variables:

NR NF - Number of Fields and record number in a record

awk '{print NR,"->",NF}' student-marks
FS – field separator
OFS – Output field separator
awk -F':' 'BEGIN{OFS="=";} {print $3,$4;}' /etc/passwd
RS - Record separator
ORS - Output Record Separator
awk 'BEGIN{ORS="=";} {print;}' student-marks
FILENAME - Name of the current input file
FNR - Number of Records relative to the current input file
awk '{print FILENAME, FNR;}' student-marks bookdetails
 awk Util
ity

String Functions in AWK:

index(string,search)
length(string)
split(string,array,separator)
substr(string,position)
substr(string,position,max)
tolower(string)
toupper(string)
 at & cro
ntab

at:
Is used to schedule commands to be executed once, at a particular time in the future.
at 1145 jan 31 abc.ksh
The atq program lists the currently queued jobs, while atrm removes jobs from the queue
-m to email once the job is finished.

Crontab:
which is used for recurring executions (e.g. once an hour, every Tuesday, January 1 every year)
# * * * * * command to execute
#│││││
#│││││
# │ │ │ │ └───── day of week (0-6)(0-6 are Sunday-Saturday, or use names; 7 is Sunday,
same as 0)
# │ │ │ └────────── month (1 - 12)
# │ │ └─────────────── day of month (1 - 31)
# │ └──────────────────── hour (0 - 23)
# └───────────────────────── min (0 - 59)
 Mis
cella
neo
us c
omm
and
s

df - The df command reports on the space left on the file system.

-g,-m,-k etc which helps you to view file system sizes in gigabytes, megabytes or kilobytes blocks resp , including t
gives total allocated space.

du - The du command outputs the number of kilobyes used by each subdirectory. Useful if you have gone over quota and
you want to find out which directory has the most files.
du -s * The -s flag will display only a summary (total size)
 Spe
cial
vari
a bles

$# number of arguments on the command line

$? exit value of the last command executed
$$ process number of the current process
$! process number of the last command done in background
$1 argument on the command line, where n is from 1 through 9, reading left to right
$0 the name of the current program
$* All the arguments are double quoted.
$@ All the arguments are individually double quoted.
 Fun
ctio
ns

• function_name ()
{ list of commands }
 Sample scripts
• Fabonacci series
• Star output
• Write multiple files scenario
• sub_mark.ksh
• cp_multi.ksh multi file copying
• Sum up the numbers from a given number (123 – 1+2+3)