lOMoARcPSD|32991115

1 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

UNIT – 5
SYSTEM PROTECTION & SECURITY
System Protection: Goals of protection, Principles and domain of protection, Access matrix,
Access control, Revocation of access rights.
System Security: Introduction, Program threats, System and network threats, Cryptography
for security, User authentication, implementing security defenses, firewalling to protect
systems and networks, Computer security classification.
Case Studies: Linux, Microsoft Windows.

NOTE:
 The main difference between protection and security is that the “Protection” focuses on
internal threats in a computer system while “Security” focuses on external threats to a
computer system.

SYSTEM PROTECTION
GOALS OF PROTECTION
 Operating system consists of a collection of objects, hardware or software. Each object
has a unique name and can be accessed through a well-defined set of operations.
 Protection problem - ensure that each object is accessed correctly and only by those
processes that are allowed to do so.
1. Obviously to prevent malicious misuse of the system by users or programs.
2. To ensure that each shared resource is used only in accordance with system
policies, which may be set either by system designers or by system administrators.
3. To ensure that errant programs cause the minimal amount of damage possible.
Note:
 Protection systems only provide the mechanisms for enforcing policies and ensuring
reliable systems. It is up to administrators and users to implement those mechanisms
effectively.

PRINCIPLES OF PROTECTION
 Programs, users and systems should be given just enough privileges to perform their
tasks-
1. The principle of least privilege dictates that programs, users, and systems should be
given just enough privileges to perform their tasks. This ensures that failures do the
least amount of harm and allow the least of harm to be done.
 For example, if a program needs special privileges to perform a task, it is better to
make it a SGID program with group ownership of "network" or "backup" or some
other pseudo group, rather than SUID with root ownership. This limits the amount
of damage that can occur if something goes wrong.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

2 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

2. Typically each user is given their own account, and has only enough privilege to modify
their own files. The root account should not be used for normal day to day activities –
The System Administrator should also have an ordinary account, and reserve use of the
root account for only those tasks which need the root privileges.

DOMAIN OF PROTECTION
 A computer can be viewed as a collection of processes and objects (both Hardware &
Software).
 The need to know principle states that a process should only have access to those
objects it needs to accomplish its task, and furthermore only in the modes for which it
needs access and only during the time frame when it needs access.
 The modes available for a particular object may depend upon its type.
Domain Structure
 A protection domain specifies the resources that a process may access.
 Each domain defines a set of objects and the types of operations that may be invoked
on each object.
 An access right is the ability to execute an operation on an object.
 A domain is defined as a set of < object, {access_right_set} > pairs.
Example,

System with three protection domains

 The association between a process and a domain may be static or dynamic.
 If the association is static, then the need-to-know principle requires a way of
changing the contents of the domain dynamically.
 If the association is dynamic, then there needs to be a mechanism for domain
switching.
 Domains may be realized in different fashions - as users, or as processes, or as
procedures.
 As users – In this case, the set of objects that can be accessed depends on the
identity of the user. Domain switching occurs when the user is changed (generally
when one user logs out and another user logs in).
 As process – In this case, the set of objects that can be accessed depends on the
identity of the process. Domain switching occurs when one process sends a message
to another process and then waits for a response.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

3 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 As procedure – In this case, the set of objects that can be accessed corresponds to
the local variables defined within the procedure. Domain switching occurs when a
procedure call is made.

ACCESS MATRIX
 General model of protection can be viewed abstractly as a matrix, called as Access
Matrix. The rows of the access matrix represent domains and the columns represent
objects.
 Each entry in the matrix consists of a set of access rights.
 The entry access(i, j) defines the set of operations that a process executing in domain D i
can invoke on object Oj.
Example 1,
Consider four domains D1 through D4 and four objects – three files F1 through F3 and one laser
printer.
Here, a process executing in domain D 1 can read files F1 and F3. A process executing in domain
D4 has the same privileges as one executing in domain D 1, in addition it can also write onto
files F1 and F3. The laser printer can be accessed only by a process executing in domain D 2.

Access Matrix
 The access matrix scheme provides a mechanism for specifying a variety of policies.
Policies of access matrix concerning protection involve which rights should be included
in the (i, j)th entry.
 We must also decide the domain in which each process executes. This policy is usually
decided by the operating system.
 The Users decide the contents of the access-matrix entries.
 Access matrix provides an appropriate mechanism for defining and implementing
strict control for both static and dynamic association between domain and processes.
 When we switch a process from one domain to another, we execute an operation
(switch) on an object (the domain).
 We can control domain switching by including domains among the objects of the
access matrix.
 Processes should be able to switch from one domain (D i) to another domain (Dj) if and
only if a switch right is given to access(i, j).
Example 2,

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

4 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

A process executing in domain D2 can switch to domain D3 or to domain D4. A process in
domain D4 can switch to D1 and one in domain D1 can switch to D2.

Access matrix for the Example 1 with domains as objects

 Allowing controlled change in the contents of the access matrix entries requires three
additional operations – copy, owner and control.
 The ability to copy rights is denoted by an asterisk, indicating that processes in that
domain have the right to copy that access within the same column, i.e. for the same
object. There are two important variations:
 If the asterisk is removed from the original access right, then the right
is transferred, rather than being copied. This may be termed a transfer right as
opposed to a copy right.
 If only the right and not the asterisk are copied, then the access right is added
to the new domain, but it may not be propagated further. That is the new
domain does not also receive the right to copy the access. This may be termed
a limited copy right.
Example 3,

Access matrix with copy rights

 The owner right adds the privilege of adding new rights or removing existing ones.
Example 4,

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

5 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

Access matrix with owner rights

 Copy and owner rights only allow the modification of rights within a column. The
addition of control rights, which only apply to domain objects, allow a process
operating in one domain to affect the rights available in other domains.
Example 5,
A process operating in domain D2 has the right to control any of the rights in domain D4.

Modified access matrix of Example 2

IMPLEMENTATION OF ACCESS MATRIX

There are several methods of implementing the access matrix -
1. Global Table

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

6 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 The simplest approach is one big global table with < domain, object, rights_set >
entries.
 Unfortunately this table is very large (even if sparse) and so cannot be kept in memory
(without invoking virtual memory techniques).
 There is also no good way to specify groupings - If everyone has access to some
resource, then it still needs a separate entry for every domain.
2. Access Lists for Objects
 Each column of the table can be kept as a list of the access rights for that particular
object, discarding blank entries.
 For efficiency a separate list of default access rights can also be kept, and checked first.
Example,
Access lists for objects for Example 1
Object F1 → Read (R), Read Write (RW)
Object F2 → Read (R)
Object F3 → Read (R), Execute (X), Read Write (RW)
3. Capability Lists for Domains
 In a similar fashion, each row of the table can be kept as a list of the capabilities of that
domain.
 Capability lists are associated with each domain, but not directly accessible by the
domain or any user process.
 Capability lists are themselves protected resources, distinguished from other data in
one of two ways:
 A tag, possibly hardware implemented, distinguishing this special type of data.
 The address space for a program may be split into multiple segments, at least one
of which is inaccessible by the program itself, and used by the operating system for
maintaining the process's access right capability list.
Example,
Capability lists for domains for Example 1
Domain D1 → Read (R), Read (R)
Domain D2 → Print
Domain D3 → Read (R), Execute (X)
Domain D4 → Read Write (RW), Read Write (RW)
4. A Lock-Key Mechanism
 Each resource has a list of unique bit patterns, termed locks.
 Each domain has its own list of unique bit patterns, termed keys.
 Access is granted if one of the domain's keys fits one of the resource's locks.
 Again, a process is not allowed to modify its own keys.

ACCESS CONTROL
V PRANAV DEPT. OF CSE
ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

7 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 Access control is a security technique that regulates who or what can view or use
resources in a computing environment.
 Three types of access control are there –
1. Mandatory access control (MAC) - This is a security model in which access rights
are regulated by a central authority based on multiple levels of security.
2. Discretionary access control (DAC) - This is an access control method in which
owners or administrators of the protected system, data or resource set the policies
defining who or what is authorized to access the resource.
3. Role-based access control (RBAC) - This is a widely used access control
mechanism that restricts access to computer resources based on individuals or
groups with defined business functions.
 The role-based security model relies on a complex structure of role assignments,
role authorizations and role permissions developed using role engineering to
regulate employee access to systems.
 RBAC systems can be used to enforce MAC and DAC frameworks.
 RBAC supports the principle of least privilege, and reduces the susceptibility to
abuse as opposed to SUID or SGID programs.

Example for RBAC,

Role-based access control in Solaris 10

REVOCATION OF ACCESS RIGHTS

 The need to revoke access rights dynamically raises several questions:
 Immediate versus delayed - If delayed, can we determine when the revocation will
take place?

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

8 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 Selective versus general - Does revocation of an access right to an object
affect all users who have that right, or only some users?
 Partial versus total - Can a subset of rights for an object be revoked, or are all
rights revoked at once?
 Temporary versus permanent - If rights are revoked, is there a mechanism for
processes to re-acquire some or all of the revoked rights?
 With an access list scheme revocation is easy, immediate, and can be selective, general,
partial, total, temporary, or permanent, as desired.
 With capabilities lists the problem is more complicated, because access rights are
distributed throughout the system. A few schemes that have been developed include:
 Reacquisition - Capabilities are periodically revoked from each domain, which must
then re-acquire them.
 Back-pointers - A list of pointers is maintained from each object to each capability
which is held for that object.
 Indirection - Capabilities point to an entry in a global table rather than to the
object. Access rights can be revoked by changing or invalidating the table entry,
which may affect multiple processes, which must then re-acquire access rights to
continue.
 Keys - A unique bit pattern is associated with each capability when created, which
can be neither inspected nor modified by the process.
 A master key is associated with each object.
 When a capability is created, its key is set to the object's master key.
 As long as the capability's key matches the object's key, then the capabilities
remain valid.
 The object master key can be changed with the set-key command, thereby
invalidating all current capabilities.
 More flexibility can be added to this scheme by implementing a list of keys for
each object, possibly in a global table.

SYSTEM SECURITY
INTRODUCTION
THE SECURITY PROBLEM
 The most common types of violations include -

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

9 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

1. Breach of Confidentiality - Theft of private or confidential information, such as
credit-card numbers, trade secrets, patents, secret formulas, manufacturing
procedures, medical information, financial information, etc.
2. Breach of Integrity - Unauthorized modification of data, which may have serious
indirect consequences.
 For example a popular game or other program's source code could be modified
to open up security holes on users systems before being released to the public.
3. Breach of Availability - Unauthorized destruction of data, often just for the "fun" of
causing havoc and for bragging rights.
 Vandalism of web sites is a common form of this violation.
4. Theft of Service - Unauthorized use of resources, such as theft of CPU cycles,
installation of daemons running an unauthorized file server, or tapping into the
target's telephone or networking services.
5. Denial of Service (DOS) - Preventing legitimate users from using the system, often
by overloading and overwhelming the system with an excess of requests for service.
 One common attack is masquerading, in which the attacker pretends to be a trusted
third party. A variation of this is the man-in-the-middle, in which the attacker
masquerades as both ends of the conversation to two targets.
 A replay attack involves repeating a valid transmission. Sometimes this can be the
entire attack, or other times the content of the original message is replaced with
malicious content.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

10 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

Standard security attacks

 There are four levels at which a system must be protected -
1. Physical - The easiest way to steal data is to pocket the backup tapes. Also, access to
the root console will often give the user special privileges, such as rebooting the
system as root from removable media. Even general access to terminals in a
computer room offers some opportunities for an attacker, although today's modern
high-speed networking environment provides more and more opportunities for
remote attacks.
2. Human - There is some concern that the humans who are allowed access to a
system be trustworthy, and that they cannot be coerced into breaching security.
However more and more attacks today are made via social engineering, which
basically means fooling trustworthy people into accidentally breaching security.
 Phishing involves sending an innocent-looking e-mail or web site designed to
fool people into revealing confidential information.
 Dumpster Diving involves searching the trash or other locations for passwords
that are written down.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

11 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 Password Cracking involves divining user’s passwords, either by watching them
type in their passwords, knowing something about them like their pet's names,
or simply trying all words in common dictionaries.
3. Operating System - The OS must protect itself from security breaches, such as
runaway processes (denial of service), memory-access violations, stack overflow
violations, the launching of programs with excessive privileges, and many others.
4. Network - As network communications become ever more important and pervasive
in modern computing environments, it becomes ever more important to protect this
area of the system.

PROGRAM THREATS
 Program threats are the threats that attack specific programs or are carried and
distributed in programs.
1. Trojan horse
 A Trojan horse is a program that secretly performs some maliciousness in addition to
its visible actions.
 Some Trojan horses are deliberately written as such, and others are the result of
legitimate programs that have become infected with viruses.
 One dangerous opening for Trojan horses is long search paths, and in particular paths
which include the current directory (“.”) as part of the path. If a dangerous program
having the same name as a legitimate program (or a common mis-spelling, such as "sl"
instead of "ls") is placed anywhere on the path, then an unsuspecting user may be
fooled into running the wrong program by mistake.
 Another classic Trojan horse is a login emulator, which records a users account name
and password, issues a "password incorrect" message, and then logs off the system. The
user then tries again (with a proper login prompt), logs in successfully, and doesn't
realize that their information has been stolen.
 Two solutions to Trojan Horses are to have the system print usage statistics on logouts,
and to require the typing of non-trappable key sequences such as Control-Alt-Delete in
order to log in.
 Spyware is a version of a Trojan horse that is often included in "free" software
downloaded off the Internet. Spyware programs generate pop-up browser windows,
and may also accumulate information about the user and deliver it to some central
site. Another common task of spyware is to send out spam e-mail messages, which then
purportedly come from the infected user.
2. Trap Door
 A Trap Door is when a designer or a programmer (or hacker) deliberately inserts a
security hole that they can use later to access the system.
 Because of the possibility of trap doors, once a system has been in an untrustworthy
state, that system can never be trusted again. Even the backup tapes may contain a
copy of some cleverly hidden back door.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

12 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 A clever trap door could be inserted into a compiler, so that any programs compiled
with that compiler would contain a security hole. This is especially dangerous, because
inspection of the code being compiled would not reveal any problems.
3. Logic Bomb
 A Logic Bomb is code that is not designed to cause havoc all the time, but only when a
certain set of circumstances occurs, such as when a particular date or time is reached
or some other noticeable event.
 A classic example is the Dead-Man Switch, which is designed to check whether a certain
person (e.g. the author) is logging in every day, and if they don't log in for a long time,
then the logic bomb goes off and either opens up security holes or causes other
problems.
4. Viruses
 A virus is a fragment of code embedded in an otherwise legitimate program, designed
to replicate itself, and wreaking havoc.
 Viruses are more likely to infect PCs than UNIX or other multi-user systems, because
programs in the latter systems have limited authority to modify other programs or to
access critical system structures.
 Viruses are delivered to systems in a virus dropper, usually some form of a Trojan
horse, and usually via e-mail or unsafe downloads.

A boot-sector computer virus

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

13 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

SYSTEM AND NETWORK THREATS
 The threats that attack the operating system or the network itself, or leverage those
systems to launch their attacks are known as System and Network threats.
1. Worms
 A worm is a process that uses the fork / spawns process to make copies of itself in order
to wreak havoc on a system.
 Worms consume system resources, often blocking out other, legitimate processes.
 Worms that propagate over networks can be especially problematic, as they can tie up
vast amounts of network resources and bring down large-scale systems.
 A worm consisted of two parts:-
1. A small program called a grappling hook, which was deposited on the target
system through one of three vulnerabilities, and
2. The main worm program, which was transferred onto the target system and
launched by the grappling hook program.

The Morris Internet worm

 The three vulnerabilities exploited by the Morris Internet worm were as follows:
1. rsh ( remote shell ) is a utility that was in common use at that time for accessing
remote systems without having to provide a password. If a user had an account on
two different computers (with the same account name on both systems), then the
system could be configured to allow that user to remotely connect from one system
to the other without having to provide a password.
 Many systems were configured so that any user (except root) on system A could
access the same account on system B without providing a password.
2. finger is a utility that allows one to remotely query a user database, to find the true
name and other information for a given account name on a given system.
 For example "finger [email protected]" would access the finger
daemon at somemachine.edu and return information regarding joeUser.
Unfortunately the finger daemon (which ran with system privileges) had the
buffer overflow problem, so by sending a special 536-character user name the

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

14 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

worm was able to fork a shell on the remote system running with root
privileges.
3. sendmail is a routine for sending and forwarding mail that also included a
debugging option for verifying and testing the system.
 The debug feature was convenient for administrators, and was often left turned
on.
 The Morris worm exploited the debugger to mail and executes a copy of the
grappling hook program on the remote system.
 Once in place, the worm undertook systematic attacks to discover user passwords:
 First it would check for accounts for which the account name and the password
were the same, such as "guest", "guest".
 Then it would try an internal dictionary of 432 favorite password choices.
 Finally it would try every word in the standard UNIX on-line dictionary to try and
break into user accounts.
 Once it had gotten access to one or more user accounts, then it would attempt to use
those accounts to rsh to other systems, and continue the process.
 With each new access, the worm would check for already running copies of itself, and 6
out of 7 times if it found one it would stop.
 Fortunately the same rapid network connectivity that allowed the worm to propagate
so quickly also quickly led to its demise - Within 24 hours remedies for stopping the
worm propagated through the Internet from administrator to administrator, and the
worm was quickly shut down.
2. Port Scanning
 Port Scanning is technically not an attack, but rather a search for vulnerabilities to
attack. The basic idea is to systematically attempt to connect to every known network
port on some remote machine, and to attempt to make contact. Once it is determined
that a particular computer is listening to a particular port, then the next step is to
determine what daemon is listening, and whether or not it is a version containing a
known security flaw that can be exploited.
 Because port scanning is easily detected and traced, it is usually launched from zombie
systems, i.e. previously hacked systems that are being used without the knowledge or
permission of their rightful owner. For this reason it is important to protect
"innocuous" systems and accounts as well as those that contain sensitive information
or special privileges.
 There are also port scanners available that administrators can use to check their own
systems, which report any weaknesses found but which do not exploit the weaknesses
or cause any problems. Two such systems are nmap (http://www.insecure.org/nmap)
and nessus (http://www.nessus.org). The former identifies what OS is found, what
firewalls are in place, and what services are listening to what ports. The latter also
contains a database of known security holes, and identifies any that it finds.

3. Denial of Service

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

15 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 Denial of Service (DOS) attacks do not attempt to actually access or damage systems,
but merely to clog them up so badly that they cannot be used for any useful work.
 Tight loops that repeatedly request system services are an obvious form of this attack.
 DOS attacks can also involve social engineering, such as the Internet chain letters that
say "send this immediately to 10 of your friends, and then go to a certain URL", which
clogs up not only the Internet mail system but also the web server to which everyone is
directed.
 Security systems that lock accounts after a certain number of failed login attempts are
subject to DOS attacks which repeatedly attempt logins to all accounts with invalid
passwords strictly in order to lock up all accounts.

CRYPTOGRAPHY FOR SECURITY

 Cryptography is technique of securing information and communications through use of
codes so that only those people for whom the information is intended can understand it
and process it. Thus preventing unauthorized access to information.
 In Cryptography the techniques which are use to protect information are obtained
from mathematical concepts and a set of rule based calculations known as algorithms
to convert messages in ways that make it hard to decode it.
 These algorithms are used for cryptographic key generation, digital signing, and
verification to protect data privacy, web browsing on internet and to protect
confidential transactions such as credit card and debit card transactions.
 The process where an ordinary plain-text is converted to cipher-text which is the text
made such that intended receiver of the text can only decode it and hence this process
is known as “Encryption”.
 The process of conversion of cipher-text to plain-text this is known as “Decryption”.

Relationship between the plain-text and the cipher-text

 If P is the plain-text file, K E is the encryption key, C is the cipher-text and E is the
encryption algorithm, then C = E(P, KE). Here, cipher-text is obtained by using the
encryption algorithm with the plain-text and encryption key as parameters.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

16 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 Similarly, P = D(C, KD) where, D is the decryption algorithm and K D is the decryption
key. Here, plain-text is obtained back by using the decryption algorithm with the
cipher-text and decryption key as parameters
Features of Cryptography -
1. Confidentiality - Information can only be accessed by the person for whom it is
intended and no other person except him can access it.
2. Integrity - Information cannot be modified in storage or transition between sender and
intended receiver without any addition to information being detected.
3. Non-repudiation - The creator/sender of information cannot deny his or her intention
to send information at later stage.
4. Authentication - The identities of sender and receiver are confirmed. As well as
destination/origin of information is confirmed.
Types of Cryptography -
In general there are three types of cryptography:
1. Symmetric Key Cryptography - It is an encryption system where the sender and
receiver of message use a single common key to encrypt and decrypt messages.
 Symmetric Key Systems are faster and simpler but the problem is that sender and
receiver have to somehow exchange key in a secure manner.
 The most popular symmetric key cryptography system is Data Encryption System
(DES).
2. Hash Functions - There is no usage of any key in this algorithm. A hash value with fixed
length is calculated as per the plain text which makes it impossible for contents of plain
text to be recovered.
 Many operating systems use hash functions to encrypt passwords.
3. Asymmetric Key Cryptography - Under this system a pair of keys is used to encrypt and
decrypt information. A public key is used for encryption and a private key is used for
decryption. Public key and Private Key are different.
 Even if the public key is known by everyone the intended receiver can only decode it
because he alone knows the private key.

USER AUTHENTICATION
 Most authentication methods are based on three general principles –
1. Something the user knows.
2. Something the user has.
3. Something the user is.
 Protection, dealt with making sure that only certain users were allowed to perform
certain tasks, i.e. that a users privileges were dependent on his or her identity.
Authentication using Passwords
 The most widely used form of authentication is to require the user to type a login name
and a password.
 Password protection is easy to understand and easy to implement.
V PRANAV DEPT. OF CSE
ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

17 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 The simplest implementation just keeps a central list of (login-name, password) pairs.
The login name typed in is looked up in the list and the typed password is compared to
the stored password. If they match, the login is allowed; if they do not match, the login
is rejected.
Password Vulnerabilities
 Passwords can be guessed.
 "Shoulder surfing" involves looking over people's shoulders while they are typing in
their password.
 "Packet sniffing" involves putting a monitor on a network connection and reading data
contained in those packets.
 Passwords can be given away to friends or co-workers, destroying the integrity of the
entire user-identification system.
One-Time Passwords
 One-time passwords resist shoulder surfing and other attacks where an observer is
able to capture a password typed in by a user.
 A variation on the password idea is to have each new user provide a long list of
questions and answers that are then stored on the server securely.
 Another variation is based on a challenge and a response. Because the challenge is
different each time, the old response will not be valid for future challenges.
 For example, the user may be in possession of a secret function f(x). The system
challenges with some given value for x, and the user responds with f(x), which the
system can then verify. Since the challenger gives a different (random) x each time,
the answer is constantly changing.
Authentication using Physical objects
 Another way of providing authentication is to use information bearing plastic cards.
 There are two types of cards –
1. Magnetic stripe cards
 These hold about 140 bytes of information written on a piece of magnetic tape glued to
the block of the card. This information can be read out by the terminal and sent to the
central computer.
 Often the information contains the user’s password (e.g. PIN code), so that the terminal
can do an identity check.
 Magnetic stripe cards are risky because the equipment to read and write them is cheap
and widespread.
2. Chip cards
 These cards contain a tiny integrated circuit on them.
 These cards are sub-divided into two categories –
1. Stored value cards - They contain a small amount of memory using ROM to allow
the value to be remembered when the card is removed from the reader and thus
power turned off.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

18 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 There is no CPU on the card, so the value stores must be changed by an external
CPU.
2. Smart cards – It is a small tamper off computer that can engage in a discussion
with a central computer to authenticate the user.
 They can be used to hold money, as do stored value cards, but with much better
security and universality.
 The advantage of smart cards is that they do not need an online connection to a
bank.
Authentication using Biometrics
 Biometrics involve a physical characteristic of the user that is not easily forged or
duplicated and not likely to be identical between multiple users.
 A typical biometric systems has two parts –
 Enrollment – In this part, the user’s characteristics are measured and the results
digitized. The features are stored in records which are kept in a central database.
 Identification – In this part, the user shows up and provides a login name. Then the
system makes the measurement again. If the new values match the ones sampled at
enrollment time, the login is accepted; otherwise it is rejected.
 Fingerprint scanners are getting faster, more accurate, and more economical.
 Palm readers can check thermal properties, finger length, etc.
 Retinal scanners examine the back of the users' eyes.
 Voiceprint analyzers distinguish particular voices.
 Difficulties may arise in the event of colds, injuries, or other physiological change

IMPLEMENTING SECURITY DEFENSES

Security Policy
 A security policy should be well thought-out, agreed upon, and contained in a living
document that everyone adheres to and is updated as needed.
 Examples of contents include how often port scans are run, password requirements,
virus detectors, etc.
Vulnerability Assessment
 Periodically examine the system to detect vulnerabilities.
 Port scanning.
 Check for bad passwords.
 Look for suid programs.
 Unauthorized programs in system directories.
 Incorrect permission bits set.
 Program checksums / digital signatures which have changed.
 Unexpected or hidden network daemons.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

19 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 New entries in startup scripts, shutdown scripts, cron tables, or other system scripts
or configuration files.
 New unauthorized accounts.
Intrusion Detection
 Intrusion detection attempts to detect attacks, both successful and unsuccessful
attempts. Different techniques vary along several axes:
 The time that detection occurs, either during the attack or after the fact.
 The types of information examined to detect the attack(s). Some attacks can only
be detected by analyzing multiple sources of information.
 The response to the attack, which may range from alerting an administrator to
automatically stopping the attack (e.g. killing an offending process), to tracing
back the attack in order to identify the attacker.
 Another approach is to divert the attacker to a honeypot, on a honeynet. The
idea behind a honeypot is a computer running normal services, but which no
one uses to do any real work. Such a system should not see any network traffic
under normal conditions, so any traffic going to or from such a system is by
definition suspicious. Honeypots are normally kept on a honeynet protected by
a reverse firewall, which will let potential attackers in to the honeypot, but will
not allow any outgoing traffic. Honeypots are closely watched, and any
suspicious activity carefully logged and investigated.
 Intrusion Detection Systems, IDSs, raise the alarm when they detect an intrusion.
Intrusion Detection and Prevention Systems, IDPs, act as filtering routers, shutting
down suspicious traffic when it is detected.
 There are two major approaches to detecting problems:
1. Signature-Based Detection scans network packets, system files, etc. looking for
recognizable characteristics of known attacks, such as text strings for messages or
the binary code for "exec /bin/sh". The problem with this is that it can only detect
previously encountered problems for which the signature is known, requiring the
frequent update of signature lists.
2. Anomaly Detection looks for "unusual" patterns of traffic or operation, such as
unusually heavy load or an unusual number of logins late at night.
 The benefit of this approach is that it can detect previously unknown attacks, so
called zero-day attacks.
 One problem with this method is characterizing what is "normal" for a given
system. One approach is to benchmark the system, but if the attacker is already
present when the benchmarks are made, then the "unusual" activity is recorded
as "the norm."
 Another problem is that not all changes in system performance are the result of
security attacks.
 To be effective, anomaly detectors must have a very low false alarm (false
positive) rate, lest the warnings get ignored, as well as a low false negative rate
in which attacks are missed.
Virus Protection
V PRANAV DEPT. OF CSE
ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

20 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 Modern anti-virus programs are basically signature-based detection systems, which
also have the ability of disinfecting the affected files and returning them back to their
original condition.
 Both viruses and anti-virus programs are rapidly evolving. For example viruses now
commonly mutate every time they propagate, and so anti-virus programs look for
families of related signatures rather than specific ones.
 Some antivirus programs look for anomalies, such as an executable program being
opened for writing.
 Avoiding bootleg, free, and shared software can help reduce the chance of catching a
virus, but even shrink-wrapped official software has on occasion been infected by
disgruntled factory workers.
 Some virus detectors will run suspicious programs in a sandbox, an isolated and secure
area of the system which mimics the real system.
 Rich Text Format, RTF, files cannot carry macros, and hence cannot carry Word macro
viruses.
 Known safe programs can be digitally signed, and periodically the files can be re-
verified against the stored digital signatures.
Auditing, Accounting, and Logging
 Auditing, accounting, and logging records can also be used to detect anomalous
behavior.
 Some of the kinds of things that can be logged include authentication failures and
successes, logins, running of SUID or SGID programs, network accesses, system calls,
etc. In extreme cases almost every keystroke and electron that moves can be logged for
future analysis.
 "The Cuckoo's Egg" tells the story of how Cliff Stoll detected one of the early UNIX
break ins when he noticed anomalies in the accounting records on a computer system
being used by physics researchers.

FIREWALLING TO PROTECT SYSTEMS AND NETWORKS

 Firewalls are devices (or sometimes software) that sit on the border between two
security domains and monitor/log activity between them, sometimes restricting the
traffic that can pass between them based on certain criteria.
 For example a firewall router may allow HTTP: requests to pass through to a web
server inside a company domain while not allowing telnet, ssh, or other traffic to
pass through.
 A common architecture is to establish a de-militarized zone, DMZ, which sort of sits
"between" the company domain and the outside world, as shown below. Company
computers can reach either the DMZ or the outside world, but outside computers can
only reach the DMZ. Perhaps most importantly, the DMZ cannot reach any of the other
company computers, so even if the DMZ is breached, the attacker cannot get to the rest
of the company network.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

21 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 Firewalls themselves need to be resistant to attacks, and unfortunately have several

vulnerabilities:
 Tunneling, involves encapsulating forbidden traffic inside of packets that are
allowed.
 Denial of service attacks addressed at the firewall itself.
 Spoofing, in which an unauthorized host sends packets to the firewall with the
return address of an authorized host.
 In addition to the common firewalls protecting a company internal network from the
outside world, there are also some specialized forms of firewalls that have been
recently developed:
 A personal firewall is a software layer that protects an individual computer. It may
be a part of the operating system or a separate software package.
 An application proxy firewall understands the protocols of a particular service and
acts as a stand-in (and relay) for the particular service. For example, and SMTP
proxy firewall would accept SMTP requests from the outside world, examine them
for security concerns, and forward only the "safe" ones on to the real SMTP server
behind the firewall.
 XML firewalls examine XML packets only, and reject ill-formed packets. Similar
firewalls exist for other specific protocols.
 System call firewalls guard the boundary between user mode and system mode,
and reject any system calls that violate security policies.

COMPUTER SECURITY CLASSIFICATION

 No computer system can be 100% secure, and attempts to make it so can quickly make
it unusable.
V PRANAV DEPT. OF CSE
ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])

 lOMoARcPSD|32991115

22 |U N I T - 5 : SYSTEM PROTECTION & SECURITY

 However one can establish a level of trust to which one feels "safe" using a given
computer system for particular security needs.
 The U.S. Department of Defense's "Trusted Computer System Evaluation Criteria"
defines four broad levels of trust, and sub-levels in some cases:
 Level D is the least trustworthy, and encompasses all systems that do not meet any
of the more stringent criteria. DOS and Windows 3.1 fall into level D, which has no
user identification or authorization, and anyone who sits down has full access and
control over the machine.
 Level C1 includes user identification and authorization, and some means of
controlling what users are allowed to access what files. It is designed for use by a
group of mostly cooperating users, and describes most common UNIX systems.
 Level C2 adds individual-level control and monitoring. For example file access
control can be allowed or denied on a per-individual basis, and the system
administrator can monitor and log the activities of specific individuals. Another
restriction is that when one user uses a system resource and then returns it back to
the system, another user who uses the same resource later cannot read any of the
information that the first user stored there. ( I.e. buffers, etc. are wiped out between
users, and are not left full of old contents. ) Some special secure versions of UNIX
have been certified for C2 security levels, such as SCO.
 Level B adds sensitivity labels on each object in the system, such as "secret", "top
secret", and "confidential". Individual users have different clearance levels, which
controls which objects they are able to access. All human-readable documents are
labeled at both the top and bottom with the sensitivity level of the file.
 Level B2 extends sensitivity labels to all system resources, including devices. B2 also
supports covert channels and the auditing of events that could exploit covert
channels.
 B3 allows creation of access-control lists that denote users NOT given access to
specific objects.
 Class A is the highest level of security. Architecturally it is the same as B3, but it is
developed using formal methods which can be used to prove that the system meets
all requirements and cannot have any possible bugs or other vulnerabilities.
Systems in class A and higher may be developed by trusted personnel in secure
facilities.
 These classifications determine what a system can implement, but it is up to
security policy to determine how they are implemented in practice. These systems
and policies can be reviewed and certified by trusted organizations, such as the
National Computer Security Center. Other standards may dictate physical
protections and other issues.

V PRANAV DEPT. OF CSE

ASST. PROFESSOR Sir CRRCOE

Downloaded by Monasri Kore ([email protected])