EE 5084

Cyber Security
Security Technology: Intrusion
Detection and Prevention Systems,
and Other Security Tools

Do not wait; the time will never be just right. Start where you stand and
work with whatever tools you may have at your command, and better
tools will be found as you go along.
NAPOLEON HILL (1883–1970) FOUNDER OF THE SCIENCE of SUCCESS
 Learning Objectives
• Categories and models of intrusion detection and
prevention systems (IDPS)
• Detection approaches employed by IDPS
• Honeypots, honeynets, and padded cell systems
• Categories of scanning and analysis tools, and
describe the specific tools used within each category

2
 Introduction

• Protection of organizations assets relies as much on

managerial controls as on technical safeguards.
• Technical solutions must be guided by policy
• Advanced technologies can be used to enhance the
security of information assets.

3
 Intrusion Detection and Prevention
Systems (IDPS) (1)
• Intrusion occurs when an attacker attempts to gain
entry into or disrupt the normal operations of an IS
• Intrusion prevention consists of activities that deter an
intrusion.
• Intrusion detection consists of procedures and systems
that identify system intrusions.
• Intrusion reaction encompasses actions undertaken
when intrusion event is detected.

4
 Intrusion Detection and Prevention
Systems (IDPS) (2)
• Intrusion correction: complete restoration of operations
to a normal state and seek to identify source and
method of intrusion
• Intrusion detection systems detect a violation of its
configuration and activate alarm.
• IDPSs enable administrators to configure systems to
notify them directly of trouble via e-mail or pagers.
• Can also be configured to notify an external security
service organization of a “break-in.”

5
 Why Use an IDPS?
• Main purposes
– To identify and report an intrusion
– Can quickly contain attack and prevent/mitigate loss or
damage
– Detect and deal with preambles to attacks
• Data collection allows the organization to examine
what happened after an intrusion and why.
• Serves as a deterrent by increasing the fear of
detection
• Can help management with quality assurance and
continuous improvement
6
 Types of IDPSs

• Network-based systems (NIDPS) - focus on

protecting network information assets.
• Host-based systems (HIDPS)
• Wireless IDPS: focuses on wireless networks
• Network behavior analysis IDPS: examines traffic flow
on a network in an attempt to recognize abnormal
patterns

7
8
 NIDPS (1)
• Focused on protecting network information assets.
• Resides on a computer or an appliance connected to a
segment of an organization’s network; looks for indications
of attacks
• When examining packets, a NIDPS looks for attack patterns
within network traffic
• Installed at specific place in the network where it can
monitor traffic going into and out of a particular network
segment

9
 NIDPS (2)

• To determine whether attack has occurred/is under way,

compare measured activity to known signatures in
knowledge base
• Done by using special implementation of TCP/IP stack:
– In the process of protocol stack verification, NIDPSs look
for invalid data packets.
– In the application protocol verification, higher-order
protocols are examined for unexpected packet behavior
or improper use.

10
 Advantages of NIDPS

• Good network design and placement of NIDPS can

enable an organization to monitor a large network with
few devices.
• NIDPSs are usually passive and can be deployed into
existing networks with little disruption to normal network
operations.
• NIDPSs are not usually susceptible to direct attack
and may not be detectable by attackers.

11
 Disadvantages of NIDPS

• Can become overwhelmed by network volume and fail

to recognize attacks
• Require access to all traffic to be monitored
• Cannot analyze encrypted packets
• Cannot reliably ascertain if attack was successful or not
• Some forms of attack are not easily discerned by
NIDPSs, specifically those involving fragmented
packets.

12
 Host-based IDPS (HIDPS)
• Resides on a particular computer or server (host) and
monitors activity only on that system
• Benchmarks and monitors the status of key system files
and detects when intruder creates, modifies, or deletes
files
• Advantage over NIDPS: can access encrypted
information traveling over network and make decisions
about potential/actual attacks
• Most HIDPSs work on the principle of configuration or
change management.

13
 Advantages of HIDPS

• Can detect local events on host systems and detect

attacks that may elude a network-based IDPS
• Functions on host system, where encrypted traffic will
have been decrypted and is available for processing
• Not affected by use of switched network protocols
• Can detect inconsistencies in how applications and
systems programs were used by examining records
stored in audit logs

14
 Disadvantages of HIDPS

• Pose more management issues

• Vulnerable both to direct attacks and attacks against
host operating system
• Does not detect multihost scanning, nor scanning of
non-host network devices
• Susceptible to some DoS attacks
• Can use large amounts of disk space
• Can inflict a performance overhead on its host systems

15
 Other Types
• Wireless NIDPS
– Monitors and analyzes wireless network traffic
– Issues associated with it include physical security, sensor range,
access point and wireless switch locations, wired network
connections, cost, AP and wireless switch locations.
• Network behavior analysis systems
– Identify problems related to the flow of traffic
– Types of events commonly detected include denial-of-service (DoS)
attacks, scanning, worms, unexpected application services, and
policy violations.
– Offer intrusion prevention capabilities that are passive, inline, and
both passive and inline

16
 IDPS Detection Methods (1)
• Signature-based detection
– Examines network traffic in search of patterns that match
known signatures
– Widely used because many attacks have clear and distinct
signatures
– Problem with this approach is that new attack patterns
must continually be added to IDPS’s database of
signatures.
• Slow, methodical attack involving multiple events might
escape detection.

17
 IDPS Detection Methods (2)

• Anomaly-based detection
– Anomaly-based detection (or behavior-based detection)
collects statistical summaries by observing traffic known
to be normal.
– When measured activity is outside baseline parameters
or clipping level, IDPS sends alert to administrator.
– IDPS can detect new types of attacks.
– Requires much more overhead and processing capacity
than signature-based detection
– May generate many false positives

18
 IDPS Detection Methods (3)

• Stateful protocol analysis

– SPA: process of comparing known normal/benign protocol
profiles against observed traffic
– Stores and uses relevant data detected in a session to
identify intrusions involving multiple requests /responses;
allows IDPS to better detect specialized, multisession
attacks (also called deep packet inspection)
– Drawbacks: analytical complexity; heavy processing
overhead; may fail to detect intrusion unless protocol
violates fundamental behavior; may interfere with normal
operations of protocol

19
 IDPS Detection Methods (4)

• Log file monitors

– Log file monitor (LFM) similar to NIDPS
– Reviews log files generated by servers, network devices,
and even other IDPSs for patterns and signatures
– Patterns that signify attack may be much easier to identify
when entire network and its systems are viewed as a
whole
– Requires considerable resources since it involves the
collection, movement, storage, and analysis of large
quantities of log data

20
 IDPS Response Behavior
• IDPS response to external stimulation depends on the
configuration and function; many response options are
available.
• IDPS responses can be classified as active or passive.
– Active response: collecting additional information about the
intrusion, modifying the network environment, taking action against
the intrusion
– Passive response: setting off alarms or notifications, collecting
passive data through SNMP traps
• Many IDPSs can generate routine reports and other
detailed documents.
• Failsafe features protect IDPS from being circumvented.

21
 IDPS Alert Types (1)
• True Positive
A legitimate attack which triggers an IDPS to produce an alarm.[2]
• False Positive
An event signaling an IDPS to produce an alarm when no attack
has taken place.[2]
• False Negative
When no alarm is raised when an attack has taken place.[2]
• True Negative
An event when no attack has taken place and no detection is
made.

22
 IDPS Alert Types(2)
• A good portion of false-positives and false-negatives can be
attributed to lack of knowledge about the environment being
monitored

• Knowledge about the environment is not up-to-date as well as

inaccurate history

23
 IDPS Alert Types(2)
• A good portion of false-positives and false-negatives can be
attributed to lack of knowledge about the environment being
monitored

• Knowledge about the environment is not up-to-date as well as

inaccurate history

• Incorporate context to reduce the rate of both false-positives

and false-negatives

• Context is additional security event data and information in relation to

the environment

24
 Selecting IDPS Approaches and
Products
• Technical and policy considerations
– What is your systems environment?
– What are your security goals and objectives?
– What is your existing security policy?
• Organizational requirements and constraints
– What requirements are levied from outside the
organization?
– What are your organization’s resource constraints?

25
 IDPSs product features and quality

• Is the product sufficiently scalable for your

environment?
• How has the product been tested?
• What user level of expertise is targeted by the
product?
• Is the product designed to evolve as the organization
grows?
• What are the support provisions for the product?

26
 Strengths of IDPSs (1)

• IDPSs perform the following functions well:

– Monitoring and analysis of system events and user
behaviors
– Testing security states of system configurations
– Baselining security state of system and tracking changes
– Recognizing patterns of system events corresponding to
known attacks
– Recognizing activity patterns that vary from normal activity

27
 Strengths of IDPSs (2)

• IDPSs perform the following functions well: (cont’d)

– Managing OS audit and logging mechanisms and data they
generate
– Alerting appropriate staff when attacks are detected
– Measuring enforcement of security policies encoded in
analysis engine
– Providing default information on security policies
– Allowing non-security experts to perform important security
monitoring functions

28
 Limitations of IDPSs (1)

• IDPSs cannot perform the following functions:

– Compensating for weak/missing security mechanisms in
protection infrastructure
– Instantaneously detecting, reporting, responding to
attack when there is heavy network or processing load
– Detecting new attacks or variants of existing attacks
– Effectively responding to attacks by sophisticated
attackers
– Automatically investigating attacks without human
intervention

29
 Limitations of IDPSs (2)

• IDPSs cannot perform the following functions (cont’d):

– Resisting attacks intended to defeat or circumvent them
– Compensating for problems with fidelity of information
sources
– Dealing effectively with switched networks

30
 Implementation of an IDPS

• An IDPS can be implemented via one of three basic

control strategies:
– Centralized: All IDPS control functions are implemented
and managed in a central location.
– Fully distributed: All control functions are applied at the
physical location of each IDPS component.
– Partially distributed: Combines the two; while individual
agents can still analyze and respond to local threats, they
report to a hierarchical central facility to enable
organization to detect widespread attacks.

31
32
33
34
 Deployment of an IDPS (1)

• IDPS deployment
– Great care must be taken when deciding where to locate
components.
– Planners must select deployment strategy that is based
on careful analysis of organization’s information security
requirements and causes minimal impact.
– NIDPS and HIDPS can be used in tandem to cover
individual systems that connect to an organization’s
network and networks themselves.

35
 Deployment of an IDPS (2)

• Deploying network-based IDPSs

– NIST recommends four locations for NIDPS sensors
• Location 1: Behind each external firewall, in the network
DMZ
• Location 2: Outside an external firewall
• Location 3: On major network backbones
• Location 4: On critical subnets

36
37
 Deployment of an IDPS (3)

• Deploying host-based IDPSs

– Proper implementation of HIDPSs can be a painstaking
and time-consuming task.
– Deployment begins with implementing most critical
systems first.
– Installation continues until either all systems are installed
or the organization reaches planned degree of coverage
it will accept.

38
 Comparison with firewall

• Firewalls limit access between networks to prevent

intrusion and do not signal an attack from inside the
network
• An IDPS evaluates a suspected intrusion and signals an
alarm

39
 Honeypots, Honeynets, and Padded
Cell Systems (1)
• Honeypots: decoy systems designed to lure potential
attackers away from critical systems
• Honeynets: several honeypots connected together on a
network segment
• Honeypots are designed to:
– Divert attacker from accessing critical systems
– Collect information about attacker’s activity
– Encourage attacker to stay on a system long enough for
administrators to document the event and perhaps
respond

40
 Honeypots, Honeynets, and Padded
Cell Systems (2)
• Padded cell system: protected honeypot that cannot be
easily compromised
• In addition to attracting attackers with tempting data, a
padded cell operates in tandem with a traditional IDPS.
• When the IDPS detects attackers, padded cell system
seamlessly transfers them to a special simulated
environment where they can cause no harm—hence
the name padded cell.

41
 Advantages of Honeypots, Honeynets,
and Padded Cell Systems
• Attackers can be diverted to targets they cannot
damage.
• Administrators have time to decide how to respond to
an attacker.
• Attackers’ actions can be easily and more extensively
monitored, and records can be used to refine threat
models and improve system protections.
• Honeypots may be effective at catching insiders who
are snooping around a network.

42
 Disadvantages of Honeypots,
Honeynets, and Padded Cell Systems
• Legal implications of using such devices are not well
understood.
• Honeypots and padded cells have not yet been shown
to be generally useful security technologies.
• An expert attacker, once diverted into a decoy system,
may become angry and launch a more aggressive
attack against an organization’s systems.
• Administrators and security managers need a high
level of expertise to use these systems.

43
 Trap-and-Trace Systems
• Use a combination of techniques to detect an intrusion
and trace it back to its source
• Trap usually consists of a honeypot or a padded cell and
alarm.
• Legal drawbacks to trap and trace
– Enticement: act of attracting attention to system by placing
tantalizing information in key locations
– Entrapment: act of luring an individual into committing a
crime to get a conviction
– Enticement is legal and ethical, entrapment is not.

44
 Enticement and Entrapment

Enticement (Legal and ethical):

• Making committing a crime more enticing, but the person has already broken the
law or at least has decided to do so.
• Honeypots can be a good way to use Enticement.
• Have open ports or services on a server that can be attacked.
• Enticement is not a valid defense.

Entrapment (Illegal and unethical):

• When someone is persuaded to commit a crime that they had no intention to
commit and is then charged with it.
• Openly advertising sensitive data and then charging people when they access them.
• Entrapment is a solid legal defense.

If there is a gray area in some cases between enticement and entrapment, it is

ultimately up to the jury to decide if it was one or the other.

45
 Active Intrusion Prevention

• Some organizations implement active

countermeasures.
• One tool (LaBrea) takes up unused IP address space
to pretend to be a computer and allow attackers to
complete a connection request, but then holds
connection open.

46
 Scanning and Analysis Tools (1)

• Scanning tools typically are used to collect information

that an attacker needs to launch a successful attack.
• Attack protocol is a logical sequence of steps or
processes used by an attacker to launch an attack
against a target system or network.
• Footprinting: process of collecting publicly available
information about a potential target

47
 Scanning and Analysis Tools (2)

• Fingerprinting: systematic survey of target organization’s

Internet addresses collected during the footprinting phase
to identify network services offered by hosts in that range
• Fingerprinting reveals useful information about the
internal structure and nature of the target system or
network to be attacked.
• These tools are valuable to the network defender since
they can quickly pinpoint the parts of the systems or
network that need a prompt repair to close vulnerabilities.

48
 Port Scanners

• Tools used by both attackers and defenders to

identify/fingerprint computers active on a network and
other useful information
• Can either perform generic scans or those for specific
types of computers, protocols, or resources
• The more specific the scanner is, the more useful its
information is to attackers and defenders.

49
50
 Firewall Analysis Tools
• Several tools automate remote discovery of firewall rules
and assist the administrator/attacker in analyzing them.
• Administrators who feel wary of using the same tools that
attackers use should remember:
– User intent dictates how gathered information will be
used.
– To defend a computer or network well, administrators
must understand ways it can be attacked.
• A tool that can help close an open or poorly configured
firewall will help the network defender minimize risk from
attack.

51
 Operating System Detection Tools

• Ability to detect a target computer’s operating system

(OS) is very valuable to an attacker.
– Once OS is known, the attacker can easily determine the
vulnerabilities to which it is susceptible.
• Many tools use networking protocols to determine a
remote computer’s OS.

52
 Vulnerability Scanners

• Active vulnerability scanners examine networks for highly

detailed information and initiate traffic to determine
security holes.
• Passive vulnerability scanners listen in on network and
identify the vulnerable versions of both server and client
software.
• Passive vulnerability scanners have the ability to find
client-side vulnerabilities typically not found in active
scanners.

53
 Packet Sniffers
• Network tool that captures copies of packets from
network and analyzes them
• Can provide network administrator with valuable
information for diagnosing and resolving networking
issues
• In the wrong hands, a sniffer can be used to
eavesdrop on network traffic.
• To use packet sniffers legally, an administrator must be
on a network that the organization owns, be under
direct authorization of owners of the network, and have
knowledge and consent of the content’s creators.

54
 Wireless Security Tools

• An organization that spends its time securing a wired

network while ignoring wireless networks is exposing
itself to a security breach.
• Security professionals must assess the risk of wireless
networks.
• A wireless security toolkit should include the ability to
sniff wireless traffic, scan wireless hosts, and assess
the level of privacy or confidentiality afforded on the
wireless network.

55
 Summary
• Intrusion detection system (IDPS) detects violation of its
configuration and activates alarm.
• Network-based IDPS (NIDPS) versus host-based IDPS
(HIDPS)
• Selecting IDPS products that best fit an organization’s
needs is challenging and complex.
• Honeypots are decoy systems; two variations are known
as honeynets and padded cell systems.
• Scanning and analysis tools are used to pinpoint
vulnerabilities in systems, holes in security components,
and unsecured aspects of a network.

56