Intrusion Detection

Jie Lin
Outline
Introduction
A Frame for Intrusion Detection System
Intrusion Detection Techniques
Ideas for Improving Intrusion Detection
What is the Intrusion Detection
Intrusions are the activities that violate the
security policy of system.
Intrusion Detection is the process used to
identify intrusions.
Types of Intrusion Detection System(1)
Based on the sources of the audit information
used by each IDS, the IDSs may be classified
into
Host-base IDSs
Distributed IDSs
Network-based IDSs
Host-based IDSs
Get audit data from host audit trails.
Detect attacks against a single host
Distributed IDSs
Gather audit data from multiple host and possibly the
network that connects the hosts
Detect attacks involving multiple hosts
Network-Based IDSs
Use network traffic as the audit data source, relieving
the burden on the hosts that usually provide normal
computing services
Detect attacks from network.

Types of Intrusion Detection System(2)
Intrusion Detection
Techniques
Misuse detection
Catch the intrusions in terms of the
characteristics of known attacks or system
vulnerabilities.
Anomaly detection
Detect any action that significantly deviates
from the normal behavior.
Misuse Detection
Based on known attack actions.
Feature extract from known intrusions
Integrate the Human knowledge.
The rules are pre-defined
Disadvantage:
Cannot detect novel or unknown attacks
Misuse Detection Methods & System
Method System
Rule-based Languages RUSSEL,P-BEST
State Transition Analysis STAT
family(STAT,USTAT,NS
TAT,NetSTAT)
Colored Petri Automata IDIOT
Expert System IDES,NIDX,P-
BEST,ISOA
Case Based reasoning AutiGUARD
Anomaly Detection
Based on the normal behavior of a subject.
Sometime assume the training audit data
does not include intrusion data.
Any action that significantly deviates from
the normal behavior is considered intrusion.

Anomaly Detection Methods & System
Method System
Statistical method IDES, NIDES, EMERALD
Machine Learning techniques
Time-Based inductive Machine
Instance Based Learning
Neural Network

Data mining approaches JAM, MADAM ID
Anomaly Detection Disadvantages
Based on audit data collected over a period
of normal operation.
When a noise(intrusion) data in the training
data, it will make a mis-classification.
How to decide the features to be used. The
features are usually decided by domain
experts. It may be not completely.
Misuse Detection vs. Anomaly Detection
Advantage Disadvantage
Misuse
Detection
Accurately and
generate much
fewer false alarm
Cannot detect
novel or unknown
attacks
Anomaly
Detection
Is able to detect
unknown attacks
based on audit
High false-alarm
and limited by
training data.
The Frame for Intrusion
Detection
Intrusion Detection Approaches
1. Define and extract the features of behavior
in system
2. Define and extract the Rules of Intrusion
3. Apply the rules to detect the intrusion
Training
Audit Data
Features Rules
Audit Data
Pattern matching
or Classification
1
3
3 2
Thinking about The Intrusion
Detection System
Intrusion Detection system is a pattern
discover and pattern recognition system.
The Pattern (Rule) is the most important
part in the Intrusion Detection System
Pattern(Rule) Expression
Pattern(Rule) Discover
Pattern Matching & Pattern Recognition.
Pattern
Extraction
Traning
Audit
Data
Feature
Extraction
Training
Data &
Knowled
ge
Machine
Learning &
Data
mining &
Statistics
methods
Expert
Knowledge
& Rule
collection
& Rule
abstraction
Pattern &
Decision
Rule
Intrusion
Detection
System
Pattern
Matching
Pattern
Recognition
Discriminate
function
Real-Time
Aduit data
Alarms
Pass
Rule Discover Method
Expert System
Measure Based method
Statistical method
Information-Theoretic Measures
Outlier analysis
Discovery Association Rules
Classification
Cluster
Pattern Matching & Pattern
Recognition Methods
Pattern Matching
State Transition & Automata Analysis
Case Based reasoning
Expert System
Measure Based method
Statistical method
Information-Theoretic Measures
Outlier analysis
Association Pattern
Machine Learning method

Intrusion Detection Techniques
Intrusion Detection Techniques
Pattern Matching
Measure Based method
Data Mining method
Machine Learning Method
Pattern Matching
KMP-Multiple patterns matching Algorithm
Using keyword tree to search
Building failure link to guarantee linear time searching
Shift-And(Or) pattern matching Algorithm
A classical approximate pattern matching algorithm
Karp-Rabin fingerprint method
Using the Modular arithmetic and Remainder theorem
to match pattern
(Such as regular expression pattern
matching)

Measure Based Method
Statistical Methods &
Information-Theoretic Measures
Define a set of measures to measure different
aspects of a subject of behavior. (Define Pattern)
Generate an overall measure to reflect the
abnormality of the behavior. For example:
statistic T
2
= M
1
2
+M
2
2
++M
n
2
weighted intrusion score = M
i
*W
i
Entropy: H(X|Y)=

P(X|Y) (-log(P(X|Y)))

Define the threshold for the overall measure
Association Pattern Discover
Goal is to derive multi-feature (attribute)
correlations from a set of records.
An expression of an association pattern:
The Pattern Discover Algorithm:
1. Apriori Algorithm
2. FP(frequent pattern)-Tree
Association Pattern Example
Association Pattern Detecting
Statistics Approaches
Constructing temporal statistical features from
discovered pattern.
Using measure-based method to detect intrusion
Pattern Matching
Nobody discuss this idea.
Machine Learning Method
Time-Based Inductive Machine
Like Bayes Network, use the probability and a
direct graph to predict the next event
Instance Based Learning
Define a distance to measure the similarity
between feature vectors
Neural Network

Classification
This is supervised learning. The class will
be predetermined in training phase.
Define the character of classes in training
phase.
A common approach in pattern recognition
system


Clustering
This is unsupervised learning. There are not
predetermined classes in data.
Given a set of measurement, the aim is that
establishes the class or group in the data. It
will output the character of each class or
group.
In the detection phase, this method will get
more time cost (O(n
2
)). I suggest this
method only use in pattern discover phase
Ideas for improving Intrusion
Detection
Idea 1: Association Pattern Detecting
Using the pattern matching algorithm to
match the pattern in sequent data for
detecting intrusion. No necessary to construct
the measure.
But its time cost is depend on the number of
association patterns.
It possible constructs a pattern tree to
improve the pattern matching time cost to
linear time
Idea 2: Discover Pattern from Rules
The exist rules are the knowledge from experts
knowledge or other system.
The different methods will measure different
aspects of intrusions.
Combine these rules may find other new patterns of
unknown attack.
For example:
Snort has a set of rule which come from different people.
The rules may have different aspects of intrusions.
We can use the data mining or machine learning method
to discover the pattern from these rule.
Pattern
Extraction
Traning
Audit
Data
Feature
Extraction
Training
Data &
Knowled
ge
Machine
Learning &
Data
mining &
Statistics
methods
Expert
Knowledge
& Rule
collection
& Rule
abstraction
Pattern &
Decision
Rule
Intrusion
Detection
System
Pattern
Matching
Pattern
Recognition
Discriminate
function
Real-Time
Aduit data
Alarms
Pass
Reference
Lee, W., & Stolfo, S.J. (2000). A framework for constructing features and
models for intrusion detection systems. ACM Transactions on Information and
System Security, 3 (4) (pp. 227-261).
Jian Pei,Data Mining for Intrusion Detection:Techniques,Applications and
Systems, Proceedings of the 20th International Conference on Data Engineering
(ICDE 04)
Peng Ning and Sushil Jajodia,Intrusion Detection Techniques. From
http://discovery.csc.ncsu.edu/Courses/csc774-S03/IDTechniques.pdf
Snort---The open source intrusion detection system. (2002). Retrieved February
13, 2003, from http://www.snort.org.
Thank you!