1

Self-Organizing Neural Networks (SONNs)

Introduction
Self-Organizing Neural Networks (SONNs) are a class of unsupervised learning models
designed to discover patterns or structures in input data without needing labeled outputs. The
most common and well-known SONN is the Self-Organizing Map (SOM), introduced by
Teuvo Kohonen, so SONNs are often referred to as Kohonen networks.
Self-Organizing Neural Networks (SONNs) or Self Organizing Map (or Kohonen Map or
SOM) is a type of Artificial Neural Network which is also inspired by biological models of
neural systems from the 1970s. It follows an unsupervised learning approach and trained its
network through a competitive learning algorithm. SOM is used for clustering and mapping (or
dimensionality reduction) techniques to map multidimensional data onto lower-dimensional
which allows people to reduce complex problems for easy interpretation. SOM has two layers,
one is the Input layer and the other one is the Output layer. Self-organization is the process by
which a system automatically adjusts its structure based on the input it receives, without
external supervision. In neural networks, this means adjusting the neurons' weights to reflect
patterns in the input data
A SONN, particularly the Self-Organizing Map, is a type of neural network that projects high-
dimensional input data into a low-dimensional (usually 2D) grid, while preserving the
topological structure of the data.
 It is unsupervised: It doesn't require labeled data.
 It is competitive: Neurons compete to be the "winner".
 It is topology-preserving: Nearby neurons in the map respond to similar inputs.
Key Components and Layers
SONNs are generally two-layered:
1. Input Layer
 Contains nodes (neurons) equal to the number of features in the input data.
 It feeds data to the neurons in the output layer.
2. Output Layer (Map Layer)
 Typically arranged in a 2D grid (e.g., 10x10).
 Each node (neuron) has a weight vector of the same dimension as the input vector.
 These neurons are connected to all inputs, forming a fully connected layer.
The architecture of the Self Organizing Map with two clusters and n input features of any sample
is given below:
 2

How do SOM works?

Let’s say an input data of size (m, n) where m is the number of training examples and n is the
number of features in each example. First, it initializes the weights of size (n, C) where C is the
number of clusters. Then iterating over the input data, for each training example, it updates the
winning vector (weight vector with the shortest distance (e.g Euclidean distance) from training
example). Weight updation rule is given by :
wij = wij(old) + alpha(t) * (xik - wij(old))
where alpha is a learning rate at time t, j denotes the winning vector, i denotes the ith feature of
training example and k denotes the kth training example from the input data. After training the
SOM network, trained weights are used for clustering new examples. A new example falls in the
cluster of winning vectors.
Algorithm
Training:
Step 1: Initialize the weights wij random value may be assumed. Initialize the learning rate α.
Step 2: Calculate squared Euclidean distance.
D(j) = Σ (wij – xi)^2 where i=1 to n and j=1 to m
Step 3: Find index J, when D(j) is minimum that will be considered as winning index.
Step 4: For each j within a specific neighborhood of j and for all i, calculate the new weight.
wij(new)=wij(old) + α[xi – wij(old)]
Step 5: Update the learning rule by using :
α(t+1) = 0.5 * t
Step 6: Test the Stopping Condition.
Below is the implementation of the above approach:

2.2 Key Characteristics

 Unsupervised learning: No labels are needed.
 Topology preservation: Neighboring input patterns are mapped to neighboring neurons.
 Competitive learning: Neurons compete to respond to input patterns.
3. Self-Organizing Map (SOM)
3.1 Architecture
The architecture of the Self Organizing Map with two clusters and n input features of any
sample is given below:
 3

 A 2D grid (or sometimes 1D/3D) of neurons.

 Each neuron has an associated weight vector of the same dimension as the input vector.
 Neurons are arranged in a topological neighborhood.
Components
Component Description
Vector of continuous real-valued features (e.g., RGB color, sensor
Input Layer
data, etc.)
1D or 2D lattice (most common: 2D) where each node (neuron)
Map/Grid of Neurons
represents a prototype vector (weight vector)
Each neuron has an associated weight vector with the same dimension
Weight Vectors
as the input
Neurons have a spatial relationship (defined by their position on the
Topology/Neighborhood
grid)

How do SOM works?

Let’s say an input data of size (m, n) where m is the number of training examples and n is the
number of features in each example. First, it initializes the weights of size (n, C) where C is the
number of clusters. Then iterating over the input data, for each training example, it updates the
winning vector (weight vector with the shortest distance (e.g Euclidean distance) from training
example). Weight updation rule is given by :
wij = wij(old) + alpha(t) * (x ik - wij(old))
where alpha is a learning rate at time t, j denotes the winning vector, i denotes the ith feature of
training example and k denotes the kth training example from the input data. After training the
SOM network, trained weights are used for clustering new examples. A new example falls in the
cluster of winning vectors.
4. Neighborhood and Topology
4.1 Neighborhood Function
Defines how much influence a neuron has over its neighbors during training. This preserves
topology, i.e., neighboring neurons in the grid learn similar input features.
Common types:
 Gaussian
 Bubble
 4

 Mexican hat
4.2 Grid Topologies
 Rectangular grid: Neurons arranged in square blocks.
 Hexagonal grid: Neurons arranged with 6 immediate neighbors—better topological
properties.
How It Works: Training Algorithm
Here's a step-by-step explanation of how SONNs (like SOM) learn:
Step 1: Initialize
 Each neuron in the map is assigned a random weight vector (same size as input vector).
Step 2: Input a Data Vector
 Present an input vector X to the network.
Step 3: Find Best Matching Unit (BMU)
 Compute the Euclidean distance between X and all the neuron weight vectors.
 The neuron with the smallest distance is the BMU (winner).

Step 4: Update Weights

 Update the weights of the BMU and its neighboring neurons to be closer to X.
 The update rule is:
w(t+1)=w(t)+α(t)⋅h(t)⋅(X−w(t))
Where:
o α(t)= learning rate (decreases over time)
o h(t)= neighborhood function (e.g., Gaussian centered on BMU)
o w(t) = current weights
Step 5: Repeat
o Repeat for many epochs, gradually reducing the learning rate and neighborhood
size. Repeat steps 2-4 for many iterations, decreasing learning rate and
neighborhood size over time.
Example: Visualizing the Process
Scenario: Customer Segmentation
Suppose we have a dataset of customers described by 4 features:
 Age
 Annual Income
 Spending Score
 Savings
Step-by-Step:
1. Input vector: [32, 55000, 60, 12000]
2. Input layer: 4 nodes (one per feature).
3. Map layer: 10x10 grid (100 neurons), each with a 4-dimensional weight vector.
4. Find the BMU among the 100 neurons.
5. Update weights of the BMU and its neighbors.
 5

6. After many iterations, clusters emerge on the map, such as:

o High income, low spending → “Savers”
o Low income, high spending → “Spenders”
These clusters are visual and interpretable.
Use Case: Network Intrusion Detection System (NIDS)
Objective:
To detect abnormal patterns in network traffic that could indicate a cyberattack or intrusion,
using SONNs for unsupervised anomaly detection.
Scenario Setup:
A system administrator wants to monitor the internal traffic of a corporate network and identify
suspicious or malicious activity. However, not all types of attacks are known beforehand, and
labeling data is expensive and incomplete.
Solution:
Use a Self-Organizing Map (SOM) to model the normal behavior of the network and detect
anomalies as deviations from that behavior.
Input Data Features:
Each input vector represents a snapshot of a network connection/session and includes features
like:
 Protocol type (TCP/UDP/ICMP)
 Packet size
 Duration
 Number of connections per minute
 Bytes sent/received
 Flag (status of session)
Each session → a vector like:
X=[1,1500,2.3,10,2048,1]X = [1, 1500, 2.3, 10, 2048, 1]X=[1,1500,2.3,10,2048,1]
(Values may be normalized and encoded appropriately)
How the SONN Works Here:
1. Input Layer: Each input vector has, say, 6 features.
2. Output Layer: A 15x15 grid of neurons (225 nodes).
3. Training Phase:
o Feed in normal (clean) network traffic.
o The SOM clusters similar traffic together based on their statistical properties.
4. Detection Phase:
o New traffic is mapped onto the SOM.
o If the Best Matching Unit (BMU) is in a rare or previously unused area of the
map, it's flagged as anomalous.
Result:
 Normal traffic maps to dense clusters in some zones of the grid.
 Abnormal or attack traffic lands in sparse or untrained regions → treated as suspicious.
 6

Example Detection:
 DDoS attack sessions may generate features that land far from typical web browsing
sessions.
 Port scanning behavior (many small, fast sessions) may cluster in a different zone.
Advantages of SONNs in Intrusion Detection:
 No prior knowledge of attack types is needed.
 Visual clustering allows the admin to explore traffic patterns.
 Unsupervised learning makes it scalable and adaptable to evolving threats.
Visualization (Example):
You could visualize the SOM as a 2D heatmap:
[ ] [ ] [ ] [ ] [ ] [!] [ ] [ ]
[][][][][][][][]
[ ] [ ] [N] [N] [N] [ ] [ ] [ ]
[ ] [ ] [N] [N] [N] [ ] [ ] [ ]
[][][][][][][][]
 [N] = Cluster of normal traffic
 [!] = Isolated node with anomalous (potentially malicious) traffic
4. Example: Clustering Colors
4.1 Task
Cluster RGB color vectors into a 2D map to group similar colors together.
4.2 Implementation
 Input: 3D RGB vectors, e.g., [255, 0, 0] for red.
 Map: 10x10 SOM grid.
 Result: Each neuron learns to represent a color. Similar colors are grouped together.
4.3 Visualization
Imagine a 10x10 grid where:
 Top-left shows red shades.
 Bottom-right shows blue-green shades.
 Neighboring colors are perceptually similar.
5. Mathematical Insights
5.1 Distance Metrics
 Euclidean distance:


5.2 Neighborhood Function
 Gaussian:

where:
o rb and rj are positions of BMU and neuron jon the grid.
 7

o σ(t): width of the neighborhood function, decreasing with time.

Self-Organizing Neural networks are particularly useful for:
 Pattern recognition
 Dimensionality reduction
 Clustering
 Feature mapping
 Data visualization
1. Pattern Recognition
Example: Handwriting Recognition
 Scenario: Identifying handwritten digits from the MNIST dataset.
o Task: Train a Self-Organizing Map (SOM) on a dataset of handwritten digits,
such as MNIST, where each digit (0-9) is represented as a 28x28 pixel grayscale
image.
o Process: The SOM will organize itself in such a way that similar handwriting
styles (e.g., the number "3" written by different people) are grouped together on
the map.
o Result: The trained SOM can recognize patterns in the data, identifying which
neurons represent different digits or styles of writing. This self-organizing
property allows it to generalize handwriting styles for recognition without needing
labeled data.
2. Dimensionality Reduction
Example: Reducing the Dimensionality of Iris Dataset
 Scenario: Reducing the 4D feature space (sepal length, sepal width, petal length, petal
width) of the Iris dataset for visualization and analysis.
o Task: Use a Self-Organizing Map (SOM) to reduce the dimensionality of the 4D
input features into a 2D grid of neurons.
o Process: Each neuron on the 2D map will learn to represent a group of similar
flowers. In the case of the Iris dataset, these clusters will typically correspond to
different types of Iris flowers (setosa, versicolor, virginica).
o Result: The map provides a visual reduction of the original 4D space into a 2D
representation while preserving the topology of the data.
3. Clustering
Example: Customer Segmentation in Marketing
 Scenario: Segmenting customers based on purchasing behavior for targeted marketing.
o Task: Use a SOM to group customers based on features such as the total amount
spent, frequency of purchases, and product categories.
o Process: The SOM will map customers with similar spending patterns close to
each other on the grid, forming clusters of similar customers.
 8

o Result: After training, the SOM can be analyzed to identify distinct customer
segments. For instance, some neurons may correspond to high-spending
customers, while others correspond to price-sensitive buyers.
4. Feature Mapping
Example: Mapping High-dimensional Sensor Data to a 2D Grid
 Scenario: A robot uses multiple sensors to measure various environmental features, and
we want to map this high-dimensional data to a 2D grid.
o Task: Use a SOM to map high-dimensional sensor inputs (e.g., temperature,
humidity, distance, light intensity) to a 2D grid of neurons.
o Process: As the SOM trains, it learns to organize the sensor data into a topology-
preserving map. Neurons in the same region of the map may correspond to similar
sensor data (e.g., regions with low temperature and high humidity).
o Result: The 2D map can be used to identify specific environmental patterns, such
as zones of high humidity or areas with particular light intensity, which can assist
in robot navigation or environment monitoring.
5. Data Visualization
Example: Visualizing High-dimensional Data of Wine Types
 Scenario: Visualizing the relationships between different types of wines based on their
chemical composition.
o Task: Apply SOM to the chemical features of various wines, which typically
involve measurements like alcohol content, acidity, pH, and phenolic compounds.
o Process: After training the SOM on this data, the neurons in the SOM grid will
represent different types of wines. Similar wines will be grouped close together
on the map, preserving the relationship between their chemical properties.
o Result: The SOM can be visualized as a 2D map where wines with similar
chemical properties are located near each other, helping researchers or
winemakers identify patterns and trends.
These examples illustrate how Self-Organizing Neural Networks can be applied across various
domains for pattern recognition, dimensionality reduction, clustering, feature mapping, and
data visualization. The key strength of SONNs lies in their ability to process and reveal patterns
in data without the need for labeled information, making them a versatile tool for exploratory
data analysis and unsupervised learning tasks.
Advantages and Limitations
Advantages
 Topology preservation: Similar inputs map to nearby neurons.
 No need for labeled data
 Dimensionality reduction: High-dimensional input is visualized in 1D or 2D.
 Intuitive visualization: Especially useful for exploring unknown data.
Limitations
 Sensitive to initialization
 9

May converge to local minima

 Parameter tuning (learning rate, neighborhood size) is critical
Summary
Feature SOM Characteristic
Input type Numerical vectors
Neuron structure Grid-based, each with a weight vector
Learning type Unsupervised, competitive learning
Output Topology-preserving 2D map
Common use cases Clustering, visualization, data mapping
Self-Organizing Neural Networks like SOMs are powerful tools for unsupervised learning
and data visualization.They rely on competitive learning and topological mapping. Widely
used in many real-world applications where interpretability and pattern discovery are crucial.