Learning Processes

Neural Networks
CSE-6701

Submitted By
Shafikul Islam
ID: 20 CSE 010
&
Joy Sarkar
ID: 20 CSE 012
Department of Computer Science & Engineering
University Of Barishal

Course Teacher
Sohely Jahan
Assistant Professor
Department of Computer Science & Engineering
University Of Barishal

BARISHAL UNIVERSITY
BARISHAL, BANGLADESH

3March, 2023
 Learning Processes
 What is Learning Process?

Artificial Neural Network (ANN) is entirely inspired by the way the biological
nervous system works. For Example, the human brain works. The most powerful
attribute of the human brain is to adapt, and ANN acquires similar
characteristics. We should understand that how exactly our brain does? It is still
very primitive, although we have a fundamental understanding of the procedure.
It is accepted that during the learning procedure, the brain's neural structure is
altered, increasing or decreasing the capacity of its synaptic connections relying
on their activity. This is the reason why more relevant information is simpler to
review than information that has not been reviewed for a long time. More
significant information will have powerful synaptic connections, and less
applicable information will gradually have its synaptic connections weaken,
making it harder to review.

A learning rule or Learning process is a technique or a mathematical logic. It

boosts the Artificial Neural Network's performance and implements this rule
over the network. Thus learning rules refreshes the weights and bias levels of a
network when a network mimics in a particular data environment.

The learning (training) process of a neural network is an iterative process in

which the calculations are carried out forward and backward through each layer
in the network until the loss function is minimized.

 Why is it important?

Being a complex adaptive system, learning in ANN implies that a processing

unit is capable of changing its input/output behavior due to the change in
environment. The importance of learning in ANN increases because of the fixed
activation function as well as the input/output vector, when a particular network

2
is constructed. Now to change the input/output behavior, we need to adjust the
weights.

 Basic Learning Mechanism

 Firstly, the neural network is stimulated by the environment.
 It experiences changes in its parameters or internal structure as a result of
the stimulation.
 It responds in a new way to the environmental stimulus or changes
because of these changes which have occurred in its parameters or internal
structure.
 Two Kinds of Learning

Parameter Learning: It involves changing and updating the connecting weights

in the neural network.

Structure Learning: It focuses on changing the structure or architecture of the

neural network.

 Types of learning Processes

3
  Supervised learning
Supervision: The training data (observations, measurements, etc.) are
accompanied by labels indicating the class of the observations. New data is
classified based on the training set
 Unsupervised learning
The class labels of training data are unknown. Given a set of measurements,
observations, etc. with the aim of establishing the existence of classes or
clusters in the data.

 Reinforcement Learning

Reinforcement Learning is a feedback-based Machine learning technique in

which an agent learns to behave in an environment by performing the actions
and seeing the results of actions. For each good action, the agent gets positive
feedback, and for each bad action, the agent gets negative feedback or penalty.

o In Reinforcement Learning, the agent learns automatically using

feedbacks without any labeled data, unlike supervised learning.
o Since there is no labeled data, so the agent is bound to learn by its
experience only.

 Error-Correction Learning

Error correction learning is a type of machine learning algorithm used in neural

networks to adjust the weights of the network's connections in order to
minimize the error between the network's output and the desired output. The
basic mathematical equation used in error correction learning is the delta rule:

Δw = α * (d - y) * x

where:

Δw = the change in weight to be applied to a particular connection

4
 α = the learning rate, which determines how much the weight should be
adjusted in response to each training example

d = the desired output for a particular training example

y = the actual output produced by the network for that training example

x = the input value associated with the connection being adjusted

The delta rule can be applied to each connection in the network to update its
weight, with the overall goal of minimizing the error across all training
examples. By iteratively adjusting the weights in this way, the network
gradually learns to produce more accurate outputs for a given set of inputs.

5
  Memory-based learning 

Memory-based learning (also called instance-based learning) is a type of non-

parametric algorithm that compares new test data with training data in order to
solve the given machine learning problem.  Such algorithms search for the
training data that are most similar to the test data and make predictions based on
these similarities.
A distinguishing feature of memory-based learning is its storage of the entire
training set.  This is computationally costly, especially if the training set is large
– the storage itself is costly, and the complexity of the model grows with a
larger data set.  However, it is advantageous because it uses less assumptions
than parametric models, so it is adaptable to problems for which the
assumptions may fail and no clear pattern is known ex ante.  (In contrast,
parametric models like linear regression make generalizations about the training
data; after building a model to predict the targets, the training data are
discarded, so there is no need to store them.)  Thus, I recommend using

6
memory-based learning algorithms when the data set is relatively small and
there is no prior knowledge or information about the underlying patterns in the
data.
Two classic examples of memory-based learning are K-nearest neighbours
classification and K-nearest neighbours regression.

 Hebbian Learning

Hebbian Learning is a type of unsupervised learning in artificial neural

networks that is based on the principle of Hebb's rule, which states that
"neurons that fire together, wire together." The rule suggests that when two
neurons are activated simultaneously, the connection between them is
strengthened, resulting in a more efficient communication pathway. Conversely,
when a neuron is activated without its corresponding partner neuron being
active, the connection between them is weakened.

In neural networks, Hebbian Learning is often used to adjust the weights

between neurons in a way that reflects the correlation between their inputs. The
basic idea is to increase the strength of connections between neurons that are

7
frequently activated together and decrease the strength of connections between
neurons that are rarely activated together.

There are several variants of Hebbian Learning that have been proposed over
the years, including:

1. Classical Hebbian Learning: In this variant, the weight between two

neurons is increased if both of them are active at the same time, and the
weight is decreased if only one of them is active.

2. Anti-Hebbian Learning: In this variant, the weight between two neurons

is decreased if both of them are active at the same time, and the weight is
increased if only one of them is active.

3. Spike-Timing-Dependent Plasticity (STDP): In this variant, the weight

between two neurons is increased if the presynaptic neuron fires just
before the postsynaptic neuron, and the weight is decreased if the
presynaptic neuron fires just after the postsynaptic neuron.

Hebbian Learning has been used in a wide range of applications, including

image recognition, speech recognition, and natural language processing. One of
the key benefits of Hebbian Learning is that it allows neural networks to learn
patterns and associations in an unsupervised manner, which can be useful in
situations where labeled training data is not available.

Hebbian learning can be implemented in a neural network using the following

steps:

1. Initialize the weights: Set the weights between the input and output
neurons to small random values.

2. Present an input: Present an input pattern to the network.

8
 3. Calculate the output: Calculate the output of the network by applying the
current weights to the input.

4. Update the weights: Update the weights between the input and output
neurons using the Hebbian learning rule, which states that the weight
between two neurons should be increased if they are both active, and
decreased if one is active and the other is not. The Hebbian learning rule
can be expressed mathematically as:

Δw_ij = η * x_i * y_j

where Δw_ij is the change in the weight between neuron i and neuron j, η is the
learning rate, x_i is the activation of neuron i, and y_j is the activation of neuron
j.

5. Repeat: Repeat steps 2-4 for each input pattern in the training set.

6. Test the network: Test the network on new input patterns to evaluate its
performance.

 covariance hypothesis

The covariance hypothesis in Hebbian learning proposes that the strength of the
connection between two neurons should be modified based on the covariance
between their activities. The basic idea is that when two neurons fire together,
the strength of their connection should be increased, and when they fire
independently, the strength of their connection should be decreased. One
advantage of the covariance hypothesis is that it provides a more precise and
nuanced way of adjusting synaptic strengths than the original Hebbian rule,
which simply strengthened connections between neurons that fired together.
The covariance hypothesis takes into account not only whether two neurons fire
together, but also the statistical relationship between their activities.

9
Covariance-based learning has been used in various neural network models and
has been shown to improve their learning performance on certain tasks.
However, it also has some limitations, such as being sensitive to the mean
activity levels of the neurons and being prone to instability if the covariance
matrix becomes singular or close to singular.

The covariance hypothesis in Hebbian learning suggests that the change in

synaptic strength between two neurons should be proportional to the covariance
between their activities. This can be implemented in a neural network using a
learning rule that updates the synaptic weights based on the covariance between
the pre- and post-synaptic activities.

One common form of the covariance-based learning rule is the Oja's rule, which
is given by:

Δwij = η(xiyj - αyj^2*wij)

where Δwij is the change in the synaptic weight between neuron i and j, η is the
learning rate, xi and yj are the pre- and post-synaptic activities, respectively, wij
is the initial synaptic weight, and α is a regularization parameter that controls
the magnitude of the weight update.

In this rule, the first term (xiyj) represents the covariance between the pre- and
post-synaptic activities, while the second term (αyj^2*wij) acts as a decay term
that prevents the weight from growing too large.

The Oja's rule can be used in a variety of neural network models, such as the
self-organizing map and the adaptive resonance theory. When used in
conjunction with other learning rules, such as the backpropagation algorithm,
the covariance-based learning rule can help improve the learning performance
of neural networks, particularly in tasks that require detecting correlations or
patterns in the input data.

10
It's worth noting that the covariance-based learning rule has some limitations
and requires careful tuning of its parameters to avoid instability and overfitting.
Additionally, other methods for implementing the covariance hypothesis in
Hebbian learning have been proposed, such as the BCM (Bienenstock-Cooper-
Munro) rule and the trace learning rule, which have their own strengths and
weaknesses.

 Competitive learning 

Competitive learning is a form of unsupervised learning in artificial Neural

Networks. The nodes compete for the right to respond to a subset of the input
data. Competitive learning works by increasing the specialization of each node
in the network. It is well suited to finding clusters within data.

Examples include: Vector quantization and Kohonen maps (self-organizing

maps)

Principles of Competitive Learning:

There are three basic elements to a competitive learning rule:

 A set of neurons that are all the same, except for some randomly
distributed synaptic weights, which respond differently to a given set of
input patterns

 A limit which is imposed on the "strength" of each neuron.

 A mechanism that permits the neurons to compete for the right to respond
to a given subset of inputs, such that only one output neuron (or only one
neuron per group), is active (i.e. "on") at a time. The neuron that wins the
competition is called a "winner-take-all" neuron.

Individual neurons of the network learn to specialize on ensembles of similar

patterns and become 'feature detectors' for different classes of input patterns.

11
The competitive networks recode sets of correlated inputs to one of a few output
neurons essentially removes the redundancy in representation.

Architecture & Implementation:

Competitive Learning is usually implemented with Neural Networks that

contain a hidden layer which is commonly known as “competitive layer”.

Every competitive neuron is described by a vector of weights :

For every input vector, the competitive neurons “compete” with each other to
see which one of them is the most similar to that particular input vector. The
winner neuron m sets its output to:

All the other competitive neurons set their output to:

Usually, in order to measure similarity the inverse of the Euclidean distance is

used.

Boltzmann Learning: These are stochastic learning processes having recurrent

structure and are the basis of the early optimization techniques used in ANN.

“A surprising feature of this network is that it uses only locally available

information. The change of weight depends only on the behavior of the two
units it connects, even though the change optimizes a global measure” - Ackley,
Hinton 1985.

Some important points about Boltzmann Machine −

● They use recurrent structure.

12
 ● They consist of stochastic neurons, which have one of the two

possible states, either 1 or 0.

● Some of the neurons in this are adaptive freestate and some are

clamped frozenstate
● If we apply simulated annealing on discrete Hopfield network, then

it would become Boltzmann Machine.

The main purpose of the Boltzmann Machine is to optimize the solution of a

problem. It is the work of the Boltzmann Machine to optimize the weights and
quantity related to that particular problem.

Architecture

The following diagram shows the architecture of Boltzmann machine. It is clear

from the diagram, that it is a two-dimensional array of units. Here, weights on
interconnections between units are –p where p > 0. The weights of self-
connections are given by b where b > 0.

Training Algorithm

13
As we know that Boltzmann machines have fixed weights, hence there will be
no training algorithm as we do not need to update the weights in the network.
However, to test the network we have to set the weights as well as to find the
consensus function CF

Boltzmann machine has a set of units Ui and Uj and has bi-directional

connections on them.

● We are considering the fixed weight say wij.

● wij ≠ 0 if Ui and Uj are connected.
● There also exists a symmetry in weighted interconnection, i.e. wij = wji.
● wii also exists, i.e. there would be the self-connection between units.
● For any unit Ui, its state ui would be either 1 or 0.

The main objective of Boltzmann Machine is to maximize the Consensus

Function

CF which can be given by the following relation

Testing Algorithm

Step 1 − Initialize the following to start the training −

Weights representing the constraint of the problem

Control Parameter T

Step 2 − Continue steps 3-8, when the stopping condition is not true.

Step 3 − Perform steps 4-7.

Step 4 − Assume that one of the state has changed the weight and choose the
integer I, J as random values between 1 and n.

14
Step 7 − Accept or reject this change as follows −

Case I − if R < AF, accept the change.

Case II − if R ≥ AF, reject the change.

Here, R is the random number between 0 and 1.

Step 9 − Test for the stopping conditions which may be as follows −

● Temperature reaches a specified value

● There is no change in state for a specified number of iterations

Credit-assignment problem: The credit-assignment problem is the problem of

assigning credit or blame for overall outcomes to each of the internal decisions
made by a learning machine and which contributed to those outcomes. (The
credit assignment problem is also referred to as the loading problem, the
problem of “loading” a given set of training data into the free parameters

15
of the network.)

In many cases, the dependence of outcomes on internal decisions is mediated by

a

sequence of actions taken by the learning machine. In other words, internal

decisions affect which particular actions are taken, and then the actions, not the
internal decisions, directly influence overall outcomes. In these situations, we
may decompose the credit-assignment problem into two subproblems (Sutton,
1984):

1. The assignment of credit for outcomes to actions. This is called the temporal

credit-assignment problem in that it involves the instants of time when the

actions

that deserve credit were actually taken.

2. ‘The assignment of credit for actions to internal decisions. This is called the
structural credit-assignment problem in that it involves assigning credit to the
internal structures of actions generated by the system.

‘The structural credit-assignment problem is relevant in the context of a

multicomponent learning machine when we must determine precisely which
particular component of the system should have its behavior altered and by how
much in order to improve overall system performance. On the other hand, the
temporal credit-assignment problem is relevant when there are many actions

16
taken by a learning machine that result in certain outcomes, and we must
determine which of these actions were responsible for the outcomes, The
combined temporal and structural credit-assignment problem faces any
distributed learning machine that attempts to improve its performance in
situations involving temporally extended behavior (Williams, 1988).

LEARNING WITH A TEACHER: Learning with a teacher, which is also

referred to as supervised learning.

In conceptual terms, we may think of the teacher as having knowledge of the

environment, with that knowledge being represented by a set of input-output
examples.

The environment is, however, the neural network of interest. Suppose now that
the teacher and the neural network are both exposed to a training vector (i...
example) drawn from the environment. By virtue of built-in knowledge, the

17
teacher is able to provide the neural network with a desired response for that
training vector. Indeed, the desired response represents the optimum action to
be performed by the neural network. The network parameters are adjusted under
the combined influence of the training vector and the error signal.

The error signal is defined as the difference between the desired response and
the actual response of the network. This adjustment is carried out iteratively in a
step-by-step fashion with the aim of eventually making the neural network
emulate the teacher; the emulation is presumed to be optimum in some
statistical sense.

In this way knowledge of the environment available to the teacher is transferred

to the neural network through training as fully as possible. When this condition
is reached, we may then dispense with the teacher and let the neural network
deal with the environment com- completely by itself.

It is a closed-loop feedback system, but the unknown environment is not in the

loop. As a performance measure for the system, ‘we may think in terms of the
mean-square error or the sum of squared errors over the training sample,
defined as a function of the free parameters of the system. This function may be
visualized as a multidimensional error-performance surface or simply an error
surface, with the free parameters as coordinates.

The true error surface is averaged over all possible input-output examples. Any
given operation of the system under the teacher's supervision is represented as a
point on the error surface. For the system to improve performance over time and
therefore learn from the teacher, the operating point has to move down
successively toward a minimum point of the error surface; the minimum point
may be a local minimum or a global minimum.

18
LEARNING WITHOUT A TEACHER :

In supervised learning, the learning process takes place under the tutelage of a
teacher.

However, in the paradigm known as learning without a teacher, as the name

implies,

there is no teacher to oversee the learning process. That is to say, there are no
labeled

examples of the function to be learned by the network. Under this second

paradigm,

two subdivisions are identified:

Reinforcement learning/Neurodynamic programming:

In reinforcement learning, the learning of an input-output mapping is performed

through continued interaction with the environment in order to minimize a

scalar

index of performance.

19
Figure 2.7 shows the block diagram of one form of a reinforcement Jearning
system built around a critic that converts a primary reinforcement signal
received from the environment into a higher quality reinforcement signal called
the heuristic reinforcement signal, both of which are scalar inputs. The system is
designed to learn under delayed reinforcement, which means that the system
observes a temporal sequence of stimuli (i.e., state vectors) also received from
the environment, which eventually result in the generation of the heuristic
reinforcement signal. The goal of learning is to minimize a cost-fo-go function,
defined as the expectation of the cumulative cost of actions taken over a
sequence of steps instead of simply the immediate cost. It may turn out that
certain actions taken earlier in that sequence of time steps are in fact the best
determinants of overall system behavior. The function of the learning machine,
which constitutes the second component of the system, is to discover these
actions and to feed them back to the environment.

Delayed-reinforcement learning is difficult to perform for two basic reasons:

1. There is no teacher to provide a desired response at each step of the

learning

20
process.

2. The delay incurred in the generation of the primary reinforcement signal

implies
that the learning machine must solve a temporal credit assignment problem. By

this we mean that the learning machine must be able to assign credit and blame

individually to each action in the sequence of time steps that led to the final out-

come, while the primary reinforcement may only evaluate the outcome.

Unsupervised Learning: In unsupervised or self-organized learning there is no

external teacher or critic to oversee the learning process, as indicated in Fig. 2.8.

Rather, provision is made for a task-independent measure of the quality of

representation that the network is required to learn, and the free parameters of
the network are optimized with respect to that measure. Once the network has
become tuned to the statistical regularities of the input data, it develops the
ability to form internal representations for encoding features of the input and
thereby create new classes automatically.

To perform unsupervised learning we may use a competitive learning rule. For

example, we may use a neural network that consists of two layers—an input
layer and a competitive layer. The input layer receives the available data. The
21
competitive layer consists of neurons that compete with each other (in
accordance with a learning rule) for the “opportunity” to respond to features
contained in the input data. In its simplest form, the network operates in
accordance with a “winner-takes-all” strategy.

LEARNING TASKS:

1. Pattern Association: An associative memory is a brainlike distributed

memory that learns by association. Association has been known to be a
prominent feature of human memory.
Association takes one of two forms: auto association or hetero
association. In auto association, a neural network is required to store a set
of patterns (vectors) by repeatedly presenting them to the network. The
network is subsequently presented as a partial description or distorted
(noisy) version of an original pattern stored in it, and the task is to
retrieve (recall) that particular pattern. Heteroassociation differs from
auto-association in that an arbitrary set of input patterns is paired with
another arbitrary set of output patterns. The auto association involves the
use of unsupervised learning, whereas the type of learning involved in the
hetero association is supervised.

Pattern Recognition: Humans are good at pattern recognition. We

receive data from the world around us via our senses and are able to
recognize the source of the data. We are often able to do 50 almost

22
immediately and with practically no effort. Humans perform pattern
recognition through a learning process: so it is with neural networks.

Pattern_recognition is formally defined as the process whereby a received

pattern signal is assigned to one of a prescribed number of classes

(categories). A neural network performs pattern recognition by first
undergoing a training session. During this, the network has repeatedly
presented a set of input patterns along with the category to which each
particular pattern belongs. Later. a new pattern is presented to the
network that has not been scen before, but which belongs to the same
population of patterns used to train the network. The network is able to
identify the class of that particular pattern because of the information it
has extracted from the training date.

Pattern recognition performed by a neural network is statistical in nature,

with the patterns being represented by points in a multidimensional
decision space. The decision space is divided into regions, each one of
which is associated with a class. The decision boundaries are determined
by the training process. The construction of these boundaries is made
statistical by the inherent variability that exists within and between
classes.

23
Function approximation:

Function approximation is a critical task in neural networks because it

allows the network to learn to map inputs to outputs. Here are some
specific reasons why function approximation is important in neural
networks:

● It enables learning: By approximating the function that maps inputs

to outputs, a neural network can learn to perform a specific task.
This is done through adjusting the weights and biases of the
network during training to minimize the error between the
predicted and actual outputs.

● It allows for complex modeling: Neural networks are capable of

approximating complex functions that may be difficult or
impossible to model with traditional methods. This makes them
well-suited for tasks such as image recognition, natural language
processing, and speech recognition.
● It enables generalization: Function approximation allows a neural
network to generalize its learned behavior to new, unseen data. By
learning the underlying function that maps inputs to outputs, the
network can make accurate predictions on data it has not seen
before.
● It allows for transfer learning: Function approximation can be used
as a starting point for transfer learning, which involves using a pre-
trained network as a starting point for a new task. By
approximating a function on a related task, the network can then be
fine-tuned for the new task with less data and time required than
training a new network from scratch.

24
 Overall, function approximation is a fundamental task in neural networks
that enables learning, complex modeling, generalization, and transfer
learning.

The ability of a neural network to approximate an unknown input-output

mapping may be exploited in two important ways:

1. System identification.

2. Inverse system:

Control:
25
Control is an important aspect of neural networks as it allows for the efficient
and effective operation of the network in achieving its goals. Here are some
specific reasons why control is important in neural networks:

● It ensures stability: Control can be used to stabilize the behavior of a

neural network. This can be achieved through techniques such as
feedback control, where the output of the network is used to adjust its
inputs or parameters to ensure that it behaves in a stable manner.
● It improves performance: Control can be used to optimize the
performance of a neural network by adjusting its parameters in real-time
to achieve the desired output. This is particularly important for tasks that
require the network to operate in dynamic and changing environments.
● It enables adaptability: Control can be used to make a neural network
adaptable to changes in its environment or task requirements. For
example, the network can be reconfigured or its parameters adjusted to
accommodate changes in the input data or to improve its performance on
a specific task.
● It ensures safety: Control can be used to ensure the safe operation of a
neural network in critical applications. For example, in autonomous
vehicles, control can be used to ensure that the vehicle operates safely
and avoids collisions.

26
Overall, control is an important aspect of neural networks that ensures stability,
improves performance, enables adaptability, and ensures safety in various
applications.

Filtering: ‘The term filter often refers to a device or algorithm used to extract
information about a prescribed quantity of interest from a set of noisy data. The
noise may arise from a variety of sources. For example, the data may have been
measured by means of noisy sensors or may represent an information-bearing
signal that has been corrupted by transmission through a communication
channel. Another example is that of a useful signal component corrupted by an
interfering signal picked up from the surrounding environment. We may use a
filter to perform three basic information processing tasks:

1. Filtering. This task refers to the extraction of information about a quantity

of
interest at discrete time 7 by using data measured up to and including
time .

2. Smoothing. This second task differs from filtering in that information

about the
quantity of interest need not be available at time n, and data measured
later than time n can be used in obtaining this information. This means
that in smoothing there is a delay in producing the result of interest.
Since, in the smoothing process, we are able to use data obtained not only
up to time 7 but also after time n, we expect smoothing to be more
accurate than filtering in some statistical sense.

3. Prediction. This task is the forecasting side of information processing.

The aim

27
 here is to derive information about what the quantity of interest will be
like at some time 7 + nin the future, for some ny > 0. by using data
measured up to

and including time n.

Beamforming: Beamforming is a spatial form of filtering and is used to

distinguish between the spatial properties of a target signal and background
noise. The device used to do the beamforming is called a beamformer.

Beamforming is commonly used in radar and sonar systems where the primary

task is 10 detect and track a target of interest in the combined presence of

receiver

noise and interfering signals (e.g. jammer). This task is complicated by two
factors.

1. The target signal originates from an unknown direction.

2. There is no a priori information available on the interfering signals.

MEMORY:

In a neurobiological context, memory refers to the relatively enduring neural

alterations indueed by the interaction of an organism with its environment.
Without such a change there can be no memory.

Furthermore, for the memory to be useful it must be accessible to the nervous

system in order to influence future behavior. However, an activity pattern must
initially be stored in memory through a learning process. Memory and learning
are intricately con-nected. When a particular activity pattern is learned, it is
stored in the brain where it can be, recalled later when required. Memory may

28
be divided into “short-term” and “long-term” memory, depending on the
retention time .

Short-term memory refers to a compilation of knowledge representing the

“current” state of the environment. Any discrepancies between knowledge
stored in short-term memory and a “new” state are used to update the short-term
memory. Long-term memory, on the other hand, refers to knowledge stored for
a long time or permanently.

In this section we study an associative memory that offers the following charac-

Teristics:

● The memory is distributed.

● Both the stimulus (key) pattern and the response (stored) pattern of an
associa-
tive memory consist of data vectors

● Information is stored in memory by setting up a spatial pattern of neural

activi-
ties across a large number of neurons.

● Information contained in a stimulus not only determines its storage

location in
memory but also an address for its retrieval.

● Although neurons do not represent reliable and low-noise computing

cells, the
‘memory exhibits a high degree of resistance to noise and damage of a
diffusive kind.

● There may be interactions between individual patterns stored in memory.

29
(Otherwise the memory would have to be exceptionally large for it to
accommo-

date the storage of a large number of patterns in perfect isolation from

cach

other.) There is therefore the distinct possibilty for the memory to make
errors

during the recall process

30