The Memory System

Fundamental Concepts

Basic Concepts

Memory Addressing
The maximum size of the Main Memory
(MM) that can be used in any computer
is determined by its addressing
scheme.
The addressing capability of the system
depends on the No. of address lines it
has
Ex: 220 = 1M memory locations
232 =4 G memory locations

If the smallest addressable unit of

information is a memory word,
the machine is called wordaddressable. If individual memory
bytes
are
assigned
distinct
addresses, the computer is called
byte-addressable. Most of the
commercial machines are byteaddressable. For example in a
byte-addressable
32-bit
computer, each memory word

Word length: number of bits actually

stored or retrieved in one memory
access .

Word Address
0

Byte Address

0 1 2 3

4 5 6 7

8 9 1011

..

Some basic concepts

CPU-Main Memory Connection

Processor
MAR
MDR

kbit
addressbus
nbit
databus

Memory

Upto2 kaddressable
locations
Wordlength= nbits

Controllines
R /W
(,MFC,etc.)

Some basic
concepts(Contd.,)
Measures for the speed of a memory:
memory access time.
memory cycle time.

Memory Access Times

The time that elapses between the initiation of an
operation and the completion of that operation
(for example, the time between READ and MFC).
It is a useful measure of the speed of the memory
unit.

Memory Cycle Time

The minimum time delay required between
the initiations of two successive memory
operations (for example, the time between
two successive READ operations). The cycle
time is usually slightly longer than the access
time.
It is an important measure of the memory
system.

An important design issue is to

provide a computer system with as
large and fast a memory as possible,
within a given cost target.
Several techniques to increase the
effective size and speed of the
memory:
Cache memory (to increase the effective speed).
Virtual memory (to increase the effective size).

Virtual Memory
In a virtual memory System, the address
generated by the CPU is referred to as a
virtual or logical address.
The corresponding physical address can be
different and the required mapping is
implemented by a special memory control
unit, often called the memory management
unit.

The Memory System

Semiconductor RAM
memories

Internal organization of Semiconductor

memory chips

Each memory cell can hold one bit of information.

Memory cells are organized in the form of an array.
One row is one memory word.
All cells of a row are connected to a common line,
known as the word line.
Word line is connected to the address decoder.
Each cell in the column is connected to Sense/write
circuits by two bit lines
Sense/write circuits are connected to the data
input/output lines of the memory chip.

organization of bit cells in memory chip

memory chip of 16 words of 8 bit each -16X8
7

FF
A0

A2

W1
Address
decoder

FF

A1

W0

Memory
cells

Sense/Write
circuit

Sense/Write
circuit

R /W

b1

b0

A3

W15

Sense/Write
circuit

Datainput /outputlines:

b7

CS

READ operation: the Sense/Write circuits

sense, or read, the information stored in
the cells selected by a word line and
transmit this information to the output lines.
write operation: the Sense/Write circuits
receive input information and store it in the
cells of the selected word

The data input and the data output of each

Sense/Write circuit are connected to a
single bi-directional data line that can be
connected to the data bus
R/W (Read/Write) input is used a specify the
required operation
control line: the CS (Chip Select) input is
used to select a given chip in a multichip
memory system

Internal Organization of
Memory Chip
It stores 128 bits.
Requires 14 external connections for
address, data and control lines
And two lines for power supply and
ground connections.
Total of 16 connections.

1K X 1 Memory Chip
Organization
10 bit address is required.
5 bits each to form the row and column
address.
A row address selects a row of 32 cells, all of
which are accessed in parallel.
One of these cells, selected by the column
address, is connected to the external data
lines by the input demultiplexer and output
multiplexers.

5-Bit
Decoder

32X32
memory cell
array

10
B
I

s/w
ckt

T
A
D
D

Two 32 to 1

Multiplexers

(input & 1 output)

S
S

R/w
CS

SRAM Cell

Static Memories -Consists circuits Capable of retaining their state

as long as power is applied.

T1

T2

Wordline
Bitlines

2 inverters are cross-connected to form a latch.

Two transistor inverters are cross connected to
implement a basic flip-flop.
The latch is connected to one word line and two bits
lines by transistors T1 and T2
T1 and T2 acts as switches can be opened or closed
under the control of word line.
When word line is at ground level, the transistors are
turned off and the latch retains its state Ex:cell is in
state 1 if logic value at point X is 1 and 0 at Y
b

T1

T2

Wordline
Bitlines

Read Operation
Word line is activated to close switches
T1 and T2.
If cell is in state 1, the signal on bit line
b is high and b1 is low.
Sense/write circuits at the end of the bit
lines monitor the state of b and b 1 and
set the output accordingly

Write Operation
The state of the cell is set by placing the
appropriate value on bit b and its
complement on b1, and then activating
the wordline.
This forces the
corresponding state

cell

into

the

The required signals on the bit lines are

generated by the Sense/Write circuit.

CMOS(Complementary metaloxidesemiconductor)
Memory Cell

CMOS Memory Cell (Contd..)

MOS technology is used extensively in Main Memory Units.
Transistor pairs ( t1,t3) and ( t4,t2) form the inverter in the latch.
State of the cell is read or write.
Ex: in state 1, voltage at point X is high by having transistors
(t3,t2) on while t4,t1 are off.
Transistors T5 and T6 act as switches that can be opened or
closed under control of the word line.
Continuous power is needed in cell to retain its state.
Hence, SRAMs are volatile in nature.
Advantage- Low power consumption, as current flows only when
cell is accessed.

Asynchronous DRAMs

SRAMS are fast bus high cost,needs several transistors.

Drams are less expensive but cells donot retain their state. So
DRAMS.information stored in dynamic memory cell in the form of a
charge on the capacitor.
Charge can maintained only for tens of milliseconds.
For longer time contents must be refreshed by restoring the capacitor
charge.

Single transistor dynamic

memory cell

Consists of a capacitor C and transistor T

To store information T is turned on and a
appropriate voltage is applied to bit line.
A sense amplifier connected to bit line detects the
charge stored in the capacitor

Internal organization of 2MX8

dynamic memory chip

RAS

Row
address
latch

A20 9 A8 0

Row
decoder

4096 512 8
cellarray

Sense/Write
circuits

Column
address
latch
CAS

Column
decoder

D7

D0

16-megabit DRAM
organized as 4KX4K array
Each row can store 512
bytes. 12 bits to select a
row, and 9 bits to select a
group in a row. Total of 21
bits.
First apply the row address,
RAS signal latches the row
address. Then apply the
CS
column address, CAS signal
R /W
latches the address.
Timing of the memory unit
is controlled
asynchronously. This is
asynchronous DRAM
A specialized memory
controller circuit provides
the necessary control
signals,
RAS and CAS, that govern
the timings.

Fast Page Mode

Suppose if we want to access the consecutive bytes
in the selected row.
This can be done without having to reselect the row.
Add a latch at the output of the sense circuits in each row.
All the latches are loaded when the row is selected.
Different column addresses can be applied to select and place different bytes on
the data lines.

Consecutive sequence of column addresses can be

applied under the control signal CAS, without
reselecting the row.
Allows a block of data to be transferred at a much faster rate than random
accesses.
A small collection/group of bytes is usually referred to as a block.

This transfer capability is referred to as the

fast page mode feature.

Synchronous DRAMs
Refresh
counter

Row
address
latch

Row
decoder

Column
address
counter

Column
decoder

Row/Column
address

Clock
R AS
CAS
R/ W
CS

Moderegister
and
timingcontrol

Operation is directly synchronized

with processor clock signal.
The outputs of the sense circuits are
connected to a latch.
During a Read operation, the
Cellarray
contents of the cells in a row are
loaded onto the latches.
During a refresh operation, the
contents of the cells are refreshed
Read/Write
circuits&latches without changing the contents of
the latches.
Data held in the latches correspond
to the selected columns are transferred
to the output.
For a burst mode of operation,
Datainput
Dataoutput
register
register
successive columns are selected using
column address counter and clock.
CAS signal need not be generated
externally. A new data is placed during
Data
raising edge of the clock

SDRAMs have different modes of operation by mode

register.
Ex: burst operations of different lengths can be specified.
In SDRAM not necessary to select successive columns,
control signals are provided internally using a column
counter and clock signal.
New data can be placed on the data line in each clock cycle.
Actions triggered by rising edge of the clock.

Timing diagram
Burst read of length 4.

Address is latch under the control of RAS.

Memory takes 2 or 3 cycles to activated the selected row.

Column address is latch under CAS.

Delay of 1 cycle, first set of data bits is placed on the data

line.
SDRAM automatically increments the column address to
access next 3 sets of bits in the selected row.
Data are placed on data lines in next 3 clock cycles.

Burst read of length 4 in an

SDRAM

Latency, Bandwidth
Memory latency is the time it takes
to transfer a word of data to or from
memory
Memory bandwidth is the number of
bits or bytes that can be transferred
in one second.

Structure of large memories -Static

memories
A0
A1

21bit
addresses

19bitinternalchipaddress

A19
A20

2bit
decoder

512K 8
memorychip

D3124

D2316

D 158

512K 8memorychip

19bit
address

8bitdata
input/output

Chipselect

Implement a memory unit of 2M

words of 32 bits each.
Use 512Kx8 static memory chips.
Each column consists of 4 chips.
Each chip implements one byte
position.
A chip is selected by setting its
chip select control line to 1.
Selected chip places its data on the
data output line, outputs of other
chips are in high impedance state.
21 bits to address a 32-bit word.
High order 2 bits are needed to
select
the row, by activating the
D
four Chip Select signals.
19 bits are used to access specific
byte locations inside each chip of the
selected row.
The R/W inputs of all chips are tied
together to provide common R/W
control
70

SIMM and DIMM

Large dynamic memory systems can be implemented using
DRAM chips in a similar way to static memory systems.
Placing large memory systems directly on the motherboard
will occupy a large amount of space.
Also, this arrangement is inflexible since the memory system cannot be expanded easily.

Packaging considerations have led to the development of

larger memory units known as SIMMs (Single In-line
Memory Modules) and DIMMs (Dual In-line Memory
Modules).
The primary difference between these two types of
modules is in the number of pins that they have. DIMMs
have twice as many pins compared to comparable SIMMs.

Such Memory module is an assembly of several memory

chips on a small board that plugs vertically onto a single
socket on the motherboard.
Occupy less space on the motherboard.
Allows for easy expansion by replacement.

Memory controller
Recall that in a dynamic memory chip, to reduce
the number of pins, multiplexed addresses are
used.
Address is divided into two parts:

High-order address bits select a row in the array.

They are provided first, and latched using RAS signal.
Low-order address bits select a column in the row.
They are provided later, and latched using CAS signal.

However, a processor issues all address bits at the

same time.
In order to achieve the multiplexing, memory
controller circuit is inserted between the processor
and memory.

Memory controller (contd..)

Row/Column
address

Address

RAS

R/ W
Request

Processor

Memory
controller

R/ W
CS

Clock

Clock

Data

38

CAS
Memory

Refresh overhead
DRAMS have to refresh.
Older DRAMS refreshing for 16 ms.
In SDRAMS(synchronous) period is 64 ms.
In SDRAM with 8K rows(8192 rows), if it takes 4 clock cycles to access each row,
it takes 8192x4=32,768 cycles to refresh all the rows.
If clock rate is 133MHz,time needed to refresh all the row is 32,768/
(133x10power6)=246x10-6
Refreshing process occupies 0.246ms in 64-ms time interval.
Refresh overhead is 0.246/64=0.0038 which is less than 0.4% of the total time
available for accessing the memory

Rambus memory
Performance of Dynamic memory is
characterized by its latency and bandwidth.
All chips use similar organization for their cell
array so latency will be same if chips are
produced using the same manufacturing
process.
But bandwidth is not only depends on memory
but also nature of connecting path to the
processor.
To increase the amount of data that can be
transferred on a speed limited bus is to increase
the width of the bus by providing more data
lines ,i.e: widening the bus.

RAMBUS
Very wide bus expensive, requires lot of
space on motherboard.
So Alternative approaches to implement a
narrow bus that is much fasterrambus.
Key features: fast signaling method used
to transfer information between chips.
Instead of using signals that have voltage
levels of either 0 or vsupply to represent
logic values the signals consist of much
smaller voltage swings around a reference
voltage vref. (above 2 v) and 2 logic values
are represented by 0.3 V swings above
and below vref differential signaling.

 Small voltage swings make it possible to have short transition

times, which allows for a high speed of transmission
Rambus provides a complete specification for the designing of
communication links, called a Rambus channel.
Data is transferred at both the edges of clock.
Rambus uses specially designed m/r chips & with multiple
banks to access more then 1 word at a time.
This memory is known as RDRAM or Rambus DRAM ( uses
synchronous DRAMs).
Original specification uses 9 data lines , 8 for data tranfer & 1
for parity checking.
A 2-channel Rambus (Direct RDRAM) has 18 data linesto
transfer 2 bytes at a time.

RAMBUS Communication

Processor or some other device serves as master &

RDRAM modules serves as slaves..
Communication between them is carried out by means of
packets transmitted on the data lines.
3 types of packets- request , acknowledge & data.
A request packet issued by the master indicates the type
of operation to be performed and contains desired
address and 8-bit count that specifies no. of bytes
involved in the transfer.
The operation type includes m/r reads & writes & reading
& writing of various control registers in RDRAM chip.
When master issues a request packet addressed slave
respond with an acknowledgement packet if it can serve
immediately otherwise it will send a busy signal by
sending negative ack & master has to try again.

The Memory System

Read-Only Memories (ROMs)

Read-Only Memories (ROMs)

SRAM and SDRAM chips are volatile:
Lose the contents when the power is turned off.

Many applications need memory devices to retain contents

after the power is turned off.
For example, computer is turned on, the operating system must be
loaded from the disk into the memory.
Store instructions which would load the OS from the disk.
Need to store these instructions so that they will not be lost after
the power is turned off.
We need to store the instructions into a non-volatile memory.

Non-volatile memory is read in the same manner as volatile

memory.
Separate writing process is needed to place information in this
memory.
Normal operation involves only reading of data, this type
of memory is called Read-Only memory (ROM).

Read-Only Memories (Contd.,)

Read-Only Memory:
A logic 0 is stored in the cell if T is connected to ground point P;
otherwise a 1 is stored
To read the state of the cell, the word line is activated, thus T is closed
and voltage on bit line drops to 0 if T is connected to gnd . If there is no
connection to gnd, the bit line remains at high volatge, indicating 1

The bit line is connected through a resistor to the power supply .

A sense ckt at the end of the bit line generates the proper output
Data are written into a ROM when it is manufactured.

Bit line
word line
T
P connected to store 0

Read-Only Memories
(Contd.,)
Programmable Read-Only Memory
(PROM):
Erasable Programmable Read-Only
Memory (EPROM)
Electrically Erasable Programmable
Read-Only Memory (EEPROM)

Read-Only Memories
(Contd.,)
Flash memory:
Has similar approach to EEPROM.
Read the contents of a single cell, but write the
contents of an entire block of cells.
Flash devices have greater density.
Higher capacity and low storage cost per bit.

Power consumption of flash memory is very low,

making it attractive for use in equipment that is
battery-driven.
Single flash chips are not sufficiently large, so
larger memory modules are implemented using
flash cards and flash drives.

Speed, Size, and Cost

A big challenge in the design of a computer
system is to provide a sufficiently large memory,
with a reasonable speed at an affordable cost.
Static RAM:
Very fast, but expensive, because a basic SRAM cell has a complex circuit making it
impossible to pack a large number of cells onto a single chip.

Dynamic RAM:
Simpler basic cell circuit, hence are much less expensive, but significantly slower
than SRAMs.

Magnetic disks:
Storage provided by DRAMs is higher than SRAMs, but is still less than what is
necessary.
Secondary storage such as magnetic disks provide a large amount
of storage, but is much slower than DRAMs.

Memory Hierarchy
Processor
Registers
Increasing
size

Primary L1
cache

Secondary L2
cache

Main
memory

Magneticdisk
secondary
memory

Fastest access is to the data held in

processor registers. Registers are at
the top of the memory hierarchy.
Relatively small amount of memory that
Increasing Increasing
speed costperbit
can be implemented on the processor
chip. This is processor cache.
Two levels of cache. Level 1 (L1) cache
is on the processor chip. Level 2 (L2)
cache is in between main memory and
processor.
Next level is main memory, implemented
as SIMMs. Much larger, but much slower
than cache memory.
Next level is magnetic disks. Huge amount
of inexepensive storage.
Speed of memory access is critical, the
idea is to bring instructions and data
that will be used in the near future as
close to the processor as possible.

The Memory System

Cache Memories

Cache Memories
Processor is much faster than the main memory.
As a result, the processor has to spend much of its time waiting while instructions
and data are being fetched from the main memory.
Major obstacle towards achieving good performance.

Speed of the main memory cannot be increased

beyond a certain point.
Cache memory is an architectural arrangement
which makes the main memory appear faster to
the processor than it really is.
Cache memory is based on the property of
computer programs known as locality of
reference.

Locality of Reference
Analysis of programs indicates that many
instructions in localized areas of a program
are executed repeatedly during some period
of time, while the others are accessed
relatively less frequently.
These instructions may be the ones in a loop, nested loop or few procedures
calling each other repeatedly.
This is called locality of reference.

Temporal locality of reference:

Recently executed instruction is likely to be executed again very soon.

Spatial locality of reference:

Instructions with addresses close to a recently instruction are likely
to be executed soon.

Cache memories
Processor

Cache

Main
memory

Processor issues a Read request, a block of words is transferred from the

main memory to the cache, one word at a time.
Subsequent references to the data in this block of words are found in the
cache.
At any given time, only some blocks in the main memory are held in the
cache. Which blocks in the main memory are in the cache is determined
by a mapping function.
When the cache is full, and a block of words needs to be transferred
from the main memory, some block of words in the cache must be
replaced. This is determined by a replacement algorithm.

Cache hit
Existence of a cache is transparent to the processor. The processor
issues Read and
Write requests in the same manner.
If the data is in the cache it is called a Read or Write hit.
Read hit:
The data is obtained from the cache.

Write hit:
Cache has a replica of the contents of the main memory.
Contents of the cache and the main memory may be updated simultaneously.
This is the write-through protocol.
Update the contents of the cache, and mark it as updated by setting a bit
known
as the dirty bit or modified bit. The contents of the main memory
are updated
when this block is replaced. This is write-back or copy-back
protocol.

Cache miss
If the data is not present in the cache, then a Read miss or Write miss
occurs.
Read miss:
Block of words containing this requested word is transferred from the
memory.
After the block is transferred, the desired word is forwarded to the processor.
The desired word may also be forwarded to the processor as soon as it is
transferred without waiting for the entire block to be transferred. This is
called load-through or early-restart.

Write-miss:
Write-through protocol is used, then the contents of the main memory are
updated directly.
If write-back protocol is used, the block containing the
addressed word is first brought into the cache. The desired word
is overwritten with new information.

Mapping functions
Mapping functions determine how
memory blocks are placed in the cache.
A simple processor example:

Cache consisting of 128 blocks of 16 words each.

Total size of cache is 2048 (2K) words.
Main memory is addressable by a 16-bit address.
Main memory has 64K words.
Main memory has 4K blocks of 16 words each.

Three mapping functions:

Direct mapping
Associative mapping
Set-associative mapping.

Direct mapping
Main
memory
Cache
tag

Block0

tag

Block1

tag

Block127

Tag

Block

Word

Mainmemoryaddress

Block j of the main memory maps to j modulo 128 of

the cache. 0 maps to 0, 129 maps to 1.
Block1
More than one memory block is mapped onto the sam
position in the cache.
May lead to contention for cache blocks even if the
cache is not full.
Block127
Resolve the contention by allowing new block to
Block128
replace the old block, leading to a trivial replacement
algorithm.
Block129
Memory address is divided into three fields:
- Low order 4 bits determine one of the 16
words in a block.
- When a new block is brought into the cache,
Block255
the the next 7 bits determine which cache
Block256 block this new block is placed in.
- High order 5 bits determine which of the possible
Block257
32 blocks is currently present in the cache. These
are tag bits.
Simple to implement but not very flexible.
Block0

Block4095

Associative mapping
Main
memory

Block0

tag

tag

Block1

Block127

Tag
12

Main memory block can be placed into any cache

position.
Memory address is divided into two fields:
- Low order 4 bits identify the word within a block.
- High order 12 bits or tag bits identify a memory
Block127
block when it is resident in the cache.
Block128 Flexible, and uses cache space efficiently.
Replacement algorithms can be used to replace an
Block129
existing block in the cache when the cache is full.
Cost is higher than direct-mapped cache because of
the need to search all 128 patterns to determine
whether a given block is in the cache.
Block1

Cache
tag

Block0

Word
4

Mainmemoryaddress

Block255
Block256
Block257

Block4095

Set-Associative mapping
Cache
tag

Main
memory

Block0

tag

Block1

tag

Block2

tag

Block3

tag

Block126

tag

Block127

Tag
6

set
6

Word
4

Mainmemoryaddress

Block0Blocks

of cache are grouped into sets.

Mapping function allows a block of the main
Block1
memory to reside in any block of a specific set.
Divide the cache into 64 sets, with two blocks per set.
Memory block 0, 64, 128 etc. map to block 0, and they
can occupy either of the two positions.
Block63Memory address is divided into three fields:
- 6 bit field determines the set number.
Block64
- High order 6 bit fields are compared to the tag
Block65
fields of the two blocks in a set.
Set-associative mapping combination of direct and
associative mapping.
Number of blocks per set is a design parameter.
Block127 - One extreme is to have all the blocks in one set,
requiring no set bits (fully associative mapping).
Block128
- Other extreme is to have one block per set, is
Block129
the same as direct mapping.

Block4095

Replacement algorithm
When there is a cache miss, it requires
space for new blocks.

Replacement algorithm determines which

block can be replaced by new blocks.

If use direct mapping, there is only one

cache block the frame can occupy.
Therefore, replacement algorithm is not
needed in this case.

Associative & Set Associative

Replacement Algorithms
Least Recently used (LRU)
Replace the block that hasn't been touched
in the longest period of time
Two way set associative simply uses a USE
bit. When one block is referenced, its USE
bit is set while its partner in the set is
cleared

First in first out (FIFO) replace block

that has been in cache longest

Associative & Set Associative

Replacement Algorithms (continued)
Least frequently used (LFU) replace
block which has had fewest hits
Random only slightly lower performance
than use-based algorithms LRU, FIFO, and
LFU

Replacement algorithm: LRU

Replaced the least recently used block in
the cache.

To determine where is LRU block, a

counter can be associated with each cache

block.

Advantage: This algorithm follows locality

principle, so it limits number of times the
block to be replaced.

Disadvantage: Implementation is more

complex.

 To use the LRU algorithm, the cache controller must track

references to all blocks as computation proceeds.
Suppose it is required to track the LRU block of a fourblock set in a set-associative cache. A 2-bit counter can be
used for each block.
When a hit occurs, the counter of the block that is
referenced is set to 0. Counters with values originally lower
than the referenced one are incremented by one, and all
others remain unchanged.
When a miss occurs and the set is not full, the counter
associated with the new block loaded from the main
memory is set to 0, and the values of all other counters are
increased by one.
When a miss occurs and the set is full, the block with the
counter value 3 is removed, the new block is put in its
place, and its counter is set to 0. The other three block
counters are incremented by one.
It can be easily verified that the counter values of occupied
blocks are always distinct

Replacement algorithm:
FIFO
The first-in block in the cache is replaced
first.

In the other word, the block that is in the

cache longest is replaced.

Advantage: Easy to implement.

Disadvantage: In some condition, blocks
are replaced too frequently.

The Memory System

Performance considerations

Performance
considerations
A key design objective of a computer system is to
achieve the best possible performance at the
lowest possible cost.
Price/performance ratio is a common measure of
success.

Performance of a processor depends on:

How fast machine instructions can be brought into the
processor for execution.
How fast the instructions can be executed.

Interleaving
Divides the memory system into a number of
memory modules. Each module has its own address buffer register
(ABR) and data buffer register (DBR).

Arranges addressing so that successive words

in the address space are placed in different
modules.
When requests for memory access involve
consecutive addresses, the access will be to
different modules.
Since parallel access to these modules is
possible, the average rate of fetching words
from the Main Memory can be increased.

Methods of address layouts

kbits

mbits

Module

Addressinmodule

MMaddress

mbits

kbits

Addressinmodule

Module

MMaddress

ABR DBR

ABR DBR

ABR DBR

ABR DBR

ABR DBR

ABR DBR

Module
0

Module
i

Module
n 1

Module
0

Module
i

Module
k
2 1

Consecutive words are placed in a

module.
High-order k bits of a memory
address determine the module.
Low-order m bits of a memory
address determine the word within
a module.
When
a
block
of
words
is
transferred from main memory to
cache, only one module is busy at
a time.

Consecutive words are located in

consecutive modules.
Consecutive addresses can be
located in consecutive modules.
While transferring a block of data,
several memory modules can be
kept busy at the same time.
--------------------------------------------------

 At the same time, however ,devices

with direct memory access (DMA)
ability may be accessing information
in other memory modules.

Effect of interleaving
Consider the time needed to transfer of block
of data from MM to cache with read miss.
Cache8 word block. Block copies to cache.
h/w properties: 1 c/c to send an address to
MM.
MM with slow DRAMS, allow the first word to
be accessed in 8 cycles. Then 4 for
subsequent . Then 1 c/c needed to send 1
word to cache.
If single module, time to load the desired
block into cache: 1+8+(7X4)+1=38 cycles.

With 4 modules
When the starting address of the block arrives
at the memory, all 4 modules begin accessing
the required data.
After 8 cycles, each module has 1 word of
data in its DBR. during the next 4 cycles these
words are transferred to cache one word at a
time. During this time next word in each
module is accessed. Then it takes 4 cycles to
transfer to cache.
So total time with interleaved memory is
1+8+4+4=17 cycles.
Interleaving reduces the block transfer time
by more than a factor of 2.

Hit Rate and Miss Penalty

The success rate in accessing information at
various levels of the memory hierarchy hit
rate / miss rate.
Ideally, the entire memory hierarchy would
appear to the processor as a single memory
unit that has the access time of a cache on
the processor chip and the size of a
magnetic disk depends on the hit rate
(>>0.9).
A miss causes extra time needed to bring
the desired information into the cache.

Hit Rate and Miss Penalty (cont.)

Tave=hC+(1-h)M
Tave: average access time experienced by the
processor
h: hit rate
M: miss penalty, the time to access information in
the main memory
C: the time to access information in the cache
Example:
Assume that 30 percent of the instructions in a
typical program perform a read/write operation,
which means that there are 130 memory accesses
for every 100 instructions executed.
h=0.95 for instructions, h=0.9 for data
= 5.04
C(without cache)=10 clock cycles, M=17 clock
cycles,C=1
Time without cache
130x10

 Compare this with ideal cache with

100% hit rate (all memory references
take 1 cycle)
Rough estimate of relative
performance for these caches is
100(0.95x1+0.05x17)+30(0.9x1+0.1x17)
= 1.98
130
Actual cache provides an environment in
which the processor effectively works
with large DRAM-based MM that appears
to be only 2 times slower than the
circuits in the cache.

How to Improve Hit Rate?

Use larger cache increased cost
Block sizes neither very small nor very large
give the best result.
Range 16 to 128 bytes have been the most
popular choices.
Miss penalty can be reduced if the Loadthrough approach is used when loading new
blocks into the cache.

Caches on the processor

chip
In high performance processors 2
levels of caches are normally used.
Avg access time in a system with 2
levels of caches is
T

ave

= h1c1+(1-h1)h2c2+(1-h1)(1-h2)M

Other Performance
Enhancements
Several other possibilities exist for
enhancing performance
Write buffer
Prefetching
Lockup-Free cache

Other Performance Enhancements

Write buffer

Write-through:
Each write operation involves writing to the main memory.
If the processor has to wait for the write operation to be complete, it slows
down the processor.
Processor does not depend on the results of the write operation.
Write buffer can be included for temporary storage of write requests.
Processor places each write request into the buffer and continues
execution.
If a subsequent Read request references data which is still in the write
buffer, then this data is referenced in the write buffer.
Write-back:
Block is written back to the main memory when it is replaced.
If the processor waits for this write to complete, before reading the new
block, it is slowed down.
Fast write buffer can hold the block to be written, and the new
block can be read first.

Other Performance Enhancements

(Contd.,)
Prefetching
New data are brought into the processor when
they are first needed.
Processor has to wait before the data transfer is
complete.
Prefetch the data into the cache before they are
actually needed, or a before a Read miss
occurs.
Prefetching can be accomplished through
software by including a special instruction in the
machine language of the processor.
Inclusion of prefetch instructions increases the
length of the programs.

Other Performance Enhancements

(Contd.,)
Lockup-Free Cache
Prefetching scheme does not work if it stops other
accesses to the cache until the prefetch is completed.
A cache of this type is said to be locked while it
services a miss.
Cache structure which supports multiple outstanding
misses is called a lockup free cache.
Since only one miss can be serviced at a time, a
lockup free cache must include circuits that keep
track of all the outstanding misses.
Special registers may hold the necessary
information about these misses.

 The hope is that prefetching will take

place while the processor is busy
executing instructions that do not
result in a read miss, thus allowing
accesses to the main memory to be
overlapped with computation in the
processor.

The Memory System

Virtual Memory

Virtual memories
Recall that an important challenge in the
design of a computer system is to provide a
large, fast memory system at an affordable
cost.
Architectural solutions to increase the effective
speed and size of the memory system.
Cache memories were developed to increase
the effective speed of the memory system.
Virtual memory is an architectural solution to
increase the effective size of the memory
system.
85

Virtual memories (contd..)

Recall that the addressable memory space
depends on the number of address bits in a
computer.
For example, if a computer issues 32-bit addresses, the addressable memory space is 4G
bytes.

Physical main memory in a computer is generally

not as large as the entire possible addressable
space.
Physical memory typically ranges from a few hundred megabytes to 1G bytes.

Large programs that cannot fit completely into the

main memory have their parts stored on secondary
storage devices such as magnetic disks.
Pieces of programs must be transferred to the main memory from secondary storage
before they can be executed.

86

Virtual memories (contd..)

When a new piece of a program is to be
transferred to the main memory, and the
main memory is full, then some other piece
in the main memory must be replaced.
Recall this is very similar to what we studied in case of cache memories.

Operating system automatically transfers

data between the main memory and
secondary storage.
Application programmer need not be concerned with this transfer.
Also, application programmer does not need to be aware of the limitations
imposed by the available physical memory.

87

Virtual memories (contd..)

Techniques that automatically move program and data
between main memory and secondary storage when
they are required for execution are called virtualmemory techniques.
Programs and processors reference an instruction or
data independent of the size of the main memory.
Processor issues binary addresses for instructions and
data.
These binary addresses are called logical or virtual addresses.

Virtual addresses are translated into physical addresses

by a combination of hardware and software subsystems.
If virtual address refers to a part of the program that is currently in the main memory, it is
accessed immediately.
If the address refers to a part of the program that is not currently in the main memory, it is first
transferred to the main memory before it can be used.

88

Virtual memory organization

Processor

Data

Cache
Data

Memory management unit (MMU) translates

virtual addresses into physical addresses.
Virtualaddress
If the desired data or instructions are in the
main memory they are fetched as described
MMU
previously.
If the desired data or instructions are not in
Physicaladdress
the main memory, they must be transferred
from secondary storage to the main memory.
MMU causes the operating system to bring
the data from the secondary storage into the
Physicaladdress
main memory.

Mainmemory
DMAtransfer
Diskstorage

89

Address translation
Assume that program and data are composed
of fixed-length units called pages.
A page consists of a block of words that occupy
contiguous locations in the main memory.
Page is a basic unit of information that is
transferred between secondary storage and
main memory.
Size of a page commonly ranges from 2K to
16K bytes.
Pages should not be too small, because the access time of a secondary storage
device is much larger than the main memory.
Pages should not be too large, else a large portion of the page may not be used,
and it will occupy valuable space in the main memory.

90

Address translation (contd..)

Concepts of virtual memory are
similar to the concepts of cache
memory.
Cache memory:
Introduced to bridge the speed gap between the processor and
the main memory.
Implemented in hardware.

Virtual memory:
Introduced to bridge the speed gap between the main memory
and secondary storage.
Implemented in part by software.
91

Address translation (contd..)

Each virtual or logical address generated by a
processor is interpreted as a virtual page number
(high-order bits) plus an offset (low-order bits) that
specifies the location of a particular byte within that
page.
Information about the main memory location of each
page is kept in the page table.
Main memory address where the page is stored.
Current status of the page.

Area of the main memory that can hold a page is

called as page frame.
Starting address of the page table is kept in a page
table base register.
92

Address translation (contd..)

Virtual page number generated by
the processor is added to the
contents of the page table base
register.
This provides the address of the corresponding entry in the
page table.

The contents of this location in the

page table give the starting address
of the page if the page is currently in
the main memory.
93

Address translation (contd..)

PTBR holds
Pagetablebaseregister
the address of
the page table. Pagetableaddress

Virtualaddressfromprocessor
Virtualpagenumber

Offset

Virtual address is
interpreted as page
number and offset.

+
PAGETABLE

PTBR + virtual
page number provide
the entry of the page
in the page table.

This entry has the starting location

of the page.

Page table holds information

about each page. This includes
the starting address of the page
Control
in the main memory.
bits

Pageframe
inmemory

Pageframe

Physicaladdressinmainmemory

94

Offset

Address translation (contd..)

Page table entry for a page also includes
some control bits which describe the status
of the page while it is in the main memory.
One bit indicates the validity of the page.
Indicates whether the page is actually loaded into the main memory.
Allows the operating system to invalidate the page without actually removing
it.

One bit indicates whether the page has been

modified during its residency in the main
memory.
This bit determines whether the page should be written back to the disk
when it is removed from the main memory.
Similar to the dirty or modified bit in case of cache memory.

95

Address translation (contd..)

Other control bits for various other
types of restrictions that may be
imposed.
For example, a program may only have read permission for a
page, but not write or modify permissions.

96

Address translation (contd..)

Where should the page table be located?
Recall that the page table is used by the MMU for
every read and write access to the memory.
Ideal location for the page table is within the MMU.

Page table is quite large.

MMU is implemented as part of the processor chip.
Impossible to include a complete page table on
the chip.
Page table is kept in the main memory.
A copy of a small portion of the page table can be
accommodated within the MMU.
Portion consists of page table entries that correspond to the most recently accessed
pages.

97

Address translation (contd..)

A small cache called as Translation Lookaside
Buffer (TLB) is included in the MMU.
TLB holds page table entries of the most recently accessed pages.

Recall that cache memory holds most recently

accessed blocks from the main memory.
Operation of the TLB and page table in the main memory is similar to the operation

of the cache and main memory.

Page table entry for a page includes:

Address of the page frame where the page resides in the main memory.
Some control bits.

In addition to the above for each page, TLB must

hold the virtual page number for each page.
98

Address translation (contd..)

Virtualaddressfromprocessor
Virtualpagenumber

TLB
Virtualpage
number

No

Control
bits

Pageframe
inmemory

=?
Yes

Miss
Hit

Pageframe
Physicaladdressinmainmemory

99

Offset

Associative-mapped TLB

High-order bits of the virtual address

generated by the processor select the
virtual page.
These bits are compared to the virtua
page numbers in the TLB.
If there is a match, a hit occurs and
the corresponding address of the page
frame is read.
If there is no match, a miss occurs
and the page table within the main
memory must be consulted.
Set-associative mapped TLBs are
found in commercial processors.
Offset

Address translation (contd..)

How to keep the entries of the TLB coherent
with the contents of the page table in the main
memory?
Operating system may change the contents of
the page table in the main memory.
Simultaneously it must also invalidate the corresponding entries in the TLB.

A control bit is provided in the TLB to invalidate

an entry.
If an entry is invalidated, then the TLB gets the
information for that entry from the page table.
Follows the same process that it would follow if the entry is not found in the TLB
or if a miss occurs.

100

Address translation (contd..)

What happens if a program generates
an access to a page that is not in the
main memory?
In this case, a page fault is said to occur.
Whole page must be brought into the main memory from the disk,
before the execution can proceed.

Upon detecting a page fault by the

MMU, following actions occur:

MMU asks the operating system to intervene by raising an exception.

Processing of the active task which caused the page fault is
interrupted.
Control is transferred to the operating system.
Operating system copies the requested page from secondary storage
to the main memory.
Once the page is copied, control is returned to the task which was
interrupted.

101

Address translation (contd..)

Servicing of a page fault requires
transferring the requested page from
secondary storage to the main
memory.
This transfer may incur a long delay.
While the page is being transferred
the operating system may:
Suspend the execution of the task that caused the page fault.
Begin execution of another task whose pages are in the main
memory.

Enables efficient use of the

102

Address translation (contd..)

How to ensure that the interrupted
task can continue correctly when it
resumes execution?
There are two possibilities:
Execution of the interrupted task must continue from the point
where it was interrupted.
The instruction must be restarted.

Which specific option is followed

depends on the design of the
processor.
103

Address translation (contd..)

When a new page is to be brought into the main
memory from secondary storage, the main
memory may be full.
Some page from the main memory must be replaced with this new page.

How to choose which page to replace?

This is similar to the replacement that occurs when the cache is full.
The principle of locality of reference (?) can also be applied here.
A replacement strategy similar to LRU can be applied.

Since the size of the main memory is relatively

larger compared to cache, a relatively large
amount of programs and data can be held in the
main memory.
Minimizes the frequency of transfers between secondary storage and main memory.

104

Address translation (contd..)

A page may be modified during its residency
in the main memory.
When should the page be written back to the
secondary storage?
Recall that we encountered a similar problem
in the context of cache and main memory:
Write-through protocol(?)
Write-back protocol(?)

Write-through protocol cannot be used, since

it will incur a long delay each time a small
amount of data is written to the disk.
105

The Memory System

Secondary Storage

Magnetic Hard Disks

Disk
Disk drive
Disk controller

Organization of Data on a
Disk

Sector3,track n

Sector0,track1
Sector0,track0

Figure5.30.Organizationofonesurfaceofadisk.

Access Data on a Disk

Sector header
Following the data, there is an errorcorrection code (ECC).
Formatting process
Difference between inner tracks and
outer tracks
Access time seek time / rotational
delay (latency time)
Data buffer/cache

Disk Controller
Processor

Mainmemory

Systembus
Diskcontroller

Diskdrive

Diskdrive

Figure5.31.Disksconnectedtothesystembus.

Aluminum

Optical Disks

Pit

Acrylic

Label

Polycarbonate plastic

Land

(a) Cross-section

Pit

Land

Reflection

Reflection

No reflection

Source

Detector

Source

Source

Detector

Detector

(b) Transition from pit to land

(c) Stored binary pattern

Figure 5.32. Optical disk.

Optical Disks

CD-ROM
CD-Recordable (CD-R)
CD-ReWritable (CD-RW)
DVD
DVD-RAM

Magnetic Tape Systems

File

File
mark

File
mark

Filegap

Record

Record
gap

Record

Record
gap

Figure5.33.Organizationofdataonmagnetictape.

7or9
bits

End

Cache Coherence Problem

A bit called as valid bit is provided for each block.

If the block contains valid data, then the bit is set to 1, else it is 0.
Valid bits are set to 0, when the power is just turned on.
When a block is loaded into the cache for the first time, the valid bit is set to 1.

Data transfers between main memory and disk occur directly bypassing the
cache.
When the data on a disk changes, the main memory block is also updated.
However, if the data is also resident in the cache, then the valid bit is set to 0.
What happens if the data in the disk and main memory changes and the writeback protocol is being used?
In this case, the data in the cache may also have changed and is indicated by
the dirty bit.
The copies of the data in the cache, and the main memory are different. This is
called the cache coherence problem.
One option is to force a write-back before the main memory is updated from
the disk.