Unit – 3

The Memory System

Some basic concepts

Programs and the Data
Fast , Large and Inexpensive Memory
Maximum size of memory that can be used in any computer is determined by addressing
Scheme

CPU-Main
Memory
Connection
6 Transistor Static
CMOS RAM Cell
DRAM – Slower & Less expensive
 Read-Only Memories (ROMs)
 SRAM and SDRAM chips are volatile:
 Lose the contents when the power is turned off.
 Many applications need memory devices to retain contents after the power is turned off.
 For example, computer is turned on, the operating system must be loaded from the disk into the
memory.
 To load OS into memory, Boot program has to be executed , since Boot pgm is quite large, most of
it is stored on the disk. Now processor has to execute some instructions that load boot pgm into
memory.
 Need to store these instructions so that they will not be lost after the power is turned off.
 We need to store these instructions into a non-volatile memory.
 Non-volatile memory is read in the same manner as volatile memory.
 But Separate writing process is needed to place information in this memory.
 Since its Normal operation involves only reading of data, this type
of memory is called Read-Only memory (ROM).
 Electrically Erasable Programmable Read-Only Memory (EEPROM):

 To erase the contents of EPROMs, they have to be exposed to ultraviolet light.

 Disadvantage with EPROM , Physically removed from the circuit.

 3 )Power consumption of flash memory is very low, making it attractive for use in equipment
that is battery-driven.

Larger Memory modules consisting of a number of chips are needed

Speed, Size, and Cost

 A big challenge in the design of a computer system is to provide a sufficiently large memory,
with a reasonable speed at an affordable cost.

 Static RAM:

 Very fast, but expensive, because a basic SRAM cell has a complex circuit making it
impossible to pack a large number of cells onto a single chip.

 Dynamic RAM:

 Simpler basic cell circuit, hence are much less expensive, but significantly slower than
SRAMs.

 Magnetic disks:

 Storage provided by DRAMs is higher than SRAMs, but is still less than what is necessary.

 Secondary storage such as magnetic disks provide a large amount

of storage, but is much slower than DRAMs.
 that will be used in
the near future as
close to the processor
as possible.
Cache memory

 Processor is much faster than the main memory.

 As a result, the processor has to spend much of its time waiting while instructions and
data are being fetched from the main memory. – Idling of Processor

 Major obstacle towards achieving good performance.

 Speed of the main memory cannot be increased beyond a certain point.

 Cache memory is an architectural arrangement which makes the main memory appear faster to
the processor than it really is.

 Cache memory is based on the property of computer programs known as “locality of reference”.

Locality of Reference

 Analysis of programs indicates that many instructions in localized areas of a program are
executed repeatedly during some period of time, while the others are accessed relatively less
frequently.

 These instructions may be the ones in a loop, nested loop or few procedures calling
each other repeatedly.

 This is called “locality of reference”.

 Temporal locality of reference:

 Recently executed instruction is likely to be executed again very soon. – whenever an

information is required by processor, this info should be brought into the cache where it
will remain until it is needed again.

 Spatial locality of reference:

 Instructions with addresses close to a recently executed instruction are likely

to be executed soon. – suggest that instead of fetching just one item from the main
memory to cache, it is useful to fetch several items that reside at adjacent address as well.

Cache memories

• When Processor issues a Read request, a block of words is transferred from the main memory
to the cache, one word at a time.

• Subsequently when the program references any of the locations in this block, the desired
contents are read directly from the cache.
• At any given time, only some blocks in the main memory are held in the cache. Which blocks in
the main memory are in the cache is determined by a “mapping function”.

• When the cache is full, and a block of words needs to be transferred

from the main memory, some block of words in the cache must be

replaced. This is determined by a “replacement algorithm”.

Cache hit

• Existence of a cache is transparent to the processor. The processor issues Read and Write
requests in the same manner. (In a Read operation, the main memory is not involved)

• If the requested data is present in the cache then it is called a Read or Write hit.

• Read hit:

 The data is obtained from the cache.

• Write hit:

 Cache has a replica of the contents of the main memory.

 Contents of the cache and the main memory may be updated simultaneously. This is
called write-through protocol.

 Update the contents of the cache, and mark it as updated by setting a bit known as the
dirty bit or modified bit. The contents of the main memory are updated when this block
is replaced from cache. This is write-back or copy-back protocol.

Cache miss

• When the addressed word in the read operation is not in the cache, then a Read miss or Write
miss occurs.

• Read miss:

 Block of words containing this requested word is transferred from the main memory
into the cache.

 After the entire block is transferred into cache, the requested word is forwarded to the
processor.

 The desired word may also be forwarded to the processor as soon as it is read from the
main memory , this transfer reduces the processor waiting , this approach is called
load-through or early-restart.
• Write-miss:

 Write-through protocol is used, then the contents of the main memory are updated
directly.

 If write-back protocol is used, the block containing the addressed word is first brought
into the cache. The desired word is overwritten with new information.

Cache memory

• Cache memory is a Random Access Memory.

• The main advantage of cache memory is its very fast speed.

• It can be accessed by the CPU at much faster speed than main memory.

Location-

• Cache memory lies on the path between the CPU and the main memory.

• It facilitates the transfer of data between the processor and the main memory at the speed
which matches to the speed of the processor.

• Data is transferred in the form of words between the cache

memory and the CPU.

• Data is transferred in the form of blocks or pages between the

cache memory and the main memory.

Purpose-

• The fast speed of the cache memory makes it extremely useful.

• It is used for bridging the speed mismatch between the fastest CPU and the main memory.

• It does not let the CPU performance suffer due to the slower speed of the main memory.

Execution Of Program-

• Whenever any program has to be executed, it is first loaded in the main memory.

• The portion of the program that is mostly probably going to be executed in the near future is
kept in the cache memory.

• This allows CPU to access the most probable portion at a faster speed.

Step-01:
 • Whenever CPU requires any word of memory, it is first searched in the CPU registers.

• If the required word is found in the CPU registers, it is read from there.

• If the required word is not found in the CPU registers, Step-02 is followed.

Step-02:

• When the required word is not found in the CPU registers, it is searched in the cache memory.

• Tag directory of the cache memory is used to search whether the required word is present in
the cache memory or not.

Now, there are two cases possible-

Case-01: If the required word is found in the cache memory, the word is delivered to the CPU.
This is known as Cache hit.

Case-02: If the required word is not found in the cache memory, Step-03 is followed. This is
known as Cache miss.

Step-03:

• When the required word is not found in the cache memory, it is searched in the main memory.

• Page Table is used to determine whether the required page is present in the main memory or
not.

• Now, there are two cases possible

Case-01:

If the page containing the required word is found in the main memory,

• The page is mapped from the main memory to the cache memory.

• This mapping is performed using cache mapping techniques.

• Then, the required word is delivered to the CPU.

Case-02:

If the page containing the required word is not found in the main memory,

• A page fault occurs.

• The page containing the required word is mapped from the secondary memory to the main
memory.
 • Then, the page is mapped from the main memory to the cache memory.

• Then, the required word is delivered to the CPU.

Multilevel Cache Organization-

• A multilevel cache organization is an organization where cache memories of different sizes are
organized at multiple levels to increase the processing speed to a greater extent.

• The smaller the size of cache, the faster its speed.

• The smallest size cache memory is placed closest to the CPU.

• This helps to achieve better performance in terms of speed.

• Mapping functions determine how memory blocks are placed in the cache.

• Three types of mapping functions:

• Direct mapping

• Associative mapping

Set-associative mapping

Direct Mapping

The following steps explain the working of direct mapped cache-

After CPU generates a memory request,

• The line number field of the address is used to access the particular line of the cache.

• The tag field of the CPU address is then compared with the tag of the cache line.

• If the two tags match, a cache hit occurs and the desired word is found in the cache.

• If the two tags do not match, a cache miss occurs.

• In case of a cache miss, the required word has to be brought from the main memory.

• It is then stored in the cache together with the new tag replacing the previous one.

• Several Words or Bytes are grouped together to form Block ( Line in Cache )

• Size of Main Memory Block and Cache Line are same

Number of blocks on main memory=Memory size/Block size

=128/8 =16 blocks=2^4 blocks Therefore Block index bits=4 bits.

 Since multiple blocks of main memory map to single cache line,CPU needs a mechanism to
identify which particular block of main memory is residing in the cache. Hence, it maintains few
additional bits called tag bits.

The number of tag bits required is computed using

the formula= number of bits in Block index - No of cache index bits

= 4-2 =2

Tags hold Most significant bits of block index.

The following steps explain the working of direct mapped cache-

After CPU generates a memory request,

• The line number field of the address (PA) is used to access the particular line of the cache.

• The tag field of the CPU address is then compared with the tag of the cache line.

• If the two tags match, a cache hit occurs and the desired word is found in the cache.

• If the two tags do not match, a cache miss occurs.

• In case of a cache miss, the required word has to be brought from the main memory.

• It is then stored in the cache together with the new tag replacing the previous one.

• Formulas : Block offset Bits = Block Size in terms of 2 ^N

• Cache address bits = Cache Size in terms of 2 ^ N

• Cache Index = Cache address bits – Block offset bits

• Block index bits = P.A bits – Block offset bits

• Number of Cache Lines (Blocks) = 2 ^ Cache Index bits

• Cache Size = 2 ^ Cache Address bits or( Block Size in Bytes * Total Number of Cache Lines )

• P.A bits = Main memory size in terms of 2 ^ N

• Tag Directory Size= Nor of Cache lines * Nor of Tag Bits

Number of bits in tag bits = Physical address - cache address bits or( block index- cache index )

Problems on direct mapping

1.Consider a direct mapped cache of size 2KB with block size 16 bytes. The main memory has
64KB. Find The number of bits needed for i) Cache index ii) Block Offset iii) Cache Address iv) Tag
bits v) Block Index and vi) Physical Address

Solution: Given Cache Size 2 KB = 2^1 * 2^10 = 11 bits for cache addressing (iii)

Block Size is 16bytes = 2^4 = 4 bits for block offset (ii)

Therefore required Cache index bits = 11 -4 = 7 bits (i)

Physical address bits = main memory size = 64 KB = 2^6 * 2 ^ 10 = 16 bits (vi)

Tag Bits = PA – CA = 16 -11 = 05 bits (iv)

Block index bits = PA – Block offset = 16 – 4 = 12 bits (v)

2.Consider a direct mapped cache of size 32KB with block size 32 bytes. The CPU generates 32-bit
address. Find The number of bits needed for cache indexing, and the number of tag bits.

Solution: PA bits are 32 bits – Means Main memory size is = 2 ^ 32 Bytes

Given, cache size=32KB=2^5*2^10=2^15bytes

Therefore Cache address=15 bits

Block size= 32 bytes=2^5 bytes.

therefore block offset=5 bits

Given physical address= 32 bits.

WKT tag bits=PA-CA

=32-15 therefore, number of tag bits=17 bits.

Problem-3: Consider a direct mapped cache of size 16 KB with block size 256 bytes. The size of main
memory is 128 KB. Find-

• Number of bits in tag

• Tag directory size

Solution:

• Cache memory size = 16 KB

• Block size = 256 bytes

• Main memory size = 128 KB

• Size of main memory=128KB

=(2^7)*(2^10) bytes

=2^17 bytes.

Therefore, number of bits in physical address=17 bits.

To compute Number of bits in block offset

We have, Block size

= 256 bytes

= 28 bytes

Therefore, block offset= 8 bits.

To compute number of bits in cache index

Total number of lines in cache

= Cache size / Line size(block size)

= 16 KB / 256 bytes

= 214 bytes / 28 bytes

= 26 lines

Thus, Number of bits in cache index = 6 bits

To compute Number of Bits in Tag:

Number of bits in tag

= Number of bits in physical address – (Number of bits in cache index + Number of bits in block
offset)

= 17 bits – (6 bits + 8 bits)

= 17 bits – 14 bits

= 3 bits

Thus, Number of bits in tag = 3 bits

Or

Tag bits= Physical address- cache address=17-14=3

To compute Tag Directory Size:

Tag directory size

= Number of tags x Tag size

= Number of lines in cache x Number of bits in tag

= 26 x 3 bits

= 192 bits

= 24 bytes

Thus, size of tag directory = 24 bytes

Problem-4:

Consider a direct mapped cache of size 512 KB with block size 1 KB. There are 7 bits in the tag. Find-

• Size of main memory

• Tag directory size

Solution:

Cache size = 512 KB= 2^9 * 2^10 = 2^19 Bytes = 19 bits Cache Addr Bits

Block Size = 1 KB = 2^10 Bytes = 10 bits Block Offset

Number of bits in physical address = 19+07 = 26 bits

Thus, Size of main memory

= 226 bytes

= 64 MB

To compute Tag Directory Size

Tag directory size

= Number of tags x Tag size

= Number of lines in cache x Number of bits in tag

= 29 x 7 bits

= 3584 bits
 = 448 bytes

Thus, size of tag directory = 448 bytes

Associative Mapping

• In associative mapping, A main memory block can be placed into any cache block position that
is freely available at that moment.

• This makes associative mapping more flexible than direct mapping. Thus space in cache can be
used more efficiently.

• The tag bits of an address received from the processor ( PA) are compared to the tag bits of
each block of the cache to see if the desired block is present. This is called associative mapping
technique.

• A new block that has to be brought into the cache has to replace an existing block only if the
cache is full.

• Thus, replacement algorithm like FCFS Algorithm, LRU Algorithm etc is employed.

• The cost of an associative cache is higher than the cost of a direct mapped cache because of the
need to search all tag patterns to determine whether a given block is in cache. A search of this
kind is called associative search.

Problem-02:

Consider a fully associative mapped cache of size 16 KB with block size 256 bytes. The size of
main memory is 128 KB. Find- Number of bits in tag & Tag directory size

Solution-Given- Cache memory size = 16 KB

Block size = Frame size = Line size = 256 bytes

Main memory size = 128 KB

We consider that the memory is byte addressable.

Number of Bits in Physical Address-

We have, Size of main memory= 128 KB= 2^7 * 2^10 = 217 bytes
Thus, Number of bits in physical address = 17 bits

Number of Lines in Cache-

Total number of lines in cache

= Cache size / Line size

= 16 KB / 256 bytes

= 214 bytes / 28 bytes

= 26 lines

Tag Directory Size-

Tag directory size

= Number of tags x Tag size

= Number of lines in cache x Number of bits in tag

= 26 x 9 bits = 576 bits

= 72 bytes

Thus, size of tag directory = 72 bytes

Problem-03:

Consider a fully associative mapped cache of size 512 KB with block size 1 KB. There are 17 bits
in the tag. Find- Size of main memory & Tag directory size

Solution-

Given- Cache memory size = 512 KB

Block size = Frame size = Line size = 1 KB

Number of bits in tag = 17 bits

In k-way set associative mapping,

• Cache lines are grouped into sets where each set contains k number of lines.

• A particular block of main memory can map to only one particular set of the cache ( Direct
Mapping).

• However, within that set, the memory block can map any cache line that is freely available
( Associative Mapping )

• So Set Associative Mapping is combination of Direct mapping and Associative mapping : Direct
mapping to identify set and Associative mapping to map within the set

• K represents numbers lines in a set

• When a processor generates the PA , it uses SI bits in PA to identify the set in Cache memory ,
then uses Tag bits of CPU address to compare with associated Tab bits in tag directory . If
matches Cache hit is said to occurred.

Virtual memories

 Recall that an important challenge in the design of a computer system is to provide a large, fast
memory system at an affordable cost.

 Architectural solutions to increase the effective speed and size of the memory system.

 Cache memories were developed to increase the effective speed of the memory system.

 Virtual memory is an architectural solution to increase the effective size of the memory system.

 Recall that the addressable memory space depends on the number of address bits in a
computer.

 For example, if a computer issues 32-bit addresses, the addressable memory space is 4G
bytes.

 Physical main memory in a computer is generally not as large as the entire possible addressable
space.

 Physical memory typically ranges from a few hundred megabytes to 1G bytes.

 Large programs that cannot fit completely into the main memory have their parts stored on
secondary storage devices such as magnetic disks.

 Pieces of programs must be transferred to the main memory from secondary storage
before they can be executed.
 Virtual memory organization
Processor
Memory management unit (MMU) translates
Virtualvirtual
address addresses into physical addresses.
If the desired data or instructions are in the
Data MMU
main memory they are fetched as described
Physicalpreviously.
address

Cache
If the desired data or instructions are not in
the main memory, they must be transferred
Data Physicalfrom secondary storage to the main memory.
address

Main memory MMU causes the operating system to bring

the data from the secondary storage into the
DMA transfer main memory.

Disk storage

Address translation

 Assume that program and data are composed of fixed-length units called pages.

 A page consists of a block of words that occupy contiguous locations in the main memory.

 Page is a basic unit of information that is transferred between secondary storage and main
memory.

 Size of a page commonly ranges from 2K to 16K bytes.

 Pages should not be too small, because the access time of a secondary storage device is
much larger than the main memory.

 Pages should not be too large, else a large portion of the page may not be used, and it
will occupy valuable space in the main memory.

 Concepts of virtual memory are similar to the concepts of cache memory.

 Cache memory:

 Introduced to bridge the speed gap between the processor and the main memory.

 Implemented in hardware.

 Virtual memory:

 Introduced to bridge the speed gap between the main memory and secondary storage.

 Implemented in part by software.

 Each virtual or logical address generated by a processor is interpreted as a virtual page number
(high-order bits) plus an offset (low-order bits) that specifies the location of a particular byte
within that page.

 Information about the main memory location of each page is kept in the page table.

 Main memory address where the page is stored.

 Current status of the page.

 Area of the main memory that can hold a page is called as page frame.

 Starting address of the page table is kept in a page table base register.

 Virtual page number generated by the processor is added to the contents of the page table base
register.

 This provides the address of the corresponding entry in the page table.

The contents of this location in the page table give the starting address of the page if the page is
currently in the main memory

 Page table entry for a page also includes some control bits which describe the status of the page
while it is in the main memory.

 One bit indicates the validity of the page.

 Indicates whether the page is actually loaded into the main memory.

 Allows the operating system to invalidate the page without actually removing it.

 One bit indicates whether the page has been modified during its residency in the main memory.

 This bit determines whether the page should be written back to the disk when it is
removed from the main memory.

 Similar to the dirty or modified bit in case of cache memory.

 A small cache called as Translation Lookaside Buffer (TLB) is included in the MMU.

 TLB holds page table entries of the most recently accessed pages.

 Recall that cache memory holds most recently accessed blocks from the main memory.

 Operation of the TLB and page table in the main memory is similar to the operation of
the cache and main memory.

 Page table entry for a page includes:

 Address of the page frame where the page resides in the main memory.

 Some control bits.

 In addition to the above for each page, TLB must hold the virtual page number for each page.