Lecture 11:

Cache Conclusion and I/O Introduction:

Storage Devices, Metrics, &
Productivity
Professor David A. Patterson
Computer Science 252
Fall 1996

DAP.F96 1

Review: IRAM Challenges

Chip

Speed, area, power, yield of logic in DRAM process?

Speed, area, power, yield of SRAM in DRAM process?
Good performance and reasonable power?
BW/Latency oriented DRAM tradeoffs?

Architecture
How to turn high memory bandwidth into performance?
Vector?
Extensive Prefetching?

Extensible IRAM: Large pgm/data solution?

Redudancy in processor to match redundancy in DRAM?

DAP.F96 2

Review: Doing Research in the

Information Age
Online at UCB
Finding articles
INSPECT database
COMP database

Printing IEEE articles

Finding Books: MELVYL and GLADIS

WWW Search Engines

Alta Vista, HotBot, Yahoo!

Computer Architecture Resources

Architecture Homepage, Benchmark Database...

DAP.F96 3

Cache Cross Cutting Issues

Superscalar CPU & Number Cache Ports
Speculative Execution and non-faulting
option on memory
Parallel Execution vs. Cache locality
Want far separation to find independent operations
vs. want reuse of data accesses to avoid misses

I/O and consistency of data between cache

and memory
Caches => multiple copies of data
Consistency by HW or by SW?
Where connect I/O to computer?
DAP.F96 4

Alpha 21064
Separate Instr & Data
TLB & Caches
TLBs fully associative
TLB updates in SW
(Priv Arch Libr)
Caches 8KB direct
mapped, write thru
Critical 8 bytes first
Prefetch instr. stream
buffer
2 MB L2 cache, direct
mapped, WB (off-chip)
256 bit path to main
memory, 4 x 64-bit
modules
Victim Buffer: to give
read priority over write
4 entry write buffer
between D$ & L2$

Instr

Data

Write
Buffer
Stream
Buffer

Victim Buffer

DAP.F96 5

Alpha Memory Performance:

Miss Rates of SPEC92

Miss Rate

10.00%

Su2cor

Spice
Nasa7

Mdljp2
Hydro2d

Wave5

Alvinn
Tomcatv

Doduc

Ear
Swm256

Fpppp

Mdljsp2
Ora

Gcc
Compress

Sc

Li
Eqntott

Espresso

I$ miss = 6%
D$ miss = 32%
L2 miss = 10%

TPC-B (db2)
TPC-B (db1)

AlphaSort

100.00%

1.00%

I $

8K

D$

8K

L2

2M

0.10%

I$ miss = 2%
D$ miss = 13%
L2 miss = 0.6%
0.01%

I$ miss = 1%
D$ miss = 21%
L2 miss = 0.3%
DAP.F96 6

Alpha CPI Components

Instruction stall: branch mispredict;
Other: compute + reg conflicts, structural conflicts
4.50
4.00
3.50

L2
I$

2.50

D$

2.00

I Stall

1.50

Other

1.00
0.50

Hydro2d

Mdljp2

Wave5

Tomcatv

Alvinn

Doduc

Swm256

Ear

Fpppp

Ora

Mdljsp2

Compress

Gcc

Sc

Eqntott

Li

Espresso

TPC-B (db1)

TPC-B (db2)

0.00

AlphaSort

CPI

3.00

DAP.F96 7

Pitfall: Predicting Cache Performance

from Different Prog. (ISA, compiler, ...)
35%
D: tomcatv

4KB Data cache miss

rate 8%,12%,
or 28%?
1KB Instr cache miss
rate 0%,3%,
Miss
or 10%?
Rate
Alpha vs. MIPS for
8KB Data:
17% vs. 10%

30%
D: gcc
D: espresso

25%

I: gcc
20%

I: espresso
I: tomcatv

15%

10%

5%

0%
1

16

Cache Size (KB)

32

64

128

DAP.F96 8

Pitfall: Simulating Too Small an

Address Trace
4.5
4

Cummlati
3.5
ve
3
Average
Memory 2.5
Access
2
Time
1.5
1
I$ = 4 KB, B=16B
0 1 2 3 4 5 6 7 8 9 1011 12
D$ = 4 KB, B=16B
L2 = 512 KB, B=128B
Instructions Executed (billions)
MP = 12, 200
DAP.F96

Motivation: Who Cares About I/O?

CPU Performance: 50% to 100% per year
Multiprocessor supercomputers 150% per year
I/O system performance limited by mechanical delays
< 10% per year (IO per sec or MB per sec)

Amdahl's Law: system speed-up limited by the slowest

part!
10% IO & 10x CPU => 5x Performance (lose 50%)
10% IO & 100x CPU => 10x Performance (lose 90%)

I/O bottleneck:
Diminishing fraction of time in CPU
Diminishing value of faster CPUs

DAP.F96 10

Storage System Issues

Historical Context of Storage I/O

Secondary and Tertiary Storage Devices
Storage I/O Performance Measures
A Little Queuing Theory
Processor Interface Issues
I/O Buses
Redundant Arrarys of Inexpensive Disks (RAID)
ABCs of UNIX File Systems
I/O Benchmarks
Comparing UNIX File System Performance
DAP.F96 11

I/O Systems
Processor

interrupts

Cache

Memory - I/O Bus

Main
Memory

I/O
Controller
Disk

Disk

I/O
Controller

I/O
Controller

Graphics

Network

Time(workload) = Time(CPU) + Time(I/O) - Time(Overlap)

DAP.F96 12

Technology Trends
CPU Performance
Mini:
40% increase
per year
RISC:
100% increase
per year

DRAM Capacity
doubles every 3
years

DAP.F96 13

Technology Trends

Disk Capacity
doubles every
3 years

Today: Processing Power Doubles Every 18 months

Today: Memory Size Doubles Every 18 months(?)

The I/O
GAP

Today: Disk Capacity Doubles Every 18 months

Disk Positioning Rate (Seek + Rotate) Doubles Every Ten Years!
DAP.F96 14

Storage Technology Drivers

Driven by the prevailing computing paradigm
1950s: migration from batch to on-line processing
1990s: migration to ubiquitous computing
computers in phones, books, cars, video cameras,
nationwide fiber optical network with wireless tails

Effects on storage industry:

Embedded storage
smaller, cheaper, more reliable, lower power

Data utilities
high capacity, hierarchically managed storage

DAP.F96 15

Historical Perspectives
1956 IBM Ramac early 1970s Winchester
Developed for mainframe computers
proprietary interfaces

Steady shrink in form factor: 27 in. to 14 in.

driven by performance demands
higher rotation rate
more actuators in the machine room

DAP.F96 16

Historical Perspective
1970s developments
5.25 inch floppy disk formfactor
download microcode into mainframe

semiconductor memory and microprocessors

early emergence of industry standard disk
interfaces
ST506, SASI, SMD, ESDI

DAP.F96 17

Historical Perspective
Early 1980s
PCs and first generation workstations

Mid 1980s
Client/server computing
Centralized storage on file server
accelerates disk downsizing
8 inch to 5.25 inch

Mass market disk drives become a reality

industry standards: SCSI, IPI, IDE
5.25 inch drives for standalone PCs
End of proprietary disk interfaces
DAP.F96 18

Historical Perspective
Late 1980s/Early 1990s:
Laptops, notebooks, palmtops
3.5 inch, 2.5 inch, 1.8 inch formfactors
Formfactor plus capacity drives market, not
performance
Challenged by DRAM, flash RAM in PCMCIA cards
still expensive, Intel promises but doesnt deliver
unattractive MBytes per cubic inch

Optical disk fails on performace (e.g., NEXT) but

finds niche (CD ROM)

DAP.F96 19

Historical Perspective
$30,000

Disk Revenue, millions

30000

$25,000

Semiconductor Memory
Revenue, millions

25000

World Population,
millions

15000

$20,000
$15,000
$10,000

20000

10000

$5,000

5000

$0
1992

1990

1988

1986

1984

1982

1980

1978

1976

1974

1972

1970

Mega
Dollars

Mega
People

Year

DAP.F96 20

Historical Perspectives
9000.0
8000.0
7000.0
6000.0
5000.0
4000.0
3000.0
2000.0
1000.0
0.0

TBytes

Disk, Terabytes

30000

Memory, Terabytes

25000

World Population,
millions

20000
15000
10000
5000
0

1988

1989

1990
Year

1991

1992

Mega
People

1.5 MBytes Disk per person on the earth sold in 1992

0.1 MBytes Memory per person on the earth sold in 1992

DAP.F96 21

CS 252 Administrivia
Midterm Quiz Wednesday October 8
5:45 - 8:45 PM in 306 Soda
2 sheets with notes
Chapters 4, 5, and Ap B + Lectures

Answer questions during lecture time Wednesday

Pizza at LaVals after quiz; how many?
8 minute project meetings for Friday October 4
(11-12:30, 2:10-3:10) in 635 Soda
Email URL of initial project home page to TA

DAP.F96 22

Alternative Data Storage

Technologies
Cap
Technology
(MB)
Conventional Tape:
Cartridge (.25") 150
IBM 3490 (.5") 800

BPI

TPI

12000
22860

104
38

Helical Scan Tape:

Video (8mm)
4600
43200
DAT (4mm)
1300
61000
D-3 (1/2")
20,000

BPI*TPI Data Xfer Access

(Million) (KByte/s) Time
1.2
0.9

92
3000

minutes
seconds

45 secs
20 secs
15 secs?

1638
1870

71
114

492
183

Magnetic & Optical Disk:

Hard Disk (5.25") 1200 33528
IBM 3390 (10.5") 3800
27940

1880
2235

63
62

3000
4250

Sony MO (5.25") 640

18796

454

88

24130

18 ms
20 ms
100 ms

DAP.F96 23

Devices: Magnetic Disks

Purpose:
Long-term, nonvolatile storage
Large, inexpensive, slow level
in the storage hierarchy

Track
Sector

Characteristics:

Cylinder

Seek Time (~15 ms avg, 1M cyc

at 50MHz)

positional latency
rotational latency

Transfer rate

About a sector per ms

(1-10 MB/s)
Blocks

Capacity

Gigabytes
Quadruples every 3 years
(aerodynamics)

Head

Platter

3600 RPM = 60 RPS => 16 ms per rev

ave rot. latency = 8 ms
32 sectors per track => 0.5 ms per sector
1 KB per sector => 2 MB / s
32 KB per track
20 tracks per cyl => 640 KB per cyl
2000 cyl => 1.2 GB

Response time
= Queue + Controller + Seek + Rot + Xfer
Service time
DAP.F96 24

Disk Device Terminology

Disk Latency = Queuing Time + Seek Time + Rotation Time + Xfer Time
Order of magnitude times for 4K byte transfers:
Seek: 12 ms or less
Rotate: 4.2 ms @ 7200 rpm (8.3 ms @ 3600 rpm )
Xfer: 1 ms @ 7200 rpm (2 ms @ 3600 rpm)

DAP.F96 25

Advantages of Small
Formfactor Disk Drives

Low cost/MB
High MB/volume
High MB/watt
Low cost/Actuator
Cost and Environmental Efficiencies
DAP.F96 26

Tape vs. Disk

Longitudinal tape uses same technology as
hard disk; tracks its density improvements
Inherent cost-performance based on geometries:
fixed rotating platters with gaps
(random access, limited area, 1 media / reader)

vs.
removable long strips wound on spool
(sequential access, "unlimited" length, multiple / reader)

New technology trend:

Helical Scan (VCR, Camcoder, DAT)
Spins head at angle to tape to improve density

DAP.F96 27

Example: R-DAT Technology

Rotating (vs. Stationary) head Digital Audio Tape
Highest areal recording density commercially available
High density due to:
high coercivity metal tape
helical scan recording method
narrow, gapless (overlapping) recording tracks

10X improvement capacity & xfer rate by 1999

faster tape and drum speeds
greater track overlap
DAP.F96 28

R-DAT Technology
2000 RPM

Four Head Recording

Helical Recording Scheme

Tracks Recorded 20 w/o guard band

Read After Write Verify

DAP.F96 29

Optical Disk vs. Tape

Optical
Disk

Helical Scan
Tape

Type

5.25"

8mm

Capacity

0.75 GB

5 GB

Media Cost $90 - $175

$8

Drive Cost $3,000

$3,000

Access

Write Once Read/Write

Robot Time 10 - 20 s

10 - 20 s

Media cost ratio optical disk vs. helical tape

= 75 : 1 to 150 : 1
DAP.F96 30

Current Drawbacks to Tape

Tape wear out:
Helical 100s of passes to 1000s for longitudinal

Head wear out:

2000 hours for helical

Both must be accounted for in economic /

reliability model
Long rewind, eject, load, spin-up times;
not inherent, just no need in marketplace (so far)

DAP.F96 31

Automated Cartridge System

STC 4400

8 feet

10 feet
6000 x 0.8 GB 3490 tapes = 5 TBytes in 1992
$500,000 O.E.M. Price
6000 x 20 GB D3 tapes = 120 TBytes in 1994
1 Petabyte (1024 TBytes) in 2000
DAP.F96 32

Relative Cost of Storage

TechnologyLate 1995/Early 1996
Magnetic Disks
5.25

9.1 GB

3.5

4.3 GB

2.5

514 MB
1.1 GB

$2129
$1985
$1199
$999
$299
$345

$0.23/MB
$0.22/MB
$0.27/MB
$0.23/MB
$0.58/MB
$0.33/MB

$1695+199
$1499+189

$0.41/MB
$0.39/MB

$700
$1300
$3600

$175/MB
$32/MB
$20.50/MB

Optical Disks
5.25

4.6 GB

PCMCIA Cards
Static RAM
Flash RAM

4.0 MB
40.0 MB
175 MB

DAP.F96 33

5 minute Class Break

Lecture Format:

1 minute: review last time & motivate this lecture

20 minute lecture
3 minutes:
discuss class manangement
25 minutes:
lecture
5 minutes:
break
25 minutes:
lecture
1 minute: summary of todays important topics

DAP.F96 34

Disk I/O Performance

300

Metrics:
Response Time
Throughput

Response
Time (ms)

200

100

100%

0%

Throughput
(% total BW)
Queue
Proc

IOC

Device

Response time = Queue + Device Service time

DAP.F96 35

Response Time vs. Productivity

Interactive environments:
Each interaction or transaction has 3 parts:

Entry Time: time for user to enter command

System Response Time: time between user entry & system
replies
Think Time: Time from response until user begins next
command
1st transaction

2nd transaction

What happens to transaction time as shrink system

response time from 1.0 sec to 0.3 sec?
With Keyboard: 4.0 sec entry, 9.4 sec think time
With Graphics: 0.25 sec entry, 1.6 sec think time

DAP.F96 36

Response Time & Productivity

conventional
0.3s
conventional
1.0s
graphics
0.3s

entry

graphics
1.0s
0.00

5.00

resp

10.00

think

15.00

Time

0.7sec off response saves 4.9 sec (34%) and 2.0 sec
(70%) total time per transaction => greater productivity
Another study: everyone gets more done with faster
response, but novice with fast response = expert with
DAP.F96 37
slow

Disk Time Example

Disk Parameters:

Transfer size is 8K bytes

Advertised average seek is 12 ms
Disk spins at 7200 RPM
Transfer rate is 4 MB/sec

Controller overhead is 2 ms
Assume that disk is idle so no queuing delay
What is Average Disk Access Time for a Sector?
Ave seek + ave rot delay + transfer time + controller overhead
12 ms + 0.5/(7200 RPM/60) + 8 KB/4 MB/s + 2 ms
12 + 4.15 + 2 + 2 = 20 ms

Advertised seek time assumes no locality: typically 1/4

to 1/3 advertised seek time: 20 ms => 12 ms
DAP.F96 38

INtroduction To Queueing
Theory
Arrivals

Departures

More interested in long term, steady state

than in startup => Arrivals = Departures
Littles Law: Mean number tasks in system =
arrival rate x mean reponse time
Applies to any system in equilibrium, as long
as nothing in black box is creating or
destroying tasks
DAP.F96 39

A Little Queuing Theory:

Litttles Theorem
Queue
waiting line
Proc

server
IOC

Device

Queuing models assume state of equilibrium:

input rate = output rate
Notation:
r
Ts
u
Tw
Tq
Lw
Lq

average number of arriving customers/second

average time to service a customer ( = 1/ Ts )
server utilization (0..1): u = r x Ts
average time/customer in waiting line
average time/customer in queue: Tq =Tw + Ts
average length of waiting line:Lw = r x Tw
average length of queue:Lq = r x Tq

Littles Law: Lq = r x Tq
Mean number customers = arrival rate x mean service
DAP.F96 40
time

A Little Queuing Theory

Queue
waiting line
Proc

server
IOC

Device

Service time completions vs. waiting time for a busy

server when randomly arriving event joins a waiting
line of arbitrary length when server is busy, otherwise
serviced immediately
A single server queue: combination of a servicing
facility that accomodates 1 customer at a time (server)
+ waiting area (waiting line): together called a queue
Server spends a variable amount of time with
customers; how do you characterize variability?
Distribution of a random variable: histogram? curve?
DAP.F96 41

A Little Queuing Theory

Queue
waiting line
Proc

server
IOC

Device

Server spends a variable amount of time with customers

Weighted mean m1 = (f1 x T1 + f2 x T2 +...+ fn x Tn)/F (F=f1 + f2...)
variance = (f1 x T12 + f2 x T22 +...+ fn x Tn2)/F m12
Changes depending on unit of measure (100 ms vs. 0.1 s)

Squared coefficient of variance: C = variance/m12

Exponential distribution C = 1 : most short relative to average, few others
long; 90% < 2.3 x average, 63% < average
Hypoexponential distribution C < 1 : most close to average,
C=0.5 => 90% < 2.0 x average, only 57% < average
Hyperexponential distribution C > 1 : further from average
C=2.0 => 90% < 2.8 x average, 69% < average

DAP.F96 42

A Little Queuing Theory:

Variable Service Time
Queue
waiting line
Proc

server
IOC

Device

Server spends a variable amount of time with customers

Weighted mean m1 = (f1xT1 + f2xT2 +...+ fnXTn)/F (F=f1+f2+...)
Squared coefficient of variance C

Disk response times C 1.5 (majority seeks < average)

Yet usually pick C = 1.0 for simplicity
Another useful value is average time must wait for server
to complete task: m1(z)
Not just 1/2 x m1 because doesnt capture variance
Can derive m1(z) = 1/2 x m1 x (1 + C)
No variance => C= 0 => m1(z) = 1/2 x m1

DAP.F96 43

A Little Queuing Theory:

Average Wait Time
Calculating average wait time Tw
If something at server, it takes to complete on average m1(z)
Chance server is busy = u; average delay is u x m1(z)
All customers in line must complete; each avg Ts
Tw = u x m1(z) + Lw x Ts = 1/2 x u x Ts x (1 + C) + Lw x Ts
Tw = 1/2 x u x Ts x (1 + C) + r x Tw x Ts
Tw = 1/2 x u x Ts x (1 + C) + u x Tw
Tw x (1 u) = Ts x u x (1 + C) /2

Tw = Ts x u x (1 + C) / (2 x (1 u))
Notation:
r
Ts
u
Tw
Lw

average number of arriving customers/second

average time to service a customer
server utilization (0..1): u = r x Ts
average time/customer in waiting line
average length of waiting line:Lw = r x Tw
DAP.F96 44

A Little Queuing Theory:

M/G/1 and M/M/1
Assumptions so far:
System in equilibrium
Time between two successive arrivals in line are random
Server can start on next customer immediately after prior
finishes
No limit to the waiting line: works First-In-First-Out
Afterward, all customers in line must complete; each avg Ts

Described memoryless Markovian request arrival (M

for C=1 exponentially random), General service
distribution (no restrictions), 1 server: M/G/1 queue
When Service times have C = 1, M/M/1 queue
Tw = Ts x u x (1 + C) /(2 x (1 u)) = Ts x u / (1 u)
Ts
u
Tw

average time to service a customer

server utilization (0..1): u = r x Ts
average time/customer in waiting line
DAP.F96 45

Note distinction between waiting time and queue delay

A Little Queuing Theory:

An Example
Suppose processor sends 10 x 8KB disk I/Os per second,
requests exponentially distrib., disk service time = 20 ms
On average, how utilized is the disk?
What is the number of requests in the waiting line?
What is the average time spent in the waiting line?
What is the average response time for a disk request?

Notation:
r
Ts
u
Tw
Tq
Lw
Lq

average number of arriving customers/second = 10

average time to service a customer = 20 ms
server utilization (0..1): u = r x Ts= 10/s x .02s = 0.2
average time/customer in waiting line = Ts x u / (1 u)
= 20 x 0.2/(1-0.2) = 20 x 0.25 = 5 ms
average time/customer in queue: Tq =Tw +Ts= 25 ms
average length of waiting line:Lw = r x Tw
= 10/s x .005s = 0.05 requests in wait line
average length of queue:Lq = r x Tq= 10/s x .025s = 0.25

DAP.F96 46

A Little Queuing Theory:

Another Example
Suppose processor sends 20 x 8KB disk I/Os per sec,
requests exponentially distrib., disk service time = 12 ms
On average, how utilized is the disk?
What is the number of requests in the waiting line?
What is the average time a spent in the waiting line?
What is the average response time for a disk request?

Notation:
r
Ts
u
Tw
Tq
Lw
Lq

average number of arriving customers/second= 20

average time to service a customer= 12 ms
server utilization (0..1): u = r x Ts= 20/s x .012s = 0.24
average time/customer in waiting line = Ts x u / (1 u)
= 12 x 0.24/(1-0.24) = 12 x 0.32 = 3.8 ms
average time/customer in queue: Tq =Tw +Ts= 16 ms
average length of waiting line:Lw = r x Tw
= 20/s x .0038s = 0.016 requests in wait line
average length of queue:Lq = r x Tq= 20/s x .016s = 0.32

DAP.F96 47

A Little Queuing Theory:

Yet Another Example
Suppose processor sends 10 x 8KB disk I/Os per second,
req. squared coef. var. = 1.5, disk service time = 20 ms
On average, how utilized is the disk?
What is the number of requests in the waiting line?
What is the average time a spent in the waiting line?
What is the average response time for a disk request?

Notation:
r
Ts
u
Tw
Tq
Lw
Lq

average number of arriving customers/second= 10

average time to service a customer= 20 ms
server utilization (0..1): u = r x Ts= 10/s x .02s = 0.2
average time/customer in waiting line = Ts x u x (1 + C) /(2 x (1 u))
= 20 x 0.2(2.5)/2(1 0.2) = 20 x 0.32 = 6.25 ms
average time/customer in queue: Tq =Tw +Ts= 26 ms
average length of waiting line:Lw = r x Tw
= 10/s x .006s = 0.06 requests in wait line
DAP.F96 48
average length of queue:Lq = r x Tq= 10/s x .026s = 0.26

Summary: Storage System Issues

Historical Context of Storage I/O

Secondary and Tertiary Storage Devices
Storage I/O Performance Measures
A Little Queuing Theory
Processor Interface Issues
I/O Buses
Redundant Arrarys of Inexpensive Disks (RAID)
ABCs of UNIX File Systems
I/O Benchmarks
Comparing UNIX File System Performance
DAP.F96 49