CS 152 Computer Architecture and

Engineering

Lecture 4 - Pipelining

Krste Asanovic
Electrical Engineering and Computer Sciences
University of California at Berkeley

http://www.eecs.berkeley.edu/~krste
http://inst.eecs.berkeley.edu/~cs152

January 31, 2012 CS152, Spring 2012

 Last time in Lecture 3
• Microcoding became less attractive as gap between
RAM and ROM speeds reduced
• Complex instruction sets difficult to pipeline, so
difficult to increase performance as gate count grew
• Load-Store RISC ISAs designed for efficient
pipelined implementations
– Very similar to vertical microcode
– Inspired by earlier Cray machines (more on these later)
• Iron Law explains architecture design space
– Trade instructions/program, cycles/instruction, and time/cycle

January 31, 2012 CS152, Spring 2012 2

 An Ideal Pipeline

stage stage stage stage

1 2 3 4

• All objects go through the same stages

• No sharing of resources between any two stages

• Propagation delay through all pipeline stages is equal

• The scheduling of an object entering the pipeline

is not affected by the objects in other stages
These conditions generally hold for industrial
assembly lines, but instructions depend on each
other!
January 31, 2012 CS152, Spring 2012 3
 Pipelined RISC-V
To pipeline RISC-V:

• First build RISC-V without pipelining

with CPI=1

• Next, add pipeline registers to reduce

cycle time while maintaining CPI=1

January 31, 2012 CS152, Spring 2012 4

 Lecture 3: Unpipelined Datapath for RISC-V
PCSel
br RegWriteEn MemWrite WBSel
rind
jabs
pc+4

0x4
Add
Add

clk

we Br Logic Bcomp?
clk
rs1
rs2
PC addr 1 rd1 we
inst wa addr
wd rd2 ALU
clk Inst. GPRs rdata
Memory Data
Imm Memory
Select wdata
ALU
Control

OpCode WASel ImmSel FuncSel Op2Sel

January 31, 2012 CS152, Spring 2012 5

 Lecture 3: Hardwired Control Table
Opcode ImmSel Op2Sel FuncSel MemWr RFWen WBSel WASel PCSel
ALU * Reg Func no yes ALU rd pc+4
ALUi IType12 Imm Op no yes ALU rd pc+4
LW IType12 Imm + no yes Mem rd pc+4
SW BsType12 Imm + yes no * * pc+4
BEQtrue BrType12 * * no no * * br
BEQfalse BrType12 * * no no * * pc+4
J * * * no no * * jabs
JAL * * * no yes PC X1 jabs
JALR * * * no yes PC rd rind

Op2Sel= Reg / Imm WBSel = ALU / Mem / PC

WASel = rd / X1 PCSel = pc+4 / br / rind / jabs

January 31, 2012 CS152, Spring 2012 6

 Pipelined Datapath
0x4
Add
we
rs1
rs2
PC addr rd1 we
rdata IR wa addr
wd rd2 ALU
GPRs rdata
Inst. Data
Memory Imm Memory
Select wdata

write
fetch decode & Reg-fetch execute memory -back
phase phase phase phase phase
Clock period can be reduced by dividing the execution of an
instruction into multiple cycles
tC > max {tIM, tRF, tALU, tDM, tRW} ( = tDM probably)

However, CPI will increase unless instructions are pipelined

January 31, 2012 CS152, Spring 2012 7
 “Iron Law” of Processor Performance
Time = Instructions Cycles Time
Program Program * Instruction * Cycle
– Instructions per program depends on source code,
compiler technology, and ISA
– Cycles per instructions (CPI) depends upon the
ISA and the microarchitecture
– Time per cycle depends upon the
microarchitecture and the base technology
Microarchitecture CPI cycle time
Lecture 2 Microcoded >1 short
Lecture 3 Single-cycle unpipelined 1 long
Lecture 4 Pipelined 1 short

January 31, 2012 CS152, Spring 2012 8

 CPI Examples
Microcoded machine Time
7 cycles 5 cycles 10 cycles
Inst 1 Inst 2 Inst 3

3 instructions, 22 cycles, CPI=7.33

Unpipelined machine
Inst 1 Inst 2 Inst 3
3 instructions, 3 cycles, CPI=1
Pipelined machine
Inst 1
Inst 2 3 instructions, 3 cycles, CPI=1
Inst 3 5-stage pipeline CPI≠5!!!
January 31, 2012 CS152, Spring 2012 9
 Technology Assumptions
• A small amount of very fast memory (caches)
backed up by a large, slower memory
• Fast ALU (at least for integers)
• Multiported Register files (slower!)

Thus, the following timing assumption is reasonable

tIM tRF tALU tDM  tRW

A 5-stage pipeline will be the focus of our

detailed design
- some commercial designs have over
30 pipeline stages to do an integer add!
January 31, 2012 CS152, Spring 2012 10
 5-Stage Pipelined Execution
0x4
Add
we
rs1
rs2
PC addr rd1 we
rdata IR wa addr
wd rd2 ALU
GPRs rdata
Inst. Data
Memory Imm Memory
Select wdata

Write
I-Fetch Decode, Reg. Fetch Execute Memory -Back
(IF) (ID) (EX) (MA) (WB)
ime t0 t1 t2 t3 t4 t5 t6 t7 .
nstruction1 IF1 ID1 EX1 MA1 WB1
nstruction2 IF2 ID2 EX2 MA2 WB2
nstruction3 IF3 ID3 EX3 MA3 WB3
nstruction4 IF4 ID4 EX4 MA4 WB4
nstruction5 IF5 ID5 EX5 MA5 W

January 31, 2012 CS152, Spring 2012 11

 5-Stage Pipelined Execution
Resource Usage Diagram

0x4
Add
we
rs1
rs2
PC addr rd1 we
rdata IR ws addr
wd rd2 ALU
GPRs rdata
Inst. Data
Memory Imm Memory
Select wdata

Write
I-Fetch Decode, Reg. Fetch Execute Memory -Back
(IF) (ID) (EX) (MA) (WB)
time t0 t1 t2 t3 t4 t5 t6 t7 ...
Resources

IF I1 I2 I3 I4 I5
ID I1 I2 I3 I4 I5
EX I1 I2 I3 I4 I5
MA I1 I2 I3 I4 I5
WB I1 I2 I3 I4 I5
January 31, 2012 CS152, Spring 2012 12
 Pipelined Execution:
ALU Instructions

0x4
IR IR IR
Add

1
we
rs1
rs2
addr rd1 A
PC we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
Select wdata
wdata

MD1 MD2

Not quite correct!

We need an Instruction Reg (IR) for each stage

January 31, 2012 CS152, Spring 2012 13

 Pipelined RISC-V Datapath
without jumps
F D E M W
IR IR IR
1
0x4
Add

WASel
RegWriteEn
we FuncSel
rs1 MemWrite
WBSel
rs2
addr rd1 A we
PC
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
wdata
Select wdata

MD1 MD2

ImmSel Op2Sel

Control Points Need to

Be Connected

January 31, 2012 CS152, Spring 2012 14

 Instructions interact with each other
in pipeline
• An instruction in the pipeline may need a
resource being used by another instruction in
the pipeline  structural hazard
• An instruction may depend on something
produced by an earlier instruction
– Dependence may be for a data value
 data hazard
– Dependence may be for the next instruction’s address
 control hazard (branches, exceptions)

January 31, 2012 CS152, Spring 2012 15

 Resolving Structural Hazards
• Structural hazard occurs when two
instructions need same hardware resource at
same time
– Can resolve in hardware by stalling newer instruction till older
instruction finished with resource
• A structural hazard can always be avoided by
adding more hardware to design
– E.g., if two instructions both need a port to memory at same
time, could avoid hazard by adding second port to memory
• Our 5-stage pipe has no structural hazards by
design
– Thanks to RISC-V ISA, which was designed for pipelining

January 31, 2012 CS152, Spring 2012 16

 Data Hazards
x4  x1 … x1 …
0x4
Add IR IR IR
1

we
rs1
rs2
addr rd1 A
PC we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
Select wdata
wdata

MD1 MD2

...
x1 x0 + 10
x4 x1 + 17 x1 is stale. Oops!
...
January 31, 2012 CS152, Spring 2012 17
 Resolving Data Hazards (1)

Strategy 1:

Wait for the result to be available by freezing

earlier pipeline stages  interlocks

January 31, 2012 CS152, Spring 2012 18

 Feedback to Resolve Hazards

FB1 FB2 FB3 FB4

stage stage stage stage

1 2 3 4

• Later stages provide dependence information to

earlier stages which can stall (or kill) instructions
• Controlling a pipeline in this manner works provided
the instruction at stage i+1 can complete without
any interference from instructions in stages 1 to i
(otherwise deadlocks may occur)

January 31, 2012 CS152, Spring 2012 19

 Interlocks to resolve Data Hazards
Stall Condition

0x4 bubble
IR IR IR
Add
1

we
rs1
rs2
addr rd1 A
PC we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
wdata
Select wdata
...
MD1 MD2
x1 x0 + 10
x4 x1 + 17
...

January 31, 2012 CS152, Spring 2012 20

 Stalled Stages and Pipeline Bubbles
time
t0 t1 t2 t3 t4 t5 t6 t7 ....
1 (x0) + 10 IF1 ID1 EX1 MA1 WB1
4 (x1) + 17 IF2 ID2 ID2 ID2 ID2 EX2 MA2
IF3 IF3 IF3 IF3 ID3
stalled stages IF4 ID4
IF5 ID5

time
t0 t1 t2 t3 t4 t5 t6 t7 ....
IF I1 I2 I3 I3 I3 I3 I4 I5
ID I1 I2 I2 I2 I2 I3 I4 I5
Resource
EX
Usage I1 - - - I2 I3 I4 I5
MA I1 - - - I2 I3 I4
WB I1 - - - I2 I3
-  pipeline bubble

January 31, 2012 CS152, Spring 2012 21

 Interlock Control Logic
stall wa
Cstall
rs2 ?
rs1

0x4 bubble
Add IR IR IR
1

we
rs1
rs2
addr rd1 A
PC we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
wdata
Select wdata

MD1 MD2

Compare the source registers of the instruction in the decode

stage with the destination register of the uncommitted
instructions.

January 31, 2012 CS152, Spring 2012 22

 Interlock Control Logic
ignoring jumps & branches
stall wa
we
Cstall
rs1 ?
rs2 we wa we wa
re1 re2 Cdest Cdest
Cre
0x4 bubble
IR IR IR
Add
1

Cdest
we
rs1
rs2
addr rd1 A
PC we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
wdata
Select wdata

MD1 MD2

Should we always stall if an rs field matches some rd?

not every instruction writes a register we
not every instruction reads a register re

January 31, 2012 CS152, Spring 2012 23

 Source & Destination Registers
rd rs1 rs2 func10 opcode ALU
rd rs1 Imm[11:0] func3 opcode ALUI/LW/JALR
Imm[11:7
] rs1 rs2 Imm[6:0] func3 opcode SW/Bcond
Jump offset[24:0] opcode

source(s) destination
ALU rd  rs1 func10 rs2 rs1, rs2 rd
ALUI rd  rs1 op imm rs1 rd
LW rd M [rs1 + imm] rs1 rd
SW M [rs1 + imm]  rs2 rs1, rs2 -
Bcond rs1,rs2 rs1, rs2 -
true: PC  PC + imm
false: PC  PC + 4
J PC  PC + imm - -
JAL x1  PC, PC  PC + imm - x1
JALR rd  PC, PC  rs1 + imm rs1 rd

January 31, 2012 CS152, Spring 2012 24

 Deriving the Stall Signal
Cdest Cre
ws = Case opcode re1 = Case opcode
JAL X1 ALU, ALUi,
else rd LW, SW, Bcond,
JALR on
we = Case opcode J, JAL off
ALU, ALUi, LW,JALR (ws  0)
JAL on re2 = Case opcode
... off ALU, SW,Bcond on
... off

Cstall
stall = ((rs1D =wsE).weE + !
t y
(rs1D =wsM).weM + n o t or
is l s
(rs1D =wsW).weW) . re1D +
h is f u l
((rs2D =wsE).weE + T e
th
(rs2D =wsM).weM +
(rs2D =wsW).weW) . re2D
January 31, 2012 CS152, Spring 2012 25
 Hazards due to Loads & Stores
Stall Condition
What if
x1+7 = x3+5 ?

0x4 bubble IR IR IR
Add
1

we
rs1
rs2
addr rd1 A
PC we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
wdata
Select wdata

MD1 MD2
...
M[x1+7]  x2 Is there any possible data hazard
x4  M[x3+5] in this instruction sequence?
...

January 31, 2012 CS152, Spring 2012 26

 Load & Store Hazards

...
M[x1+7]  x2 x1+7 = x3+5  data hazard
x4  M[x3+5]
...

However, the hazard is avoided because our

memory system completes writes in one cycle !

Load/Store hazards are sometimes resolved in the

pipeline and sometimes in the memory system
itself.

More on this later in the course.

January 31, 2012 CS152, Spring 2012 27
 CS152 Administrivia
• Quiz 1 on Feb 14 will cover PS1, Lab1, lectures 1-5,
and associated readings.
• Section on Friday will review pipelining.

January 31, 2012 CS152, Spring 2012 28

 Resolving Data Hazards (2)

Strategy 2:

Route data as soon as possible after it is

calculated to the earlier pipeline stage  bypass

January 31, 2012 CS152, Spring 2012 29

 Bypassing
t0 t1 t2 t3 t4 t5 t6 t7 ....
+ 10 IF1 ID1 EX1 MA1 WB1
+ 17 IF2 ID2 ID2 ID2 ID2 EX2 MA2
IF3 IF3 IF3 IF3 ID3
stalled stages IF4 ID4 EX4
IF5 ID5
Each stall or kill introduces a bubble in the pipeline
 CPI > 1
A new datapath, i.e., a bypass, can get the data from
the output of the ALU to its input
time t0 t1 t2 t3 t4 t5 t6 t
x0 + 10 IF1 ID1 EX1 MA1 WB1
 x1 + 17 IF2 ID2 EX2 MA2 WB2
IF3 ID3 EX3 MA3 W
IF4 ID4 EX4 MA4 W
IF5 ID5 EX5 MA5 W

January 31, 2012 CS152, Spring 2012 30

 Adding a Bypass
stall

x4  x1... x1 ...
0x4 bubble
E M W
IR IR IR
Add
1

ASrc
we
rs1
rs2
A
PC addr D rd1 we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
wdata
Select wdata

MD1 MD2

... When does this bypass help?

(I1) x1 x0 + 10 x1 M[x0 + 10] JAL 500
(I2) x4 x1 + 17 x4 x1 + 17 x4 x1 + 17
yes no no
January 31, 2012 CS152, Spring 2012 31
 The Bypass Signal
Deriving it from the Stall Signal

stall = ( ((rs1D =wsE).weE + (rs1D =wsM).weM + (rs1D =wsW).weW).re1D

+((rs2D =wsE).weE + (rs2D =wsM).weM + (rs2D =wsW).weW).re2D )

ws = Case opcode we = Case opcode

JAL X1 ALU, ALUi, LW, JALR (ws  0)
else rd JAL on
... off
ASrc = (rs1D=wsE).weE.re1D Is this correct?

No because only ALU and ALUi instructions can benefit

from this bypass
Split weE into two components: we-bypass, we-stall

January 31, 2012 CS152, Spring 2012 32

 Bypass and Stall Signals
Split weE into two components: we-bypass, we-stall
we-bypassE = Case opcodeE we-stallE = Case opcodeE
ALU, ALUi  (ws  0) LW, JALR  (ws  0)
... off JAL on
... off

ASrc = (rs1D =wsE).we-bypassE . re1D

stall = ((rs1D =wsE).we-stallE +

(rs1D=wsM).weM + (rs1D=wsW).weW). re1D
+((rs2D = wsE).weE + (rs2D = wsM).weM + (rs2D = wsW).weW). re2D

January 31, 2012 CS152, Spring 2012 33

 Fully Bypassed Datapath
stall PC for JAL, ...

0x4 bubble
E M W
IR IR IR
Add
ASrc 1

we
rs1
rs2
A
PC addr D rd1 we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
wdata
Select wdata
BSrc
MD1 MD2

Is there still
a need for the
stall signal ? stall = (rs1D=wsE). (opcodeE=LWE).(wsE0 ).re1D
+ (rs2D=wsE). (opcodeE=LWE).(wsE0 ).re2D

January 31, 2012 CS152, Spring 2012 34

 Pipeline CPI Examples
Measure from when first instruction
Time finishes to when last instruction in
sequence finishes.
Inst 1
3 instructions finish in 3 cycles
Inst 2
Inst 3 CPI = 3/3 =1

Inst 1
Inst 2 3 instructions finish in 4 cycles
Bubble CPI = 4/3 = 1.33
Inst 3
Inst 1
Bubble 1
Inst 2 3 instructions finish in 5cycles
Inst
Bubble
3 2 CPI = 5/3 = 1.67
Inst 3
January 31, 2012 CS152, Spring 2012 35
 Resolving Data Hazards (3)
Strategy 3:

Speculate on the dependence. Two cases:

Guessed correctly  do nothing

Guessed incorrectly  kill and restart

…. We’ll later see examples of this approach in

more complex processors.

January 31, 2012 CS152, Spring 2012 36

 Speculation that load value=zero
stall PC for JAL, ...

0x4 bubble
E M W
IR IR IR
Add
ASrc 1

Guess_zero
we
rs1
rs2
0
A
PC addr D rd1 we
inst IR wa ALU Y addr
wd rd2
Inst GPRs B rdata
Memory Data
Imm Memory R
wdata
Select wdata
BSrc
MD1 MD2

Guess_zero= (rs1D=wsE). (opcodeE=LWE).(wsE0 ).re1D

Also need to add circuitry to remember that this was a guess and flush
pipeline if load not zero!
Not worth doing in practice – why?
January 31, 2012 CS152, Spring 2012 37
 Control Hazards
• What do we need to calculate next PC?

– For Jumps
» Opcode, PC and offset
– For Jump Register
» Opcode, Register value, and PC
– For Conditional Branches
» Opcode, Register (for condition), PC and offset
– For all other instructions
» Opcode and PC
• have to know it’s not one of above

January 31, 2012 CS152, Spring 2012 38

 PC Calculation Bubbles
time
t0 t1 t2 t3 t4 t5 t6 t7 ....
x0 + 10 IF1 ID1 EX1 MA1 WB1
x2 + 17 IF2 IF2 ID2 EX2 MA2 WB2
IF3 IF3 ID3 EX3 M
IF4 IF4 ID4 E

time
t0 t1 t2 t3 t4 t5 t6 t7 ....
IF I1 - I2 - I3 - I4
ID
Resource I1 - I2 - I3 - I4
EX
Usage I1 - I2 - I3 - I4
MA I1 - I2 - I3 - I4
WB I1 - I2 - I3 -
-  pipeline bubble

January 31, 2012 CS152, Spring 2012 39

 Speculate next address is PC+4
PCSrc (pc+4 / jabs / rind/ br)
stall

Add
E M
0x4 bubble
Add IR IR

Jump? I1

PC addr
inst IR

104 Inst
Memory I2

I1 096 ADD A jump instruction kills (not stalls)

I2 100 J 304 the following instruction
I3 104 ADD kill
I4 304 ADD
How?

January 31, 2012 CS152, Spring 2012 40

 Pipelining Jumps
PCSrc (pc+4 / jabs / rind/ br)
stall
To kill a fetched
instruction -- Insert
a mux before IR
Add
E M
0x4 bubble
Add IR IR

Jump? II21 I1

IRSrcD
Any
addr
interaction
PC bubble
inst IR between
304
104 Inst
bubble
I2
stall and
Memory
jump?
IRSrcD = Case opcodeD
I1 096 ADD
J, JAL bubble
I2 100 J 304
... IM
I3 104 ADD kill
I4 304 ADD
January 31, 2012 CS152, Spring 2012 41
 Jump Pipeline Diagrams
time
t0 t1 t2 t3 t4 t5 t6 t7 ...
(I1) 096: ADD IF1 ID1 EX1 MA1 WB1
(I2) 100: J 304 IF2 ID2 EX2 MA2 WB2
(I3) 104: ADD IF3 - - - -
(I4) 304: ADD IF4 ID4 EX4 MA4 WB4

time
t0 t1 t2 t3 t4 t5 t6 t7 ....
IF I1 I2 I3 I4 I5
ID I1 I2 - I4 I5
Resource
Usage EX I1 I2 - I4 I5
MA I1 I2 - I4 I5
WB I1 I2 - I4 I5
-  pipeline bubble

January 31, 2012 CS152, Spring 2012 42

 Pipelining Conditional Branches
PCSrc (pc+4 / jabs / rind / br)
stall

Add
E M
0x4 bubble IR IR
Add

BEQ? I1
Taken?

IRSrcD

PC addr bubble A
inst IR
ALU Y
104 Inst
Memory I2

I1 096 ADD
Branch condition is not known until
I2 100 BEQ x1,x2 +200
the execute stage
what action should be taken in the
I3 104 ADD
decode stage ?
I4 304 ADD
January 31, 2012 CS152, Spring 2012 43
 Pipelining Conditional Branches
PCSrc (pc+4 / jabs / rind / br)
stall
?

Add
E Bcond? M
0x4 bubble
IR IR
Add

I2 I1
Taken?
IRSrcD

PC addr bubble A
inst IR
ALU Y
108 Inst
Memory I3

If the branch is taken

I1 096 ADD - kill the two following instructions
I2 100 BEQ x1,x2 +200 - the instruction at the decode stage
I3 104 ADD is not valid
I4 304 ADD  stall signal is not valid
January 31, 2012 CS152, Spring 2012 44
 Pipelining Conditional Branches
PCSrc (pc+4/jabs/rind/br) stall

Add
E Bcond? M
IRSrcE
0x4 bubble
IR IR
Add

Jump? I2 I1
Taken?
PC
IRSrcD
PC addr bubble A
inst IR
ALU Y
108 Inst
Memory I3

If the branch is taken

I1:096 ADD - kill the two following instructions
I2:100 BEQZ x1,x2 +200 - the instruction at the decode stage
I3:104 ADD is not valid
I4:304 ADD  stall signal is not valid
January 31, 2012 CS152, Spring 2012 45
 Branch Pipeline Diagrams
(resolved in execute stage)

time
t0 t1 t2 t3 t4 t5 t6 t7 ....
(I1) 096: ADD IF1 ID1 EX1 MA1 WB1
(I2) 100: BEQZ +200 IF2 ID2 EX2 MA2 WB2
(I3) 104: ADD IF3 ID3 - - -
(I4) 108: IF4 - - -
(I5) 304: ADD IF5 ID5 EX5 MA5

time
t0 t1 t2 t3 t4 t5 t6 t7 ....
IF I1 I2 I3 I4 I5
ID I1 I2 I3 - I5
Resource
Usage EX I1 I2 - - I5
MA I1 I2 - - I5
WB I1 I2 - - I5
-  pipeline bubble

January 31, 2012 CS152, Spring 2012 46

 Reducing Branch Penalty
(resolve in decode stage)
• One pipeline bubble can be removed if an extra
comparator is used in the Decode stage – issues?

0x4 Taken?
Add

Bcomp
we
rs1
rs2 A
PC addr D rd1
inst IR wa ALU
wd rd2
Inst GPRs B
Memory
Imm
Select
BSrc MD1

January 31, 2012 CS152, Spring 2012 47

 Acknowledgements
• These slides contain material developed and
copyright by:
– Arvind (MIT)
– Krste Asanovic (MIT/UCB)
– Joel Emer (Intel/MIT)
– James Hoe (CMU)
– John Kubiatowicz (UCB)
– David Patterson (UCB)

• MIT material derived from course 6.823

• UCB material derived from course CS252

January 31, 2012 CS152, Spring 2012 48