PCI Express 4.

0 Electrical Previews Parts I & II

Dan Froelich, Intel Corporation, EWG Member Gerry Talbot, AMD, EWG Co-Chair

Copyright 2012, PCI-SIG, All Rights Reserved

Disclaimer
The information in this presentation refers specifications still in the development process. This presentation reflects the current thinking of various PCI-SIG workgroups, but all material is subject to change before the specifications are released.

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

Overview
PCIe 4.0 motivations and assumptions Choice of data rate Channel Pathfinding studies
CEM connector Channel improvements Channel HVM simulations

Silicon Pathfinding studies

Tx/Rx specifications Die-pad cap mitigation

Specification update
Reorganization of electrical section Design collateral to be included Specification release timeline

Next steps for PCIe 4.0 PCIe 3.0 ECRs

PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

PCIe 4.0 Motivations and Assumptions

Looking forward we continue to see a requirement to increase I/O bandwidth
Graphics, Networking, Storage

Motivations for PCIe 2.x->3.0 apply equally for 3.0->4.0 Given the eco-system impact of a new generation 2x increase in delivered bandwidth is required Highly desirable to extend PCIe 3.0 infrastructure and PHY architecture for another generation
Moving to a new infrastructure such as electrical or optical waveguides breaks backwards compatibility Highly desirable to preserve current usage models With incremental improvements 3.0 PHY architecture is capable of significantly higher data rates
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

EWGs Analysis Approach

PHY assumed to have similar EQ solution to 3.0 Silicon performance assumed to scale with data rate
Jitter performance, Return loss, Tx bandwidth This assumption is currently being refined

Evaluate CEM channels to determine maximum data rate

Two data rates 16GT/s and 24GT/s initially considered Determine channel topology limiters Evaluate mitigation techniques Investigate component improvements

Conclusion from pathfinding work

Existing CEM topologies do not support 24GT/s 16GT/s is the PCIe 4.0 data rate

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

Channel Topologies
Target max length PCIe 3.0 server channel is ~20 with 1 or 2 connectors Pathfinding for 16GT/s shows ~12-14 with 1 or 2 connectors possible Even with reduced channel length mitigation is required
Improvements to the CEM connector launch Clean up via transitions Minimize crosstalk Center channel impedance ~85ohm

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

Client Topology
Bottom-side microstrip route
Add-in card package Root package Socket CEM connector

Motherboard Swept length

30mil pair-pair space

Add-in card

Top-side microstrip route

Root package Socket

Add-in card package

CEM connector

Motherboard Swept length Break-out

Swept length

5mil pair-pair space

PCI-SIG Developers Conference

30mil pair-pair space

7

Copyright 2012, PCI-SIG, All Rights Reserved

Add-in card

Server 2-connector Topology

PKG

acLen {1-3}

CONN

rsrLen {1-3}

PKG

SKT

mbLen {4-10} stripline, 0.015 LossTan, 70,85,100 ohms

CAP

CONN

Stripline route assumed as this has worst via stubs

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

CEM Connector Improvements

Existing CEM connector has three problems for 16GT/s signaling
Insertion loss increase at 8GHz Return loss at 8GHz Significant peak in FEXT at 8GHz

Significant improvements can be made by changing footprint

Improves the launch into the connector structure

Under review by connector vendors

Goal is to minimize cost impact
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

Possible Performance Improvement

Current CEM connector
IL RL RL

Improved CEM connector

IL

FEXT FEXT

Substantial improvement in IL, FEXT and RL by creating a true differential launch from board into connector

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

10

Modifications to Footprint

Possible CEM connector launch improvements under consideration

PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

11

SMT Measurement Result

-1.3dB

Existing SMT connectors get close to the through hole-launch Additional measurements in progress on improved footprints
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

12

PCB Etch Loss/Inch

Microstrip Stripline

LossTangent=0.01 LossTangent=0.01

LossTangent=0.025

LossTangent=0.025

Channel performance is approximately proportional to PCB loss Loss tangent is more significant at 16GT/s
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

13

Loss Tangent Impact Eye Height

Swept loss tangent form 0.01, 0.015, 0.025

Loss tangent

Loss Tan of 0.025 was assumed for 8GT/s Changing to 0.015 is of medium risk/cost in future Using to 0.015 increases solution space by about 2 Changing to 0.010 starts to have diminishing returns for the cost
Copyright 2012, PCI-SIG, All Rights Reserved

PCI-SIG Developers Conference

14

Bidir TDR of 2-connector Server Channel

FCLGA Package

LGA Socket

BGA Package

Connector transitions clearly visible in TDR data

PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

15

Via Stub Impact

For 2-connector server channel there are 4 main vias
Two for main board to go to stripline and back up to AC cap
- 11 and 58 mills

Two for the connectors

- 0 mills and 58 mills

Worst combination can reduce the total routing length by >3

11 58

Via Stub height

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

16

Tool Improvements Needed

Channel response at >8GHz is affected by large number of features in the channel
Pre-layout evaluation of topology choices is a complicated multi-dimensional problem Need to be able to quickly build and test many different options Large number of HVM permutations need to be evaluated to determine robustness of solution

Seasim has been enhanced to allow EWG members to efficiently evaluate these options
Once validated this tool will be made available to the PCI-SIG membership for 4.0 channel compliance
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

17

Improvements to Seasim
Addition of a GUI form based interface
Underlying config file interface to seasim is unchanged A simple form based dialogue tool added Tab based interface to group config controls by context Ability to save and load configurations Launch (and kill) seasim from GUI

Touchstone channel modeling

A set of touchstone files can be cascaded to form die-pad to die-pad channel A vector of left hand and right hand ports define connections between S-parameters Rx port and set of Tx ports define step responses to be generated Tx amplitude and Gaussian bandwidth can be specified Addition of a PCIe 4.0 include file for simulation conditions

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

18

Seasim Channel
Allows whatif analysis on the channel components by changing the touchstone files that are concatenated together for the channel
Different analyses can be selected as the channel is tuned Either a pre-saved config can be loaded or the pciegen4.inc for normal sim conditions The other tabs allow simulation conditions to be changed from the default config
19

PCI-SIG Developers Conference

Client HVM Sweeps

At 16GT/s small changes in the channel have a big impact on eye opening An end-end reflection peaks and nulls when flight time changes by 62.5ps or ~0.42 Impedance variations between motherboard and add-in card introduce low frequency reflections that interact with the adapted EQ solution Different root complex package responses vary significantly Topology differences between top and bottom microstrip routing impact reflections Different connector models impact channel reflections To capture solution space sensitivity channel parameters can be swept using seasim Set of component touchstone files built Seasim sweep capability allows large number of cases to be studied
- In this example ~15,000 cases tested
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

20

Seasim HVM Sweep

Seasim can be used to define HVM sweeps The channel model can be swept to represent manufacturing variations of impedance or loss To consider the impact of different PCB layout the length of different channel segments can be swept

Seasim will launch jobs in parallel then collect results and plot them

PCI-SIG Developers Conference

21

Example Client Channel Simulation

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

22

Client Channel HVM Sweep

Min eye height 47mV

Min eye width 0.38UI

Max IL -21dB

Variables swept: Motherboard: - Length: 1-11 - Impedance 70/100ohm Root package - Length 10-30mm - Impedance 80/90ohm - Loss hi/lo EP package - Length 10/30mm - Impedance 80/90 - Loss hi/lo AIC etch - 70/90 15,360 cases

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

23

Server 2-conn Sweep

Eye Height

Total Length inches

Eye Width

Insertion loss @8GHz

PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

24

Channel Recommendations
The CEM form factor is the most important usage model for PCI Express
Can be extended by another generation with improvements to CEM connector launch

Current CEM channels are electrically very complex beyond 8GT/s

Discontinuities and crosstalk from packages, sockets, vias, etch, coupling capacitors, CEM connector Non-monotonic frequency domain behavior yields unpredictable data rate scaling

To extend current infrastructure requires enabling SIG membership to design and build cleaner channels
Tuning via launches, minimizing layer transitions, careful layer choices For longer reach channels, back-drilling, lower loss materials and repeaters will be required

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

25

Passive Tx/Rx Improvements

PCIe 3.0 Tx/Rx BW constrained by CPAD Requires a BW > 10 GHz Two options considered
Reduce CPAD to ~ 400 fF Add T-coils to Tx and Rx

T-coils also improve RL, reducing reflections Analysis indicates that T-coils on Tx and Rx decrease IL by ~ 5 dB

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

26

T-coil Model
Blue block is normal driver model in time domain simulator
CPAD = CTX - CE. (Tx or Rx pad capacitance) CESD = 0.35pF (ESD capacitance) CTX = 0.9 pf RSERIES = 2W (Metalization resistance)
CESD/12

Example of metal stackup

1250*CESD

1250*CESD

pin
RSERIES

RSERIES RTX

-417*CESD

80 fF CTX-CESD CESD

pin

T-coil lumped element Driver Model

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

27

T-coil Effect on IL
Blue curve is Tx package with 0.9pF Cpad Green is with 0.35pF of the 0.9pF compensated for with T-coil. Red is with Cpad of 0.55pF (0.9-0.35pf) The magnitude of the ripple on the IL with the 0.9pF cpad (blue) is driving the IL down to -10dB at 8GHz

CTX = 0.9 pf, no T-coil CTX = 0.55 pf, no T-coil Tcoil, CPAD= 0.55pf, CESD= 0.35pf

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

28

T-coil Effect on RL
-1.5 dB -2.5dB -5.0dB

CTX = 0.9 pf CTX= 0.55 pf T-coil, CPAD= 0.55pf, CESD= 0.35pf

RL improves with t-coil for two reasons:

RESD decoupled from high speed signal path RSERIES adds a small amount of DC resistance to the signal path

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

29

Platform Topology
Variables
main board 7-9, stripline, tand 0.015, Z: 70-100 Riser card 1.5-3.5, mstrip, tand 0.015, Z: 70-100 Addin Card 1.5-3.5, mstrip, tand 0.015, Z: 70-100 Connector is EWG web site, 8mm 5_26_2011 Tx package is 10-25mm with socket Rx package is 10-20mm BGA 3 tap EQ, pre, cur, post 800mV swing 0.9-0.55pF CTLE ADC: 4 to 12 dB, fp1 1.5-4 GHz 2,4,6,8 tap DFE
PKG

EQ training: exhaustive grid search Vias

Motherboard via stub 11 or 58 mill Conn via stub 0 or full 64 mill At both Tx and Rx ESD Cap is 0.35pF for both Rx-cpad = Tx-cpad-0.1pF RSERIES = 2W

Models

T-coils

Tx

Rx

1.5-3.0

CONN

PKG

SKT

Len=7- 9 Zo= 70,85,100 ohms

CAP

CONN

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

30

Effect on IL
0.9/0.8pF CPAD introduces too much rolloff in Tx and Rx
Time constant for 0l;9 pf, 50W = 45 ps vs. UI of 40 ps

0.55/0.45 pf CPAD reduces IL slightly, but still exceeds the 25 dB limit Adding T-coils to Tx and Rx decreases IL to 25 dB, which can be equalized
CTX = 0.9 pf CTX= 0.55 pf T-coil, CPAD= 0.55pf, CESD= 0.35pf

-25 dB -27 dB

-30 dB

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

31

Pulse Response Comparison

C=0.55pF Tx yields the largest pulse height C=0.9 pf with T-coil gives the least amount of pre and post cursor interference
0.9 pf Tx

0.55 pf Tx
Tcoil

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

32

Eye Height Comparison

0.9 pf, with T-Coil

0.9 pf with T-coil

0.9 pf no T-coil

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

0.9pF, no T-coil

Top is with T-Coil 0.9pF Tx Total T-coil 0.35pF in ESD tand=0.015, short via stubs Increasing # of DFE taps reduces delta

33

Eye Height Distribution 15

2-tap and 6-tap DFE with and without T-coils, length=15 total
2 DFE mean Tcoil 0.9pF 45mV 32mV 6 DFE mean 53mV 47mV Tcoil 0.9pF 2 DFE min 11mV 0mV 6 DFE min 27mV 22mV

2-tap DFE

6-tap DFE

2-tap DFE +T-coil

PCI-SIG Developers Conference

6-tap DFE + T-coil

Copyright 2012, PCI-SIG, All Rights Reserved

34

Eye Width Comparison

0.9 pf, with T-Coil

0.9 pf with T-coil

0.9pF, no T-coil

Top is with T-Coil 0.9pF Tx Total T-coil .35pF in ESD tand=0.015, short via stubs Increasing #DFE taps reduces delta Eye Height target of 25mV is current limiter with 8 taps DFE

0.9 pf no T-coil
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

35

Eye Width Distribution

2-tap and 6-tap DFE with and without T-coils, length=15
2 DFE mean T-coil 0.9pF 0.43UI 0.37UI 6 DFE mean 0.48UI 0.45UI T-coil 0.9pF 2 DFE min 0.23UI 0.04UI 6 DFE min 0.35UI 0.28UI

2-tap DFE

6-tap DFE

2-tap DFE +T-coil

PCI-SIG Developers Conference

6-tap DFE + T-coil

Copyright 2012, PCI-SIG, All Rights Reserved

36

0.9pF with T-coil vs. 0.55pF no t-coil

EH mean Tcoil 0.55pF 40mV 40mV EW mean 0.41UI 0.41UI

C=0.55 pf Eye width

C=0.9 pf Eye width

C=0.55 pf Eye height

PCI-SIG Developers Conference

C=0.9 pf Eye height

Copyright 2012, PCI-SIG, All Rights Reserved

37

Tx/Rx Circuit Improvements

Tx jitter and Rx timing uncertainty must scale approximately with data rate
Some parameters such as Tx PWJ may need to scale more than a factor of 2x wrt. PCIe 3.0

At this time we have not obtained precise jitter estimates for Rx or Rx circuits Estimates will go into the Rev 0.3 specification

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

38

Equalization Capability
Only minor enhancements to Tx or Rx equalization are anticipated for PCIe 4.0
The channel IL at Nyquist for PCIe 4.0 is not appreciably worse than for PCIe 3.0 Proposed eq capabilities
- Tx FFE: One pre and one post cursor tap. Retain the same presets and coefficient range/resolution - Reference Receiver (actual implementations may have more) - Rx CTLE: Same resolution, but DC gain may be increased - Rx: DFE, increase the number of taps to 2-3, retain same tap range and magnitude - Training method would remain the same, although only certain presets would be used
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

39

Clocking and RefClk

Add support for independent SSC Implications include the following
Model Rx CDR must suppress SSC spurs RefClk filter mask is changed to reflect updated model CDR

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

40

Repeater/Retimer/Re-driver
Active component for channel extension will be important in more systems for PCIe 4.0 Allowed architectures and compliance for channel extension may need to be specified for PCIe 4.0 Areas for investigation
Interaction with TX Equalization negotiation protocol Clocking Electrical specifications Models for simulation

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

41

PCIe 4.0 Electrical Spec

Undergoing reorganization to achieve a more regularized format
8.0G/Ts methodology and parameters will be applied to 16G/Ts Same parameter definitions retroactively applied to 2.5G/Ts and 5.0G/Ts 2.5G/Ts and 5.0G/Ts parameters values will be defined to guarantee interoperability
- Some tightening of certain Tx parameters is likely

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

42

Tx Jitter Specifications
Parameter TTX-UTJ TTX-UDJDD TTX-UPW-TJ TTX-UPW-DJDD TTX-DDJ Description Tx uncorrelated total jitter Tx uncorrelated deterministic jitter Total uncorrelated PWJ Deterministic DjDD uncorrelated PWJ Data dependent jitter 2.5 GT/s 100 (max) 40 (max) 75 (max) 32 (max) 60 (max) 5.0 GT/s 50 (max) 20 (max) 38 (max) 16 (max 30 (max) 8.0 GT/s 31.25 (max) 12 (max) 24 (max) 10 (max) 18 (max) 16 GT/s 15.62 (max) 6 (max) 12 (max) 5.0 (max) 9.0 (max)

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

43

Other PCIe 4.0 Tx Parameters

Tx differential and common mode RL masks
Extend 2.5-4.0 GHz PCIe 3.0 limits to 8 Ghz while retaining the same respective values Makes sense given that Tx/Rx RL at 8G is improved via T-coils

Redefine Tx behavior during EIEOS

Optimization may be needed studies ongoing
Double the # of symbols in the high and low intervals Turn off de-emphasis to avoid hitting boost limit

Others?

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

44

Transmitter Equalization
Max PCIe 4.0 channel IL remains approx the same s for PCIe 3.0 Plan is to retain same equalization presets
Training will require that only a subset of the presets be used (P7 and P8)

Equalization coefficient range and resolution also are intended to remain unchanged EIEOS signaling will likely change such that no TxEQ is applied during the EIEOS interval

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

45

Transmitter Jitter Spec

PCIe 4.0 uses same jitter parameters as PCIe 3.0
TTX-UPW-TJ, TTX-UPW-DJDD, TTX-DDJ, TTX-UTJ and TTX-UDJDD Jitter will need to scale approximately with bitrate De-embedding approach will likely remain the same

PCIe 1.x and PCIe 2.x jitter parameters will be recast into the same form as the PCIe 3.0 parameters
Backward compatibility will be guaranteed Some PCIe 1.x/2.x parameters will be effectively tightened Example: PCIe 2.x TMIN-PULSE parameter will be converted into TTX-UPW-TJ and TTX-UPW-DJDD
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

46

PCIe 4.0 Rx Specification

Will continue to rely upon a stressed eye approach where both EH and EW are stressed
Calibration channels IL will need to be reduced to yield ~24 dB at 8 GHz Behavioral package model needs to comprehend reduced CPAD or include T-coil models Behavioral DFE model to have increased number of taps, at least 2 More capable CTLE model

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

47

PCIe 4.0 Design Collateral

Will be included in PCIe 4.0 spec in response to customers requests
A purely normative spec does not give sufficient information to allow many developers to successfully implement designs.

Some design collateral will be included as separate subsections for Tx, Rx, channel, etc. CEM specific collateral will be included in CEM spec

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

48

PCIe 4.0 Design Collateral

CEM s-parameter masks
CEM connector: IL, RL, FEXT Tx and Rx: RL Baseboard: IL, RL adapter: IL, RL

Reference models for Tx die/pkg, channel, Rx die/pkg Others

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

49

PCIe 3.0 ECNs

Support independent Tx and Rx RefClks with SSC on each
This is an optional requirement Certain cabled applications may not route RefClk, so adapters RefClk is independent from root ports Reduce pincount and minimize EMI/RFI

Rx CDR needs to filter SSC

20 db/dec insufficient to filter RefClk spurs and meet the 1 ps RMS RefClk jitter spec Previously a 1st order CDR was acceptable

Support will likely be required for PCIe 4.0 base spec compliant silicon
PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

50

Next Steps
EWG work
Start scoping Tx, Rx jitter parameters

Rev 0.3 spec release

Common parameters across PCIe 1.x-PCIe 4.0 Estimated values for all PCIe 4.0 parameters Merge PCIe 1.x, PCIe 2.x specific material into PCIe 3.0 spec format

PCI-SIG Developers Conference

Copyright 2012, PCI-SIG, All Rights Reserved

51

Thank you for attending the PCI-SIG Developers Conference 2012

For more information please go to www.pcisig.com

PCI-SIG Developers Conference Copyright 2012, PCI-SIG, All Rights Reserved

52