PCI Express

In the last PCIe learning session we have seen how the PCIe devices Link up and how the
devices are discovered by the Software or the OS and how the Bus number, Device number and
Function number (BDF) are assigned. In this session we will try to understand how the PCIe
packets are formed at each layer and what are the additional overheads added at each layer before
it is sent to the link partner and the popular 8b/10b encoding scheme.

Higher Layer (Software)

(Config Rd/Wr, Mem Rd/Wr, Message Req/Compl)

Transaction Layer Packet (TLP) Transaction Layer Packet (TLP)

Header Data Payload ECRC Header Data Payload ECRC

Transaction Layer
Link Layer Packet DLLP packets Link Layer Packet DLLP packets
Replay Sequence ACK/NAK/ Sequence ACK/NAK/
Buffer TLP LCRC CRC TLP LCRC CRC
Number Update/FC Number Update/FC

Data Link Layer

Weighted Arbiter /
DeMux
Mux

Phy Layer Packet Phy Layer Packet

Training Training
START Link Layer START Link Layer
END Sequences / END Sequences /
(STP) Packet (STP) Packet
Ordered Sets Ordered Sets

Weighted Arbiter / Electronicspedia©

DeMux
Mux

8b/10b or 128b/130b Encode 8b/10b or 128b/130b Decode

Serializer (SERDES) De-Serializer (SERDES)

Differential Driver (Analog) Differential Receiver (Analog)

Tx+ Tx- Physical Layer Rx+ Rx-

Figure: PCIe packet formation at each layer

RAVIKUMAR JAVALI 1

ELECTRONICSPEDIA

RAVIJAVALI
 As shown in the above diagram, the software indicates the Transaction Layer to form TLP
packet, it can be Config Rd/Wr, Memory Rd/Wr or Message Requests. The actual data to be sent
across is called “Data Payload” To this Data Payload there will be a Header gets added indicating
the packet type, payload size and few other attributes. Also, it can add optional ECRC and pass
the information as a TLP packet to Data Link Layer.

Data Link Layer adds the sequence number to the TLP received from Transaction layer
and calculates the Link CRC and adds to the packet. Data link layer also has some other packets
called DLLPs, which are required for the flow control management. Data Link layer also keeps
the copy of the TLP packets that are sent across, this will be resent / replayed if the other link
partner gives Negative Acknowledgment (NAK), otherwise they will be flushed once we receive
the ACK. These three types of packets are arbitrated/muxed (based on implementation scheme)
and sent to the Physical Layer as a Link layer packets.

Physical Layer adds the START and END frame to the packet received from the Data Link
Layer. Physical Layer also has its own Training Sequences (TS1/TS2) and Ordered Sets to send
across to link partner for the link training, maintaining the clock frequency to match pclk and for
link low power states. These packets are arbitrated/muxed and further sent to the Encoding scheme
either 8b/10b for PCIe Gen1/2 or 128b/130b for Gen3 and higher speeds. Then the encoded data
is serialized by passing it through the Parallel to Serial converter (SERDES). The serialized data
would be in binary 1’s and 0’s, which will converted into appropriate differential voltage levels
(Tx+/-) by the differential driver.

The Receiver side of the link partner receives these packets in serialized manner which will
deserialized to get the parallel data and then it is passed through the Decoder and the all the START
and END framed added at the PHY layer will be removed and the packet is sent to the DL Layer.

DL layer calculates the CRC of the received packet and compares with the LCRC value
attached with the received packet, if it matches then ACK DLLP will be sent to the Link partner if
LCRC is not matching then NAK will be sent. Also Sequence number and LCRC will be removed
before passing the data to Transaction layer.

RAVIKUMAR JAVALI 2

ELECTRONICSPEDIA

RAVIJAVALI
 Transaction layer receives the packet for and calculates the optional ECRC (similar to the
way it is done in Data Link Lyer LCRC). Data will be consumed / handled appropriately.

8b/10b Encoding-Decoding
Do you know why the 8b/10b encode/decode scheme is used in PCIe?

8b/10b encoding scheme was invented by IBM. For every 8bit of data the scheme encodes it into
10bit of data. The 10bit of data is a special encoded data it provides below advantages.

1. PCIe protocol embeds the clock into the data and sends it across to link partner. Meaning
there is no separate indication from transmitter side to receiver side (link partner) about
the clock frequency the transmitter is running at. This information will be embedded
inside the data. This is done with the help of 8b/10b encoding. At the receiver side the
clock will be recovered after decoding.
2. It helps in maintain the DC balance. Maintaing the DC balance is necessary for the high-
speed serial data communications. 8b/10b encoding scheme ensures that the values are
encoded in a way that it will have even number of 1’s and 0’s. Or the disparity of the bits
will not be more than 2.

Every 8b data is split as 5b + 3b. 5bit data is encoded into 6bit and 3bit data is encoded into 4bit
resulting in total 10bit encoded data.

Eg: In PCIe, the PHY layer adds something called COM character which has 8bit value of 0xBC

In binary this will be 10111100b which will be encoded as either 1100000101 or 0011111010.
The 10bit encoded values are complement to each other. The 10bit encoded helps the receiver
with a symbol lock and bit lock

1100000_101 or 0011111_010. The first 7 digits set is called a symbol and when the receiver
sees this pattern it is called symbol lock. And then the next transition 101 or 010 is for the clock
recovery which is called bit lock.

RAVIKUMAR JAVALI 3

ELECTRONICSPEDIA

RAVIJAVALI
As shown in the below figure the 8bit data is fed to the Encoder, the encoder has encoding
equations (which is mentioned in the PCIe Spec) which will be used to encode the data into 10bit
and the 10b bit encoded it fed to the Serializer (Parallel to Serial Converter) which serializes the
data at 10x clock frequency and sent across.

H G F E D C B A 8b data

8b/10b encoding module

j i h g f e d c b a 10b data

j i h g f e d c b a
Serial Data will be sent at a
speed x10 clock frequency

If the PCIe is in Gen1 speed, assuming the data width is 8bit parallel data, which will converted
into 10bit. The serialized data will be sent at 2.5GHz to get the Gen1 speed of 2.5GT/s. The
parallel interface will be working at 250MHz.

If the parallel interface data width is changed to 32bit then the parallel clock needs to run at
62.5MHz to get the 2.5GT/s for Gen1 speed.

62.5MHz * 40b (this is because 32bit will be converted to 40bit) = 2.5 GT/s

RAVIKUMAR JAVALI 4

ELECTRONICSPEDIA

RAVIJAVALI