February 2004

WHITE PAPER
February 2004
™

PCI EXPRESS TECHNOLOGY

Jim Brewer, Dell Business and Technology Development
Joe Sekel, Dell Server Architecture and Technology

Formerly known as 3GIO, PCI Express is

the open standards- based successor Contents
to PCI and its variants for server- and PCI Bus .............................................. ...................1
Client Systems............................... ...................2
client-system I/O interconnects. Un-
Server Systems............................. ...................3
like PCI and PCI-X, which are based PCI Express Technology...................... ...................4
on 32- and 64-bit parallel buses, PCI PCI Express Advanced Features.......... ...................5
Express uses high-speed serial link tech- Advanced Power Management...... ...................6
nology similar to that found in Gigabit1 Ethernet, Serial Support for Realtime Data Traffic... ...................6
Hot Plug and Hot Swap.................. ...................6
ATA (SATA), and Serial-Attached SCSI (SAS). PCI Ex-
Data Integrity and Error Handling... ...................6
press reflects an industry trend to replace legacy shared PCI Express Form Factors.................... ...................6
parallel buses with high-speed point-to-point serial bus- PCI Express Standard and Low-Profile Cards.....6
es. PCI Express Mini Card.................... ...................8
ExpressCard................................... ...................8
The new bus technology is expected to allow the PCI PCI Express Server I/O Module...... ...................9
Express transmission rates to keep pace with processor Sample PCI Express Architectures...... ...................9
and I/O advances for the next 10 years or more. Sys- Client Systems............................... ...................9
tems with PCI Express will begin appearing around the Portable Computers....................... ...................9
middle of 2004. Server Systems............................. ...................10
Enabling Future Modular Designs........ ...................10
PCI Express has the following advantages over PCI: Conclusion ......................................... ...................11

• Serial technology providing scalable performance.

• High bandwidth—Initially, 5–80 gigabits per second features. For instance, PCI Express will initially be de-
(Gbps) peak theoretical bandwidth, depending on ployed as a replacement for the AGP8X graphics bus in
the implementation. client systems, providing high bandwidth and support
• Point-to-point link dedicated to each device, instead for multimedia traffic. It will also coexist with and ulti-
of the PCI shared bus. mately replace the PCI-X bus in server systems.
• Opportunities for lower latency (or delay) in server In this white paper, we begin with a review of the PCI
architectures, because PCI Express provides a bus and its variants (PCI-X and AGP) in client and server
more direct connection to the chip set Northbridge2 systems. The paper continues with a discussion of PCI
than PCI-X. Express technology, including its strengths, advanced
• Small connectors and, in many cases, easier imple- features, and form factors. We conclude with its impact
mentation for system designers. on computer system architectures.
• Advanced features—Quality of service (QoS) via
isochronous channels for guaranteed bandwidth PCI Bus
delivery when required, advanced power manage-
Since its inception in 1992, the PCI bus has become the
ment, and native hot plug/hot swap support.
I/O backbone of nearly every computing platform. The
PCI Express will replace the PCI, PCI-X, and AGP parallel original 33-MHz, 32-bit implementation delivers a peak
buses gradually over the next decade. It will initially re- theoretical bandwidth of 133 megabytes per second
place buses that need the additional performance or (MB/sec). Over time, the industry has evolved the plat-

1. This term does not connote an actual operating speed of 1 Gbps. For high-speed transmission, connection to a Gigabit Ethernet server and network infrastructure is required.
2. The term, Northbridge, refers to the controller for the processor bus, memory bus, AGP bus, and the link to the Southbridge. The term, Southbridge, refers to the I/O device controller.

Read Dell’s technology white papers @ www.dell.com/r&d 1

www.dell.com/r&d
PCI Express Technology

form architecture by offloading various functions to of a typical client PC system and the bandwidth of its
higher-bandwidth PCI derivatives, including AGP and I/O and graphics buses.
PCI-X, both of which are PCI variants. Table 1 presents
the peak bandwidth of the PCI, PCI-X, and AGP buses.

Peak 32-Bit Peak 64-Bit

Bus and Frequency Transfer Rate Transfer Rate
33-MHz PCI 133 MB/sec 266 MB/sec
66-MHz PCI 266 MB/sec 532 MB/sec
100-MHz PCI-X Not applicable 800 MB/sec
133-MHz PCI-X Not applicable 1 GB/sec
AGP8X 2.1 GB/sec Not applicable

Table 1. Bandwidth of PCI, PCI-X, and AGP Buses

Figure 1. Typical Client System Architecture
A close examination of PCI signaling technology reveals
a multidrop,3 parallel bus that is reaching its perfor- Client-System Bottlenecks
mance limits. The PCI bus cannot be easily scaled up in
Several client-system buses can limit system perfor-
frequency or down in voltage. In addition, the PCI bus
mance because of CPU, memory, and I/O device ad-
does not support features such as advanced power
vances: the PCI bus, AGP bus, and the link between the
management, native hot plugging/hot swapping of pe-
Northbridge and Southbridge.
ripherals, or QoS to guarantee bandwidth for real-time
operations. Finally, all of the available bandwidth of the PCI Bus. The PCI bus provides up to 133 MB/sec to con-
PCI bus is limited to one direction (send or receive) at a nected I/O devices. A number of I/O devices can satu-
time. Many communications networks support simulta- rate or consume a high percentage of this bandwidth.
neous bidirectional traffic, which minimizes message When more than one of these devices are active, the
latency. shared PCI bus is quickly stressed beyond its limits.

Figure 2 shows many of the contributors to the PCI bus

Client Systems
bottleneck. This figure shows the bandwidth required
The original PCI bus was designed to support 2D graph- by various communications, video, and external devices
ics, higher-performance disk drives, and local area net- that are serviced by the PCI bus. It can be seen that the
working. Not long after PCI was introduced, the multidrop, shared PCI bus is hard pressed to keep up
increasing bandwidth requirements of 3D graphics sub- with today's devices. This situation worsens with up-
systems outstripped the 32-bit, 33-MHz PCI bus band- coming peripheral devices with even higher data rates.
width. As a result, Intel and several graphics suppliers For example, Gigabit Ethernet requires a bandwidth
created the AGP specification, which defined a dedicat- of 125 MB/sec, which effectively saturates the
ed high-speed PCI bus for graphics operations. The 133-MB/sec PCI bus. The IEEE 1394b bus has a maxi-
AGP bus offloaded graphics traffic from the PCI system mum bandwidth of 100 MB/sec, which can saturate a
bus and freed up bandwidth for other communications standard PCI bus.
and I/O operations. In addition, Intel recently added
AGP. Over the past decade, video performance require-
dedicated USB 2.0 and Serial ATA links to the South-
ments have approximately doubled every two years.
bridge in its chip sets, further reducing the I/O demands
During this time, the graphics bus has transitioned from
on the PCI bus. Figure 1 shows the internal architecture
PCI to AGP, and from AGP to AGP2X, AGP4X, and finally

3. On a multidrop bus, all devices attached to it are connected to the same set of wires. When a device is using the PCI bus, no other device can communicate over the bus. All connected
devices must share the bus and wait their turn before sending or receiving data.

2
February 2004

The PCI Special Interest Group (PCI SIG) has been devel-
oping the PCI-X 2.0 specification, which will effectively
create a 64-bit, 266-MHz PCI-X bus with double the data
rate of the 133-MHz PCI-X bus. However, there are sig-
nificant design issues associated with extending these
parallel PCI-X bus variants. The connectors are large and
expensive, and stringent design requirements drive up
the cost of system boards significantly as frequencies
are increased. In addition, to avoid excessive electrical
loading at the higher speeds of PCI-X 2.0, only one I/O
device can be attached in a point-to-point configuration
to the PCI-X bus. It cannot be implemented as a shared
bus.

Server-System Bottlenecks
Figure 3 shows the internal system interconnects in a
Figure 2. Bandwidth of Devices Serviced by PCI Bus typical dual-processor server system. In this architec-
ture, high-bandwidth expansion is provided via a propri-
today's AGP8X. AGP8X operates at 2.134 gigabytes per
etary interface between the Northbridge and PCI-X
second (GB/sec). Despite this bandwidth, the progres-
bridge chips. Multiple PCI-X buses connect to high-
sive performance demands on the AGP bus are putting
speed expansion slots, 10-Gigabit Ethernet, and SAS/
considerable pressure on board design and intercon-
SATA drives. This architecture has some drawbacks.
nection costs. Like the PCI bus, extending the AGP bus
The proprietary PCI-X bridge chips connect multiple par-
becomes more difficult and expensive as frequencies
allel PCI-X buses to the chip set's proprietary serial inter-
increase.
connect. This approach is expensive, inefficient, and
Link Between Northbridge and Southbridge. Conges- introduces latency between I/O devices and the North-
tion on the PCI bus also affects the link between the bridge. For example, the approach connects a serial 10-
Northbridge and the Southbridge. SATA drives and USB Gbps fabric to a point-to-point, 64-bit parallel bus that is,
devices further stress this link. A higher-bandwidth link in turn, connected via a proprietary PCI-X bridge chip to
will be required in the future. a proprietary serial interconnect into the Northbridge.

Server Systems
In servers, the original 32-bit, 33-MHz PCI bus was ex-
tended to a 64-bit, 66-MHz bus with a bandwidth of 532
MB/sec. The 64-bit bus was recently extended to 100
and 133 MHz, referred to as PCI-X. The PCI-X bus con-
nects the server-system (and the high-end, dual-proces-
sor workstation-system) chip set to expansion slots,
Gigabit Ethernet controllers, and Ultra320 SCSI control-
lers embedded on the system board. A 64-bit PCI-X bus
at 133 MHz delivers 1 GB/sec of peak bandwidth be-
tween the system chip set and the I/O device. This is
sufficient bandwidth for the majority of immediate serv-
er I/O requirements, including Gigabit Ethernet,
Ultra320 SCSI, and 2-GB/sec Fibre Channel. However,
like PCI, PCI-X is a shared bus and is likely to require a Figure 3. Current Dual-Processor Server Architecture
higher-bandwidth alternative in 2004.

3
www.dell.com/r&d
PCI Express Technology

In addition, next-generation external server I/O technol- store (and flat address space) model. The layered archi-
ogies are expected to require much greater bandwidth tecture is discussed in the sidebar below, “PCI Express
than a 133-MHz PCI-X bus can provide. These technolo- Layered Architecture.”
gies include system-area fabrics such as 10-Gigabit
The PCI Express architecture defines a high-perfor-
Ethernet, 10-Gbps Fibre Channel, and 4x Infiniband.
mance, point-to-point, scalable, serial bus. A PCI Ex-
They also include future higher-speed hard-drive inter-
press link consists of dual simplex channels, each
faces such as 3-Gbps SATA and SAS. In the case of a
implemented as a transmit pair and a receive pair for si-
10-Gbps fabric, each 10-Gbps port will be able to trans-
multaneous transmission in each direction. Each pair
mit bidirectional data at a peak bandwidth of 2 GB/sec.
consists of two low-voltage, differentially driven pairs of
The 133-MHz PCI-X bus delivers a maximum of 1 GB/
signals. A data clock is embedded in each pair, using an
sec in one direction at a time. This suggests that the
8b/10b clock-encoding scheme to achieve very high
133-MHz PCI-X bus could throttle the peak bandwidth of
data rates. Figure 4 compares the PCI and PCI Express
these fabrics by as much as 50 percent. Although PCI-X
links.
2.0 at 266 MHz would double the PCI-X peak bandwidth
to 2 GB/sec, it would still fall short of the total 4 GB/sec
required by a dual-ported 10-Gbps fabric controller.

Dell believes that client and server systems require a re-

placement bus for the parallel PCI bus and its variants.

PCI Express Technology

PCI Express provides a scalable, high-speed, serial I/O
bus that maintains backward compatibility with PCI ap-
plications and drivers. The PCI Express layered architec-
ture supports existing PCI applications and drivers by Figure 4. PCI Versus PCI Express
maintaining compatibility with the existing PCI load-

PCI Express Layered Architecture

Configuration/Operating System Layer—Leverages the standard mechanisms defined
in the PCI Plug-and-Play specification for device initialization, enumeration, and configu-
ration. This layer communicates with the software layer by initiating a data transfer
between peripherals or receiving data from an attached peripheral. PCI Express is
designed to be compatible with existing operating systems, but future operating system
support is required for many of the technology’s advanced features.
Software Layer—Generates read and write requests to peripheral devices. PCI Express
maintains initialization and runtime software compatibility with PCI. Like PCI, the PCI
Express initialization model allows the operating system to discover add-in hardware
devices and allocate system resources. PCI Express retains the PCI configuration space
and the programmability of I/O devices. In fact, all operating systems will boot without
modification on a PCI Express system. The PCI runtime software model is also preserved,
enabling existing software to execute unchanged.
Transaction Layer—Transports read and write requests from the software layer to the
link layer using a packet-based protocol, and matches response packets to the original software requests. The transaction layer supports 32-bit and extended 64-bit
memory addressing. It also supports PCI memory, I/O, and configuration address spaces, as well as a new message space for in-band messages such as interrupts and
resets. This message space eliminates the need for numerous PCI and PCI-X sideband signals.
Link Layer—Adds sequencing and error detection cyclic redundancy codes (CRCs) to the data packets to create a reliable data transfer mechanism between the system
chip set and the I/O controller.
Physical Layer—Implements the dual simplex PCI Express channels. Implementations are flexible and various technologies and frequencies may be used. In this way,
initial silicon technology can be replaced easily with future implementations that are backward compatible. For example, fiber-optic technology might be used to increase
the data transfer rate.
Mechanical Layer—Defines various form factors for peripheral devices.

4
February 2004

The bandwidth of a PCI Express link can be scaled by In contrast to PCI, PCI Express has minimal sideband
adding signal pairs to form multiple lanes between the signals and the clocks and addressing information are
two devices. The specification supports x1, x4, x8, and embedded in the data. Because PCI Express is a serial
x16 lane widths and stripes the byte data across the technology with few sideband signals, it provides a very
links accordingly. Once the two agents at each end of high bandwidth per I/O connector pin compared to PCI.
the PCI Express link negotiate lane widths and frequen- This is designed to provide more efficient, smaller, and
cy of operation, the striped data bytes are transmitted cheaper connectors. Figure 5 compares the bandwidth
with 8b/10b encoding. per I/O connector pin of PCI, PCI-X, AGP, and PCI Ex-
press.
The basic “x1” link has
PCI Express “Coded” and
a peak raw bandwidth “Unencoded” Bandwidth
of 2.5 Gbps. Because
PCI Express bandwidth is commonly
the bus is bidirectional expressed as “encoded” bandwidth. PCI
(that is, data can be Express uses 8b/10b encoding, which
encodes 8-bit data bytes into 10-bit trans-
transferred in both di- mission characters. This approach improves
rections simultaneous- the physical signal so that bit synchroniza-
ly), the effective raw tion is easier, design of receivers and
transmitters is simplified, error detection is
data transfer rate is 5 improved, and control characters can be
Gbps. Table 2 summa- distinguished from data characters.
Figure 5. Comparison of I/O Bus Bandwidth Per Pin
rizes the encoded and The “encoded” bandwidth of a basic x1 PCI PCI Express technology achieves high data rates reli-
unencoded data rates Express lane is 5 Gbps. However, a more
accurate bandwidth figure is the “unen- ably by using low-voltage differential signaling. In this
(see sidebar) of x1, x4,
coded” bandwidth, which is 80 percent of 5 approach, the signal is sent from the source to the re-
x8, and x16 implemen- Gbps or 4 Gbps. Table 2 presents both
ceiver over two lines. One contains a “positive” image
tations, which are de- encoded and unencoded PCI Express band-
width. In this paper, we follow the common and the other, a “negative” or “inverted” image of the
fined in the initial industry practice of citing the higher signal. The lines are routed using strict routing rules so
generation of PCI encoded bandwidth figures.
that any noise that affects one line also affects the other
Express.
line. The receiver collects both signals, inverts the neg-
ative version back to the positive and sums the two col-
lected signals, which effectively removes the noise.
PCI Express Encoded Data
Implementation Rate Unencoded Data Rate The original PCI Express specification defines graphics
x1 5 Gbps 4 Gbps (500 MB/sec) cards with up to 75 watts of power. In addition, a new
x4 20 Gbps 16 Gbps (2 GB/sec) high-end PCI Express graphics specification is under
x8 40 Gbps 32 Gbps (4 GB/sec) development that defines cards of up to 150 watts.
x16 80 Gbps 64 Gbps (8 GB/sec) These higher power levels accommodate the require-
Table 2. PCI Express Bandwidth ments of graphics adapters, which currently peak at 41
watts for mainstream AGP cards and 110 watts for AGP
Future implementations of PCI Express will raise the Pro 110 cards.
channel communication frequency to even higher lev-
els. For example, a second generation of PCI Express PCI Express Advanced Features
could increase the communication frequency by a fac- PCI Express has advanced features that will be phased
tor of 2 or more. in as operating system and device support is developed
and as customer applications require them:
Because it is a point-to-point architecture, the entire
bandwidth of each PCI Express bus is dedicated to the • Advanced power management
device at the end of the link. Multiple PCI Express devic- • Support for real-time data traffic
es can be active without interfering with each other. • Hot plug and hot swap
• Data integrity and error handling

5
www.dell.com/r&d
PCI Express Technology

Advanced Power Management dressed pressing requirements of server and portable

computer platforms:
PCI Express has “active-state” power management,
which lowers power consumption when the bus is not • It is often difficult or impossible to schedule down-
active (that is, no data is being sent between compo- time on a server to replace or install peripheral
nents or peripherals). On a parallel interface such as cards. The ability to hot plug I/O devices minimizes
PCI, no transitions occur on the interface until data downtime.
needs to be sent. In contrast, high-speed serial interfac- • Portable computer users need the ability to hot plug
es such as PCI Express require that the interface be ac- cards that provide I/O functions such as mobile disk
tive at all times so that the transmitter and receiver can drives and communications.
maintain synchronization. This is accomplished by con-
PCI Express has native support for hot plugging and hot
tinuously sending idle characters when there is no data swapping I/O peripherals. No sideband signals are re-
to send. The receiver decodes and discards the idle quired and a unified software model can be used for all
characters. This process consumes additional power,
PCI Express form factors.
which impacts battery life on portable and handheld
computers. Data Integrity and Error Handling
To address this issue, the PCI Express specification cre- PCI Express supports link-level data integrity for all
ates two low-power link states and the active-state types of transaction- and data-link packets. Thus, it is
power management (ASPM) protocol. When the PCI Ex- suitable for end-to-end data integrity for high-availability
press link goes idle, the link can transition to one of the applications, particularly those running on server sys-
two low-power states. These states save power when tems. PCI Express also supports PCI error handling and
the link is idle, but require a recovery time to resynchro- has advanced error reporting and handling to help im-
nize the transmitter and receiver when data needs to be prove fault isolation and recovery solutions.
transmitted. The longer the recovery time (or latency),
the lower the power usage. The most frequent imple- PCI Express Form Factors
mentation will be the low-power state with the shortest
A number of PCI Express form factors address the re-
recovery time.
quirements of client, server, and portable computer
platforms:
Support for Real-Time Data Traffic
• Standard and low-profile cards: desktops, worksta-
Unlike PCI, PCI Express includes native support for iso-
tions, and servers
chronous (or time-dependent) data transfers and vari-
• Mini card: portable computers
ous QoS levels. These features are implemented via
• ExpressCard: portable computers and desktops
“virtual channels” that are designed to guarantee that
• Server I/O module (SIOM) that is currently being de-
particular data packets arrive at their destination in a giv-
fined by PCI SIG
en period of time. PCI Express supports multiple isoch-
ronous virtual channels—each an independent PCI Express Standard and Low-Profile Cards
communications session—per lane. Each channel may
have a different QoS level. This end-to-end solution is Current PCI standard and low-profile cards are used in a
designed for applications that require real-time delivery variety of platforms, including servers, workstations,
and desktops. PCI Express also defines standard and
such as real-time voice and video.
low-profile cards that can replace or coexist with legacy
Hot Plug and Hot Swap PCI cards. These cards have the same dimensions as
PCI cards and are equipped with a rear bracket to ac-
PCI-based systems do not have native (or built-in) sup-
commodate external cable connections.
port for hot plugging or hot swapping I/O cards. In-
stead, a few limited server and PC Card hot plug, hot The differences between the PCI and PCI Express cards
swap implementations were developed as add-ons to lie in their I/O connectors. A x1 PCI Express connector
PCI after the original bus definition. These solutions ad- has 36 pins, compared to the 120 pins on a standard PCI

6
February 2004

connector. Figure 6 compares PCI and PCI Express low-

profile cards. The x1 PCI Express connector shown is
much smaller than the connector on the PCI card. Next
PCI Express
to the PCI Express connector is a small tab that pre- Implementation x1 Slot x4 Slot x8 Slot x16 Slot
cludes it from being inserted into a PCI slot. The stan- x1 Card Required Required Required Required
dard and low-profile form factors also support x4, x8, x4 Card No Required Allowed Allowed
and x16 implementations. x8 Card No Allowed* Required Allowed
x16 Card No No No Required

*These implementations will have an x8 connector on a wired x4 slot. This means

that the slot will accept x8 cards, but run at x4 speeds.

Table 3. PCI Express Card Interoperability

Transition to PCI Express Cards

Client system boards will gradually migrate from the
PCI connector to the x1 PCI Express connector. Work-
stations will migrate from PCI to x1 PCI Express connec-
Figure 6. Comparison of PCI Express and PCI tors, and from PCI-X to x4 PCI Express connectors. The
Low-Profile Cards
AGP8X connector will be replaced with a x16 PCI Ex-
Figure 7 compares the size of PCI Express connectors press connector. Unlike AGP, this connector can be
to the PCI, AGP8X, and PCI-X connectors they will re- used for other PCI Express cards if a PCI Express graph-
place on the system board. ics card is not required.

Servers will gradually migrate from PCI-X connectors to

primarily x4 and x8 connectors. Beginning in 2004, cus-
tomers should expect a mix of PCI Express and PCI/PCI-
X slots in server systems. This approach will allow cus-
tomers to adopt new technology, while maintaining leg-
acy support.

Figure 8 compares the I/O connectors on a typical cur-

rent client system board to those on a transitional PCI
Express system board. The PCI system board contains
five standard PCI slots and one AGP slot. The PCI Ex-
press system board also has six I/O slots, but only three
are PCI slots. Two are x1 PCI Express connectors and
one is a x16 PCI Express connector that replaces the
AGP8X slot. The PCI Express connectors on the system
board are black to distinguish them from off-white PCI
and brown AGP slots.
Figure 7. PCI Express System Board Connector Size for
Standard and Low-Profile Cards

Table 3 shows the interoperability requirements of stan-

dard and low-profile PCI Express cards. A x1 card can
be used in all four system board slots: x1, x4, x8, and
x16. When a x1 card is inserted into a higher-bandwidth
slot, the link layer negotiates the link down to the x1
data transfer rate.

7
www.dell.com/r&d
PCI Express Technology

(Source: Intel)
Figure 8. Comparison of PCI and Transitional PCI Express
System Boards Figure 9. PCI Express Mini Versus Mini PCI

The first devices that will migrate to PCI Express cards A PCI Express Mini Card socket on the system board
will be those that require the bandwidth. For client sys- must support both a x1 PCI Express link and a USB 2.0
tems, these devices will include graphics, 1394, Gigabit link. A PCI Express Mini Card can use either PCI Express
Ethernet, and TV tuner cards. For server systems, or USB 2.0 (or both). USB 2.0 support will help during
Ultra320 SCSI RAID cards, Fibre Channel host bus the transition to PCI Express, because peripheral ven-
adapters (HBAs), and 1- and 10-Gigabit Ethernet cards dors will need time to design PCI Express into their chip
will be available initially. The cost of these cards is ex- sets. During the transition, PCI Express Mini Cards can
pected to be comparable to (and, in some cases, lower) be quickly implemented using USB 2.0.
than PCI-X alternatives. Other cards are expected to
gradually migrate to PCI Express, but it will be many ExpressCard
years before inexpensive and low-bandwidth cards ExpressCard is a small, modular add-in card designed to
such as modems are migrated. Similar to the transition replace the PC Card over the next few years. The Ex-
from the ISA to PCI bus, systems with both PCI and PCI pressCard specification was developed by the Personal
Express will exist for many years. Computer Memory Card International Association (PC-
MCIA). The ExpressCard form factors shown in Figure
PCI Express Mini Card 10 are designed to provide a small, less-expensive, and
The PCI Express Mini Card replaces the Mini PCI card, higher-bandwidth replacement for the PC Card. Like the
which is a small internal card functionally identical to PCI Express Mini Card, an ExpressCard module can
standard desktop computer PCI cards. Mini PCI cards support a x1 PCI Express and a USB 2.0 link. Its low cost
are used mainly to add communications functions to also makes it feasible for small form-factor desktop sys-
portable computers that are built- or customized-to-or- tems. The ExpressCard module also has low power re-
der. The PCI Express Mini Card is half the size of the quirements and is hot pluggable. It is likely to be used
Mini PCI card as shown in Figure 9. This allows system for communications, hard-disk storage, and emerging
designers to include one or two cards, depending on I/O technologies. ExpressCard modules are expected in
the size constraints of a particular portable computer. the second half of 2004.

8
February 2004

Sample PCI Express System

Architectures
The following sections present examples of PCI Ex-
press architectures for client and server systems.

Client Systems
Figure 11 shows how PCI Express could be implement-
ed in a client system. Initially, a x16 PCI Express link will
replace the AGP bus between the graphics subsystem
and the Northbridge. A PCI Express variant could also
replace the link between the Northbridge and South-
Figure 10. ExpressCard Modules
bridge, relieving the bottleneck between peripheral I/O
devices and the Northbridge. There will also be multiple
PCI Express Server I/O Module PCI Express links off the Southbridge for the network in-
terface controller (NIC), 1394 devices, and other periph-
The SIOM specification is currently being defined.
erals. The Southbridge will continue to support legacy
SIOMs are expected with the second generation of the
PCI slots.
PCI Express technology. The PCI Express SIOM will pro-
vide a robust form factor that can be easily installed or
replaced. It will be modular, allowing I/O cards to be in-
stalled and serviced in a system while it is still operating
and without opening the chassis.

The SIOM is a more radical form factor change than oth-

er PCI Express form factors. It will solve many of the
problems with PCI and PCI-X cards in servers. It will be
hot pluggable and its cover will protect the internal
components. These features are designed to make the
cards more reliable in data center environments where
many people handle cards.

The module is also designed with forced-air cooling in

mind because high-speed server devices tend to gener-
ate a lot of heat. The cooling air can originate from the
back, top, or bottom of the module. This flexibility offers Figure 11. Sample Desktop Architecture
system designers more options when evaluating ther-
mal solutions for rack-mounted systems equipped with This architecture highlights several key implications for
SIOMs. customers. Desktop systems will have both PCI and PCI
The largest SIOM form factor will accommodate rela- Express buses for a long time. To minimize confusion
tively complicated functions and should be able to le- during the transition, PCI cards cannot be inadvertently
verage the full range of PCI Express links. inserted into PCI Express slots, nor can PCI Express
cards be inserted into legacy PCI slots. In addition, PCI
Express enables widespread adoption of Gigabit4
Ethernet, 10-Gigabit Ethernet, 1394b, or other high
speed devices in client systems. It will also support in-

4. This term does not connote an actual operating speed of 1 Gbps. For high-speed transmission, connection to a Gigabit Ethernet server and network infrastructure is required.

9
www.dell.com/r&d
PCI Express Technology

creasing bandwidth requirements of graphics sub-

systems.

Portable Computers
Figure 12 shows how PCI Express could be implement-
ed in a portable computer system. Like desktop sys-
tems, PCI Express will replace the AGP bus, and a PCI
Express variant is a candidate to replace the link be-
tween the Northbridge and Southbridge. In addition,
PCI Express could be used to replace the PCI bus be-
tween the Northbridge and the build-to-order/custom-
ize-to-order (BTO/CTO) slot. This slot currently
accommodates Mini PCI cards, but in new systems it Figure 13. Sample Server Architecture
may be used for PCI Express Mini Cards.
date the peak bandwidth required for a dual-ported
10-Gbps controller.
• Lower implementation cost. More slots and em-
bedded I/O devices can be connected to the sys-
tem chip set with fewer bridge chips and fewer
signal routing requirements on the system board.
• Lower latency. Transmission latency between I/O
devices and the CPU and memory can be reduced
by eliminating the PCI-X bridge chip.

Initial generations of PCI Express servers will also in-

clude PCI-X slots for legacy PCI-X cards.

Figure 12. Sample Portable Computer Architecture Enabling Future Modular Designs
The PCI SIG is also working on the PCI Express cable
specification. Because PCI Express has high data rates
The PCI bus between the Northbridge and the docking and low-pin-count connectors, it is likely to be used as
station could also migrate to PCI Express. A x1 Express- a high-speed interconnect between components in cli-
Card slot that uses a USB 2.0 link will replace the PC ent and server systems. Modular systems with sepa-
Card slot. Finally, individual PCI Express links will re- rate high-speed components can be connected with
place the PCI bus that supports integrated peripheral PCI Express cables. Figure 14 illustrates the concept of
devices such as Gigabit Ethernet, audio, and graphics. a “split” system that separates components that gener-
ate heat such as processors, memory, and graphics
Server Systems
from other components such as removable storage,
Figure 13 shows how PCI Express could be implement- display devices, and I/O ports. It may also make sense
ed in a dual-processor server architecture. PCI Express to separate high-end graphics subsystems, which re-
can help to significantly reduce server system complex- quire more power and generate heat, from the main
ity. PCI Express links for I/O devices and slots are processor chassis. This approach would make it easier
placed directly off the Northbridge. This approach is ex- to deliver appropriate power and cooling to the graphics
pected to provide the following potential advantages: subsystem.
• Higher bandwidth for next-generation I/O such
as 10-Gbps Ethernet and x4 Infiniband fabrics.
For example, a x8 PCI Express link can accommo-

10
February 2004

Figure 14. Examples of Split Systems That Separate Processor From I/O

Conclusion For More Information

PCI Express is the open standards-based successor to • Intel white paper: “Creating a Third Generation I/O
PCI. It is designed to provide a reliable and scalable Interconnect,” www.intel.com/technology/pciexpress/
high-speed serial interconnect that maintains backward devnet/docs/WhatisPCIExpress.pdf
compatibility with PCI. Like PCI, it will be implemented • PCI SIG: www.pcisig.com
in a broad range of existing platforms, including servers,
portable computers, desktop systems, and worksta-
tions. It will also enable innovative modular computer
system designs.

Dell has been a strong supporter of PCI Express tech-

nology, participating fully in the development of the
specification and planning for its inclusion in Dell™
products. Dell will begin transitioning Dell platforms to
PCI Express in 2004 when chip set support and PCI Ex-
press devices are introduced.

THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT
IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.

© 2004 Dell Inc. All rights reserved.

Trademarks used in this text: Dell and the DELL logo are trademarks of Dell Inc.; Intel is a registered trademark of Intel Corporation. Other trademarks and trade
names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell Inc. disclaims any proprietary interest
in trademarks and trade names other than its own.

11