PCI Express System Architecture

Nathan Chien
March 27 ’07

A Reliable Business Partner

 Agenda
• Basic Overview
• Features
• Device Layers & Packets
• Link Initialization & Training

A Reliable Business Partner

 Technology Shift to PCI Express
Desktop
PCI PCI Express
Notebook
PCI PCI Express
Server PCI PCI-X PCI Express

Industrial PC PCI PCI Express

Networking PCI or Proprietary PCI Express

Telecom

PCI or Other PCI Express

Embedded

2005 2006 2007

– All major markets are transitioning to PCI Express

• Driven by new Intel chipset rollout, first market segments to
transition are PC and Notebook
A Reliable Business Partner
 PCI Express Topology
Upstream port, Downstream port

A Reliable Business Partner

 Physical Layer PCI v.s PCI-Express

• PCI uses a shared parallel bus

– Bandwidth shared between devices on the bus
– Only one device may own the bus at any time
– Large number of parallel signals
– Wait states may be added by Initiator or Target
• PCI-Express uses a serial point to point interconnect
– Full bandwidth dedicated to that link
– No need for arbitration
– Low number of signals
– No wait states

A Reliable Business Partner

 Performance – PCI v.s PCI Express

PCI /PCI-X 32bit 32bit 64bit 64bit

Bus 33Mhz 66Mhz 66Mhz 133Mhz
Bandwidth 0.132 0.264 0.528 1.064
(GBytes/s)

• Bandwidth(GB) = Frequency(Mhz) x bit(Bytes)

A Reliable Business Partner

 PCI Express Throughput

PCI Express Link x1 x2 x4 x8 x12 x16 x32

Width(Lanes)
Bandwidth 0.5 1 2 4 6 8 16
(GBytes/s)

• 2.5Gbits/sec/lane/direction transfer rate

• Direction-Transmitter and Receiver(full duplex)
• Divide by 10-bits per Byte (8b/10b encoding)
– Embedded Clock and Error Detection
• Multiply by number of lanes

A Reliable Business Partner

 Key Definitions

• Port
– A group of transmitters and
receivers located on the same chip
PHY
that define a link
Port
• Lane
– A set of differential signal pairs, Lane Link
one pair for transmission and one
pair for reception
Port
• Link
PHY
– A dual-simplex communications
path between two components
Example: 4 Lanes
– A xN link is composed of N lanes
A Reliable Business Partner
 Features

• Point to point
• low voltage differential signaling
• Lane reversal and polarity inversion
• Speed, lane width, lane reversal and polarity negotiation
at initialization
• Packet based transaction(Serial interconnect Technology)
• Flow control
• Transaction Layer Packet acknowledgements (Ack/Nak)
• Support VCs, TCs

A Reliable Business Partner

 Low Voltage Differential Signaling

D+ D- D+ D-

A Differential Pair A Differential Pair

(Transmitter) (Receiver)

A Differential Pair
In each direction
= one Lane
This is an x1 Link
There are four signals

VDIFF :P-P

800mV~1200mV
Vcm DC: 0~3.6V

A Reliable Business Partner

Link Configuration

A Reliable Business Partner

 Lane Reversal
Neither device A nor B
Device B supports Lane
supports Lane Reversal
Reversal

Device A Device A
(Upstream Device) (Upstream Device)

0123 0123 0123 0123

3210 3210 3210 3210

Device B Device B
(Downstream Device) (Downstream Device)

A Reliable Business Partner

 Polarity Inversion

Device A
(Upstream Device)
D+ D- D+ D-
After Polarity Inversion
D- D+
After Polarity Inversion

D+ D- Before Polarity Inversion

D- D+ D- D+
Device B
(Downstream Device)

• lane reversal and polarity negotiation at initialization

A Reliable Business Partner
The Protocol Layers

A Reliable Business Partner

 Device Layers
PCI Express Device A PCI Express Device B

Device Core Device Core

PCI Express Core PCI Express Core

Logic Interface Logic Interface
TX RX TX RX
Transaction Layer TLP Transaction Layer

Data Link Layer DLLP Data Link Layer

Physical Layer PLP Physical Layer

TX RX TX RX

Link
A Reliable Business Partner
 PCI Express Layers

Start Seq Header Payload ECRC LCRC End

Transaction Layer
Data Link Layer
Physical Layer

ƒPackets are transmitted serially

ƒCredit based Flow Control
ƒEach device transmits periodic flow control
packets to inform transmitter of buffer space
ƒEnsures that receiving device has buffer space
for the packet

A Reliable Business Partner

 Transaction Layer
Start Seq Header Payload ECRC LCRC End
Transaction Layer
Data Link Layer
Physical Layer
• Responsible for;
– Storing negotiated and programmed configuration information
– Managing link flow control
– Enforcing ordering and Quality of Service(QoS)
– Power management control/status
• Header
Information may include;
• Address/Routing
• Data transfer Length
• Transaction descriptor
• End to End CRC checking provides additional security(Optional)
A Reliable Business Partner
 QoS General Information

• ACK/NAK ensure correct delivery of data

• Replay Timer allows Retry Buffer to be re-sent
• Can re-train the link in the event of catastrophic
failure
– Only one or some of the lanes may be corrupted
– Retraining may allow fall back to lower
bandwidth system.

A Reliable Business Partner

 Data Link Layer Packets

Start Seq Header Payload ECRC LCRC End

Transaction Layer
Data Link Layer
Physical Layer
• Responsible for;
– Integrity of TLP’s
• Link-level error detection and re-transmission of bad
TLP’s
– Initialization and updates of credit based flow control
mechanism
• Classes of DLLPs
– Transaction Layer Packet acknowledgements (Ack/Nak)
– Power management
– Flow Control (Flow Control packets)
A Reliable Business Partner
 ACK/NAK Protocol
• “Reliable” transport of TLPs from one device to
another device across Link.
From Transaction Layer To Transaction Layer

Data Link Layer Data Link Layer

TLP DLLP TLP
Seq. TLP LCRC DLLP ACK/NAK Seq. TLP LCRC
ACK/NAK
Replay
Buffer
De-Mux
De-Mux Mux Error
Mux Check
DLLP
ACK/NAK
Tx Rx Tx Rx

Link
TLP
Seq. TLP LCRC

A Reliable Business Partner

PCI Express Mechanicals

A Reliable Business Partner

Desktop & Server Form Factor

• Motherboard usage
• Supports I/O &
graphics
• First available form
factor
• Systems will ship in
1H04

16X
8X
4X
1X
A Reliable Business Partner
 MiniCard Form Factor

• Replacement for Mini-

PCI
• Wired & wireless comm

Current Mini-PCI Card New Mini PCI Express Cards

A Reliable Business Partner
 ExpressCard Form Factor

• Replacement for CardBus

• Modular expansion
• Host will support PCI Express
& USB 2.0
• For notebooks

Current CardBus New ExpressCard Types

A Reliable Business Partner
 Routing Advantages of PCI

PCI-X
64 Bit

A Reliable Business Partner

 Routing Advantages of PCI Express

• Board area reduced by 53%

• Reduction of number of board layers

A Reliable Business Partner