than the CPU core to one or two orders of magnitude slower. 2.

2 DRAM Access Technical Details

In the section introducing DRAM we saw that DRAM chips multiplex the addresses in order to
save resources int the form of address pins. We also saw that accessing DRAM cells takes time
since the capacitors in those cells do not discharge instantaneously to produce a stable signal; we
also saw that DRAM cells must be refreshed. Now it is time to put this all together and see how all
these factors determine how the DRAM access has to happen. We will concentrate on current
technology; we will not discuss asynchronous DRAM and its variants as they are simply not
relevant anymore. Readers interested in this topic are referred to [3] and [19]. We will also not talk
about Rambus DRAM (RDRAM) even though the technology is not obsolete. It is just not widely
used for system memory. We will concentrate exclusively on Synchronous DRAM (SDRAM) and
its successors Double Data Rate DRAM (DDR). Synchronous DRAM, as the name suggests,
works relative to a time source. The memory controller provides a clock, the frequency of which
determines the speed of the Front Side Bus (FSB) – the memory controller interface used by the
DRAM chips. As of this writing, frequencies of 800MHz, 1,066MHz, or even 1,333MHz are
available with higher frequencies (1,600MHz) being announced for the next generation. This does
not mean the frequency used on the bus is actually this high. Instead, today’s buses are double- or
quad-pumped, meaning that data is transported two or four times per cycle. Higher numbers sell
so the manufacturers like to advertise a quad-pumped 200MHz bus as an “effective” 800MHz bus.
For SDRAM today each data transfer consists of 64 bits – 8 bytes. The transfer rate of the FSB is
therefore 8 bytes multiplied by the effective bus frequency (6.4GB/s for the quad-pumped
200MHz bus). That sounds a lot but it is the burst speed, the maximum speed which will never be
surpassed. As we will see now the protocol for talking to the RAM modules has a lot of downtime
when no data can be transmitted. It is exactly this downtime which we must understand and
minimize to achieve the best performance. 2.2.1 Read Access Protocol Figure 2.8 shows the
activity on some of the connectors of a DRAM module which happens in three differently colored
phases. As usual, time flows from left to right. A lot of details are left out. Here we only talk about
the bus clock, RAS and CAS signals, and the address and data buses. A read cycle begins with the
memory controller making the row address available on the address CLK RAS CAS Row Addr
Col Addr Address Data Out Data Out Data Out Data Out DQ tRCD CL Figure 2.8: SDRAM Read
Access Timing bus and lowering the RAS signal. All signals are read on the rising edge of the
clock (CLK) so it does not matter if the signal is not completely square as long as it is stable at the
time it is read. Setting the row address causes the RAM chip to start latching the addressed row.
The CAS signal can be sent after tRCD (RAS-to-CAS Delay) clock cycles. The column address is
then transmitted by making it available on the address bus and lowering the CAS line. Here we
can see how the two parts of the address (more or less halves, nothing else makes sense) can be
transmitted over the same address bus. Now the addressing is complete and the data can be
transmitted. The RAM chip needs some time to prepare for this. The delay is usually called CAS
Latency (CL). In Figure 2.8 the CAS latency is 2. It can be higher or lower, depending on the
quality of the memory controller, motherboard, and DRAM module. The latency can also have
half values. With CL=2.5 the first data would be available at the first falling flank in the blue area.
With all this preparation to get to the data it would be wasteful to only transfer one data word. This
is why DRAM modules allow the memory controller to specify how much data is to be
transmitted. Often the choice is between 2, 4, or 8 words. This allows filling entire lines in the
caches without a new RAS/CAS sequence. It is also possible for the memory controller to send a
new CAS signal without resetting the row selection. In this way, consecutive memory addresses
can be read from or written to significantly faster because the RAS signal does not have to be sent
and the row does not have to be deactivated (see below). Keeping the row “open” is something the
memory controller has to decide. Speculatively leaving it open all the time has disadvantages with
real-world applications (see [3]). Sending new CAS signals is only subject to the Command Rate
of the RAM module (usually specified as Tx, where x is a value like 1 or 2; it will be 1 for high-
performance DRAM modules which accept new commands every cycle). In this example the
SDRAM spits out one word per cycle. This is what the first generation does. DDR is able to
transmit two words per cycle. This cuts down on the transfer time but does not change the latency.
In princi8 Version 1.0 What Every Programmer Should Know About Memory ple, DDR2 works
the same although in practice it looks different. There is no need to go into the details here. It is
sufficient to note that DDR2 can be made faster, cheaper, more reliable, and is more energy
efficient (see [6] for more information). 2.2.2 Precharge and Activation Figure 2.8 does not cover
the whole cycle. It only shows parts of the full cycle of accessing DRAM. Before a new RAS
signal can be sent the currently latched row must be deactivated and the new row must be
precharged. We can concentrate here on the case where this is done with an explicit command.
There are improvements to the protocol which, in some situations, allows this extra step to be
avoided. The delays introduced by precharging still affect the operation, though. CLK RAS tRP
CAS WE Col Addr Row Addr Col Addr Address Data Out Data Out DQ tRCD CL Figure 2.9:
SDRAM Precharge and Activation Figure 2.9 shows the activity starting from one CAS signal to
the CAS signal for another row. The data requested with the first CAS signal is available as
before, after CL cycles. In the example two words are requested which, on a simple SDRAM,
takes two cycles to transmit. Alternatively, imagine four words on a DDR chip. Even on DRAM
modules with a command rate of one the precharge command cannot be issued right away. It is
necessary to wait as long as it takes to transmit the data. In this case it takes two cycles. This
happens to be the same as CL but that is just a coincidence. The precharge signal has no dedicated
line; instead, some implementations issue it by lowering the Write Enable (WE) and RAS line
simultaneously. This combination has no useful meaning by itself (see [18] for encoding details).
Once the precharge command is issued it takes tRP (Row Precharge time) cycles until the row can
be selected. In Figure 2.9 much of the time (indicated by the purplish color) overlaps with the
memory transfer (light blue). This is good! But tRP is larger than the transfer time and so the next
RAS signal is stalled for one cycle. If we were to continue the timeline in the diagram we would
find that the next data transfer happens 5 cycles after the previous one stops. This means the data
bus is only in use two cycles out of seven. Multiply this with the FSB speed and the theoretical
6.4GB/s for a 800MHz bus become 1.8GB/s. That is bad and must be avoided. The techniques
described in section 6 help to raise this number. But the programmer usually has to do her share.
There is one more timing value for a SDRAM module which we have not discussed. In Figure 2.9
the precharge command was only limited by the data transfer time. Another constraint is that an
SDRAM module needs time after a RAS signal before it can precharge another row (denoted as
tRAS). This number is usually pretty high, in the order of two or three times the tRP value. This is
a problem if, after a RAS signal, only one CAS signal follows and the data transfer is finished in a
few cycles. Assume that in Figure 2.9 the initial CAS signal was preceded directly by a RAS
signal and that tRAS is 8 cycles. Then the precharge command would have to be delayed by one
additional cycle since the sum of tRCD, CL, and tRP (since it is larger than the data transfer time)
is only 7 cycles. DDR modules are often described using a special notation: w-x-y-z-T. For
instance: 2-3-2-8-T1. This means: w 2 CAS Latency (CL) x 3 RAS-to-CAS delay (tRCD) y 2
RAS Precharge (tRP) z 8 Active to Precharge delay (tRAS) T T1 Command Rate There are
numerous other timing constants which affect the way commands can be issued and are handled.
Those five constants are in practice sufficient to determine the performance of the module, though.
It is sometimes useful to know this information for the computers in use to be able to interpret
certain measurements. It is definitely useful to know these details when buying computers since
they, along with the FSB and SDRAM module speed, are among the most important factors
determining a computer’s speed. The very adventurous reader could also try to tweak a system.
Sometimes the BIOS allows changing some or all these values. SDRAM modules have
programmable registers where these values can be set. Usually the BIOS picks the best default
value. If the quality of the RAM module is high it might be possible to reduce the one or the other
latency without affecting the stability of the computer. Numerous overclocking websites all around
the Internet provide ample of documentation for doing this. Do it at your own risk, though and do
not say you have not been warned. 2.2.3 Recharging A mostly-overlooked topic when it comes to
DRAM access is recharging. As explained in section 2.1.2, DRAM cells must constantly be
refreshed. This does not happen