What every programmer should know about memory, Part 1 [LWN.net]

LWN<br>.net<br>News from the source

Content Weekly Edition<br>Archives<br>Search<br>Kernel<br>Security<br>Events calendar<br>Unread comments

LWN FAQ<br>Write for us

Edition Return to the Front page

User:<br>Password: |

Log in /<br>Subscribe /<br>Register

What every programmer should know about memory, Part 1

This article brought to you by LWN subscribers

Subscribers to LWN.net made this article — and everything that<br>surrounds it — possible. If you appreciate our content, please<br>buy a subscription and make the next<br>set of articles possible.

September 21, 2007

This article was contributed by Ulrich Drepper

[Editor's introduction: Ulrich Drepper recently approached us asking if<br>we would be interested in publishing a lengthy document he had written on<br>how memory and software interact. We did not have to look at the text for<br>long to realize that it would be of interest to many LWN readers. Memory<br>usage is often the determining factor in how software performs, but good<br>information on how to avoid memory bottlenecks is hard to find. This<br>series of articles should change that situation.

The original document prints out at over 100 pages. We will be splitting<br>it into about seven segments, each run 1-2 weeks after its predecessor.<br>Once the entire series is out, Ulrich will be releasing the full text.

Reformatting the text from the original LaTeX has been a bit of a<br>challenge, but the results, hopefully, will be good. For ease of online<br>reading, Ulrich's footnotes have been placed {inline in the text}.<br>Hyperlinked cross-references (and [bibliography references]) will not be<br>possible until the full series is published.

Many thanks to Ulrich for allowing LWN to publish this material; we hope<br>that it will lead to more memory-efficient software across our systems in<br>the near future.]

1 Introduction

In the early days computers were much simpler. The various components<br>of a system, such as the CPU, memory, mass storage, and network<br>interfaces, were developed together and, as a result, were quite<br>balanced in their performance. For example, the memory and network<br>interfaces were not (much) faster than the CPU at providing data.

This situation changed once the basic structure of computers<br>stabilized and hardware developers concentrated on optimizing<br>individual subsystems. Suddenly the performance of some components of<br>the computer fell significantly behind and bottlenecks developed.<br>This was especially true for mass storage and memory subsystems which,<br>for cost reasons, improved more slowly relative to other components.

The slowness of mass storage has mostly been dealt with using software<br>techniques: operating systems keep most often used (and most likely to<br>be used) data in main memory, which can be accessed at a rate orders of<br>magnitude faster than the hard disk. Cache storage was added to the<br>storage devices themselves, which requires no changes in the operating system to<br>increase performance. {Changes are needed, however, to<br>guarantee data integrity when using storage device caches.} For the<br>purposes of this paper, we will not go into more details of software<br>optimizations for the mass storage access.

Unlike storage subsystems, removing the main memory as a bottleneck<br>has proven much more difficult and almost all solutions require changes to<br>the hardware. Today these changes mainly come in the following forms:

RAM hardware design (speed and parallelism).

Memory controller designs.

CPU caches.

Direct memory access (DMA) for devices.

For the most part, this document will deal with CPU caches and some<br>effects of memory controller design. In the process of exploring<br>these topics, we will explore DMA and bring it into the larger<br>picture. However, we will start with an overview of the design for<br>today's commodity hardware. This is a prerequisite to understanding<br>the problems and the limitations of efficiently using memory<br>subsystems. We will also learn about, in some detail, the different types<br>of RAM and illustrate why these differences still exist.

This document is in no way all inclusive and final. It is limited to<br>commodity hardware and further limited to a subset of that hardware.<br>Also, many topics will be discussed in just enough detail<br>for the goals of this paper. For such topics, readers are recommended to<br>find more detailed documentation.

When it comes to operating-system-specific details and solutions,<br>the text exclusively<br>describes Linux. At no time will it contain any information about<br>other OSes. The author has no interest in discussing the implications<br>for other OSes. If the reader thinks s/he has to use a different OS<br>they have to go to their vendors and demand they write documents<br>similar to this one.

One last comment before the start. The text contains a number of<br>occurrences of the term “usually” and other, similar qualifiers.<br>The technology discussed here exists in many, many<br>variations in the real world and this paper only addresses the most<br>common,...