Computing Scaling Laws

Computing has consistently grown faster, and a handful of empirical observations — dubbed "laws", but not really — turned out to describe that growth surprisingly well. This article explores the most influential of these scaling laws, what they got right, where they broke down, and what they reveal about the physical limits of trying to cram ever more computation into tight spaces without starting a fire.

Transistors

Moore's Law states that the number of transistors in CPUs will double every two years (sometimes incorrectly quoted as the speed doubling every two years).

Figure 1: Source: Transistor Count

The transistor density increase was initially supported by advances in the manufacturing processes, allowing for ever smaller transistor dimensions ("process node").

Table 1: Process node size shrinking over the years

Year<br>Product<br>Milestone<br>Process Node

1971<br>Intel 4004<br>First commercial microprocessor<br>10,000nm

1978<br>Intel 8086<br>Foundation of the x86 architecture<br>3,000nm

1989<br>Intel 486<br>First with on-chip floating point unit<br>1,000nm

1993<br>Intel Pentium<br>Superscalar architecture<br>800nm

2000<br>Intel Pentium 4<br>Peak of clock frequency scaling<br>180nm

2004

Dennard Scaling breaks down, clock speeds plateau<br>90nm

2006<br>Intel Core 2 Duo<br>Industry shifts to multicore<br>65nm

2012<br>Intel Ivy Bridge<br>First 3D FinFET transistors<br>22nm

2017<br>Apple A11<br>fFirst with dedicated neural engine<br>10nm

2019

TSMC volume production with EUV<br>7nm

2020<br>Apple M1<br>ARM displaces x86 in personal computing<br>5nm

2022

Intel cost per transistor increases for first time<br>4nm

2023<br>Apple M3<br>First consumer chip on 3nm node<br>3nm

2024

TSMC N2 gate-all-around transistors enter production<br>2nm

Figure 2: Equally scaled scanning electron microscope images of semiconductor device cross-sections, showing a 28nm, a 40nm, a 65nm, and 150nm process node. — Ludwig, Matthias & Purice, Dinu & Lippmann, Bernhard & Bette, Ann-Christin & Lenz, Claus. (2021). Towards Fully Automated Verification of Semiconductor Technologies. 10.1201/9781003337232-13.

Moore's Law encountered fundamental material and manufacturing constraints as process nodes approached atomic scales:

90nm (2004)Dennard Scaling held that shrinking transistors reduced voltage and current proportionally, keeping power density constant. At 90nm, the gate-oxide thickness was down to only a few atomic layers, and leakage current became significant as electrons tunnelled through the oxide via quantum effects. This is the era of "speed demons" like the Pentium 4 series, where clock frequencies plateaued at approximately 3–4 GHz before processors started to melt. The industry then transitioned to multicore architectures to sustain throughput through parallelism. A prime example of defeating frequency limits by exploiting parallelism was the Cell processor featured in the PlayStation 3, released in 2006.

7nm (2019)As feature sizes pushed to 7nm and below, deep ultraviolet (DUV) immersion lithography could only keep up through increasingly complex multi-patterning, which drove up defect rates and cost. Extreme Ultraviolet (EUV) lithography — pioneered by ASML, the sole manufacturer of EUV scanners — was introduced for the most critical layers. TSMC was the first to deploy it in high-volume production, with its N7+ process in 2019. The added manufacturing complexity pushed cost per transistor up significantly.

3nm (2023)Effective gate dielectrics thinned to just one to two atomic layers. At this scale, electron behaviour is dominated by quantum effects, including tunnelling through potential barriers, which compromises the deterministic switching behaviour that binary logic requires, turning manufacturing into a major challenge.

Process scaling alone could not sustain performance gains, so current innovation happens at the architecture level with the use of specialised accelerators (GPUs, TPUs, NPUs). That means gains became workload-specific, and require deliberate architectural choices.

Efficiency

Koomey's law states the number of computations per joule of energy dissipated doubles about every 2.6 years.<br>Here's the numerical computing performance vs. power rating of the top 500 most energy efficient supercomputers over the last decade:

Figure 3: Source: Top500

During the Dennard Scaling era, Koomey's Law and Moore's Law advanced together since denser transistors means higher efficiency. When Dennard Scaling broke down around 2004, the two diverged. Transistor counts continued rising but power density increased, temporarily slowing efficiency gains. Koomey's Law continued at a reduced rate, sustained by architectural and systems-level improvements rather than process scaling.

Figure 4: A Cray liquid-cooled compute blade. — Colin Heshmat / Cray Shasta Supercomputer — C.H.

Bandwidth

Nielsen's "Law of Internet Bandwidth" states that users' bandwidth grows by 50% per year.

Figure 5: Sources: N-ISDN, 3G, 4G, V.90, V.34, ADSL

Bandwidth grows more slowly than transistor density....