JP's Website · 2026-08-22 · Why aren't my two Cortex-A9 cores cache coherent?

Why aren't my two Cortex-A9 cores cache coherent?

Posted on 2026-08-22

Contents

The Hardware

Loading Code

The Memory Management Unit

Kinds of Memory

The actual issue

One of my favourite things to do when I have some down time is to grab a random devkit from my pile of random devkits, and try and write some code that will run on it. I generally try and avoid existing SDKs, or installing new tools. I just want to see how little I can write and have something run on the board and have an observable output. If that goes OK, I might go overboard and start writing some little drivers for a few peripherals here and there. If it really gets out of control I might end up having written a whole new Operating System, with VGA output and SD Card support. But usually I'm OK with Hello, World on a UART or a blinking LED.

I had some time off over the past few weeks and, looking for a change from playing with ARM7, I picked up my Terasic DE0-Nano-SOC. I've had this board for about 7 years but I hadn't really done anything with it. It's similar to the DE10 Nano board that is used by the MiSTER retro emulation project, but not compatible.

The Hardware

The Terasic DE0-Nano-SOC is a small devkit released in around 2015 from what I can tell. It has:

An Altera Cyclone-V System on Chip

1 GiB DDR3 SDRAM

An RJ45 Ethernet port

A 5V power input (barrel-jack)

A mini-USB programming port

A micro-USB port for an on-board UART to USB Serial convertor

An SD Card slot

Arduino Uno style headers

Sadly what it doesn't have is an active cooler, because when running it gets really hot. Like, too hot to touch. I've wired up a spare 80mm fan which seems to do more than enough to keep it cool.

The Altera Cyclone-V SoC is a combination of a standard Arm SoC (the Hard Processor System) and an Altera FPGA (with 40K logic elements). In particular, I was interested in the two Arm Cortex-A9 processors.

The SoC has a built-in Boot ROM that can load a preloader from some special sectors on an SD Card. That preloader will run from On-Chip RAM (OCRAM) and initialise the external DDR3 SDRAM, before loading a full copy of U-Boot from the SD Card into SDRAM and then executing it from there. I didn't want to muck around with all that, so I just downloaded the disk image from https://soc.terasic.com and wrote it to a spare card. It all seemed to boot OK.

By default it boots into a fairly old Linux kernel (with a root partition stored on the SD Card), but I deleted the kernel causing it to stop at the U-Boot prompt. The on-board USB to Serial adapter means I can just hook up my PC to the micro-USB port, and use minicom on /dev/tty.usbserial- and interact with the board.

The preloader prints this:

U-Boot SPL 2013.01.01 (Dec 29 2014 - 15:29:15)<br>BOARD : Terasic DE0_Nano_SoC Version-A Board<br>################################################################################<br>##################################= =#<br>########=-=#######################= -- =#<br>######- =#######################= ####- =###### =#<br>######- ###############- -=##=##= ####- ######## =#<br>####= =##- -### ##= -####### ###### ####- #####- - =#<br>#### ## ### ## -##= -=--=###= ####==# ####- #### =#<br>######- ###- ##- -### =####= =====###= ####- #### #### =#<br>######- ### =##### #####= ####-=###= -=###- ####- ##### =#<br>######- -#= - =## #####= #### -###= #=-=#### ####- ########- ##<br>####### =##= ## #####= -########= ####### ####- =###### =#<br>#########==######====####==#######= - --- -- ---- =#<br>##################################= =#<br>################################################################################<br>BOARD : Terasic DE0_Nano_SoC Version-A Board<br>CLOCK: EOSC1 clock 25000 KHz<br>CLOCK: EOSC2 clock 25000 KHz<br>CLOCK: F2S_SDR_REF clock 0 KHz<br>CLOCK: F2S_PER_REF clock 0 KHz<br>CLOCK: MPU clock 925 MHz<br>CLOCK: DDR clock 400 MHz<br>CLOCK: UART clock 100000 KHz<br>CLOCK: MMC clock 50000 KHz<br>CLOCK: QSPI clock 3613 KHz<br>SDRAM: Initializing MMR registers<br>SDRAM: Calibrating PHY<br>SEQ.C: Preparing to start memory calibration<br>SEQ.C: CALIBRATION PASSED<br>SDRAM: 1024 MiB<br>ALTERA DWMMC: 0<br>It's just a small copy of U-Boot, linked to run from the small amount of On-Chip RAM (OCRAM).

The full U-Boot then prints this:

U-Boot 2013.01.01 (Dec 30 2014 - 12:07:34)

CPU : Altera SOCFPGA Platform<br>BOARD : Terasic DE0_Nano_SoC Version-A Board<br>################################################################################<br>##################################= =#<br>########=-=#######################= -- =#<br>######- =#######################= ####- =###### =#<br>######- ###############- -=##=##= ####- ######## =#<br>####= =##- -### ##= -####### ###### ####- #####- - =#<br>#### ## ### ## -##= -=--=###= ####==# ####- #### =#<br>######- ###- ##- -### =####= =====###= ####- #### #### =#<br>######- ### =##### #####= ####-=###= -=###- ####- ##### =#<br>######- -#= - =## #####= #### -###= #=-=#### ####- ########- ##<br>####### =##= ## #####= -########= ####### ####- =######...