Harvesting Ethereum Traces Without an Archive Node | Fables for Robots

2026-08-11 · 7 min read · Dragan Milic

Harvesting Ethereum Traces Without an Archive Node

How I got every call trace back to genesis out of a patched geth, on a very ordinary machine.

#ethereum

#web3

Intro #<br>If you have ever needed the full history of Ethereum call traces, you know the problem: they are not stored anywhere.

Logs are easy, they sit in the receipts. Internal calls do not. A value transfer that never shows up as its own transaction, a delegatecall three frames deep, a storage slot a contract read before it decided what to do. None of that is in any database. The only way to see it is to execute the block.

Lately I had a task that needed exactly that, for every block since genesis.

The standard answer is to run an archive node.

Why I Did Not Want an Archive Node #<br>An archive node keeps every intermediate state the chain has ever been in. You can go back to any block, reconstruct the world as it was at that moment, and re-execute against it.

That is a great capability. It is also a lot of disk, a sync measured in weeks, and a machine you have to keep running afterwards. Renting one is not cheap either.

But the part that bothered me was different. What an archive node really sells you is the ability to ask a question you have not thought of yet. That is why it keeps everything, it does not know what you will want.

I knew what I wanted. Call traces, every block, once, written somewhere else. The generality was the expensive part and I was going to throw it away.

The Observation #<br>Then something fairly obvious hit me.

A full sync from genesis executes every single block. It has to. That is what execution is: take the state after block N, apply the transactions of block N+1, get the state after block N+1. There is no shortcut and no sampling.

So every intermediate state gets built anyway, in order, on every syncing node. An archive node does not see more history than a full node. It just refuses to throw the state away as it goes past.

Which means I did not need to store any state at all. I only needed to be there at the right moment, while the block was being executed.

Patching Geth #<br>Geth is already doing the execution. I only needed somewhere to put my code.

That place is writeHeadBlock in core/blockchain.go, which runs when a block becomes the new head. Everything I needed goes at the end of it:

func (bc *BlockChain) writeHeadBlock(block *types.Block) {<br>// ... existing head-writing logic ...<br>bc.currentBlock.Store(block.Header())<br>headBlockGauge.Update(int64(block.NumberU64()))

// add your block processing here!

What makes that spot useful is not the block, you can get a block anywhere. It is that the state the block was executed against is still sitting in the trie database. Nothing has to be reconstructed. It is just there, for as long as you are standing in that function.

So you can run whatever tracers you like over the block right there and store the results somewhere else. I used mostly the ones geth already ships, callTracer with logs enabled and prestateTracer in diff mode, plus two I wrote for this job. One of them, the keccak256 preimage tracer, is in go-ethereum upstream now.

Why It Blocks #<br>Whatever you put there runs synchronously. The sync does not continue until it is done.

That looks like a mistake, and it is the most important part of the whole thing.

Hand the block to a worker and return immediately, and the sync runs ahead. Tracing falls behind. Then the state your tracer needs gets pruned while the tracer is still using it. The obvious fix is to hold that state until the tracer catches up, and congratulations, you have just written an archive node by accident.

Blocking removes the race by removing the concurrency. The sync runs exactly as fast as tracing allows. That is slower, and that is the bill. In exchange the tracer always looks at state that is guaranteed to still be there.

You also get backpressure for free. If whatever you write the traces to gets slow, your code takes longer to return, and the sync slows down with it. Nothing piles up in memory and nothing quietly drops a block.

Reorgs #<br>Backfilling old blocks and following the head look like two different problems. Old blocks are settled history. The head reorgs, so a block you just traced may turn out to have never happened.

It turns out you do not have to do anything about it.

writeHeadBlock runs whenever a block becomes the head, and on a reorg geth rewinds the head to the common ancestor and then writes the new canonical chain forward. So your code simply runs again with a block number lower than the one it saw a moment ago, and walks forward from there.

Seen from inside that function, a reorg is just the block number jumping backwards and some blocks being processed a second time.

That does move the problem downstream. Whatever consumes the traces has to key on block number and hash, not on the number alone,...