Building a DynamoDB conformance suite - Martin Hicks

Journal

13th April 2026

AWS doesn't ship a conformance suite for DynamoDB emulators, so I built one. 526 tests, ground-truthed against real DynamoDB. Dynoxide passes all of them. DynamoDB Local fails 42.

Update, July 2026. Since I wrote this, the DynamoDB conformance suite has grown from the 526 tests below to 954, and it now has a name and a board of its own: Parity Suite. The current standings are live at the DynamoDB conformance board. The rest of the article is left as it was, a record of where the suite started.

When I started building Dynoxide - a DynamoDB-compatible engine in Rust - I kept running into the same problem. I'd implement an operation, write tests against my understanding of DynamoDB's behaviour, and ship it. Then someone (usually me) would discover that real DynamoDB does something slightly different. A validation error with a different message. A condition expression that fails where I expected it to succeed. An edge case in number precision that I'd never considered.

Every emulator ships with "DynamoDB compatible" and you're expected to take their word for it. I needed a way to answer a simple question: does Dynoxide actually behave like DynamoDB?

No public conformance suite exists for DynamoDB - not from AWS, not from the community.

So I built one.

The idea

The principle is straightforward. Write a test. Run it against real DynamoDB. Record the result. That recorded result becomes the ground truth. Then run the same test against an emulator and compare.

If the emulator's response matches DynamoDB's, the test passes. If it doesn't, the emulator has a conformance gap. No opinions, no guesswork about what the documentation means - just "does this behave the same way the real thing does?"

I built it as a standalone project - dynamodb-conformance - so anyone can clone it, point it at their own DynamoDB-compatible endpoint, and get results. It uses the AWS SDK v3 for TypeScript and vitest as the test runner. Nothing exotic.

Tiers

Not all DynamoDB operations are equally important. If your emulator can't do a basic PutItem, nothing else matters. If it doesn't perfectly replicate the error message format for an obscure validation edge case, that's less critical.

So the suite is split into three tiers.

Tier 1 covers core CRUD operations - CreateTable, PutItem, GetItem, UpdateItem, DeleteItem, Query, Scan, BatchGetItem, BatchWriteItem. These are the operations every application uses. If an emulator fails here, it's not usable for real development.

Tier 2 covers advanced features - transactions, PartiQL, streams, tags, TTL, and table updates like billing mode changes. These are features that production applications rely on but that you might not hit in a simple prototype.

Tier 3 covers edge cases and error fidelity - validation ordering, exact error message formatting, API limits, and legacy API behaviour. The things that only matter when you need your emulator to be indistinguishable from the real service.

This tiered structure means a result like "100% Tier 1, 95% Tier 2, 80% Tier 3" actually tells you something useful. It means core operations work perfectly, advanced features are mostly there, and edge cases have some gaps. A flat "92% overall" hides whether the failures are in PutItem or in obscure validation corner cases.

Ground truth

The key design decision was making DynamoDB itself the source of truth rather than my interpretation of the documentation.

Every test in the suite has been run against a real DynamoDB table in eu-west-2. The expected values in the tests aren't what I think DynamoDB should return based on reading the docs - they're what DynamoDB actually returns. This matters more than you'd expect. DynamoDB's documentation is good, but it doesn't cover every edge case. Sometimes the only way to know what DynamoDB does is to try it and see.

A scheduled CI job runs the full suite against real DynamoDB weekly, so if AWS changes something - a new validation, a different error message, a behavioural tweak - the ground truth stays current. The suite can't go stale the way a set of hand-written expected values would.

Results

Here's where it gets interesting.

Target<br>Tier 1 (267)<br>Tier 2 (93)<br>Tier 3 (166)<br>Overall

DynamoDB (eu-west-2)<br>100%<br>100%<br>100%<br>526 / 526

Dynoxide<br>100%<br>100%<br>100%<br>526 / 526

LocalStack<br>98.9%<br>95.7%<br>81.9%<br>489 / 526

DynamoDB Local<br>98.9%<br>90.3%<br>81.9%<br>484 / 526

Dynalite<br>98.1%<br>10.8%<br>92.8%<br>426 / 526

Dynoxide passes every test. 526 out of 526, across all three tiers.

DynamoDB Local - AWS's own emulator - fails 42. LocalStack fails 37. Dynalite, which hasn't been actively maintained for a while, fails 57 (and that Tier 2 number of 10.8% tells you it's missing entire feature categories like transactions and PartiQL).

I want to be careful about what this means and what it doesn't.

526 tests is a lot, but DynamoDB is a massive service. There are aspects of it that the suite doesn't cover - DAX...