Audn vs Codex vs Aikido on the Same Repo — 159 Findings, and All Three Agreed on Exactly 3 | Audn.AI Security Research<br>Home →

Audn vs Codex vs Aikido on the Same Repo — 159 Findings, and All Three Agreed on Exactly 3

Every security vendor publishes a benchmark where they win.

So here's ours, with the part nobody publishes attached: the 63 findings we couldn't confirm, the 9 findings Codex caught that we structurally cannot see, the 23 findings Aikido found alone — including a class we don't scan for at all — and the blind spots that survive even with all three tools running.

The full interactive report is live, unedited, with every row from all three sides:

→ blog.audn.ai/audn-vs-codex-vs-aikido

(The original two-way Audn-vs-Codex A/B is still up at blog.audn.ai/audn-vs-codex.)

The setup was simple. We pointed Audn Agentic WhiteBox at a fork of OWASP Juice Shop — the deliberately-vulnerable app the whole industry uses as a punching bag. Then we ran the official OpenAI Codex security review over the same source. Then we ran Aikido , a commercial SAST + SCA + secrets + CI platform, over the same repo. Same target, ozgurozkan123/juice-shop. Three very different machines looking at it.

The headline numbers are 103, 20, and 36. Those are also the least interesting things in the report.

The Result That Should Worry You

Between the three tools there were 159 findings .

The three-way intersection — issues that all three independently flagged — is three issues :

JWT signature is not verified — Audn traces the hard-coded key and algorithm confusion in lib/insecurity.ts; Codex flags a newly-introduced global registration of forged JWTs; Aikido's SAST flags the same files by pattern.

Unpinned third-party GitHub Actions holding the repo token — image_actions.yml, all three, from three different angles.

Remote fetch without integrity check in CI — Audn's remote installer lead, Codex's lockfile/scripts findings, Aikido's ci.yml flag.

Three tools. 159 findings. Three unanimous. When a live red-team, a diff reviewer, and a commercial SAST/SCA platform all land on the same thing independently, treat it as certainly real — and treat the other 156 as evidence that these tools are not substitutes for each other.

Audn WhiteBoxCodexAikidoWhat it is SAST fused with a live red-team runCommit-diff code reviewSAST + SCA + secrets + CI (ASPM)Findings 1032036Severity mix 18 Crit / 24 High / 52 Med / 5 Low / 4 Info8 High / 7 Med / 2 Low / 3 Info3 Crit / 13 High / 8 Med / 12 LowReproduced live 18 00Dependency CVEs (SCA) NoNoYes — 3 Secrets scanning Partial (source)NoYes — 17 Diff-aware NoYes NoRemediation Prose + CVSS + attack pathsGenerated git diff Fix-time estimate + guidanceRuntime 2h 28m + live targetLight, diff-scoped~52 seconds Sweet spot Pentest / exploitability proofPR gate / regressionsContinuous baseline<br>Where Each One Actually Shined

Audn shined on: proving things

103 findings with CWE, file:line, CVSS vectors, and 32 derived attack paths . But the number that matters isn't 103 — it's 18 , the findings the agent didn't just report but fired at the running app and reproduced :

server.ts:280 — unauthenticated public access logs containing change-password URLs with cleartext current and new passwords . Not a code smell. Pulled off the live box.

A spoofable X-Forwarded-For header bypassing the password-reset rate limit — an account-takeover enabler with no static signature at all. The agent found it by getting rate-limited and then working out how to not be. No pattern scanner will ever find this, because there's no pattern.

routes/search.ts:21 — SQL injection, fired and confirmed. NoSQL injection in trackOrder.ts and showProductReviews.ts. XSS via models/product.ts. Hard-coded credentials in users.yml, 7ms.yml, and routes/login.ts — four of them, all validated live.

The other Audn-exclusive lane is authorization logic , which pattern scanners structurally cannot reason about: 13 broken-access-control/IDOR findings (basket IDOR exposing other users' carts, coupon updates missing ownership checks, duplicate BasketId bypassing ownership), 3 race conditions (wallet debit permitting concurrent overspending), and the business-logic set (client clock controlling coupon validity, unanchored coupon validation accepting unbounded discounts). Aikido found zero IDOR findings. Not because it's bad — because no regex knows who's supposed to own a basket.

Codex shined on: what the change broke

Codex reviewed commit 160f306 as a diff. Nearly every finding is framed as "introduced by this commit." It caught nine things nobody else did — and adding Aikido didn't cover a single one of them:

2FA temporary JWTs accepted as bearer auth. A commit stopped inserting no-data JWTs into authenticatedUsers to fix a crash — but never started rejecting them. They still sail through as valid auth. A red-team walking the app from outside would never trip over this. A diff reader catches it instantly.

Unbounded complaint similarity scan...