Model Genome: Fingerprinting Whether an LLM Was Trained From Scratch or Derived

Log In<br>Sign Up

Back to Articles<br>a]:hidden">

Model Genome: Fingerprinting Whether an LLM Was Trained From Scratch or Derived

Community Article Published<br>August 8, 2026

Upvote 12

+6

Proto_AGI mayafree Follow

TL;DR — When a lab announces a "self-developed, from-scratch" foundation model, how can an outsider verify the claim using only public artifacts? We built a reproducible pipeline that fingerprints a model on three axes — architecture (config.json), tokenizer (vocabulary overlap), and weights (embedding CKA) — and combined them into a single at-a-glance genotype. Along the way we hit two instructive traps: row-wise embedding cosine is useless because of rotational invariance , and even CKA cannot cleanly separate continued-pretraining from from-scratch — so config + tokenizer remain the primary evidence. We applied the exact same yardstick to the public foundation models of nine Korean organizations. Try it live: Model Genome Korea.

1. The question

Building a large language model on top of an open-weight base (Qwen, Llama, DeepSeek, Mistral) is a legitimate, industry-standard practice. But it is different from training a foundation model from scratch — and vendors do not always make the distinction explicit. When several labs released DeepSeek-rivaling "self-developed" models in late July 2026 (e.g. LG K-EXAONE 2.0, 750B), the debate spilled into Chinese tech communities as well — a Zhihu thread (→ link) crossed 2.7M views. The natural question followed: from scratch, or derived?

This is answerable, objectively, from public files. Here is how.

2. Axis 1 — Architecture fingerprint (config.json)

Every transformers checkpoint ships a config.json. A handful of fields form a surprisingly discriminative signature:

model_type

vocab_size

hidden_size

intermediate_size

num_hidden_layers

num_attention_heads / num_key_value_heads

import requests

def arch_fingerprint(repo):<br>c = requests.get(f"https://huggingface.co/{repo}/resolve/main/config.json",<br>headers={"User-Agent": "genome/1.0"}).json()<br>return {k: c.get(k) for k in<br>("model_type", "vocab_size", "hidden_size",<br>"intermediate_size", "num_hidden_layers",<br>"num_attention_heads", "num_key_value_heads")}

The shape tuple (hidden_size, intermediate_size, num_hidden_layers, heads, kv) is effectively a fingerprint of the reference architecture. When a model's tuple matches a foreign open-weight exactly , that is strong evidence the architecture was adopted rather than designed independently. Examples we measured:

Model<br>shape (h · i · L · heads · kv)<br>Exact match

a 7B commercial model<br>3584 · 18944 · 28 · 28 · 4<br>Qwen2.5-7B

a 72B commercial model<br>8192 · 29568 · 80 · 64 · 8<br>Qwen2.5-72B

a 14B VLM<br>5120 · 17408 · 40 · 40 · 8<br>Qwen3-14B

an 8B model<br>4096 · 14336 · 32 · 32 · 8<br>Llama-3.1-8B

a MoE model<br>7168 · 18432 · 61 · (moe 2048)<br>DeepSeek-V3

A single coincidental field means nothing; five simultaneously is a fingerprint.

3. Axis 2 — Tokenizer fingerprint (a paternity test)

Architecture alone can mislead. A model can copy a foreign architecture but train a genuinely new tokenizer, or vice-versa. The tokenizer is measured directly from tokenizer.json, comparing the vocabulary sets with a min-overlap ratio:

def vocab_set(repo):<br>j = requests.get(f"https://huggingface.co/{repo}/resolve/main/tokenizer.json").json()<br>v = j["model"]["vocab"] # BPE: {token: id}<br>return set(v.keys())

def tok_overlap(a, b):<br>A, B = vocab_set(a), vocab_set(b)<br>return len(A & B) / min(len(A), len(B)) # 1.0 == subset

This immediately surfaces things config hides. One model matched Qwen2.5-7B's architecture exactly , yet its tokenizer overlapped Qwen by only ~0.38 — a "foreign brain, own language" case: the architecture was adopted, but a new Korean tokenizer was trained. Conversely, some VLMs reused a base tokenizer verbatim (overlap = 1.000), confirming a straight fine-tune.

A practical trap: min(|A|,|B|) in the denominator (not the union) is what makes a reduced vocabulary that is a strict subset of a larger one score ~1.0 — the correct signal for "carved out of the base."

4. Axis 3 — Weights fingerprint (the hard one)

The gold-standard question is: were the weights trained from scratch, or continued-pretrained on a foreign base? This is where two instructive traps live.

Trap 1 — row-wise cosine is useless

The naive idea: load embed_tokens.weight from both models, and for shared tokens, average the row-wise cosine similarity. If they share lineage, embeddings should be similar.

They are not — even when they obviously share lineage. We measured near-zero mean cosine for both a known from-scratch model and a known Llama-derivative. The reason is rotational invariance : a Transformer's hidden space has no privileged basis, so two models can encode identical information under an arbitrary orthogonal rotation. Row-wise cosine sees rotation as dissimilarity. It cannot distinguish lineage.

Trap 2 — CKA...