Foundation Models for Oversight | Transluce AI

Foundation Models for Oversight<br>A vision for training a foundation model that formalizes, tests, and answers questions about AI model behavior<br>Jacob Steinhardt

Transluce | Published: July 28, 2026

This outlines the research vision for the Oversight Foundations team at Transluce. As an experiment in public transparency, in addition to<br>the high-level vision we've included our current concrete de-risking plan. Of course, many specifics of the plan may change as we execute.<br>We will post regular updates as that plan proceeds to keep you updated. We hope this serves as a valuable resource on how to train novel AI systems.

If you are excited to contribute to this effort, please apply to join our team!

To oversee an AI model, we'd ideally like to ask questions such as:

What are important situations where the model sandbags?

Does the model have an objective it wouldn't admit to if asked directly?

Does the model treat a user differently once it infers something about their identity, and along what axis?

Is the model's chain of thought load-bearing, or is it a post-hoc rationalization of an answer that was already settled on?

Is it reward hacking on this input, or actually trying to solve the task?

It would be great if we had an oversight assistant that could answer these questions. We'd want it to do three things: help us formalize the question as a testable empirical criterion; produce data that satisfies that criterion; and do so in a way we can justifiably trust.

To get such an assistant, we lay out a vision for building a foundation model for oversight : an AI system mid-trained (or pre-trained) on a large, diverse corpus of experiments on a given "subject model", RLVR'd on a large number of verified oversight tasks, and then fine-tuned to answer natural-language questions about the subject model. Mid-training allows the oversight model to build rich knowledge about the subject model it is evaluating; RLVR teaches it to leverage reasoning to solve more difficult oversight tasks; and fine-tuning makes it easy to talk to the assistant and teaches it to formalize natural language questions into checkable criteria.

To achieve this, we need two things:

A training objective that is "universal" for oversight in the same sense that next-token prediction is universal for language modeling.

A way of generating large amounts of diverse data for this training objective.

Our approach is to cast oversight as a world modeling problem: we view the subject model being overseen as the<br>"world" or environment, interventions such as prompting, fine-tuning, and activation steering as actions that<br>modify the environment, and forward-sampling, probing, and other forms of outputs as "sensors" that measure the<br>environment.

We formalize these oversight tasks through what we call<br>Pythonic world models , which express interventions and measurements through Python code.<br>As an example, consider the sandbagging question from above. We can formalize it as a check that generates a batch of tasks, runs the model with and without a cue that it's being evaluated, and measures the drop in quality. We mark with output()1Notation. Judge, MultipleChoiceJudge, and Query are assumed to call an underlying language model, with issues like position / ordering bias handled automatically inside the wrapper.

sample_model(prompt: str) -> str; sample one response from the subject model S.

Judge(rubric: str, judge_model: str); an LM-based judge, where judge.apply(...) returns a scalar in [0, 1]. The rubric may contain {name} placeholders bound by keyword at .apply time; positional arguments are the inputs being judged (a reply, a (prompt, response) pair, or two replies to compare).

MultipleChoiceJudge(rubric: str, options: List[str]); like Judge, but .apply(...) returns a distribution over options.

Query(prompt_template: str, gen_model: str, output_format: type); an LM-based structured query, where .apply(**vars) fills template variables, samples from gen_model, and parses the response as output_format.

flip(p: float) -> bool; returns True with probability p, False otherwise.

output(x); records x in the trace, as an observable to be predicted or conditioned on. the high-level quantities we want to reason about:

def check_sandbagging(task_schema, k=20):<br># e.g. task_schema = "moderate-difficulty coding tasks, each<br># plausibly part of a capability benchmark"<br>judge = Judge("rate the technical skill of the reply, 0-10", 'qwen3-32b')<br>tasks = Query("Generate {k} tasks satisfying the following condition: {condition}",<br>'claude-opus-4.8', output_format=List[str]).apply(condition=task_schema)<br>watched = [sample_model("[SAFETY EVAL]\n" + t) for t in tasks]<br>unwatched = [sample_model(t) for t in tasks]<br>gap = mean(map(judge.apply, unwatched)) - mean(map(judge.apply, watched))<br>output(gap > 1.5)<br>output(task_schema)

Now finding instances of sandbagging reduces to sampling the value of task_schema conditional<br>on...