How we use /goal to find bugs in Patch the Planet - The Trail of Bits BlogPage content

Codex’s /goal feature amplifies bug hunting, but getting good results requires the right prompt, the right scope, and the right number of outcomes per run. For Patch the Planet, our joint initiative with OpenAI to find and fix bugs in open-source software, we pointed Codex at some of the most widely used, heavily audited codebases in the world, like Rust, curl, and zlib. One tool came up again and again in our internal bug-report channels: /goal, which hands Codex an open-ended objective and lets it work independently toward a success condition. Here are a few highlights:<br>/goal found every Rust bug we submitted, including a soundness hole and a miscompilation now patched in Rust 1.98, from a single variant-analysis pipeline.<br>It turned every project&rsquo;s past CVEs into Semgrep rules that had to fire on the vulnerable version and stay silent on the patched one, then flagged 11 variant hits across multiple projects.<br>It uncovered two potential high-severity privilege-escalation bugs in Keycloak&rsquo;s SAML component during a discovery run.<br>Over the first few weeks of Patch the Planet, our engineers independently converged on three techniques for using /goal. We found that getting the most out of /goal means treating the prompt as a set of specific success criteria, not a set of instructions. (Note that this blog post uses /goal to refer to goal-based prompting in general. Codex can also set goals for itself through a tool call, and that&rsquo;s how we recommend everyone use it; we rarely type the slash command ourselves.)<br>1. Let Codex write the goal<br>The art of using /goal is prompt design, and we found that Codex knows Codex the best. Internally, our single most repeated /goal tip was to use Codex to help write each /goal prompt. We hand Codex threat model files and the context about what we’re looking for, and then tell it to write the goal prompt. As mentioned before, /goal is a tool Codex can invoke on itself, and a few engineers stopped typing goals by hand entirely.<br>$goal-prompt based on threat model write goal to find single critical issue (RCE) exploitable by remote attacker for kubernetes-client. the kubernetes-client is used in normal config, malicious remote users exploits.

Figure 1: A meta-prompt from one of our engineers asking Codex to create a goal prompt. Results are shown in figure 2.This works because Codex knows the target and its own tendencies better than we can specify up front. It can translate a threat model into concrete, testable success criteria, name the code paths worth prioritizing, and phrase the outcome precisely enough that a run actually converges. A goal written this way tends to be tighter than one we&rsquo;d write cold, and it takes a fraction of the time.<br>Letting the model draft the goal also closes a gap we&rsquo;d otherwise miss. Any outcome you define can be satisfied in ways you didn&rsquo;t intend, and the model is often the first to spot where the easy outs are.<br>Now when we ask Codex to draft a goal, we ask it to red-team its own goal by identifying the ways a future model might be lazy in its approach, and to revise the criteria to remove them before the run starts. We also built tooling that makes it easier for Codex to verify its own work. For example, we noticed Codex has a tendency to skip reading the entire codebase even when explicitly asked. We built aicov, a tool that tracks what lines of code Codex has actually read, so it can’t “cheat.”<br>This is an iterative process. As we find more shortcuts a model takes, we exclude them from the next version of the prompt.<br>2. Define the outcome, not the path<br>A good goal names the outcome, defines it precisely, and then enforces persistence:<br>/goal Audit the kubernetes-client repository in this workspace to find exactly one previously unreported critical remote code execution vulnerability reachable in normal/default client configuration by a malicious remote user or server that controls only network/API responses, Kubernetes objects the client legitimately fetches, or other remote data accepted during normal use.<br>First build a concise threat model of realistic remote attacker entry points and trust boundaries, then prioritize code paths involving deserialization, YAML/JSON/protobuf parsing, dynamic imports/eval/template execution, archive/file extraction, auth redirects, generated client hooks, websocket/exec/attach/port-forward streams, and subprocess or filesystem effects. Do not assume attacker control of local kubeconfig, CLI arguments, environment variables, installed plugins, source code, credentials, privileged cluster/admin access, or prior code execution; explicitly reject findings that rely on those preconditions. Before accepting a candidate, search local known-findings files plus current open issues/PRs for duplicates, then produce a minimal safe proof that demonstrates attacker-controlled code execution or a direct RCE...