Sure, Your Claude Is Doing Amazing Things. Prove It. • RUNME<br>July 6, 2026

ai

Sebastian Huckleberry

Sure, Your Claude Is Doing Amazing Things. Prove It.<br>F*** Your Skills (And Mine Too) [1]. That was the line that got closest to the truth. Not "skills are useless." Not "agents are fake." Something more uncomfortable: if we are going to pass around folders of instructions, scripts, references, tool expectations, and workflow habits as reusable agent behavior, then both the agent's skill and the author's skill need to be inspected like code.

Not as prompts. Not as magic. Code.

Why don't we test them like code: smoke tests, integration tests, unit tests?

Because an agent skill can look useful and still fail like any agent workflow: wrong activation, wrong tool, stale context, missing sources, leaky boundaries, or a polished artifact that only passed because the agent got lucky once.

The "mine, too" part matters. It turns the joke back on the people writing these workflows. Before a skill proves anything to a registry, a teammate, or a user, it has to prove something to its author. A skill is not just markdown an agent consumes; it is a little software object made of assumptions, provenance, maintenance habits, and judgment. Install it, and you inherit the author's operational folklore.

That is the problem runme eval (v3.17) is trying to make boring: replace repeated vibe checks with proof that the workflow continues to work after the folklore leaves home. Skills are just the tip of the iceberg. The same task eval pattern applies anywhere instructions, tools, context, memory, and final artifacts have to keep working together. runme eval brings that discipline into the local repo, where the workflow already lives, using Harbor's eval model underneath [2].

The reason this extends beyond skills is that "your Claude" is no longer just a monolithic model. Or your Codex. Or ChatGPT. Or Cursor. Or OpenCode. Once these tools become part of your workflow, the thing that matters is the adoption layer between the base model and the work itself:

agent harnesses

repo instructions such as CLAUDE.md or AGENTS.md

repo-local and user-wide skills

tools, retrieval systems, and MCP server configuration

memory systems

secrets and redaction boundaries

the task trajectory the agent chooses

the final artifact it produces

This post is not about evaluating bespoke agents built with LLM or agent SDKs, nor about security problems such as supply-chain attacks, malicious skills, or secret exfiltration. Those are separate problems. This is about the popular AI harnesses people already use in real repos: deploying skills, instructions, tools, and workflow habits into agents such as Claude, Codex, Cursor, ChatGPT, or OpenCode, then proving that behavior continues to work.

We depend on that layer like infrastructure, but still validate it like copy-paste prompts: screenshots, demos, lucky runs, and personal rituals. The Show Us Your Agent Skills segment on Hamel Husain's skill scepticism makes a similar point: public skills should be read like code, with provenance, maintenance, and constraints checked before anyone treats them as reusable infrastructure [3]. The same discipline applies to the full workflow: smoke tests, focused tests for the pieces, integration tests for tools and context, and regression history for trajectories that used to work.

runme eval starts with the inner loop: record the task, run the workflow, keep the evidence, and make improvement history part of the repo where Claude is already configured and doing the work. That lowers the barrier for people getting started, while keeping the path open to the outer loop: running the same task evals in CI or scaled-out benchmark jobs. It builds on the same insight behind Harbor [2] and benchmarks like Terminal-Bench 2.0 [4], SWE-Bench Verified, and others: agent work can be captured as tasks, run through a harness, and judged by repeatable evidence instead of vibes. That makes Harbor a useful bridge between serious benchmark infrastructure and repo-local workflow regression tests.

Let's make that concrete with a small workflow that has real-world failure modes.

Read the Eval docs

A Concrete Eval Loop

Skills are still a useful starting point because they are bounded enough to inspect, but messy enough to fail like larger workflows. Skills.sh already lists close to a million published skills, and many of them look like single-commit artifacts: generated once, rarely installed, and not actively maintained. I have done my share of skill-creator dumping, too.

Choose the Unit Under Test

As with software tests, the author chooses the boundary of what is under test. That boundary might be one skill, a full agent workflow, a tool integration, or the final artifact a team depends on. runme eval defaults to ./evals/tasks because it assumes the common case is a task or workflow integration that belongs with the repo, not only a standalone skill artifact. You can still...