Encoded Judgment: What the SKILL.md Pattern Shares with Runbooks — and Where the Analogy Breaks
Every time you start a new session with Claude, you start from zero. The model has no memory of the last run, no accumulated judgment from the previous session, no operational context carried forward. If you want Claude to behave consistently — to follow the same approach, apply the same tradeoffs, produce output in the same format — you have to tell it again.
Skills are the answer to that problem. A SKILL.md file is a document Claude reads at the start of a task that encodes the reasoning, constraints, and output expectations for a specific workflow. The model loads it, applies it, and produces outputs that are — in theory — consistent across sessions without requiring the practitioner to re-establish context every time.
That is a simple description of a pattern that turns out to be more interesting than it first appears.
This Is an Older Problem
The problem skills are solving — how do you make repeatable what a skilled operator does intuitively — is not new. Operations teams have been working on it for decades. Runbooks. Standard operating procedures. Decision trees. Incident response playbooks. All of these are attempts to take judgment that lives in someone's head and give it a form that other people (or systems) can follow without having to re-derive it from first principles.
A SKILL.md file sits in this lineage. It is documentation that runs. Instead of a human operator reading an SOP and applying it, Claude reads a SKILL.md and applies it. The underlying pattern — encode the judgment, reduce the variance — is the same.
What the runbook analogy reveals is that the hard part has never been writing down the steps. The hard part is deciding what level of detail is necessary, what the operator can be trusted to reason through without explicit instruction, and what happens when conditions fall outside what was anticipated. Those problems do not go away when the operator is an AI. They become more visible.
What a Skill Actually Encodes
A well-written skill does not encode a procedure. It encodes judgment — the decisions that a practitioner has already made and wants to not remake every session. The steps are often incidental. What matters is the reasoning behind the steps: what to do when the input is ambiguous, what the output must contain, what the quality bar actually looks like, what to skip if a condition is not met.
This distinction changes what goes in the file. A procedure-focused skill reads like a script: do this, then this, then this. A judgment-focused skill reads like a brief: here is the problem space, here is what good looks like, here are the decisions that have already been made, here are the ones you will encounter and how to navigate them.
The difference in output quality is significant. A procedure-focused skill produces correct outputs in the case it was written for. A judgment-focused skill handles variation — edge cases, missing inputs, format ambiguity — without requiring the practitioner to anticipate every scenario in advance. The brief form is harder to write. It requires knowing what you actually know, not just what steps you follow.
The Constraint Traditional Documentation Does Not Face
Runbooks can be as long as they need to be. A 40-page SOP is inconvenient but functional. A 40-page SKILL.md is a problem.
Every byte Claude reads from a skill file is context window space that is not available for the actual task. A verbose skill does not just slow things down — it crowds out the working memory needed to do the work. The leanness constraint is not a preference. It is a structural pressure that shapes every skill design decision.
This creates a tension that has no equivalent in traditional operational documentation: the more precisely you specify the behavior, the less room there is for the model to operate. Specificity is a resource. You spend it carefully or you pay for it in output quality.
The question is where to spend it. Not every part of a workflow needs explicit instruction. Some steps Claude handles reliably without guidance — standard formatting, common error patterns, basic reasoning tasks. The judgment calls are where variance appears: what to include in the output, how to handle partial inputs, when to stop and flag versus when to proceed. Those are where specificity earns its context cost.
The Specificity Tension in Practice
In practice, the tradeoff shows up as a design question every time you write a skill: at what point does more instruction reduce variance without meaningfully cutting into available context?
There is no universal answer. It depends on the model, the task, the acceptable variance range, and how much output has been observed to diverge across sessions. Some workflows tolerate significant variation in execution — the output is evaluated at the end and the path does not matter. Others are brittle: a formatting error early in the chain breaks downstream processing, and the skill needs to be explicit about the format even if it seems obvious.
The honest calibration process involves running the skill, observing where variance appears, and adding specification at the friction points — not before. Pre-emptive over-specification is the most common failure mode. It produces skills that are expensive to load, rigid in execution, and difficult to maintain as the workflow evolves.
Where Logging Fits In
One requirement that surfaces consistently in practical skill design is logging: capturing enough reasoning context that a session can be reviewed, debugged, or handed off without starting the analysis from scratch. This is the same requirement that makes runbooks include decision logs and incident reports include timeline reconstructions.
It collides directly with the leanness constraint. More logging instruction in the skill means more context spent on structure. More logging output means more of the session occupied with recording rather than doing.
The resolution is not a rule but a design question: what does the logging actually serve? If it serves QA — checking that the output is correct — a light log is sufficient and review happens after. If it serves debugging — understanding why an output deviated — the reasoning chain needs to be captured at the point of decision, not reconstructed afterward. These are different requirements and they produce different skill designs. Treating them as the same produces a log that is too heavy for QA and too shallow for debugging.
Where the Runbook Analogy Breaks
A human operator reading an SOP brings accumulated judgment that is not in the document. They notice when something looks wrong. They draw on experience outside the procedure to catch gaps. They update their mental model of the system as they work through it.
Claude, reading a skill, applies what the skill contains. It does not have accumulated operational experience with your specific environment. It does not remember that last Tuesday's run produced a malformed output because of a specific input pattern. What is not in the skill file is not available to the session unless it is provided at runtime.
This is not a limitation to work around — it is a constraint to design with. The skill needs to contain what a practitioner would know going in that Claude cannot be expected to know. Not every fact, but the facts that change the behavior. The operational specifics that make your workflow different from the general case. Experienced runbook writers learn to write for the operator who knows nothing except what the document contains. Skill writers are learning the same lesson for a different kind of operator.
An Open Question About Calibration Over Time
Skills as they exist today are calibrated to current model capability. The more explicit instruction a skill contains, the less the model needs to reason through on its own. That is the practical tradeoff: more specification, less variance, more context cost.
As models improve, that tradeoff shifts. A skill that is correctly specified for today's model may be overspecified for next year's — paying a context cost for instructions the model would have handled correctly without them. The inverse is also possible: some tasks that require heavy specification today may run cleanly with minimal guidance as the underlying reasoning capability improves.
This suggests that skill design is not a one-time exercise. Skills are calibrated at a point in time against a specific capability level. They will need revisiting as that capability changes — not because the workflow changed, but because the relationship between explicit instruction and model behavior is not static. What the right maintenance cadence looks like for a library of skills maintained across years, through multiple model generations, is a question the field does not have a settled answer to yet.
