How Much to Say: The Decisions That Determine Whether Your SKILL.md Works

How Much to Say: The Decisions That Determine Whether Your SKILL.md Works

You have decided to use Claude to automate a repeatable workflow using the skill system. The choice is made. What is left is a series of decisions about what actually goes in the SKILL.md file — and those decisions have direct consequences for how the skill performs across sessions, how much context it consumes on load, and whether it can be debugged when outputs start drifting.

None of these decisions are obvious the first time through. The field does not have settled conventions yet. What follows is the reasoning I have worked through on my own skills — the decisions I had to make consciously to stop relying on luck.

What Does the Skill Need to Carry vs. What Can Claude Supply?

This is the first and most consequential decision. Every word in the skill file costs context. The question is which words earn that cost.

The easy answer — put everything in — produces a skill that loads slowly, crowds the working memory Claude needs for the actual task, and becomes a maintenance burden whenever the workflow changes. The other easy answer — trust Claude to figure it out — produces variance that compounds across sessions until you cannot reliably predict what the output will look like.

The useful answer requires distinguishing between two categories of content. The first is procedural steps — the sequence of actions Claude needs to take. Claude handles these reliably without exhaustive instruction in most cases. A skill that lists every step in prose form is usually spending context on content that adds little. The second category is judgment calls — the decisions that change the output in ways you care about. What to include when input is ambiguous. What the quality bar actually looks like. When to stop and flag rather than proceed. These are where variance appears, and these are where specification earns its cost.

In a homelab context, where you are both the builder and the reviewer, you can start with a minimal skill and let variance teach you where guidance is needed. Each session where the output drifts from what you wanted is a calibration signal. You add specificity at those friction points and nowhere else. The iteration cost is low because you catch the problems and you understand the workflow.

In an enterprise context, where the skill may run unattended or produce outputs that feed downstream systems, starting lean is harder to justify. Variance in early runs has a cost that is higher than the time saved by writing a shorter initial file. Starting more explicit — specifying the judgment calls you can anticipate — and then pruning what does not affect output quality is the more defensible approach. You establish a baseline first. Then you find out what Claude does not need.

The honest calibration regardless of context: write the brief, not the script. Describe the problem space, the quality bar, and the decisions that matter. Leave the sequence to Claude unless the sequence has been a source of failure.

What Is the Logging Actually For?

Logging is one of those requirements that looks like one thing but is actually three different things depending on what you need it to do. Getting this wrong is expensive — the wrong logging design either costs more context than it needs to, or fails to capture what you actually need when something goes wrong.

QA logging serves output validation. At the end of a session, you want enough recorded to confirm that the output is correct — the right files were produced, the right format was used, the key decisions were made. This is lightweight. A structured summary at session end, referencing what was done and what was produced. The context cost is low and the value is reliable if your QA process happens promptly.

Debugging logging serves root cause analysis. When an output deviates — a draft comes out in the wrong format, a pipeline step produces an error, a draft gets flagged for content that should not be there — you need to understand not just what happened but why Claude made the choice it did at the specific decision point where things went wrong. This requires capturing reasoning at the moment of the decision, not reconstructing it afterward. The context cost is higher. The value is only realized when you need it, but when you need it, lightweight QA logging is useless.

Audit trail logging serves accountability. Timestamps, inputs, outputs, decisions made, files moved. If a run has compliance implications — if you need to be able to show what happened and when to a third party — this is the format. Highest context cost. Almost never necessary in a homelab context unless you are deliberately building operational discipline into your practice.

In a homelab context, QA logging is usually the right call. You are the reviewer. You will catch problems in the output. If something goes badly wrong, you can reconstruct the session from context. Debugging logging is worth adding only if you are running complex pipelines where failure modes are non-obvious and re-running the session to diagnose is expensive.

In an enterprise context, the risk profile of the workflow determines the logging level. If a failed skill run has business consequences — missed content, broken pipelines, downstream errors that someone else has to fix — debugging-level logging is worth the context cost. If there are compliance or audit requirements attached to the process, build the audit trail in from the start. Retrofitting it later into a skill that was not designed for it is painful.

The skill design implication: decide what the logging serves before you write the logging instructions. Then write instructions that produce exactly that and nothing more. Do not default to verbose logging because it feels safer. Logging instructions that exceed what you will actually use are context spent on nothing.

How Precisely Do You Need to Specify the Output Format?

This is the decision where over-specification consistently earns its cost. Output format is a brittle point in any automated workflow — a wrong quote character, markdown inside a field that expects HTML, a missing required field, or an escaped character handled incorrectly will break downstream processing in ways that are annoying to diagnose and easy to prevent.

The temptation is to trust that Claude will produce correct JSON, correct HTML, or correct markdown without explicit instruction because the format is familiar. That trust is mostly warranted for simple cases. It is not warranted for nested structures, for fields with specific escaping requirements, or for formats where there is a meaningful difference between valid and correct — where Claude can produce output that parses without error but breaks the consumer.

Specify the output format completely. Include the exact structure, the field names, the expected types, and any escaping requirements. If the downstream system cannot handle markdown inside a JSON body field, say so explicitly. If double quotes inside HTML must be escaped as " or as ", say which. If the file naming convention matters to the publisher, define it.

The homelab vs enterprise distinction matters less here than in other decisions. A format requirement either works or it does not, regardless of operating context. What changes between contexts is the consequence of getting it wrong. In a homelab, a formatting error surfaces immediately when you try to use the output and you fix it. In an enterprise context, the error may propagate before anyone catches it. But the specification in the skill should be complete in either case — the only variable is how urgently you discover that it was not.

Include a quality gate in the skill. A checklist of format requirements that Claude checks against before declaring a draft complete. This is low context cost and catches errors at the point of production rather than at the point of consumption.

How Do You Iterate When the Skill Drifts?

Every skill drifts eventually. The workflow changes, the input structure changes, the model behavior shifts, or you discover that a judgment call you left unspecified is producing variance you care about. The question is how you notice and how you respond.

The failure mode to avoid is treating drift as a sign that the skill was written wrong. It was not. Skills are calibrated at a point in time. Drift is the normal signal that the calibration needs an update. Building the expectation of revision into how you think about skills changes the maintenance posture from reactive to routine.

The practical iteration pattern that works: run the skill, review the output against what you expected, identify the specific decision point where the output diverged, add specification at that point and only that point. Do not rewrite the skill from scratch because one section drifted. Surgical additions at the friction points preserve everything that was working and target what was not.

In a homelab context, this iteration loop is short. You run the skill, you review the output, you update the file, you run it again. The turnaround is measured in minutes. The cost of a drift is low.

In an enterprise context, the iteration loop is longer because each change to a skill that other processes depend on carries more risk. The pattern here is to run the updated skill against a representative test case before deploying it, and to keep the change set narrow. Wide rewrites of production skills are a risk even when they seem necessary. Narrow targeted updates are more recoverable if the change introduces a new problem.

What to Carry Into Your Own Skill Designs

The decisions above are not a checklist to complete before shipping a skill. They are questions to return to as the skill evolves. A skill that answered them well at the start may need different answers after twenty runs.

The durable reasoning: specificity is a resource, not a virtue. Spend it on the judgment calls that produce variance you care about, not on steps that Claude handles reliably. Logging serves a purpose — decide what that purpose is before writing the instruction. Format requirements are not optional specification — they are the brittle layer between your skill and everything downstream. And drift is expected, not a failure. The skill that never needs revision is the one that never gets used enough to reveal where it falls short.

What none of this answers is how the right calibration changes as the models improve. A skill that is correctly specified for today's Claude may be overspecified for the next version. That recalibration question sits underneath all of it and does not have a settled answer yet.

Ryan Mckernan
Author

Ryan Mckernan

IT service manager & infrastructure architect turning real-world IT messes into practical, documented fixes. I build systems, streamline ops, and share field notes at Painfully Useful—tested, refined, and repeatable.