Harness Notes
Lecture 02

What a harness actually is

A prompt file is not a harness. A harness is the engineering infrastructure around the model that turns capability into reliable execution.

Notes from Learn Harness Engineering - Lecture 02

harness_modeloutside model weights
AI Agent
Instructions

AGENTS.md / CLAUDE.md

State

PROGRESS.md / commits

Tools

shell / files / tests

Environment

deps / services / versions

If it is not model weights, it is harness.

OpenAI frames the job as designing environments, expressing intent, and building feedback loops. Anthropic calls the Claude Agent SDK a general-purpose agent harness. Different emphasis, same point: infrastructure determines how much model capability becomes useful work.

Prompt file

Text the agent reads. Useful, but it cannot run tests, preserve state, control environment, or close the feedback loop by itself.

vs.

Harness

Instructions, tools, environment, state, and feedback working together so the agent can act, observe, recover, and improve.

Core concepts

The lecture turns "harness" from a vague label into an auditable engineering object.

HarnessEverything in the engineering infrastructure outside model weights.
Repo as sourceAnything the agent cannot see effectively does not exist.
Map, not manualAGENTS.md should route the agent, not become an encyclopedia.
ConstrainUse executable invariants instead of micromanaging every implementation detail.
AblationRemove one subsystem at a time and observe which removal causes the biggest drop.

The five subsystems

Missing any subsystem leaves the agent awkward to use. Feedback usually has the lowest investment and highest return.

Instructions

Overview, stack, run commands, hard constraints, and links to deeper docs.

Tools

Enough shell, file, and test access to do real work under least privilege.

Environment

Self-describing dependencies, versions, services, Docker or devcontainers.

State

Progress files and commits so long tasks survive session boundaries.

Feedback

Explicit verification commands: tests, typecheck, lint, and full checks.

How to improve it

Start with verification commands.

List exactly how to test, typecheck, lint, and run full verification.

Use controlled exclusions.

Keep model and task fixed, remove one subsystem, and measure the drop.

Attribute failures.

Ablation supports diagnosis, but bottlenecks require failure records and root-cause attribution.

Audit regularly.

Harnesses rot like code. Pay down harness debt deliberately.

Verification commands:
- Tests: pytest tests/ -x
- Type check: mypy src/ --strict
- Lint: ruff check src/
- Full verification: make check

Same model, better harness.

A TypeScript + React team improved GPT-4o results by adding harness components one stage at a time. The model did not change.

Stage 1

README only

Wrong package manager, naming drift, no tests.

20%
Stage 2

AGENTS.md

Stack, conventions, architecture decisions.

60%
Stage 3

Verification

yarn test && yarn lint && yarn build.

80%
Stage 4

Progress state

Templates for completed and incomplete work.

80-100%
TAKEAWAY

A harness is five subsystems: instructions, tools, environment, state, and feedback.