Only a full pipeline run counts as real verification
Unit tests prove isolated parts. End-to-end tests prove the parts still work after they are wired together.
Notes from Learn Harness Engineering - Lecture 10
Only the bottom path observes the real system
The defect lives between components.
In the Electron export example, the renderer, preload script, and service layer each pass unit tests. The first real click exposes wrong path format, stuck progress, memory leakage, permission differences, and missing error propagation.
Unit-test view
Fast, precise, and isolated. Useful for local logic, but blind to the contracts between mocked components.
E2E view
Slower, but it exercises the real path through UI, bridge, service, operating system, and side effects.
Unit tests have systematic blind spots.
Isolation is the design philosophy that makes unit tests fast. It is also why they cannot catch defects created by wiring, shared state, real resources, or environment differences.
Interface mismatch
Renderer sends a relative file path while preload expects an absolute path.
State propagation
Schema changes but cached ORM entries still reflect the old table shape.
Resource lifecycle
File handles, database connections, and sockets leak across component boundaries.
Environment dependency
Mocked tests hide configuration, latency, permissions, and service availability.
E2E tests change how agents write code.
The lecture's overlooked point: when agents know the real flow will be run, they stop optimizing for the fastest green check and start respecting integration pressure.
Component interactions
The agent asks how this interface connects upstream and downstream.
Architectural boundaries
Executable tests force the agent to follow boundary rules instead of copying bad local patterns.
Error paths
Failure scenarios push exception handling, UI feedback, and propagation into the implementation.
Core concepts
Remember the gradient: each wider test layer catches a larger class of defects.
Define boundaries before E2E.
Without architectural intent, a full-pipeline run only proves that the whole mess runs. For agent-built codebases, constraints need to exist on day one because agents copy existing patterns.
Rule: dependencies flow forward through fixed layers. Cross-domain concerns enter through explicit provider interfaces. Any other dependency is forbidden and mechanically enforced.
Promote review feedback into the harness.
A repeated review comment should not stay a comment. Turn it into a check with an error message that teaches the agent how to repair the violation.
Review finds: renderer imports fs directly.
Add a direct filesystem import check.
Error explains how to move access to preload.
Harness runs the check automatically.
Next violation fails immediately.
How to apply it
Require E2E for cross-component work.
Validation hierarchy: unit, integration, then E2E when multiple components change.
Automate architecture rules.
Convert constraints such as "renderer must not access filesystem" into lint or test checks.
Write repair instructions.
Failure messages need what, why, and fix steps so the agent can self-correct.
Grow the harness from reviews.
Every recurring defect category becomes a new permanent automated check.
ERROR: Found direct import of 'fs' in src/renderer/App.tsx:12
WHY: Renderer process has no access to Node.js APIs for security
FIX: Move file operations to src/preload/file-ops.ts
and call via window.api.readFile()Electron export case: E2E caught all five.
Unit tests for renderer, preload, and service all passed. The end-to-end flow found every system-level defect.
| Defect | Description | Unit | E2E |
|---|---|---|---|
| Interface mismatch | Inconsistent file path format. | Missed | Caught |
| State propagation | Export progress not sent back to UI via IPC. | Missed | Caught |
| Resource leak | Large file export handles not released. | Missed | Caught |
| Permission issue | Different permissions in packaged environment. | Missed | Caught |
| Error propagation | Service layer exceptions did not reach UI layer. | Missed | Caught |