Generated from cat-harness/skills/sdlc/sdlc-core/test-engineer.md — do not edit here.
✎ Edit this page’s source

Test Engineer Skill

Role

Write and maintain unit tests for the content-authoring pipeline. Tests are TypeScript using Bun’s built-in test runner (bun:test).

Test infrastructure

Path	Purpose
`scripts/tests/*.test.ts`	Test files (bun discovers `*.test.ts`)
`scripts/tests/helpers.ts`	Shared helpers: paths, project discovery, parsing
`scripts/tests/report.ts`	Test report generator (outputs `TestReport` JSON)
`scripts/tests/run-tests.sh`	Shell entry point (delegates to `bun test`)
`schemas/formalization-types.ts`	TypeScript types for test definitions, results, reports, coverage
`content/schema/`	Content-object types, Zod schemas, builder functions
`pipeline/validate.ts`	Content validation pipeline (schema + constraints + AST)
`pipeline/render.ts`	Block → output rendering + AST validation
`pipeline/build.ts`	Full document build from content objects

Dependency graph for scoped testing

Use the content uses[] fields (and any formalization-import graph) to scope test runs:

Identify changed blocks — which .ts/.md files were modified
Trace downstream dependents — via uses[] in content objects
Run targeted tests — only validate/build the changed block + its dependents
Report impact — show how many blocks are affected by a change

The dependency graph comes from:

uses[] field in content .ts manifests (narrative dependency)
any formalization import statements (formal-layer dependency)
dependency-graph.svg generated by CI

When to write a test

A bug was found — write a regression test before fixing.
A new script or config file is added — test its contract.
A rendering macro is changed — test extraction still works.
A new formal-layer project is added — auto-discovered, but verify.
A review skill detects a potential error — it will ask the user whether to write a test. If yes, this skill writes it.

How to write a test

// In scripts/tests/<suite>.test.ts
import { describe, test, expect } from "bun:test";
import { REPO_ROOT, ... } from "./helpers";

describe("Feature under test", () => {
  test("specific behavior", () => {
    // Arrange
    // Act
    // Assert
    expect(result).toBe(expected);
  });
});

Conventions

One test file per concern: create new files for new concerns.
Use helpers.ts: Don’t hardcode paths or re-implement discovery.
Informational vs. blocking: Tests that report data quality (coverage gaps, label mismatches) should log and pass. Tests that verify contracts (JSON validity, required fields, file existence) should fail.
Comment the failure point: Each test should document what bug or failure it prevents.
Types from schemas/formalization-types.ts: Use TestResult, TestReport, CoverageEntry etc. when generating structured output.

A test never reads `test/results/` as a fixture

Derived QA results are leaving main for the orphan qa-reports branch (arc 3fva): the checkout’s <instance>/test/results/** is a working copy, still committed until bean 5hox removes it, and the record is the commit-keyed entry qa-publish stores. A test that reads that tree is reading a file that will not be there — and the readers audit (docs/proposals/qa-readers-audit-2026-10-01.md §5.5) measured what that costs: 15 tests in 6 files fail only when the corpus is absent, and one, “no critical criterion is failing”, passes when it is absent because it iterates over zero files.

Build the fixture, or run the producer in-process over one, and assert on what it computed. A verdict is not a fixture: the corpus is supposed to change (see qa-witness).
If a test must read a stored result, read it through qa-store (readQa) and treat miss, corrupt and unknown as states to assert, never as an empty pass.
A loop over a corpus needs a “there are some” guard beside it, or it is green over nothing.
Never write into test/attestations/ from a test. It is the judgement store on main; a test that needs one builds it in a temporary root.

Adding a new formal-layer project

When a new formal-layer library is added to the build manifest:

discoverProjects() in helpers.ts auto-discovers it
the project test suite automatically tests it (directory, files, audits)
No manual test registration needed

Test categories (from `TestCategory` type)

Category	What it tests
`formal-compile`	formal-layer files exist, have imports, are non-empty
`formal-library`	toolchain, build manifest, dependencies declared
`formal-gap`	unproved-gap count per project/file
`render-coverage`	render tags resolve, statement coverage, consistency
`doc-structure`	labels, cross-refs, environments
`schema-validity`	JSON manifests match their schemas
`infrastructure`	server config, scripts, deploy, .gitignore
`content-validation`	content objects pass Zod + constraint + AST checks

Content-object tests

Test content objects using the validation pipeline:

import { validateObjects } from "../../pipeline/validate";

describe("Content objects", () => {
  test("all objects pass schema validation", async () => {
    const result = await validateObjects("content/<paper>/<section>/");
    const errors = result.issues.filter(i => i.level === "error");
    expect(errors).toHaveLength(0);
  });

  test("rendered output passes AST validation", async () => {
    // The validation pipeline includes AST checks automatically
    const result = await validateObjects("content/<paper>/<section>/");
    expect(result.valid).toBe(true);
  });
});

Running tests

# All tests
bun test                              # from scripts/tests/
cat-harness-tools/scripts/tests/run-tests.sh          # from repo root

# By suite
cat-harness-tools/scripts/tests/run-tests.sh formal     # formal-layer project tests
cat-harness-tools/scripts/tests/run-tests.sh coverage   # render coverage
cat-harness-tools/scripts/tests/run-tests.sh infra      # Infrastructure

# JSON report (TestReport schema)
cat-harness-tools/scripts/tests/run-tests.sh --json
bun run report.ts --out test-report.json

Report output

bun run report.ts outputs JSON conforming to TestReport in schemas/formalization-types.ts. This report is consumed by:

CI pipeline — build status, pass/fail gates
Publication pipeline — coverage statistics for output artifacts
Status overlay — formalization/coverage status overlay

The report includes by_category breakdown so each pipeline stage can extract just the tests relevant to it.

Processes that run this skill

process	step(s) that name it
Test-plan execution	Execute every case, or skip it with a reason