Skip to content

feat: runtime response-schema validation in dev/test#37

Merged
ethanj merged 1 commit intomainfrom
runtime-response-validation
Apr 22, 2026
Merged

feat: runtime response-schema validation in dev/test#37
ethanj merged 1 commit intomainfrom
runtime-response-validation

Conversation

@ethanj
Copy link
Copy Markdown
Contributor

@ethanj ethanj commented Apr 22, 2026

Summary

Closes the gap Codex flagged: `check:openapi` only catches schema↔spec drift, not formatter↔schema drift. A formatter change that forgets to update the schema would still slip past the gate. This PR adds a dev/test-mode middleware that wraps `res.json()` and parses the outgoing body against the route's declared response schema — prod is a pass-through no-op, so zero per-request cost in production.

What's in the PR

  • `src/middleware/validate-response.ts` — the wrapper. Throws loudly in dev/test when a 2xx body doesn't match its schema; no-op when `NODE_ENV=production`. Error messages point at the right fix (update the formatter, or update the schema + regenerate the spec).
  • `src/routes/response-schema-map.ts` — the route→schema map. Keyed by Express's router-relative `${method} ${route.path}` so parameterized paths like `/:id` resolve correctly.
  • Wired into both `createMemoryRouter` and `createAgentRouter` via `router.use(validateResponse(...))` before any handler.

Real drift caught on first run (both fixed here)

  1. `Date` leaking into search memory `created_at`. Express serializes `Date` → ISO string, so the wire shape is string, but `res.json()` sees the raw Date before serialization. Added an `IsoDateString` preprocess that accepts both.
  2. `NaN` leaking into search similarity/score/importance. `JSON.stringify(NaN)` writes `null` on the wire, so the wire shape is `number | null`, but `res.json()` sees NaN. Added a `NumberOrNaN` preprocess that converts NaN → null and validates as nullable number.

Schemas now report accurate wire types (string for dates, nullable number for NaN-possible floats). OpenAPI spec regenerates with zero further diff.

Test plan

  • `npx tsc --noEmit` clean
  • All 1037 core tests pass with the middleware active
  • `npm run generate:openapi` produces zero diff after the schema fixes
  • Verified the validator throws on drift — the Date and NaN issues above were caught by it on the first test run
  • CI green end-to-end

Codex flagged that `check:openapi` only catches schema↔spec drift —
a formatter change that forgets to update schemas still slips past.
This closes that gap with a dev/test-mode middleware that wraps
`res.json()` and parses the outgoing body against the route's
declared response schema. Prod is a pass-through no-op.

- src/middleware/validate-response.ts — the wrapper. Throws loudly
  in dev/test when a 2xx body doesn't match its schema; no-op when
  NODE_ENV=production.
- src/routes/response-schema-map.ts — the route→schema map. Keyed
  by Express's router-relative `${method} ${route.path}` so the
  lookup works for parameterized paths like `/:id`.
- Wired into both createMemoryRouter and createAgentRouter via
  `router.use(validateResponse(...))` before any handler.

The validator caught two real formatter↔schema drift issues on its
first run, both fixed here:

- `Date` leaking into search memory `created_at`. Express serializes
  Date → ISO string, so the wire shape is string, but `res.json()`
  sees the raw Date before serialization. Added an `IsoDateString`
  preprocess that accepts both and validates as string.
- `NaN` leaking into search similarity/score/importance.
  `JSON.stringify(NaN)` writes `null` on the wire, so the wire shape
  is `number | null`, but `res.json()` sees NaN. Added a `NumberOrNaN`
  preprocess that converts NaN → null and validates as nullable number.

Schemas now report the accurate wire types (string, nullable number)
and the OpenAPI spec regenerates with zero further drift. All 1037
core tests pass with the middleware active.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ethanj ethanj merged commit ee7ddc8 into main Apr 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant