opsinjs

For testing

The manual and automated passes that make up an accessibility test plan for a health screen, and how to reproduce each one.

What we guarantee

One thing, and it is worth stating precisely: automated testing finds a minority of accessibility defects, and in health UI it finds the least dangerous ones. An automated pass will catch a missing label and a low-contrast pair. It will not catch a status pill whose colour and whose word disagree, a trend arrow pointing the wrong way, or a stale reading presented as current. Those are the defects that hurt somebody.

So the guarantee is about the shape of the test plan rather than a tool: every accessibility property this system claims is written on the relevant page as a reproducible check with an expected result and a described failure, so a tester who has never used a screen reader can still produce a trustworthy answer.

What you own

The plan itself, its schedule, and the decision about what blocks a release. Also:

  • Choosing the tested matrix. Which screen reader and browser combinations matter is a function of who uses your product. Ours is on Screen readers and it is a proposal, not a prescription.
  • Test data that includes the bad cases. Most health screens are tested with a comfortable number in the middle of the range. Test the empty case, the single-reading case, the stale case, the out-of-range case, the implausible case (a value no human produces, arriving from a broken sensor) and the case where two sources disagree.
  • Regression coverage of the announcement contract. A live region that stops announcing is invisible to every visual test you own.

How to check

Six passes. Run them in this order; each is cheaper than the one after it and catches different defects.

The automated pass

Run an automated rule engine over every route in a real browser. Treat its output as a lower bound: zero violations means "no violations of the rules this engine can express", which is a much smaller claim than "accessible". Record the engine and its version in the result, because the rule set changes between versions and an unversioned pass cannot be reproduced.

The keyboard pass

Complete the primary task with the keyboard only, then complete the recovery task. Recovery means cancelling it, correcting a validation error, dismissing a dialog. Expected result: every stop shows a visible focus indicator, no stop is obscured by sticky chrome, and no sequence leaves you unable to progress. Failure looks like a focus indicator that disappears behind a bottom bar, which is SC 2.4.11.

The zoom and text-size pass

Set the browser to 400% zoom at a 1280 px viewport width. That is the SC 1.4.10 Reflow condition, and it is equivalent to a 320 px viewport. Separately, set text size to 200% without zooming. Expected result: no horizontal scrolling of the page as a whole, no clipped text, no truncated numeric value. Failure in a health screen usually looks like a reading truncated to its first two digits. Method on Text resizing and zoom.

The greyscale pass

Apply a full desaturation filter and repeat the primary task. Expected result: the status of every value is still readable, because the word and the icon are still there. Failure looks like two pills that were red and amber becoming the same grey. Method on Colour independence.

The preference pass

Run the task four times with prefers-reduced-motion, prefers-reduced-transparency, prefers-contrast and forced colours turned on in turn. Expected result: the information does not change, only its presentation. Failure looks like a state change that was carried entirely by a transition, so with motion reduced nothing appears to happen at all.

The screen-reader pass

Complete the task with a screen reader on a device you have actually configured. Do not do it with the developer-tools accessibility tree, which shows you the data rather than the experience. Record what was announced, in what order, and what was silent. A silent success is a defect; a reader who submits a log entry and hears nothing does not know whether it saved.

Measured results

This table has not been generated. It is produced by scripts/run-evals.mts. Run pnpm run generate and reload.

Nothing on this site types a measured number by hand, so an ungenerated table shows this rather than an example.

There are no published test results for this system. Playwright is in no package.json, so a normal clone does not have it and scripts/capture-registry.mts detects its absence and no-ops rather than pretending to produce artefacts. The one job that installs it at job time is .github/workflows/nightly.yml, and it runs the accessibility rig's browser half against the built /view routes. That half measures hit area, survival at 1.3x and 200% with no truncated value, and it publishes nothing: it fails on what is wrong and records nothing about what passed. When a suite exists, its dated results appear here and on Conformance report.

Known gaps

  • No test suite exists, automated or manual, for the design system itself.
  • The regression gate covers rules, not criteria. pnpm run check:a11y blocks a pull request on the rules in scripts/check-a11y.mts, and the nightly adds the three browser measurements above. Between them they touch a handful of the criteria on Conformance report and none of the ones a person has to judge. Axe does not run over a rendered page anywhere, and no other rule engine does either. The nightly is on a schedule, so it goes red the morning after a merge rather than before it.
  • No assistive technology has been used against this codebase. The docs site itself is unaudited; see This site's accessibility.

Updates to this page

Last read through against the system on 2026-09-02. Due for review every 6 months; expiry is reported by pnpm run check:freshness.

On this page