The accessibility checks a scanner cannot automate

Deque, who build axe-core, analysed over 13,000 pages and nearly 300,000 issues across more than 2,000 audits, and found that on average 57 percent of accessibility issues were completely covered by automated testing. That is a vendor study of their own engine, so treat it as a ceiling rather than a floor — but it is the best public number there is, and the rest needs something that can operate the page.

The part a scanner structurally cannot do is finish a task. Whether focus survives opening a dialog, whether a custom dropdown can be operated without a mouse, whether an error message is announced at all — those need an agent in the page.

computer.a11y() returns the accessibility tree as YAML: roles, accessible names and structure. It is broadly the same information a screen reader consumes, and it is what the model reads instead of guessing from pixels. A button with no accessible name is obvious in the tree and invisible in a screenshot.

Audit the page with computer.a11y(): report missing landmarks, unlabelled controls, images without alt text and buttons without accessible names. Then tab through the page with computer.keypress('Tab') and report anything unreachable by keyboard or with no visible focus.

Keyboard traps live in dialogs and menus, so give those their own run:

Open the main dialog or menu on the page. Tab through it and report whether focus is trapped inside while it is open, whether Escape closes it, and where focus goes when it closes. Say which element had focus at each step.

A scanner is fast, deterministic and good at the rule-checkable half. This is good at the operate-the-page half. They do not overlap much, which is the point — use both and treat this as the fast, repeatable part of a manual pass.

• Model — a fast model (Claude Haiku 4.5, GPT-5.6 Luna) is fine for a flow you run often. Reach for a deeper model when a run keeps getting stuck on the same step.
• Devices — laptop is enough for a first pass. Add phone when the flow is mostly used on mobile.
• Step budget — 30 scripts covers most flows. Raise it for long, multi-page journeys.
• Reasoning effort — passed straight through to the model. Higher effort costs more and thinks longer before acting.

• A verdict: pass, fail or blocked.
• Named checks, each marked passed or not, so you can see what was actually asserted.
• Findings rated critical, high, medium, low or info, with the evidence behind each one.
• A screenshot for every step, and one session video for the whole run.
• Console errors, uncaught page errors and failed or 4xx/5xx requests, collected automatically.

Worth knowing before you point it at something important.

• This is not a WCAG conformance audit. There is no axe-core run, no contrast ratio maths and no VPAT.
• It is not a substitute for testing with real assistive technology and real disabled users.
• It reads the tree a screen reader would announce, which is close to but not the same as hearing it announced.