Axe vs WAVE vs Lighthouse (and Pa11y): We Tested All Four on the Same Broken Page

Short version: trust axe if you write code, WAVE if you edit content, and treat Lighthouse’s accessibility score as a smoke alarm — useful when it goes off, meaningless as proof of safety. To test that advice, we ran all three tools (plus Pa11y) against the same intentionally broken web page on July 3, 2026, and got five different issue counts, ranging from 41 to 75. Those disagreements aren’t bugs. They tell you exactly what each tool is for.

We ran all four tools on the same broken page

For a fair fight we used the W3C’s own Before and After Demonstration — a fake city-guide homepage the Web Accessibility Initiative deliberately built with accessibility failures, so anyone can rerun this test on the same URL.

Our setup: axe-core 4.12.1 run headlessly with its WCAG 2.1 A/AA rule tags, Lighthouse 13.4.0 with default settings, Pa11y 9.1.1 with default settings, and WAVE via WebAIM’s online report at wave.webaim.org. Same page, same day. Here’s what came back:

One broken page. Five answers. If you’ve ever run two checkers on your own site and wondered which number is “real,” this is why: they all are — each tool is just counting a different thing.

Why did the counts disagree so much?

Three reasons, and none of them is that a tool is “wrong.”

They count different units. Axe groups findings by rule (5 rules, 44 affected elements). WAVE counts every flagged icon on the page. Lighthouse counts failed audits and scores them. Comparing raw totals across tools is comparing apples to icons.

They use different engines. Lighthouse’s accessibility audits are powered by axe-core — the JSON report from our run literally credits “axe-core 4.12.1” as its engine — but Google runs a curated subset with its own scoring on top. Pa11y defaults to a completely different engine, HTML_CodeSniffer, testing against WCAG 2 AA. WAVE uses WebAIM’s own ruleset. Even the same engine in two wrappers disagreed: raw axe found 44 issues, Pa11y running axe found 46, because each wrapper decides which rules to switch on.

They handle uncertainty differently. The starkest gap was color contrast: WAVE flagged 38 contrast errors, while axe failed only 2 elements and pushed 2 more into a “needs review” bucket rather than guessing. Same page, 19x difference — purely a difference in how each tool treats cases it can’t measure with confidence. (For what the rule actually requires, see our color contrast guide.)

The tools even disagreed on severity. The page’s unlabeled dropdown menu was a critical violation in axe and merely an alert in WAVE. Lighthouse, meanwhile, failed the page on two things our axe run didn’t check at all: a missing main landmark and undersized touch targets, the latter a WCAG 2.2 rule outside the 2.1 tags we tested.

What is axe, and why do developers trust it most?

Axe-core is an open-source rules engine from Deque Systems, and it makes an unusual promise: it “returns zero false positives (bugs notwithstanding),” per the axe-core documentation. When axe can’t be certain something is broken, it files it under “needs review” instead of crying wolf — exactly what we saw with those two contrast items.

That conservatism is why axe is the engine embedded in Lighthouse, browser DevTools extensions, and most CI testing libraries. Deque also claims axe finds “on average 57% of WCAG issues automatically” — a number worth unpacking, which we do below.

Trust it for: developer workflows. Every flag is actionable, rules map cleanly to WCAG criteria, and it slots into automated test suites so regressions get caught before launch.

What is WAVE best at?

WAVE is WebAIM’s checker, available as a browser extension and a free online report. Instead of a list, it paints icons directly onto your rendered page — red for errors, yellow for alerts that need a human decision. That visual approach is why WAVE found the most issues in our test: it surfaces everything it sees and asks you to judge the ambiguous cases.

WebAIM is also refreshingly honest about the limits. Its help documentation states: “WAVE cannot tell you if your web content is accessible. Only a human can determine true accessibility” — and notes that WAVE never declares a page “passed.”

Trust it for: content editors, designers, and owners who want to see problems in place. If “image-alt violation on node 14” means nothing to you, a red icon on the actual photo will.

Is the Lighthouse accessibility score reliable?

Use it as a floor, never a ceiling. Lighthouse — built into Chrome’s DevTools, no install needed — grades each accessibility audit pass/fail and combines them into “a weighted average of all accessibility audits,” with weighting “based on axe user impact assessments,” per Google’s scoring documentation. The same page lists an entire section of manual checks that don’t affect the score at all.

Our broken test page scored 50/100. That sounds alarming — and it should — but the inverse is the trap: a page can score 100 while remaining unusable for a screen-reader user, because the score only covers the automated subset. And as our test showed, Lighthouse makes its own picks about which axe rules to run — it failed two audits our tagged axe run excluded — which is why its count didn’t match raw axe despite sharing the engine.

Trust it for: a free, zero-setup smoke test. A low score means real problems. A perfect score means almost nothing on its own.

Where does Pa11y fit in?

Pa11y is the quiet workhorse of the four: a free, open-source command-line tool that loads pages and reports issues, built for automation rather than for humans clicking around. Its documentation shows the default runner is HTML_CodeSniffer testing WCAG 2 AA, with axe available as a swap-in — which is how we produced both a 41 and a 46 from the same tool.

Because it’s scriptable, Pa11y (and its Pa11y CI companion) is the typical choice for teams that want every code deployment to fail automatically if new accessibility errors appear. If nobody at your business uses a terminal, skip it — the other three cover you.

Which tool should you actually use?

What none of these tools will catch

The best coverage figure in the industry comes from the maker of axe itself. Deque’s 2021 study of roughly 300,000 issues across 13,000+ audited pages found automated testing identified 57% of accessibility issues by volume — and that number is the optimistic one, measured by issue count rather than by how many WCAG criteria a machine can evaluate.

The missing 43% is the stuff lawsuits are made of: whether your alt text is accurate rather than merely present, whether keyboard focus gets trapped in your popup, whether a screen reader announces your form errors. We break down the specific failure modes in what automated scans miss, and our field guide to manual vs automated testing shows how the two halves fit together. Notably, the issues these tools do catch reliably — missing alt text, contrast, unlabeled fields — are the same ones topping the list of most common accessibility issues, so running any of them is far better than running none.

The fastest path if you’re not technical

If you’ve read this far and your honest reaction is “I’m not installing four tools,” that’s reasonable. Here’s the short version:

1. Run one automated check today — our free accessibility checker scans any URL from the browser, no extension or DevTools required.
2. Fix the machine-findable errors first: alt text, contrast, labels. They’re the highest-volume offenders and the easiest to verify.
3. Then get human eyes on the rest. A manual accessibility audit covers the keyboard, screen-reader, and judgment-call issues that all four tools above openly admit they can’t test — and if the findings need fixing, remediation repairs the actual code.

Four good tools gave us five different answers on one broken page. The lesson isn’t to distrust them — it’s to use the right one for your role, and to remember that every vendor on this list, from Deque to WebAIM to Google, says the same thing in their own docs: the machine finds the first half. A human finds the rest.

Want the first half done for you right now? Start with a free scan.