Open QR testing
QR degradation benchmark
This is the public test method behind our QR reliability work. It defines the conditions, exact-pass rule and reporting limits before results are published, so the benchmark cannot quietly change to fit a preferred conclusion.
The benchmark records whether a decoder recovered the exact original payload under one controlled degradation at a time. A browser decode is evidence for that decoder and condition only; it is not a universal guarantee for phones, printers or materials.
Status: method v1 is defined; results are not claimed yet
We are publishing the test shape before publishing a leaderboard or conclusion. The first result set will keep automated browser-decoder observations separate from real-phone and physical-print tests, with test dates and implementation versions recorded alongside the data.
Download the machine-readable v1 benchmark method.
What the benchmark varies
| Dimension | Why it is isolated |
|---|---|
| Blur | Tests softened module boundaries without pretending every soft-looking image failed because of blur. |
| JPEG compression | Measures artefacts introduced by lossy re-encoding separately from optical blur. |
| Quiet-zone loss | Reduces the clear border from the four-module reference down to no border. |
| Effective resolution | Reduces image pixels per QR module to expose marginal raster resolution. |
| Contrast | Tests foreground/background separation without mixing it with resizing. |
| Logo obstruction | Tests controlled centre coverage; it does not treat an error-correction level as permission to cover a fixed percentage. |
| Skew | Applies increasing geometric rotation/perspective conditions independently. |
| Print scaling | Checks size/reproduction changes separately from the digital source image. |
Symbols and payloads
The corpus covers short and long URLs, plain text, Wi-Fi and contact payloads across L, M, Q and H error-correction levels. QR version, module count and actual test settings are recorded per case rather than assuming one symbol represents every QR code.
Pass rule
- The test condition is generated from a known source QR.
- The named decoder attempts to decode that exact rendered or captured artefact.
- A pass requires the recovered payload to match the source payload exactly.
- A failed decode remains a failure; the benchmark does not infer or repair missing data and then call the original a pass.
- Browser, phone-camera and printed-proof results are reported as separate evidence sets.
What the benchmark will not claim
- No universal “scannability score”.
- No guarantee that every phone or camera will reproduce another decoder's result.
- No claim that error correction means a fixed percentage of the visible QR can safely be hidden or damaged.
- No malware or destination-safety judgement based on whether a QR decodes.
- No physical-print conclusion until the relevant printed artefact has actually been tested.
Reproducibility
Published result packs will include the method version, test date, decoder/browser or phone details, degradation value and exact-match result. Aggregate tables will be derived from those case rows rather than hand-entered marketing figures.