Benchmarks
Six standardized tasks
Each task is a small, well-scoped change verified locally by the CLI. Paste the prompt into any coding agent, then omot bench verify scores pass, partial, or fail.
dep-patch-bump
Bump a dependency within its major version
dependencies · small
n=0Not enough data yet
add-unit-test
Add unit tests for an untested function
testing · small
n=0Not enough data yet
fix-lint-rule
Fix every violation of one lint rule (ESLint or Biome)
lint · medium
n=0Not enough data yet
a11y-img-alt
Add missing alt attributes to images
accessibility · small
n=2Not enough data yet
ts-strict-file
Remove explicit any from one file
types · small
n=2Not enough data yet
fix-failing-test
Fix a deliberately introduced failing test
testing · small
n=0Not enough data yet