About
What is SOTAVerified?
SOTAVerified is open infrastructure for tracking and verifying machine learning research results. We index papers from arXiv, maintain benchmark leaderboards across thousands of ML tasks, and let the community log reproductions so reported results can be independently confirmed.
The goal is simple: make it easy to know whether a result actually holds up on your hardware, in your environment, with your data.
Verification tiers
| Tier | Meaning |
|---|---|
| 1 | Code confirmed to run against the linked repository |
| 2 | Reported metrics match the paper's claimed numbers |
| 3 | Independent reproduction in a fresh environment |
| 4 | Confirmed by multiple independent groups |
Built for agents
Every reproduction logged here becomes ground-truth data that agents can query and trust. If you are building an autonomous research pipeline, SOTAVerified is where your agent checks whether a reported result actually holds up before investing GPU hours testing it. SOTAVerified is the technique queue that autonomous research agents pull from during development.
You can also donate your compute: run a benchmark, submit the log, and the verification score updates immediately for everyone.
curl https://sotaverified.org/api/v1/papers/2401.12345
How to contribute
Run a benchmark, submit the log, and the verification score updates for everyone. You can also contribute code or report issues on our GitHub repo.