Chart Question Answering
Question Answering task on charts images
Papers
No papers found.
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | ChartPaLI-5B + PaLM 2-S | 1:1 Accuracy | 81.3 | — | Unverified |
| 2 | Gemini Ultra | 1:1 Accuracy | 80.8 | — | Unverified |
| 3 | DePlot+FlanPaLM+Codex (PoT Self-Consistency) | 1:1 Accuracy | 79.3 | — | Unverified |
| 4 | ChartPaLI-5B | 1:1 Accuracy | 77.3 | — | Unverified |
| 5 | DePlot+Codex (PoT Self-Consistency) | 1:1 Accuracy | 76.7 | — | Unverified |
| 6 | ScreenAI 5B (4.62 B params, w/ OCR) | 1:1 Accuracy | 76.7 | — | Unverified |
| 7 | SMoLA-PaLI-X Specialist Model | 1:1 Accuracy | 74.6 | — | Unverified |
| 8 | SMoLA-PaLI-X Generalist Model | 1:1 Accuracy | 73.8 | — | Unverified |
| 9 | MatCha4096 + LaMenDa | 1:1 Accuracy | 72.64 | — | Unverified |
| 10 | PaLI-X (Single-task FT w/ OCR) | 1:1 Accuracy | 72.3 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | MatCha4096 + LaMenDa | 1:1 Accuracy | 92.89 | — | Unverified |
| 2 | MatCha | 1:1 Accuracy | 91.5 | — | Unverified |
| 3 | DePlot+FlanPaLM+Codex (PoT Self-Consistency) | 1:1 Accuracy | 66.6 | — | Unverified |
| 4 | VL-T5-OCR | 1:1 Accuracy | 66 | — | Unverified |
| 5 | CRCT | 1:1 Accuracy | 55.7 | — | Unverified |
| 6 | VisionTapas-OCR | 1:1 Accuracy | 53.9 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | vlt5 - 11th ep FineTune | 1:1 Accuracy | 0.31 | — | Unverified |
| 2 | Matcha-chartQA | 1:1 Accuracy | 0.26 | — | Unverified |
| 3 | crct- 11th ep FineTune | 1:1 Accuracy | 0.24 | — | Unverified |
| 4 | vlt5 - baseline | 1:1 Accuracy | 0.19 | — | Unverified |
| 5 | crct - baseline | 1:1 Accuracy | 0.18 | — | Unverified |