Factual Inconsistency Detection in Chart Captioning
Detect factual inconsistency between charts and captions.
Papers
No papers found.
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Bard (before Gemini) | Kendall's Tau-c | 0.29 | — | Unverified |
| 2 | GPT-4V | Kendall's Tau-c | 0.22 | — | Unverified |
| 3 | ChartVE | Kendall's Tau-c | 0.22 | — | Unverified |
| 4 | LLaVA-1.5-13B | Kendall's Tau-c | 0.21 | — | Unverified |
| 5 | DePlot + GPT-4 | Kendall's Tau-c | 0.11 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | GPT-4V | Kendall's Tau-c | 0.21 | — | Unverified |
| 2 | DePlot + GPT-4 | Kendall's Tau-c | 0.12 | — | Unverified |
| 3 | Bard | Kendall's Tau-c | 0.11 | — | Unverified |
| 4 | ChartVE | Kendall's Tau-c | 0.09 | — | Unverified |
| 5 | LLaVA-1.5-13B | Kendall's Tau-c | 0.06 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | ChartVE | Kendall's Tau-c | 0.18 | — | Unverified |
| 2 | GPT-4V | Kendall's Tau-c | 0.16 | — | Unverified |
| 3 | DePlot + GPT-4 | Kendall's Tau-c | 0.13 | — | Unverified |
| 4 | LLaVA-1.5-13B | Kendall's Tau-c | 0 | — | Unverified |
| 5 | Bard (before Gemini) | Kendall's Tau-c | -0.01 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | ChartVE | Kendall's Tau-c | 0.18 | — | Unverified |