SOTAVerified

"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection

2017-05-01ACL 2017Code Available1· sign in to hype

William Yang Wang

Code Available — Be the first to reproduce this paper.

Reproduce

Code

Abstract

Automatic fake news detection is a challenging problem in deception detection, and it has tremendous real-world political and social impacts. However, statistical approaches to combating fake news has been dramatically limited by the lack of labeled benchmark datasets. In this paper, we present liar: a new, publicly available dataset for fake news detection. We collected a decade-long, 12.8K manually labeled short statements in various contexts from PolitiFact.com, which provides detailed analysis report and links to source documents for each case. This dataset can be used for fact-checking research as well. Notably, this new dataset is an order of magnitude larger than previously largest public fake news datasets of similar type. Empirically, we investigate automatic fake news detection based on surface-level linguistic patterns. We have designed a novel, hybrid convolutional neural network to integrate meta-data with text. We show that this hybrid approach can improve a text-only deep learning model.

Tasks

Benchmark Results

DatasetModelMetricClaimedVerifiedStatus
LIARHybrid CNNs (Text + All)Test Accuracy0.27Unverified
LIARCNNsTest Accuracy0.27Unverified
LIARHybrid CNNs (Text + Speaker)Test Accuracy0.25Unverified
LIARBi-LSTMsTest Accuracy0.23Unverified

Reproductions