SOTAVerified

On the Benefit of Combining Neural, Statistical and External Features for Fake News Identification

2017-12-11Code Available0· sign in to hype

Gaurav Bhatt, Aman Sharma, Shivam Sharma, Ankush Nagpal, Balasubramanian Raman, Ankush Mittal

Code Available — Be the first to reproduce this paper.

Reproduce

Code

Abstract

Identifying the veracity of a news article is an interesting problem while automating this process can be a challenging task. Detection of a news article as fake is still an open question as it is contingent on many factors which the current state-of-the-art models fail to incorporate. In this paper, we explore a subtask to fake news identification, and that is stance detection. Given a news article, the task is to determine the relevance of the body and its claim. We present a novel idea that combines the neural, statistical and external features to provide an efficient solution to this problem. We compute the neural embedding from the deep recurrent model, statistical features from the weighted n-gram bag-of-words model and handcrafted external features with the help of feature engineering heuristics. Finally, using deep neural layer all the features are combined, thereby classifying the headline-body news pair as agree, disagree, discuss, or unrelated. We compare our proposed technique with the current state-of-the-art models on the fake news challenge dataset. Through extensive experiments, we find that the proposed model outperforms all the state-of-the-art techniques including the submissions to the fake news challenge.

Tasks

Benchmark Results

DatasetModelMetricClaimedVerifiedStatus
FNC-1Bhatt et al.Weighted Accuracy83.08Unverified
FNC-1Baseline based on skip-thought embeddings (Bhatt et al., 2017)Weighted Accuracy76.18Unverified
FNC-1Baseline based on word2vec + hand-crafted features (Bhatt et al., 2017)Weighted Accuracy72.78Unverified
FNC-1Neural baseline based on bi-directional LSTMs (Bhatt et al., 2017)Weighted Accuracy63.11Unverified

Reproductions