DiscoSense: Commonsense Reasoning with Discourse Connectives
2022-10-22Code Available0· sign in to hype
Prajjwal Bhargava, Vincent Ng
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/prajjwal1/discosenseOfficialIn paperpytorch★ 4
Abstract
We present DiscoSense, a benchmark for commonsense reasoning via understanding a wide variety of discourse connectives. We generate compelling distractors in DiscoSense using Conditional Adversarial Filtering, an extension of Adversarial Filtering that employs conditional generation. We show that state-of-the-art pre-trained language models struggle to perform well on DiscoSense, which makes this dataset ideal for evaluating next-generation commonsense reasoning systems.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| HellaSwag | ELECTRA-Large 335M (fine-tuned on DiscoSense and HellaSwag) | Accuracy | 91.5 | — | Unverified |
| HellaSwag | ELECTRA-Large 335M (fine-tuned on HellaSwag) | Accuracy | 86.9 | — | Unverified |