SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 651–700 of 747 papers

TitleStatusHype
Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI—0
SQALER: Scaling Question Answering by Decoupling Multi-Hop and Logical Reasoning—0
Logical Assessment Formula and Its Principles for Evaluations with Inaccurate Ground-Truth Labels—0
One-Step Abductive Multi-Target Learning with Diverse Noisy Samples and Its Application to Tumour Segmentation for Breast CancerCode0
A Survey on State-of-the-art Techniques for Knowledge Graphs Construction and Challenges ahead—0
A Survey of Knowledge Enhanced Pre-trained Models—0
NAIL: A Challenging Benchmark for Na\"ive Logical Reasoning—0
Logic Pre-Training of Language Models—0
Truth Table Deep Convolutional Neural Network, A New SAT-Encodable Architecture - Application To Complete Robustness—0
Efficient Training and Inference of Hypergraph Reasoning Networks—0
Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy LogicCode0
What Makes Reading Comprehension Questions Difficult? Investigating Variation in Passage Sources and Question Types—0
Counterfactual Adversarial Learning with Representation InterpolationCode0
Sinoledge: A Knowledge Engine based on Logical Reasoning and Distributed Micro Services—0
From Statistical Relational to Neurosymbolic Artificial Intelligence: a Survey—0
Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning—0
Knowledge Informed Semantic Parsing for Conversational Question Answering—0
Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning—0
Reasoning with Transformer-based Models: Deep Learning, but Shallow ReasoningCode0
Techniques for Symbol Grounding with SATNetCode0
Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer LearningCode0
Probabilistic Sufficient ExplanationsCode0
The General Theory of General Intelligence: A Pragmatic Patternist Perspective—0
Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution—0
Context Transformer with Stacked Pointer Networks for Conversational Question Answering over Knowledge GraphsCode0
Neural Sequence-to-grid Module for Learning Symbolic RulesCode0
Bayes Meets Entailment and Prediction: Commonsense Reasoning with Non-monotonicity, Paraconsistency and Predictive Accuracy—0
A Closer Look at the Robustness of Vision-and-Language Pre-trained Models—0
Neurosymbolic AI: The 3rd Wave—0
Handling Noisy Labels via One-Step Abductive Multi-Target Learning and Its Application to Helicobacter Pylori Segmentation—0
Neural Software AnalysisCode0
Axiom Learning and Belief Tracing for Transparent Decision Making in Robotics—0
Few-shot Visual Reasoning with Meta-analogical Contrastive Learning—0
Learning Syllogism with Euler Neural-Networks—0
Medical idioms for clinical Bayesian network development—0
Multi-source Meta Transfer for Low Resource Multiple-Choice Question Answering—0
Matrix Shuffle-Exchange Networks for Hard 2D TasksCode0
A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning—0
Mathematical Reasoning via Self-supervised Skip-tree Training—0
Bayesian Entailment Hypothesis: How Brains Implement Monotonic and Non-monotonic Reasoning—0
Unifying Neural Learning and Symbolic Reasoning for Spinal Medical Report Generation—0
Multi-Step Inference for Reasoning Over Paragraphs—0
Extending Automated Deduction for Commonsense Reasoning—0
From Statistical Relational to Neuro-Symbolic Artificial Intelligence—0
Improving Certified Robustness via Statistical Learning with Logical ReasoningCode0
Cognitive Argumentation and the Suppression Task—0
HypoML: Visual Analysis for Hypothesis-based Evaluation of Machine Learning Models—0
A (Simplified) Supreme Being Necessarily Exists, says the Computer: Computationally Explored Variants of Gödel's Ontological Argument—0
Generating Programmatic Referring Expressions via Program SynthesisCode0
Bridging Machine Learning and Logical Reasoning by Abductive LearningCode0
Show:102550
← PrevPage 14 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified