SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 401–450 of 747 papers

TitleStatusHype
Type-dependent Prompt CycleQAG : Cycle Consistency for Multi-hop Question Generation—0
Unifying Neural Learning and Symbolic Reasoning for Spinal Medical Report Generation—0
Unifying Structure Reasoning and Language Model Pre-training for Complex Reasoning—0
Unleash LLMs Potential for Recommendation by Coordinating Twin-Tower Dynamic Semantic Token Generator—0
Unveiling Scoring Processes: Dissecting the Differences between LLMs and Human Graders in Automatic Scoring—0
VERUS-LM: a Versatile Framework for Combining LLMs with Symbolic Reasoning—0
VGRP-Bench: Visual Grid Reasoning Puzzle Benchmark for Large Vision-Language Models—0
Video Token Sparsification for Efficient Multimodal LLMs in Autonomous Driving—0
VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge—0
VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RL—0
Wait, but Tylenol is Acetaminophen... Investigating and Improving Language Models' Ability to Resist Requests for Misinformation—0
What is the Title of this Paper? Solving logic puzzles using algorithms—0
What Makes Machine Reading Comprehension Questions Difficult? Investigating Variation in Passage Sources and Question Types—0
What Makes Reading Comprehension Questions Difficult? Investigating Variation in Passage Sources and Question Types—0
Why should we ever automate moral decision making?—0
XAgents: A Framework for Interpretable Rule-Based Multi-Agents Cooperation—0
WatME: Towards Lossless Watermarking Through Lexical Redundancy—0
ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning—0
Take A Step Back: Rethinking the Two Stages in Visual Reasoning—0
Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation—0
GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussion—0
Towards Geometry Problem Solving in the Large Model Era: A Survey—0
3DAxisPrompt: Promoting the 3D Grounding and Reasoning in GPT-4o—0
AbductionRules: Training Transformers to Explain Unexpected Inputs—0
Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution—0
A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models—0
A Closer Look at the Robustness of Vision-and-Language Pre-trained Models—0
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT—0
Action is the primary key: a categorical framework for episode description and logical reasoning—0
A Densely Connected Criss-Cross Attention Network for Document-level Relation Extraction—0
Algorithmic Phase Transitions in Language Models: A Mechanistic Case Study of Arithmetic—0
Aligning CodeLLMs with Direct Preference Optimization—0
A Mechanistic Interpretation of Syllogistic Reasoning in Auto-Regressive Language Models—0
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos—0
AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models—0
Analyzing Large language models chatbots: An experimental approach using a probability test—0
An Empirical Study of Retrieval Augmented Generation with Chain-of-Thought—0
A Neuro-Symbolic Approach to Multi-Agent RL for Interpretability and Probabilistic Decision Making—0
A New Algorithmic Decision for Categorical Syllogisms via Caroll's Diagrams—0
A New Fundamental Evidence of Non-Classical Structure in the Combination of Natural Concepts—0
An Explainable Fast Deep Neural Network for Emotion Recognition—0
Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs—0
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning—0
A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning—0
Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games—0
Are LLMs Rigorous Logical Reasoner? Empowering Natural Language Proof Generation with Contrastive Stepwise Decoding—0
A Report on the llms evaluating the high school questions—0
Argumentation Synthesis following Rhetorical Strategies—0
A separation logic for sequences in pointer programs and its decidability—0
AS-ES Learning: Towards Efficient CoT Learning in Small Models—0
Show:102550
← PrevPage 9 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified