SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 251–300 of 747 papers

TitleStatusHype
Efficient but Vulnerable: Benchmarking and Defending LLM Batch Prompting Attack—0
City-LEO: Toward Transparent City Management Using LLM with End-to-End Optimization—0
BTPK-based interpretable method for NER tasks based on Talmudic Public Announcement Logic—0
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts—0
Are LLMs Rigorous Logical Reasoner? Empowering Natural Language Proof Generation with Contrastive Stepwise Decoding—0
Algorithmic Phase Transitions in Language Models: A Mechanistic Case Study of Arithmetic—0
Bridging Technology and Humanities: Evaluating the Impact of Large Language Models on Social Sciences Research with DeepSeek-R1—0
Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation—0
Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games—0
KnowGraph: Knowledge-Enabled Anomaly Detection via Logical Reasoning on Graph Data—0
Dynamic In-Context Learning from Nearest Neighbors for Bundle Generation—0
Brainstorming Brings Power to Large Language Models of Knowledge Reasoning—0
Judgment of Thoughts: Courtroom of the Binary Logical Reasoning in Large Language Models—0
Do Large Language Models Understand Logic or Just Mimick Context?—0
A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models—0
KARGEN: Knowledge-enhanced Automated Radiology Report Generation Using Large Language Models—0
Knowledge Authoring for Rules and Actions—0
Knowledge-based Reasoning and Learning under Partial Observability in Ad Hoc Teamwork—0
LaMOuR: Leveraging Language Models for Out-of-Distribution Recovery in Reinforcement Learning—0
Do Large Language Models Truly Grasp Mathematics? An Empirical Exploration From Cognitive Psychology—0
Do Large Language Models Mirror Cognitive Language Processing?—0
Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought—0
Does Entity Abstraction Help Generative Transformers Reason?—0
Boosting Deductive Reasoning with Step Signals In RLHF—0
DMWM: Dual-Mind World Model with Long-Term Imagination—0
A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning—0
Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning—0
Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution—0
Is writing style predictive of scientific fraud?—0
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning—0
Bi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining—0
A Densely Connected Criss-Cross Attention Network for Document-level Relation Extraction—0
Discrete JEPA: Learning Discrete Token Representations without Reconstruction—0
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models—0
Is ChatGPT a Good Personality Recognizer? A Preliminary Study—0
Discourse-Aware Graph Networks for Textual Logical Reasoning—0
Dialogue-based Explanations for Logical Reasoning using Structured Argumentation—0
Beyond LLMs: Advancing the Landscape of Complex Reasoning—0
Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI—0
Detection-based Intermediate Supervision for Visual Question Answering—0
Beware of Words: Evaluating the Lexical Diversity of Conversational LLMs using ChatGPT as Case Study—0
Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs—0
Is writing style predictive of scientific fraud?—0
JAMES: Normalizing Job Titles with Multi-Aspect Graph Embeddings and Reasoning—0
DetectGPT-SC: Improving Detection of Text Generated by Large Language Models through Self-Consistency with Masked Predictions—0
Bayes Meets Entailment and Prediction: Commonsense Reasoning with Non-monotonicity, Paraconsistency and Predictive Accuracy—0
Deliberate Reasoning for LLMs as Structure-aware Planning with Accurate World Model—0
De-fine: Decomposing and Refining Visual Programs with Auto-Feedback—0
Interleaved Reasoning for Large Language Models via Reinforcement Learning—0
Towards Geometry Problem Solving in the Large Model Era: A Survey—0
Show:102550
← PrevPage 6 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified