SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 251–300 of 747 papers

TitleStatusHype
Story3D-Agent: Exploring 3D Storytelling Visualization with Large Language Models—0
SarcasmBench: Towards Evaluating Large Language Models on Sarcasm Understanding—0
CHECKWHY: Causal Fact Verification via Argument StructureCode1
A Mechanistic Interpretation of Syllogistic Reasoning in Auto-Regressive Language Models—0
Large Language Models Might Not Care What You Are Saying: Prompt Format Beats Descriptions—0
LLMI3D: Empowering LLM with 3D Perception from a Single 2D Image—0
Can Large Language Models Reason? A Characterization via 3-SAT—0
P3: A Policy-Driven, Pace-Adaptive, and Diversity-Promoted Framework for data pruning in LLM Training—0
Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO DatasetCode0
Automated Theorem Provers Help Improve Large Language Model Reasoning—0
Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation—0
Leveraging Large Language Models with Chain-of-Thought and Prompt Engineering for Traffic Crash Severity Analysis and Inference—0
Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation—0
CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledge—0
Take A Step Back: Rethinking the Two Stages in Visual Reasoning—0
Logic Distillation: Learning from Code Function by Function for Planning and Decision-making—0
An Empirical Study of Retrieval Augmented Generation with Chain-of-Thought—0
Step-by-Step Reasoning to Solve Grid Puzzles: Where do LLMs Falter?Code0
An Explainable Fast Deep Neural Network for Emotion Recognition—0
NeedleBench: Can LLMs Do Retrieval and Reasoning in Information-Dense Context?Code9
Leveraging large language models for nano synthesis mechanism explanation: solid foundations or mere conjectures?Code0
Hypergraph Multi-modal Large Language Model: Exploiting EEG and Eye-tracking Modalities to Evaluate Heterogeneous Responses for Video UnderstandingCode1
Analyzing Large language models chatbots: An experimental approach using a probability test—0
Why should we ever automate moral decision making?—0
R^2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical ReasoningCode1
ElecBench: a Power Dispatch Evaluation Benchmark for Large Language ModelsCode1
LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual ContextsCode1
Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games—0
Unveiling Scoring Processes: Dissecting the Differences between LLMs and Human Graders in Automatic Scoring—0
PUZZLES: A Benchmark for Neural Algorithmic ReasoningCode1
Scaling Synthetic Data Creation with 1,000,000,000 PersonasCode11
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts—0
Categorical Syllogisms Revisited: A Review of the Logical Reasoning Abilities of LLMs for Analyzing Categorical Syllogism—0
LLM-ARC: Enhancing LLMs with an Automated Reasoning Critic—0
Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language ModelsCode0
Large Language Models Are Cross-Lingual Knowledge-Free ReasonersCode0
Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy—0
Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference—0
Pathformer: Recursive Path Query Encoding for Complex Logical Query Answering—0
The neural correlates of logical-mathematical symbol systems processing resemble that of spatial cognition more than natural language processing—0
Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language ModelsCode0
VideoVista: A Versatile Benchmark for Video Understanding and ReasoningCode1
Program Synthesis Benchmark for Visual Programming in XLogoOnline Environment—0
Scaling Synthetic Logical Reasoning Datasets with Context-Sensitive Declarative GrammarsCode0
City-LEO: Toward Transparent City Management Using LLM with End-to-End Optimization—0
Ontology Embedding: A Survey of Methods, Applications and ResourcesCode2
A Peek into Token Bias: Large Language Models Are Not Yet Genuine ReasonersCode1
Evaluating ChatGPT-4 Vision on Brazil's National Undergraduate Computer Science ExamCode0
Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMsCode2
Large Language Models are Limited in Out-of-Context Knowledge ReasoningCode0
Show:102550
← PrevPage 6 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified