SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 401–450 of 747 papers

TitleStatusHype
Why should we ever automate moral decision making?—0
Analyzing Large language models chatbots: An experimental approach using a probability test—0
Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games—0
Unveiling Scoring Processes: Dissecting the Differences between LLMs and Human Graders in Automatic Scoring—0
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts—0
Categorical Syllogisms Revisited: A Review of the Logical Reasoning Abilities of LLMs for Analyzing Categorical Syllogism—0
LLM-ARC: Enhancing LLMs with an Automated Reasoning Critic—0
Large Language Models Are Cross-Lingual Knowledge-Free ReasonersCode0
Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language ModelsCode0
Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy—0
Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference—0
Pathformer: Recursive Path Query Encoding for Complex Logical Query Answering—0
The neural correlates of logical-mathematical symbol systems processing resemble that of spatial cognition more than natural language processing—0
Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language ModelsCode0
Program Synthesis Benchmark for Visual Programming in XLogoOnline Environment—0
Scaling Synthetic Logical Reasoning Datasets with Context-Sensitive Declarative GrammarsCode0
City-LEO: Toward Transparent City Management Using LLM with End-to-End Optimization—0
Evaluating ChatGPT-4 Vision on Brazil's National Undergraduate Computer Science ExamCode0
Dual Thinking and Logical Processing -- Are Multi-modal Large Language Models Closing the Gap with Human Vision ?Code0
Large Language Models are Limited in Out-of-Context Knowledge ReasoningCode0
Improving Multi-hop Logical Reasoning in Knowledge Graphs with Context-Aware Query Representation LearningCode0
LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct LanguagesCode0
LGR2: Language Guided Reward Relabeling for Accelerating Hierarchical Reinforcement Learning—0
Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negation—0
On the Hardness of Probabilistic Neurosymbolic LearningCode0
How Truncating Weights Improves Reasoning in Language Models—0
Bi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining—0
Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks—0
Disentangling Logic: The Role of Context in Large Language Model Reasoning CapabilitiesCode0
A Synergistic Approach In Network Intrusion Detection By Neurosymbolic AI—0
Logical Reasoning with Relation Network for Inductive Knowledge Graph Completion—0
Brainstorming Brings Power to Large Language Models of Knowledge Reasoning—0
A Closer Look at Logical Reasoning with LLMs: The Choice of Tool MattersCode0
PathReasoner: Modeling Reasoning Path with Equivalent Extension for Logical Question Answering—0
Is a 3D-Tokenized LLM the Key to Reliable Autonomous Driving?—0
RLSF: Reinforcement Learning via Symbolic Feedback—0
Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning—0
LLM+Reasoning+Planning for supporting incomplete user queries in presence of APIs—0
STAR: A Benchmark for Situated Reasoning in Real-World Videos—0
MetaReflection: Learning Instructions for Language Agents using Past Reflections—0
MathDivide: Improved mathematical reasoning by large language models—0
Logical Negation Augmenting and Debiasing for Prompt-based Methods—0
Towards a Theoretical Understanding of the 'Reversal Curse' via Training DynamicsCode0
Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning—0
SuperCLUE-Fin: Graded Fine-Grained Analysis of Chinese LLMs on Diverse Financial Tasks and Applications—0
Cantor: Inspiring Multimodal Chain-of-Thought of MLLM—0
Aligning Knowledge Graphs Provided by Humans and Generated from Neural Networks in Specific TasksCode0
Reinforcement Learning from Multi-role Debates as Feedback for Bias Mitigation in LLMs—0
Reason from Fallacy: Enhancing Large Language Models' Logical Reasoning through Logical Fallacy Understanding—0
CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues—0
Show:102550
← PrevPage 9 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified