SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 151–200 of 747 papers

TitleStatusHype
KnowRA: Knowledge Retrieval Augmented Method for Document-level Relation Extraction with Comprehensive Reasoning Abilities—0
OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and ReasoningCode4
SecBench: A Comprehensive Multi-Dimensional Benchmarking Dataset for LLMs in Cybersecurity—0
A Survey on Large Language Model Acceleration based on KV Cache ManagementCode3
Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-AugmentationCode1
StructTest: Benchmarking LLMs' Reasoning through Compositional Structured Outputs—0
Aristotle: Mastering Logical Reasoning with A Logic-Complete Decompose-Search-Resolve FrameworkCode0
Formal Language Knowledge Corpus for Retrieval Augmented Generation—0
Logical Consistency of Large Language Models in Fact-checking—0
SPHERE: A Hierarchical Evaluation on Spatial Perception and Reasoning for Vision-Language ModelsCode0
WiseAD: Knowledge Augmented End-to-End Autonomous Driving with Vision-Language ModelCode1
Reasoning-Aware Query-Focused Summarization over Multi-Table Data—0
RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World ScenariosCode1
Federated In-Context LLM Agent Learning—0
Algorithmic Phase Transitions in Language Models: A Mechanistic Case Study of Arithmetic—0
FlashRNN: Optimizing Traditional RNNs on Modern HardwareCode2
Training Large Language Models to Reason in a Continuous Latent SpaceCode5
Can OpenAI o1 outperform humans in higher-order cognitive thinking?—0
Who Speaks Next? Multi-party AI Discussion Leveraging the Systematics of Turn-taking in Murder Mystery GamesCode0
Guidance is All You Need: Temperature-Guided Reasoning in Large Language Models—0
MTMT: Consolidating Multiple Thinking Modes to Form a Thought Tree for Strengthening LLM—0
ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable CompressionCode1
Reverse Thinking Makes LLMs Stronger Reasoners—0
SentiXRL: An advanced large language Model Framework for Multilingual Fine-Grained Emotion Classification in Complex Text Environment—0
Learning for Long-Horizon Planning via Neuro-Symbolic Abductive ImitationCode0
Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs—0
Meaningless is better: hashing bias-inducing words in LLM prompts improves performance in logical reasoning and statistical learning—0
HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluator—0
Object-centric proto-symbolic behavioural reasoning from pixelsCode0
Interactive Visual Assessment for Text-to-Image Generation Models—0
XAgents: A Framework for Interpretable Rule-Based Multi-Agents Cooperation—0
Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic CorpusCode2
Testing Uncertainty of Large Language Models for Physics Knowledge and Reasoning—0
Large Language Models (LLMs) as Traffic Control Systems at Urban Intersections: A New Paradigm—0
Evaluating Creativity and Deception in Large Language Models: A Simulation Framework for Multi-Agent BalderdashCode0
LLaVA-CoT: Let Vision Language Models Reason Step-by-StepCode7
Building Trustworthy AI: Transparent AI Systems via Large Language Models, Ontologies, and Logical Reasoning (TranspNet)—0
Symbolic-AI-Fusion Deep Learning (SAIF-DL): Encoding Knowledge into Training with Answer Set Programming Loss Penalties by a Novel Loss Function Approach—0
Knowledge Authoring with Factual English, Rules, and Actions—0
OpenAI-o1 AB Testing: Does the o1 model really do good reasoning in math problem solving?—0
How Transformers Solve Propositional Logic Problems: A Mechanistic Analysis—0
Formal Logic-guided Robust Federated Learning against Poisoning Attacks—0
The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant UnitsCode1
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by TencentCode5
LogiCity: Advancing Neuro-Symbolic AI with Abstract Urban SimulationCode1
On Memorization of Large Language Models in Logical Reasoning—0
Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic ApproachCode0
Neuro-symbolic Learning Yielding Logical ConstraintsCode1
Combining Domain-Specific Models and LLMs for Automated Disease Phenotyping from Survey Data—0
Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs—0
Show:102550
← PrevPage 4 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified