SOTAVerified

Logical Reasoning

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 301–350 of 747 papers

TitleStatusHype
Hint-before-Solving Prompting: Guiding LLMs to Effectively Utilize Encoded KnowledgeCode0
HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation PredictionCode0
LogicPro: Improving Complex Logical Reasoning via Program-Guided LearningCode0
Context Transformer with Stacked Pointer Networks for Conversational Question Answering over Knowledge GraphsCode0
Exploring Self-supervised Logic-enhanced Training for Large Language ModelsCode0
Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language ModelsCode0
Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI ModelsCode0
Logical Tasks for Measuring Extrapolation and Rule ComprehensionCode0
Atomic Inference for NLI with Generated Facts as AtomsCode0
Logical Reasoning over Natural Language as Knowledge Representation: A SurveyCode0
A Neural Divide-and-Conquer Reasoning Framework for Image Retrieval from Linguistically Complex TextCode0
Conditional Logical Message Passing Transformer for Complex Query AnsweringCode0
LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct LanguagesCode0
Semantic RL with Action Grammars: Data-Efficient Learning of Hierarchical Task AbstractionsCode0
GOTaxon: Representing the evolution of biological functions in the Gene OntologyCode0
Large Language Models are Limited in Out-of-Context Knowledge ReasoningCode0
Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic ApproachCode0
Generating Programmatic Referring Expressions via Program SynthesisCode0
Leveraging large language models for nano synthesis mechanism explanation: solid foundations or mere conjectures?Code0
Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language ModelsCode0
Generating by Understanding: Neural Visual Generation with Logical Symbol GroundingsCode0
GammaE: Gamma Embeddings for Logical Queries on Knowledge GraphsCode0
Learning the meanings of function words from grounded language using a visual question answering modelCode0
Learning Symmetric Rules with SATNetCode0
Learning for Long-Horizon Planning via Neuro-Symbolic Abductive ImitationCode0
A Structured Unplugged Approach for Foundational AI Literacy in Primary EducationCode0
From Babbling to Fluency: Evaluating the Evolution of Language Models in Terms of Human Language AcquisitionCode0
Investigating the Robustness of Natural Language Generation from Logical Forms via Counterfactual SamplesCode0
Language Model Guided Interpretable Video Action ReasoningCode0
Climate Finance BenchCode0
Language models show human-like content effects on reasoning tasksCode0
Adaptive Rectification Sampling for Test-Time Compute ScalingCode0
Large Language Models Are Cross-Lingual Knowledge-Free ReasonersCode0
One-Step Abductive Multi-Target Learning with Diverse Noisy Samples and Its Application to Tumour Segmentation for Breast CancerCode0
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts—0
City-LEO: Toward Transparent City Management Using LLM with End-to-End Optimization—0
First Experiments with a Flexible Infrastructure for Normative Reasoning—0
Few-shot Visual Reasoning with Meta-analogical Contrastive Learning—0
FEVO: Financial Knowledge Expansion and Reasoning Evolution for Large Language Models—0
Federated In-Context LLM Agent Learning—0
ChatGPT is a Remarkable Tool -- For Experts—0
Assessing the Reasoning Abilities of ChatGPT in the Context of Claim Verification—0
Federated Neural Graph Databases—0
FaiRR: Faithful and Robust Deductive Reasoning over Natural Language—0
ChatABL: Abductive Learning via Natural Language Interaction with ChatGPT—0
Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogism—0
Extending Automated Deduction for Commonsense Reasoning—0
Assessing SATNet's Ability to Solve the Symbol Grounding Problem—0
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos—0
Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning—0
Show:102550
← PrevPage 7 of 15Next →
All datasetsLingOlyBIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTreeWinograd AutomaticBIG-bench (Logical Fallacy Detection)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Claude OpusDelta_NoContext28.8—Unverified
2GPT-4oDelta_NoContext25.1—Unverified
3Gemini 1.5 ProDelta_NoContext23.4—Unverified
4GPT-4Delta_NoContext21.5—Unverified
5Command R+Delta_NoContext11.6—Unverified
6GPT-3.5Delta_NoContext11.2—Unverified
7Mixtral 8x7BDelta_NoContext6.4—Unverified
8Llama 3 8BDelta_NoContext4.9—Unverified
9Llama 3 70BDelta_NoContext2.9—Unverified
10Gemma 7BDelta_NoContext2.2—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, Direct)Accuracy64.8—Unverified
2PaLM 2 (few-shot, k=3, CoT)Accuracy57.2—Unverified
3OPT 66B (few-shot, k=3)Accuracy54—Unverified
4PaLM 540B (few-shot, k=3)Accuracy53.6—Unverified
5GPT-NeoX 20B (few-shot, k=3)Accuracy52.8—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy52.8—Unverified
7Chinchilla-70B (few-shot, k=5)Accuracy52.1—Unverified
8Bloomberg GPT 50B (few-shot, k=3)Accuracy50.8—Unverified
9Gopher-280B (few-shot, k=5)Accuracy50.7—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy84.9—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy65.8—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy48.7—Unverified
4PaLM 540B (few-shot, k=3)Accuracy44.5—Unverified
5Gopher-280B (few-shot, k=5)Accuracy40.6—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy40.41—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy37.67—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy33.56—Unverified
9OPT 66B (few-shot, k=3)Accuracy28.08—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy91.2—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy61.2—Unverified
3Chinchilla-70B (few-shot, k=5)Accuracy59.7—Unverified
4Gopher-280B (few-shot, k=5)Accuracy49.2—Unverified
5PaLM 540B (few-shot, k=3)Accuracy38—Unverified
6BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
7Bloomberg GPT (few-shot, k=3)Accuracy34.8—Unverified
8OPT 66B (few-shot, k=3)Accuracy31.2—Unverified
9GPT-NeoX (few-shot, k=3)Accuracy26—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM 2 (few-shot, k=3, CoT)Accuracy100—Unverified
2PaLM 2 (few-shot, k=3, Direct)Accuracy96.4—Unverified
3PaLM 540B (few-shot, k=3)Accuracy39.6—Unverified
4BLOOM 176B (few-shot, k=3)Accuracy36.8—Unverified
5Chinchilla-70B (few-shot, k=5)Accuracy32—Unverified
6Bloomberg GPT (few-shot, k=3)Accuracy29.2—Unverified
7OPT 66B (few-shot, k=3)Accuracy23.6—Unverified
8GPT-NeoX (few-shot, k=3)Accuracy21.2—Unverified
9Gopher-280B (few-shot, k=5)Accuracy19—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy44—Unverified
2PaLM-540B (few-shot, k=5)Accuracy42.4—Unverified
3PaLM-62B (few-shot, k=5)Accuracy36.5—Unverified
4Gopher-280B (few-shot, k=5)Accuracy35.1—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLM-540B (few-shot, k=5)Accuracy73.9—Unverified
2Chinchilla-70B (few-shot, k=5)Accuracy68.3—Unverified
3PaLM-62B (few-shot, k=5)Accuracy65.4—Unverified
4Gopher-280B (few-shot, k=5)Accuracy61—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy 83.7—Unverified
2RuGPT-3 LargeAccuracy 40.7—Unverified
3RuGPT-3 MediumAccuracy 38—Unverified
4RuGPT-3 SmallAccuracy 34—Unverified
#ModelMetricClaimedVerifiedStatus
1Human benchmarkAccuracy87—Unverified
2RuGPT-3 SmallAccuracy57.9—Unverified
3RuGPT-3 MediumAccuracy57.2—Unverified
4RuGPT-3 LargeAccuracy55.5—Unverified
#ModelMetricClaimedVerifiedStatus
1Chinchilla-70B (few-shot, k=5)Accuracy72.1—Unverified
2Gopher-280B (few-shot, k=5)Accuracy58.9—Unverified