Logical Reasoning

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 101–125 of 747 papers

Title	Date	Tasks	Status	Hype
Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation	Feb 10, 2025	Logical Reasoning	CodeCode Available	1
LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles	Aug 21, 2023	Logical Reasoning	CodeCode Available	1
Explicit Planning Helps Language Models in Logical Reasoning	Mar 28, 2023	Logical ReasoningMultiple-choice	CodeCode Available	1
Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language Models	Feb 16, 2025	Language ModelingLanguage Modelling	CodeCode Available	1
Improving Large Language Models in Event Relation Logical Prediction	Oct 13, 2023	counterfactualEvent Relation Extraction	CodeCode Available	1
COLLIE: Systematic Construction of Constrained Text Generation Tasks	Jul 17, 2023	Logical ReasoningSentence	CodeCode Available	1
Advancing Abductive Reasoning in Knowledge Graphs through Complex Logical Hypothesis Generation	Dec 25, 2023	Knowledge GraphsLogical Reasoning	CodeCode Available	1
Logical Message Passing Networks with One-hop Inference on Atomic Formulas	Jan 21, 2023	Complex Query AnsweringGraph Representation Learning	CodeCode Available	1
Logic and the 2-Simplicial Transformer	May 1, 2020	Deep Reinforcement LearningInductive Bias	CodeCode Available	1
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text	May 8, 2021	Data AugmentationLogical Reasoning	CodeCode Available	1
ExAIS: Executable AI Semantics	Feb 20, 2022	Logical Reasoningvalid	CodeCode Available	1
Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical Reasoning	May 21, 2023	Abstract Meaning RepresentationContrastive Learning	CodeCode Available	1
Certified Deductive Reasoning with Language Models	Jun 6, 2023	Logical Reasoningvalid	CodeCode Available	1
Chain of Images for Intuitively Reasoning	Nov 9, 2023	Common Sense ReasoningLanguage Modelling	CodeCode Available	1
Fact-driven Logical Reasoning for Machine Reading Comprehension	May 21, 2021	Logical ReasoningMachine Reading Comprehension	CodeCode Available	1
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming	May 5, 2023	Logical Reasoning	CodeCode Available	1
ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers	Oct 13, 2021	Logical ReasoningQuestion Answering	CodeCode Available	1
AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension	Mar 16, 2022	Logical ReasoningMachine Reading Comprehension	CodeCode Available	1
Automatic Curriculum Expert Iteration for Reliable LLM Reasoning	Oct 10, 2024	HallucinationLogical Reasoning	CodeCode Available	1
Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration	Apr 17, 2025	Geometry Problem SolvingLarge Language Model	CodeCode Available	1
A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices	Mar 9, 2022	Logical Reasoning	CodeCode Available	1
AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models	Feb 24, 2025	Logical ReasoningMultiple-choice	CodeCode Available	1
ElecBench: a Power Dispatch Evaluation Benchmark for Large Language Models	Jul 7, 2024	FairnessGeneral Knowledge	CodeCode Available	1
CHECKWHY: Causal Fact Verification via Argument Structure	Aug 20, 2024	Fact VerificationLogical Reasoning	CodeCode Available	1
AbductionRules: Training Transformers to Explain Unexpected Inputs	Mar 23, 2022	Common Sense ReasoningLogical Reasoning	CodeCode Available	1

Show:10 25 50

← PrevPage 5 of 30Next →

All datasets LingOly BIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTree Winograd Automatic BIG-bench (Logical Fallacy Detection)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Claude Opus	Delta_NoContext	28.8	—	Unverified
2	GPT-4o	Delta_NoContext	25.1	—	Unverified
3	Gemini 1.5 Pro	Delta_NoContext	23.4	—	Unverified
4	GPT-4	Delta_NoContext	21.5	—	Unverified
5	Command R+	Delta_NoContext	11.6	—	Unverified
6	GPT-3.5	Delta_NoContext	11.2	—	Unverified
7	Mixtral 8x7B	Delta_NoContext	6.4	—	Unverified
8	Llama 3 8B	Delta_NoContext	4.9	—	Unverified
9	Llama 3 70B	Delta_NoContext	2.9	—	Unverified
10	Gemma 7B	Delta_NoContext	2.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, Direct)	Accuracy	64.8	—	Unverified
2	PaLM 2 (few-shot, k=3, CoT)	Accuracy	57.2	—	Unverified
3	OPT 66B (few-shot, k=3)	Accuracy	54	—	Unverified
4	PaLM 540B (few-shot, k=3)	Accuracy	53.6	—	Unverified
5	GPT-NeoX 20B (few-shot, k=3)	Accuracy	52.8	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	52.8	—	Unverified
7	Chinchilla-70B (few-shot, k=5)	Accuracy	52.1	—	Unverified
8	Bloomberg GPT 50B (few-shot, k=3)	Accuracy	50.8	—	Unverified
9	Gopher-280B (few-shot, k=5)	Accuracy	50.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	84.9	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	65.8	—	Unverified
3	Chinchilla-70B (few-shot, k=5)	Accuracy	48.7	—	Unverified
4	PaLM 540B (few-shot, k=3)	Accuracy	44.5	—	Unverified
5	Gopher-280B (few-shot, k=5)	Accuracy	40.6	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	40.41	—	Unverified
7	Bloomberg GPT (few-shot, k=3)	Accuracy	37.67	—	Unverified
8	GPT-NeoX (few-shot, k=3)	Accuracy	33.56	—	Unverified
9	OPT 66B (few-shot, k=3)	Accuracy	28.08	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	91.2	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	61.2	—	Unverified
3	Chinchilla-70B (few-shot, k=5)	Accuracy	59.7	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	49.2	—	Unverified
5	PaLM 540B (few-shot, k=3)	Accuracy	38	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	36.8	—	Unverified
7	Bloomberg GPT (few-shot, k=3)	Accuracy	34.8	—	Unverified
8	OPT 66B (few-shot, k=3)	Accuracy	31.2	—	Unverified
9	GPT-NeoX (few-shot, k=3)	Accuracy	26	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	100	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	96.4	—	Unverified
3	PaLM 540B (few-shot, k=3)	Accuracy	39.6	—	Unverified
4	BLOOM 176B (few-shot, k=3)	Accuracy	36.8	—	Unverified
5	Chinchilla-70B (few-shot, k=5)	Accuracy	32	—	Unverified
6	Bloomberg GPT (few-shot, k=3)	Accuracy	29.2	—	Unverified
7	OPT 66B (few-shot, k=3)	Accuracy	23.6	—	Unverified
8	GPT-NeoX (few-shot, k=3)	Accuracy	21.2	—	Unverified
9	Gopher-280B (few-shot, k=5)	Accuracy	19	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Chinchilla-70B (few-shot, k=5)	Accuracy	44	—	Unverified
2	PaLM-540B (few-shot, k=5)	Accuracy	42.4	—	Unverified
3	PaLM-62B (few-shot, k=5)	Accuracy	36.5	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	35.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM-540B (few-shot, k=5)	Accuracy	73.9	—	Unverified
2	Chinchilla-70B (few-shot, k=5)	Accuracy	68.3	—	Unverified
3	PaLM-62B (few-shot, k=5)	Accuracy	65.4	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	61	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Human benchmark	Accuracy	83.7	—	Unverified
2	RuGPT-3 Large	Accuracy	40.7	—	Unverified
3	RuGPT-3 Medium	Accuracy	38	—	Unverified
4	RuGPT-3 Small	Accuracy	34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Human benchmark	Accuracy	87	—	Unverified
2	RuGPT-3 Small	Accuracy	57.9	—	Unverified
3	RuGPT-3 Medium	Accuracy	57.2	—	Unverified
4	RuGPT-3 Large	Accuracy	55.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Chinchilla-70B (few-shot, k=5)	Accuracy	72.1	—	Unverified
2	Gopher-280B (few-shot, k=5)	Accuracy	58.9	—	Unverified