Logical Reasoning

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 276–300 of 747 papers

Title	Date	Tasks	Status	Score
Logical Tasks for Measuring Extrapolation and Rule Comprehension	Nov 14, 2022	Inductive BiasLogical Reasoning	CodeCode Available	5
Hint-before-Solving Prompting: Guiding LLMs to Effectively Utilize Encoded Knowledge	Feb 22, 2024	Logical Reasoning	CodeCode Available	5
HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction	Oct 10, 2024	Binary ClassificationCitation Prediction	CodeCode Available	5
Large Language Models are Limited in Out-of-Context Knowledge Reasoning	Jun 11, 2024	AttributeLogical Reasoning	CodeCode Available	5
Context Transformer with Stacked Pointer Networks for Conversational Question Answering over Knowledge Graphs	Mar 13, 2021	Conversational Question AnsweringKnowledge Graphs	CodeCode Available	5
Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach	Oct 29, 2024	Logical Reasoning	CodeCode Available	5
Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language Models	Jun 18, 2024	Logical Reasoning	CodeCode Available	5
LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages	Jun 10, 2024	Logical Reasoning	CodeCode Available	5
Learning Symmetric Rules with SATNet	Jun 28, 2022	Logical ReasoningRubik's Cube	CodeCode Available	5
Learning the meanings of function words from grounded language using a visual question answering model	Aug 16, 2023	Logical ReasoningQuestion Answering	CodeCode Available	5
Learning for Long-Horizon Planning via Neuro-Symbolic Abductive Imitation	Nov 27, 2024	Imitation LearningLogical Reasoning	CodeCode Available	5
A Neural Divide-and-Conquer Reasoning Framework for Image Retrieval from Linguistically Complex Text	May 3, 2023	Image RetrievalLogical Reasoning	CodeCode Available	5
Conditional Logical Message Passing Transformer for Complex Query Answering	Feb 20, 2024	Complex Query AnsweringKnowledge Graphs	CodeCode Available	5
Semantic RL with Action Grammars: Data-Efficient Learning of Hierarchical Task Abstractions	Jul 29, 2019	Hierarchical Reinforcement LearningLogical Reasoning	CodeCode Available	5
GOTaxon: Representing the evolution of biological functions in the Gene Ontology	Feb 16, 2018	Logical Reasoning	CodeCode Available	5
Language models show human-like content effects on reasoning tasks	Jul 14, 2022	Language ModellingLogical Reasoning	CodeCode Available	5
Generating Programmatic Referring Expressions via Program Synthesis	Jan 1, 2020	Enumerative SearchLogical Reasoning	CodeCode Available	5
Language Model Guided Interpretable Video Action Reasoning	Apr 2, 2024	Action RecognitionDecision Making	CodeCode Available	5
Large Language Models Are Cross-Lingual Knowledge-Free Reasoners	Jun 24, 2024	Cross-Lingual TransferLogical Reasoning	CodeCode Available	5
Generating by Understanding: Neural Visual Generation with Logical Symbol Groundings	Oct 26, 2023	DisentanglementLogical Reasoning	CodeCode Available	5
GammaE: Gamma Embeddings for Logical Queries on Knowledge Graphs	Oct 27, 2022	Knowledge GraphsLogical Reasoning	CodeCode Available	5
Atomic Inference for NLI with Generated Facts as Atoms	May 22, 2023	Logical ReasoningNatural Language Inference	CodeCode Available	5
Logical Reasoning over Natural Language as Knowledge Representation: A Survey	Mar 21, 2023	Logical ReasoningSurvey	CodeCode Available	5
JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models	Jan 24, 2025	Logical Reasoning	CodeCode Available	5
Interpret Your Decision: Logical Reasoning Regularization for Generalization in Visual Classification	Oct 6, 2024	ClassificationDomain Generalization	CodeCode Available	5

Show:10 25 50

← PrevPage 12 of 30Next →

All datasets LingOly BIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTree Winograd Automatic BIG-bench (Logical Fallacy Detection)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Claude Opus	Delta_NoContext	28.8	—	Unverified
2	GPT-4o	Delta_NoContext	25.1	—	Unverified
3	Gemini 1.5 Pro	Delta_NoContext	23.4	—	Unverified
4	GPT-4	Delta_NoContext	21.5	—	Unverified
5	Command R+	Delta_NoContext	11.6	—	Unverified
6	GPT-3.5	Delta_NoContext	11.2	—	Unverified
7	Mixtral 8x7B	Delta_NoContext	6.4	—	Unverified
8	Llama 3 8B	Delta_NoContext	4.9	—	Unverified
9	Llama 3 70B	Delta_NoContext	2.9	—	Unverified
10	Gemma 7B	Delta_NoContext	2.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, Direct)	Accuracy	64.8	—	Unverified
2	PaLM 2 (few-shot, k=3, CoT)	Accuracy	57.2	—	Unverified
3	OPT 66B (few-shot, k=3)	Accuracy	54	—	Unverified
4	PaLM 540B (few-shot, k=3)	Accuracy	53.6	—	Unverified
5	GPT-NeoX 20B (few-shot, k=3)	Accuracy	52.8	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	52.8	—	Unverified
7	Chinchilla-70B (few-shot, k=5)	Accuracy	52.1	—	Unverified
8	Bloomberg GPT 50B (few-shot, k=3)	Accuracy	50.8	—	Unverified
9	Gopher-280B (few-shot, k=5)	Accuracy	50.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	84.9	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	65.8	—	Unverified
3	Chinchilla-70B (few-shot, k=5)	Accuracy	48.7	—	Unverified
4	PaLM 540B (few-shot, k=3)	Accuracy	44.5	—	Unverified
5	Gopher-280B (few-shot, k=5)	Accuracy	40.6	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	40.41	—	Unverified
7	Bloomberg GPT (few-shot, k=3)	Accuracy	37.67	—	Unverified
8	GPT-NeoX (few-shot, k=3)	Accuracy	33.56	—	Unverified
9	OPT 66B (few-shot, k=3)	Accuracy	28.08	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	91.2	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	61.2	—	Unverified
3	Chinchilla-70B (few-shot, k=5)	Accuracy	59.7	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	49.2	—	Unverified
5	PaLM 540B (few-shot, k=3)	Accuracy	38	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	36.8	—	Unverified
7	Bloomberg GPT (few-shot, k=3)	Accuracy	34.8	—	Unverified
8	OPT 66B (few-shot, k=3)	Accuracy	31.2	—	Unverified
9	GPT-NeoX (few-shot, k=3)	Accuracy	26	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	100	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	96.4	—	Unverified
3	PaLM 540B (few-shot, k=3)	Accuracy	39.6	—	Unverified
4	BLOOM 176B (few-shot, k=3)	Accuracy	36.8	—	Unverified
5	Chinchilla-70B (few-shot, k=5)	Accuracy	32	—	Unverified
6	Bloomberg GPT (few-shot, k=3)	Accuracy	29.2	—	Unverified
7	OPT 66B (few-shot, k=3)	Accuracy	23.6	—	Unverified
8	GPT-NeoX (few-shot, k=3)	Accuracy	21.2	—	Unverified
9	Gopher-280B (few-shot, k=5)	Accuracy	19	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Chinchilla-70B (few-shot, k=5)	Accuracy	44	—	Unverified
2	PaLM-540B (few-shot, k=5)	Accuracy	42.4	—	Unverified
3	PaLM-62B (few-shot, k=5)	Accuracy	36.5	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	35.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM-540B (few-shot, k=5)	Accuracy	73.9	—	Unverified
2	Chinchilla-70B (few-shot, k=5)	Accuracy	68.3	—	Unverified
3	PaLM-62B (few-shot, k=5)	Accuracy	65.4	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	61	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Human benchmark	Accuracy	83.7	—	Unverified
2	RuGPT-3 Large	Accuracy	40.7	—	Unverified
3	RuGPT-3 Medium	Accuracy	38	—	Unverified
4	RuGPT-3 Small	Accuracy	34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Human benchmark	Accuracy	87	—	Unverified
2	RuGPT-3 Small	Accuracy	57.9	—	Unverified
3	RuGPT-3 Medium	Accuracy	57.2	—	Unverified
4	RuGPT-3 Large	Accuracy	55.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Chinchilla-70B (few-shot, k=5)	Accuracy	72.1	—	Unverified
2	Gopher-280B (few-shot, k=5)	Accuracy	58.9	—	Unverified