Logical Reasoning

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 126–150 of 747 papers

Title	Date	Tasks	Status	Hype
Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting	May 11, 2023	AllArithmetic Reasoning	CodeCode Available	1
Wasserstein-Fisher-Rao Embedding: Logical Query Embeddings with Local Comparison and Global Transport	May 6, 2023	Knowledge GraphsLogical Reasoning	CodeCode Available	1
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming	May 5, 2023	Logical Reasoning	CodeCode Available	1
Complex Logical Reasoning over Knowledge Graphs using Large Language Models	May 2, 2023	Knowledge GraphsLogical Reasoning	CodeCode Available	1
Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4	Apr 7, 2023	Logical ReasoningNatural Language Inference	CodeCode Available	1
Explicit Planning Helps Language Models in Logical Reasoning	Mar 28, 2023	Logical ReasoningMultiple-choice	CodeCode Available	1
Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases	Mar 26, 2023	Link PredictionLogical Reasoning	CodeCode Available	1
Natural Language Reasoning, A Survey	Mar 26, 2023	Logical ReasoningMathematical Reasoning	CodeCode Available	1
Domain Specific Question Answering Over Knowledge Graphs Using Logical Programming and Large Language Models	Mar 3, 2023	Knowledge GraphsLogical Reasoning	CodeCode Available	1
ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models	Feb 14, 2023	Decision MakingLesion Segmentation	CodeCode Available	1
A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity	Feb 8, 2023	Code GenerationHallucination	CodeCode Available	1
Logical Message Passing Networks with One-hop Inference on Atomic Formulas	Jan 21, 2023	Complex Query AnsweringGraph Representation Learning	CodeCode Available	1
Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-shot Logical Reasoning over Text	Jan 8, 2023	Contrastive LearningLogical Reasoning	CodeCode Available	1
Large Language Models are Better Reasoners with Self-Verification	Dec 19, 2022	Arithmetic ReasoningCommon Sense Reasoning	CodeCode Available	1
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning	Dec 15, 2022	Instruction FollowingLanguage Modeling	CodeCode Available	1
Counterfactual reasoning: Do language models need world knowledge for causal understanding?	Dec 6, 2022	counterfactualCounterfactual Reasoning	CodeCode Available	1
UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression	Dec 6, 2022	Geometry Problem SolvingLogical Reasoning	CodeCode Available	1
NQE: N-ary Query Embedding for Complex Query Answering over Hyper-Relational Knowledge Graphs	Nov 24, 2022	Complex Query AnsweringKnowledge Graphs	CodeCode Available	1
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning	Sep 29, 2022	Logical ReasoningMath	CodeCode Available	1
Neural Methods for Logical Reasoning Over Knowledge Graphs	Sep 28, 2022	BenchmarkingKnowledge Graphs	CodeCode Available	1
FOLIO: Natural Language Reasoning with First-Order Logic	Sep 2, 2022	Language ModelingLanguage Modelling	CodeCode Available	1
Semantic Probabilistic Layers for Neuro-Symbolic Learning	Jun 1, 2022	Hierarchical Multi-label ClassificationLogical Reasoning	CodeCode Available	1
TFLEX: Temporal Feature-Logic Embedding Framework for Complex Reasoning over Temporal Knowledge Graph	May 28, 2022	Knowledge GraphsLogical Reasoning	CodeCode Available	1
On the Paradox of Learning to Reason from Data	May 23, 2022	Logical Reasoning	CodeCode Available	1
Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoning	May 2, 2022	Logical ReasoningMachine Reading Comprehension	CodeCode Available	1

Show:10 25 50

← PrevPage 6 of 30Next →

All datasets LingOly BIG-bench (Formal Fallacies Syllogisms Negation)BIG-bench (Penguins In A Table)BIG-bench (Reasoning About Colored Objects)BIG-bench (Temporal Sequences)BIG-bench (Logic Grid Puzzle)BIG-bench (StrategyQA)RuWorldTree Winograd Automatic BIG-bench (Logical Fallacy Detection)

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Claude Opus	Delta_NoContext	28.8	—	Unverified
2	GPT-4o	Delta_NoContext	25.1	—	Unverified
3	Gemini 1.5 Pro	Delta_NoContext	23.4	—	Unverified
4	GPT-4	Delta_NoContext	21.5	—	Unverified
5	Command R+	Delta_NoContext	11.6	—	Unverified
6	GPT-3.5	Delta_NoContext	11.2	—	Unverified
7	Mixtral 8x7B	Delta_NoContext	6.4	—	Unverified
8	Llama 3 8B	Delta_NoContext	4.9	—	Unverified
9	Llama 3 70B	Delta_NoContext	2.9	—	Unverified
10	Gemma 7B	Delta_NoContext	2.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, Direct)	Accuracy	64.8	—	Unverified
2	PaLM 2 (few-shot, k=3, CoT)	Accuracy	57.2	—	Unverified
3	OPT 66B (few-shot, k=3)	Accuracy	54	—	Unverified
4	PaLM 540B (few-shot, k=3)	Accuracy	53.6	—	Unverified
5	GPT-NeoX 20B (few-shot, k=3)	Accuracy	52.8	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	52.8	—	Unverified
7	Chinchilla-70B (few-shot, k=5)	Accuracy	52.1	—	Unverified
8	Bloomberg GPT 50B (few-shot, k=3)	Accuracy	50.8	—	Unverified
9	Gopher-280B (few-shot, k=5)	Accuracy	50.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	84.9	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	65.8	—	Unverified
3	Chinchilla-70B (few-shot, k=5)	Accuracy	48.7	—	Unverified
4	PaLM 540B (few-shot, k=3)	Accuracy	44.5	—	Unverified
5	Gopher-280B (few-shot, k=5)	Accuracy	40.6	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	40.41	—	Unverified
7	Bloomberg GPT (few-shot, k=3)	Accuracy	37.67	—	Unverified
8	GPT-NeoX (few-shot, k=3)	Accuracy	33.56	—	Unverified
9	OPT 66B (few-shot, k=3)	Accuracy	28.08	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	91.2	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	61.2	—	Unverified
3	Chinchilla-70B (few-shot, k=5)	Accuracy	59.7	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	49.2	—	Unverified
5	PaLM 540B (few-shot, k=3)	Accuracy	38	—	Unverified
6	BLOOM 176B (few-shot, k=3)	Accuracy	36.8	—	Unverified
7	Bloomberg GPT (few-shot, k=3)	Accuracy	34.8	—	Unverified
8	OPT 66B (few-shot, k=3)	Accuracy	31.2	—	Unverified
9	GPT-NeoX (few-shot, k=3)	Accuracy	26	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM 2 (few-shot, k=3, CoT)	Accuracy	100	—	Unverified
2	PaLM 2 (few-shot, k=3, Direct)	Accuracy	96.4	—	Unverified
3	PaLM 540B (few-shot, k=3)	Accuracy	39.6	—	Unverified
4	BLOOM 176B (few-shot, k=3)	Accuracy	36.8	—	Unverified
5	Chinchilla-70B (few-shot, k=5)	Accuracy	32	—	Unverified
6	Bloomberg GPT (few-shot, k=3)	Accuracy	29.2	—	Unverified
7	OPT 66B (few-shot, k=3)	Accuracy	23.6	—	Unverified
8	GPT-NeoX (few-shot, k=3)	Accuracy	21.2	—	Unverified
9	Gopher-280B (few-shot, k=5)	Accuracy	19	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Chinchilla-70B (few-shot, k=5)	Accuracy	44	—	Unverified
2	PaLM-540B (few-shot, k=5)	Accuracy	42.4	—	Unverified
3	PaLM-62B (few-shot, k=5)	Accuracy	36.5	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	35.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLM-540B (few-shot, k=5)	Accuracy	73.9	—	Unverified
2	Chinchilla-70B (few-shot, k=5)	Accuracy	68.3	—	Unverified
3	PaLM-62B (few-shot, k=5)	Accuracy	65.4	—	Unverified
4	Gopher-280B (few-shot, k=5)	Accuracy	61	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Human benchmark	Accuracy	83.7	—	Unverified
2	RuGPT-3 Large	Accuracy	40.7	—	Unverified
3	RuGPT-3 Medium	Accuracy	38	—	Unverified
4	RuGPT-3 Small	Accuracy	34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Human benchmark	Accuracy	87	—	Unverified
2	RuGPT-3 Small	Accuracy	57.9	—	Unverified
3	RuGPT-3 Medium	Accuracy	57.2	—	Unverified
4	RuGPT-3 Large	Accuracy	55.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Chinchilla-70B (few-shot, k=5)	Accuracy	72.1	—	Unverified
2	Gopher-280B (few-shot, k=5)	Accuracy	58.9	—	Unverified