Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Code

• github.com/microsoft/guidance
  none★ 21,361
• github.com/guidance-ai/guidance
  none★ 21,360
• github.com/thudm/chatglm2-6b
  pytorch★ 15,640
• github.com/srush/minichain
  pytorch★ 1,233
• github.com/lupantech/chameleon-llm
  none★ 1,140
• github.com/lastmile-ai/aiconfig
  none★ 1,082
• github.com/rlqja1107/torch-LLM4SGG
  pytorch★ 116
• github.com/scofield7419/thor-isa
  pytorch★ 109
• github.com/imnearth/coat
  none★ 100
• github.com/coldmist-lu/erroranalysis_prompt
  none★ 92

Abstract

We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking. For instance, prompting a 540B-parameter language model with just eight chain of thought exemplars achieves state of the art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.

Tasks

Benchmark Results

Reproductions