Boost LLM Reasoning with Few‑Shot Chain‑of‑Thought Prompting Techniques

Few‑shot Chain‑of‑Thought (CoT) prompting applies the chain‑of‑thought technique in low‑sample scenarios by providing the model with a small number of examples that include explicit reasoning steps, guiding it to generate coherent logical processes and improving accuracy on complex tasks like math, logical inference, and multi‑step decision making.

Standard Few‑shot CoT Prompt Structure

Instruction: Clearly state the task goal (e.g., “Solve the following math problem”).

Few‑shot examples: Each example contains an input, a step‑by‑step reasoning process, and a final answer, with the reasoning clearly broken down.

Problem to solve: The new question the model must answer.

Instruction: Please solve the following math problem, first write the reasoning steps, then give the answer.

Example 1:
Input: Xiao Ming has 5 apples, Xiao Hong has 3 more than Xiao Ming, Xiao Gang has twice as many apples as Xiao Hong. How many apples does Xiao Gang have?
Reasoning:
1. Compute Xiao Hong's apples: 5 + 3 = 8
2. Compute Xiao Gang's apples: 8 × 2 = 16
Answer: 16

Example 2:
Input: A bookshelf contains 12 story books, science books are 4 fewer than story books, comic books are three times the number of science books. How many comic books are there?
Reasoning:
1. Compute science books: 12 - 4 = 8
2. Compute comic books: 8 × 3 = 24
Answer: 24

Problem to solve:
Input: A stationery store has 20 pencils, pens are 5 fewer than pencils, ballpoint pens are four times the number of pens. How many ballpoint pens are there?

Simple few‑shot CoT faces issues such as low example quality, high manual cost, error‑prone reasoning, poor adaptability of fixed examples, and limited model self‑evolution.

Contrastive CoT Prompting

This method mimics human learning from both correct and incorrect examples. It automatically generates “wrong reasoning steps” by extracting key information from a correct chain of thought and shuffling their order, then presents the model with the problem, the correct reasoning, and the erroneous reasoning for contrastive learning.

Complexity‑based Prompting

The idea is to select examples with more and more complex reasoning steps, as they tend to yield better performance than simple examples. Complexity can be measured by the number of reasoning steps, problem length, or formula length. The same principle can be applied to model outputs by voting among the most complex generated reasoning chains.

Active Prompting

Instead of randomly picking fixed examples, this approach selects the questions the model is most uncertain about. The workflow consists of four steps:

Measure model uncertainty by feeding many unlabeled questions and generating multiple answers per question.

Compute disagreement (e.g., multiple distinct answers) as an uncertainty indicator.

Calculate entropy or other confidence metrics; higher entropy indicates greater uncertainty.

Optionally consider variance or self‑assessment confidence, though these can be misleading.

After ranking questions by uncertainty, human annotators provide detailed chain‑of‑thought explanations for the top‑uncertain items, creating custom examples that are placed before test questions. The model then reasons step‑by‑step using these examples and can further improve accuracy via self‑consistency (multiple generations and majority voting).

Memory‑of‑Thought Prompting (MoT)

MoT addresses the need for high‑quality annotated data or costly fine‑tuning by enabling the model to generate and store its own “thought memories” for later reuse. It consists of two phases:

Pre‑thinking phase: The model solves many unlabeled problems, generates multiple reasoning paths, votes for the most consistent answer, filters out low‑confidence paths using answer entropy, and stores high‑confidence problem‑reasoning‑answer triples as external memory.

Recall phase: When faced with a test question, the model retrieves semantically similar memories (e.g., via SBERT), selects the most helpful ones, and incorporates them into a new chain‑of‑thought to produce the final answer.

Automatic CoT Prompting (Auto‑CoT)

Auto‑CoT eliminates manual example creation by clustering questions and selecting diverse representative examples. The two main steps are:

Cluster the dataset into distinct categories (e.g., addition/subtraction vs. multiplication/division) to ensure varied problem types.

From each cluster, pick a representative question and generate a reasoning chain using zero‑shot CoT. Apply simple filters (e.g., limit length, keep reasoning steps ≤ 5) and combine the representative question + reasoning chain pairs into the prompt.

These advanced prompting strategies significantly improve large language model reasoning performance while reducing manual effort and cost.