Chain of Thought Prompting: 9 Templates and When to Skip It

Chain of thought prompting asks a language model to write out the intermediate steps between a question and its answer instead of jumping straight to the answer. On a model that reasons only in its output, those written steps are where the reasoning happens, so the answer that follows them is usually better than the one the model would have given cold.

The one rule this page keeps coming back to: chain of thought is a technique for models that don't think before they speak. The reasoning models that arrived in 2024 and after do that thinking on their own, and both OpenAI and Anthropic now tell you to leave the "think step by step" line out. Below is what the technique is, where it came from, nine templates you can paste, and a table that says which models still want one.

What Is Chain of Thought Prompting?

A standard prompt gives the model a question and takes whatever comes back. A chain of thought prompt gives the same question and asks for the working first: what's known, what's needed, each calculation or inference in order, and only then the answer. The model's output becomes a short derivation, and the final line is conditioned on everything above it rather than on the question alone.

That matters because a model generating text one token at a time can't backtrack. If the first words of its answer are wrong, the rest of the answer is built on them. Writing the steps out moves the hard part earlier, where a mistake shows up as a wrong intermediate line instead of a wrong conclusion, and gives the model more tokens of its own text to condition on before it commits. It also gives you something to check: a wrong answer with visible steps is debuggable, a wrong answer alone is a coin flip.

Here's the difference on one question. A standard prompt:

A bakery sells muffins at $3 each and cookies at $1.50 each. A customer buys 4 muffins and 6 cookies and pays with a $50 bill. How much change do they get?

The chain of thought version of the same prompt:

A bakery sells muffins at $3 each and cookies at $1.50 each. A customer buys 4 muffins and 6 cookies and pays with a $50 bill. How much change do they get?

Think through it step by step: first the cost of the muffins, then the cookies, then the total, then the change. Show each step on its own line, then give the final answer.

The second prompt produces four lines of arithmetic and a total before the answer. On a small or older model, the first prompt is where the "$29" answers come from.

Where It Came From: Three Papers

The technique has a paper trail, all of it from 2022, and the numbers below belong to the models and benchmarks in those papers. They don't transfer to the models you're using today, which is the point of the "when to skip it" section further down.

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (Wei et al., January 2022)

The paper that named the technique. Wei and colleagues at Google defined a chain of thought as "a series of intermediate reasoning steps" and showed that putting a handful of worked examples with their reasoning into the prompt made large models reason better on arithmetic, commonsense and symbolic tasks. Their headline result: eight chain of thought exemplars given to a 540-billion-parameter model reached state-of-the-art accuracy on the GSM8K math word problem benchmark. The effect showed up at scale; the paper reports it as a property of large models. Source: arXiv 2201.11903.

"Let's think step by step" (Kojima et al., May 2022)

Four months later, Kojima and colleagues showed you didn't need the worked examples. Appending a single sentence, "Let's think step by step," before the answer turned a plain prompt into a zero-shot chain of thought prompt. On text-davinci-002 the line raised accuracy on MultiArith from 17.7% to 78.7% and on GSM8K from 10.4% to 40.7%. That one sentence is where most people's idea of chain of thought prompting comes from. Source: arXiv 2205.11916.

Self-consistency (Wang et al., March 2022)

Between the two, Wang and colleagues asked what happens if you don't trust a single chain. Self-consistency samples several reasoning paths for the same question and takes the answer the most paths agree on, on the reasoning that a correct answer can be reached in more ways than a wrong one. On GSM8K it improved chain of thought results by 17.9 percentage points. Template 5 below is the by-hand version of this idea. Source: arXiv 2203.11171.

9 Chain of Thought Prompt Templates

CoT prompting comes in a few shapes, and the shape depends on what the model is and what the task needs. The first eight templates are for models without built-in reasoning, or for reasoning models with thinking turned off. The ninth is what the same request looks like when the model reasons on its own. Brackets mark the parts to fill in.

1. Zero-shot chain of thought

The Kojima sentence, with a format line so you can find the answer.

[Your question or task]

Let's think step by step. Write each step on its own line, then end with "Final answer:" followed by the answer only.

2. Few-shot chain of thought

Two solved examples teach the reasoning style. Keep the examples close to the real task in length and difficulty.

Here are two solved examples. Follow the same reasoning style for the new question.

Q: [example question 1]
A: [step 1]. [step 2]. [step 3]. So the answer is [answer 1].

Q: [example question 2]
A: [step 1]. [step 2]. So the answer is [answer 2].

Q: [your question]
A:

3. Structured chain of thought with tags

For anything you'll parse with code. The reasoning goes in one tag, the answer in another, so you can extract the answer and keep or drop the working.

Solve the problem below. First work through it step by step inside <reasoning> tags: list what you know, what you need, and each calculation. Then give only the final answer inside <answer> tags, one line, no explanation.

Problem: [paste problem]

4. Plan first, then answer

For multi-part tasks: writing, analysis, anything with more than three moves. The plan is the chain; the answer follows it.

Before you answer, write a numbered plan of the steps you'll take to [task], one line each. Then carry out the plan in order, marking each step as you go. Finish with the result in the format: [format].

Task: [task details]

5. Self-consistency by hand

Three independent attempts, then a vote. Slower, and worth it when a wrong answer is expensive.

Solve the problem below three separate times, each time from scratch and by a different route if one exists. Label them Attempt 1, 2 and 3, and show the steps for each. Then compare the three final answers and report the one that at least two attempts agree on. If all three differ, say so and explain which you trust and why.

Problem: [paste problem]

6. Verify your steps

Chain of thought plus a check pass. The second half catches arithmetic slips and skipped conditions.

Work through [task] step by step. After your final answer, go back over each step and check it against the original question: did you use every number and condition given, and does each step follow from the one before? If a check fails, correct the step and update the answer.

[Task details]

7. Chain of thought for a business decision

Options, criteria, a scored comparison, then a recommendation. The structure stops the model from picking first and justifying after.

Help me decide [decision]. Reason through it in this order:
1. List the realistic options (at least three).
2. List the criteria that matter, with a weight for each.
3. Score each option against each criterion, with one line of reasoning per score.
4. Total the scores and state the recommendation, plus the single risk most likely to change it.

Context: [budget, constraints, timeline, anything the decision depends on]

8. Chain of thought for debugging code

Hypotheses before fixes. The model names what could be wrong and tests each one against the evidence before it touches the code.

Here is a bug report and the code involved. Before proposing a fix: (1) restate what the code should do, (2) list three possible causes of the behavior described, (3) for each, say what evidence in the code or report supports or rules it out, (4) pick the most likely cause. Only then propose the smallest change that fixes it, and explain why it fixes it.

Bug report: [description]
Code: [paste code]

9. The reasoning-model version

No "think step by step." Goal, constraints and output format, and let the model's own reasoning do the rest. This is the shape to use on current reasoning models.

[Task in one or two sentences.]

Constraints: [limits, must-haves, what to avoid]
Output: [the exact format you want back, and nothing else]

A Worked Example: One Problem, Three Prompts

Take a pricing question: a freelancer wants to know what hourly rate to charge to clear a target income. The three prompts below show how the shape of the request changes the shape of the answer. The responses are illustrations of what each prompt asks for, not transcripts of a run.

The standard prompt asks for the number:

I want to earn $90,000 a year as a freelance designer, working 48 weeks a year and billing 25 hours a week. My business costs are $12,000 a year. What hourly rate should I charge?

A model answering this cold tends to return one figure with a sentence of justification, and the figure is often right. When it's wrong, nothing in the answer shows you where.

The zero-shot chain of thought version adds one line:

I want to earn $90,000 a year as a freelance designer, working 48 weeks a year and billing 25 hours a week. My business costs are $12,000 a year. What hourly rate should I charge?

Let's think step by step, then give the rate on its own line.

Now the answer carries its working: total needed is 90,000 plus 12,000, that's 102,000; billable hours are 48 times 25, that's 1,200; the rate is 102,000 divided by 1,200, which is 85 dollars an hour. If the model forgot the costs, the missing line would be visible.

The structured version is the one to use in an app, because the answer is in a tag you can read with code:

I want to earn $90,000 a year as a freelance designer, working 48 weeks a year and billing 25 hours a week. My business costs are $12,000 a year.

Inside <reasoning> tags, work out the total income needed, the billable hours, and the rate. Inside <answer> tags, give the hourly rate as a number only.

Same arithmetic, same 85, but the reasoning and the answer are separable, and a 20% buffer for unbilled time is an easy next instruction to add. On a reasoning model, the first prompt plus the output line from template 9 gets you the same answer without the extra instruction, which is the subject of the next section.

When Not to Use Chain of Thought

The research above was done on models that only reason in their output. The models most people use now have a separate reasoning phase before the answer, and for those, a prompt that asks for step-by-step thinking is at best redundant. OpenAI's guide for its reasoning models has a line that says "Avoid chain-of-thought prompts," explaining that since the models reason internally, asking them to "think step by step" is unnecessary, and elsewhere that the instruction "may not enhance performance (and can sometimes hinder it)." Anthropic's prompting guidance says that on its newer Claude models you should rely on the built-in thinking instead, and that a prompt asking the model to write out its reasoning, for example in thinking tags, "may be declined." Manual chain of thought stays as the fallback when thinking is turned off.

Model type	Write CoT in the prompt?	Why	Source
Reasoning model with thinking on (OpenAI o-series and GPT-5 family, Claude with thinking)	No	The model reasons before it answers; "think step by step" is redundant and can hinder	OpenAI reasoning best practices, Anthropic prompting best practices
Reasoning-capable model with thinking turned off	Yes, as a fallback	Anthropic's guidance: ask it to think through the problem before answering and put the result in answer tags	Anthropic prompting best practices
Chat model without a reasoning phase (older or small instruction-tuned models)	Yes	Written steps are the only reasoning the model gets; this is where the 2022 results came from	Kojima et al., Wei et al.
Local or small open models	Yes, with few-shot examples	Zero-shot CoT was reported as an effect of scale; worked examples help smaller models more than the single sentence does	Wei et al.
Any model, when you need the reasoning in the output for review or audit	Yes, structured	Asking for the steps in a tag gives you a record to check, whatever the model does internally	this page, template 3

The practical rule: if the model has a thinking mode and it's on, write template 9. If it doesn't, or you've switched it off for speed, write one of templates 1 to 8. If you're unsure which kind of model is behind an API, send the same question both ways once and compare the outputs.

Chain of Thought vs Few-Shot, ReAct and Tree of Thoughts

Chain of thought sits in a family of prompting techniques, and the names get mixed up because several of them combine. The short version of each, and where it's covered on this site:

Technique	What it adds	Use it when	Read more
Few-shot prompting	Examples of the output you want, with or without reasoning	The format or style matters more than the reasoning	few-shot prompting, and the basics of zero-shot, one-shot and few-shot prompts
Chain of thought	Written reasoning steps before the answer	One question, one answer, and the model doesn't reason on its own	this page
ReAct	Reasoning steps alternated with actions (a search, a tool call) and their results	The answer depends on information the model has to fetch	ReAct prompting, and combining CoT with ReAct
Tree of thoughts	Several candidate reasoning paths, evaluated and pruned, instead of one chain	Planning and puzzle-type problems where the first path is often a dead end	tree of thoughts

Two more relatives are worth a line. Meta prompting asks the model to write or improve the prompt itself, chain of thought included. Prompt chaining splits one task into several calls, each with its own prompt, which is what template 4 turns into once the plan and the execution become separate requests. For the wider set, see more advanced techniques.

Frequently Asked Questions

What is chain-of-thought prompting in simple words?

It's asking the model to show its work. Instead of "what's the answer," you ask "work it out step by step, then tell me the answer." The steps the model writes become the reasoning that leads to its answer, which makes the answer more reliable on models that don't reason privately, and makes mistakes visible when it isn't.

How does chain of thought work?

A language model produces its output one token at a time, and each token is conditioned on everything before it. When the prompt asks for steps, the model writes intermediate lines first, and the final answer is generated with those lines in its context, so it's an answer to the derivation rather than a guess at the question. The 2022 papers measured this on math and logic benchmarks, where the derivation is most of the work.

Do I need chain of thought prompting with reasoning models?

No. A reasoning model does its step-by-step work in a thinking phase before it writes the answer, and both OpenAI and Anthropic say a "think step by step" instruction is unnecessary there and can get in the way. Give the goal, the constraints and the output format, as in template 9. The exception is when you've turned thinking off, where the manual version comes back as a fallback.

Is chain of thought the same as showing the model's reasoning?

They overlap but aren't the same. Chain of thought is a prompt that asks for written steps, and on a model without a reasoning phase those steps are the reasoning. A reasoning model's thinking is a separate phase that may or may not be shown to you, and asking it to copy that thinking into the answer is the thing Anthropic says may be declined. If you need a record of the reasoning for review, template 3 asks for a written derivation in the output regardless of what the model did internally.

Does chain of thought work on small models?

The original paper reported the gains as something that appeared at large model scale, with its headline result on a 540-billion-parameter model, and smaller models in the same study benefited less. For a small or local model, the few-shot form with worked examples (template 2) is the better bet than the single zero-shot sentence, and the only way to know for your model is to test both on your own questions.

Your Move

Pick one question you ask a model often and run it three ways: cold, with template 1, and with template 9. Whichever reads best tells you what kind of model you're talking to and which templates on this page belong in your prompts. If the step-by-step version wins, keep template 3 handy for anything an application has to parse.

Put this into practice: the prompt engineering guide covers the rest of the techniques with examples. Want all of it in one place? The Complete AI Bundle puts the whole prompt library, Custom GPTs and n8n workflow templates behind one payment, with lifetime access.