In-context learning

The model doesn’t update its weights when you talk to it. But within a single context, it can be taught to do things it wouldn’t do by default.

That’s in-context learning (ICL).

What ICL actually is

ICL was named and formalized in the 2020 GPT-3 paper, Language Models are Few-Shot Learners (Brown et al.). The key finding: if you put a few examples of a task into the prompt, sufficiently large models can generalize from those examples to new instances, without any fine-tuning or gradient updates.

Before ICL, adapting a model to a new task meant training or fine-tuning. With ICL, you just showed the model what you wanted and it figured out the pattern. The model’s weights didn’t change; its behavior on that turn did.

Few-shot prompting

The most common form of ICL. Include 2–5 examples of input-output pairs in the prompt, then ask the model to produce the next output. Works well when:

• The task has a clear pattern examples can teach
• Outputs should follow a specific format or style
• Edge cases matter; include one or two in your examples

The sweet spot is usually two to five examples. More isn’t always better: too many examples burn context budget and can over-anchor the model on the specific shape of the examples. Your examples are more expensive than your instructions.

Chain-of-thought

Chain-of-thought prompting (CoT), introduced by Wei et al. (2022), showed that asking the model to “think step by step” (or giving it reasoning-style examples) improves performance on multi-step tasks. The benefit was initially an emergent property specific to CoT, appearing only in models above roughly 100 billion parameters; smaller models didn’t reliably benefit.

In 2026, explicit CoT prompting matters less than it did. Modern models do step-by-step reasoning implicitly when the task warrants it, especially with higher reasoning effort. You rarely need to write “let’s think step by step” anymore. But asking explicitly still helps when:

• The task involves math or formal logic
• You want the reasoning visible for review
• You’re building a pipeline that separates “reasoning” from “final answer”

Prompt-as-program

The broader frame: the prompt is the most expressive surface you have. A prompt can contain instructions, examples, context, constraints, format specifications, and personas. You’re not “configuring” the model; you’re writing a program the model executes.

Modern Claude models are so strong at zero-shot that you rarely need elaborate few-shot prompting. Describe the task clearly, give the model the relevant context, and it usually figures out the rest. Reach for few-shot or CoT when clear description isn’t enough.

Where ICL still matters a lot

Even as models get better at zero-shot:

• Format adherence: one or two examples are still the most reliable way to pin down a specific output format.
• Tone and voice: matching a company’s writing style is easier with examples than with a style guide.
• Domain-specific conventions: an example of a correctly-formatted legal clause or medical note teaches more than a rule.

Resources

• Brown et al.: Language Models are Few-Shot Learners (GPT-3), the original ICL paper
• Wei et al.: Chain-of-Thought Prompting Elicits Reasoning
• Prompt Engineering Guide, a practical reference for few-shot, CoT, and beyond
• Anthropic: Prompt engineering overview