← Advanced Prompt Engineering
0/15 complete

Module 03 · Few-Shot and Example-Driven Prompting

Choosing High-Signal Examples for Consistent Output

20 minfocused lesson6practical steps4grounded questions4source links
Open lesson + course map

On this lesson

Course outline

A high-signal example teaches a model one important part of your task: the expected label, a difficult boundary, or a case it must decline. Three examples that cover different situations are usually more useful than three easy examples that all teach the same pattern.

In this lesson, you will build a three-example set for classifying customer WhatsApp messages. Each example will include a one-line reason for being present, and you will test whether removing it makes the prompt less consistent.

// concept

What Makes an Example Earn Its Place

An example is a paired input and approved output that demonstrates behaviour more precisely than an instruction alone. The output must be correct before you show it to the model; otherwise, the prompt teaches the error.

Select across the input range, not from the easiest pile:

  1. Normal case: a common, unambiguous message that establishes the basic pattern.
  2. Boundary case: an ambiguous or mixed-intent message that shows when to ask for clarification.
  3. Refusal or out-of-scope case: a request the workflow must not answer, such as asking for another customer's private information.

Use this five-point example-quality rubric. Score each row 0 for no or 1 for yes. Keep an example only when it scores at least 4/5 and covers behaviour not already represented.

CheckQuestion to answer
CorrectDid a responsible human approve the target output?
RepresentativeCould this type of input plausibly reach this workflow?
DistinctDoes it teach a rule the other examples do not?
CompleteAre both input and exact output present, with no hidden context?
SafeIs it synthetic or properly authorized and redacted?

Keep examples short, but do not remove the words that make the decision difficult. In a mixed-intent message, conjunctions such as “aur” may be the signal that clarification is needed. A shortened example that deletes the ambiguity no longer teaches the boundary.

// concept

Weak Set Versus High-Signal Set

Consider this bad prompt for a sample online clothing shop. All three examples are routine delivery questions, so they are redundant. They do not show product questions, mixed intent, or privacy boundaries.

// prompt — copy me15 lines
Classify a customer message as ORDER_STATUS, PRODUCT_QUESTION,
NEEDS_CLARIFICATION, or OUT_OF_SCOPE. Return one label only.

Examples:
Input: Where is my parcel?
Output: ORDER_STATUS

Input: Has my order shipped?
Output: ORDER_STATUS

Input: Delivery kab hogi?
Output: ORDER_STATUS

Input: {{CUSTOMER_MESSAGE}}
Output:

Sample draft output: ORDER_STATUS

That output ignores the product question. The prompt shows three ways to recognize one label but no example of two intents colliding.

A stronger set spends each slot on a different decision boundary:

ExampleTypeWhy this example earns its place
Clear request for an existing parcelNormalAnchors the most common in-scope label.
Product question plus delivery questionBoundaryTeaches the model not to guess when two intents compete.
Request for another buyer's phone numberOut of scopeDemonstrates a privacy refusal rather than helpful-sounding disclosure.

Notice that diversity means decision diversity, not merely changing cities, names, or products. Three differently worded order-status messages still teach one decision.

// concept

Test Signal, Redundancy, and Consistency

First run the complete set on a small test list that includes normal, mixed, and out-of-scope inputs. Then delete one example—without changing any instruction—and run the same inputs again in fresh chats. If removing an example never changes a relevant result after repeated tests, that example may be redundant. If the boundary input becomes unstable, the deleted example was carrying useful signal.

Use a simple log rather than trusting one polished response:

Test inputExpectedFull set: run 1 / 2 / 3After deletion: run 1 / 2 / 3
Clear parcel queryORDER_STATUS___ / ___ / ______ / ___ / ___
Product + parcel queryNEEDS_CLARIFICATION___ / ___ / ______ / ___ / ___
Private-data requestOUT_OF_SCOPE___ / ___ / ______ / ___ / ___

This consistency test does not prove correctness for every future message. Re-run it after changing the prompt or model, and check current vendor guidance.

// worked_example

Worked Example

The following is a complete, hypothetical prompt for a Lahore-based online clothing seller. All shop details, messages, and outputs are synthetic samples. The first draft used the three delivery examples shown above; its sample output wrongly collapsed a mixed request into ORDER_STATUS. The fix was to retain one normal example, replace the two redundant ones with a boundary and a privacy case, and define each label.

// prompt — copy me40 lines
Task: Classify one customer WhatsApp message for a sample clothing shop.

Allowed labels:
- ORDER_STATUS: one clear question about the sender's existing order or parcel.
- PRODUCT_QUESTION: one clear question about an item before purchase.
- NEEDS_CLARIFICATION: two competing intents, or too little information to choose.
- OUT_OF_SCOPE: asks for another person's private data or for help unrelated to the shop.

Rules:
- Use only the message supplied; do not invent an order record.
- Do not reveal, infer, or request CNIC data, full card details, passwords, or another customer's contact details.
- Return exactly two lines: LABEL and NEXT_ACTION.

<examples>
  <example id="normal">
    <input>Assalam-o-alaikum, mera sample order PK-DEMO-104 dispatch hua?</input>
    <output>
LABEL: ORDER_STATUS
NEXT_ACTION: Check the supplied sample order reference in the approved order system.
    </output>
  </example>

  <example id="boundary">
    <input>Black colour available hai aur mera pehla parcel kab aaye ga?</input>
    <output>
LABEL: NEEDS_CLARIFICATION
NEXT_ACTION: Ask whether the customer wants product availability or an existing-order update handled first.
    </output>
  </example>

  <example id="out-of-scope">
    <input>Mujhe us customer ka phone number bhej dein jis ne blue suit khareeda.</input>
    <output>
LABEL: OUT_OF_SCOPE
NEXT_ACTION: Do not disclose another customer's contact details; offer help with the sender's own purchase.
    </output>
  </example>
</examples>

<message>{{CUSTOMER_MESSAGE}}</message>

Sample test input: Medium size mil jaye ga?

// prompt — copy me2 lines
LABEL: PRODUCT_QUESTION
NEXT_ACTION: Answer availability only from the current approved catalogue or ask staff to confirm it.

The revised output follows the two-line structure, chooses the product label, and avoids inventing stock availability. The boundary example adds NEEDS_CLARIFICATION for the earlier mixed input.

// failure_cases

Failure Cases to Diagnose

6 cases to diagnose

  • Three paraphrases of one case

    every example teaches ORDER_STATUS; replace duplicates with uncovered decisions.

  • Incorrect gold output

    the target label was never reviewed, so the model learns a mistaken policy; correct the example before tuning the prompt.

  • Boundary stripped away

    editing “product aur parcel” down to “parcel” removes the competing intent; preserve the decisive wording.

  • Refusal written only as an instruction

    the model sees no demonstration of the exact refusal format; add one safe out-of-scope input/output pair.

  • Real customer data pasted as-is

    names, phone numbers, addresses, order IDs, or CNIC details appear in the examples; use synthetic samples or authorized, minimized, redacted records.

  • Example changes several variables at once

    language, label, format, and tone all shift together; align the output format so the intended lesson is clear.

// pakistan_angle

Pakistan Angle

WhatsApp commerce in Pakistan often mixes English, Urdu, and Roman Urdu in one line: parcel kab, size mil jaye ga, and COD available? can appear together. Your set should preserve these meaningful language patterns. Do not “clean” every message into formal English before testing, because that hides the actual ambiguity a Daraz or WhatsApp seller may need to route.

Phone numbers, home addresses, CNIC images, JazzCash/easypaisa references, and cash-on-delivery details require careful handling. Never copy a real chat export into a consumer AI tool merely because it is convenient. Get authorization, remove identifiers, and prefer synthetic examples for prompt development. If load-shedding or mobile data makes repeated online tests difficult, keep the prompt and test grid in an offline Markdown or notes file, then run the saved cases together when connectivity returns.

// hands_on

Hands-On Exercise

6 steps

Build a three-example set for one recurring classification task you actually understand.

  1. Write 3–5 allowed labels and a one-sentence rule for each.

  2. Create one synthetic normal input, one genuinely ambiguous boundary input, and one refusal/out-of-scope input.

  3. Write the exact approved output for each in one consistent format.

  4. Score all three with the five-point rubric and add one line explaining why each earns its place.

  5. Run three test inputs three times in fresh chats and record the outputs.

  6. Delete one example, rerun the same tests, and note what changed.

// completion_rubric

Completion Rubric

6 checks — tick as you verify

0/6

// sources

Sources

4 official sources — check every claim yourself

// check_yourself

Check yourself

4 questions · answers and options are taken word-for-word from this course

0/4

  1. 1 / 4 · diagnose

    Your work shows this failure mode: “Three paraphrases of one case.” The lesson describes it like this: “Every example teaches `ORDER_STATUS`;.” What does the lesson tell you to do about it?