Prompt Engineering for Precision: Criteria, Constraints, and Structured Output

1. Explicit Categorical Criteria vs. Vague Instructions

Vague instructions like "be conservative" or "only report high-confidence findings" fail because they do not define objective thresholds. Replace adjectives with numeric or behavioral criteria.

Classify lead quality as low, medium, or high. Be conservative.

HIGH
- Revenue is greater than $10M, AND
- Company has an active AI initiative or open data/ML roles.
Example: "$25M revenue and hiring an ML platform lead."

MEDIUM
- Revenue is $2M-$10M, OR
- Revenue is unknown but company has a clear AI adoption signal.
Example: "$6M revenue and recently published an AI case study."

LOW
- Revenue is below $2M, OR
- No AI adoption signal is present.
Example: "$900k revenue and no technical hiring signal."

If evidence supports two categories, choose the higher category.
If revenue and AI signal are both missing, output needs_clarification.

1a. Behavioral Thresholds and Escape Hatches

Behavioral thresholds make classification reproducible: "production is down for all users" is stronger than "severe outage." Escape hatches like needs_clarification (equivalently named unclear or other in exam scenarios) prevent the model from forcing a fuzzy input into a false category.

2. Few-Shot Examples for Generalization

Few-shot examples are the best way to handle ambiguous cases where two tools, fields, or categories look reasonable.

2a. The Reasoning Column

High-quality few-shot examples include a Why: line. The reasoning teaches the rule behind the pattern, so the model generalizes to novel cases rather than merely matching examples.

Example 1
Input: "Acme reports $14M ARR and is hiring a VP of AI Platform."
Output: {"lead_quality": "HIGH", "annual_revenue": "$14M", "ai_signal": "hiring VP of AI Platform"}
Why: Revenue is greater than $10M and the hiring signal confirms active AI investment.

Example 2
Input: "BetaCo launched a chatbot pilot, but revenue is not disclosed."
Output: {"lead_quality": "MEDIUM", "annual_revenue": null, "ai_signal": "chatbot pilot"}
Why: Revenue is missing, but a clear AI adoption signal qualifies for MEDIUM rather than LOW.

Example 3
Input: "Gamma LLC has $800k revenue and no AI-related hiring or projects."
Output: {"lead_quality": "LOW", "annual_revenue": "$800k", "ai_signal": null}
Why: Revenue is below $2M and there is no AI signal.

2b. Negative Triggers

Include examples that distinguish acceptable patterns from genuine issues. Negative examples reduce false positives by teaching what not to flag or extract.

3. Structured Output via JSON Schemas

There are two ways to get schema-shaped data back. The exam guide frames it as tool use: define an extraction tool whose input_schema is the output schema you want, force Claude to call it with tool_choice, and read the data from the tool_use block's input. On Opus 5.5 and Sonnet 5.5 that forced tool_choice returns HTTP 400, so current code uses structured outputs instead: pass the schema as output_config.format and the response text is guaranteed to be valid JSON matching it.

import json

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=16000,
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "company_name": {"type": "string"},
                    "annual_revenue": {"type": ["string", "null"]},
                    "lead_quality": {"type": "string", "enum": ["HIGH", "MEDIUM", "LOW"]}
                },
                "required": ["company_name", "annual_revenue", "lead_quality"],
                "additionalProperties": False
            }
        }
    },
    messages=[{"role": "user", "content": document_text}]
)

# Check for a refusal or a cut-off before parsing, then read the JSON text.
if response.stop_reason in ("refusal", "max_tokens"):
    ...  # handle: the output may be absent or truncated
data = json.loads(next(b.text for b in response.content if b.type == "text"))

When the structured data must come from a tool call (for example, an agent that also acts), keep tool_choice on auto, add strict: true to the tool definition so its inputs conform to your schema exactly, and check that the tool_use block is present.

3a. Choosing the Right tool_choice

The exam tests the tool_choice modes and when each applies, so know all three. Remember that the two forcing modes (any and tool) return HTTP 400 on Opus 5.5 and Sonnet 5.5; on those models the equivalent guarantee comes from structured outputs or from a code-side check.

• {"type": "auto"}: the model decides. It may answer in plain text instead of calling a tool, so this is not sufficient when you need guaranteed structured output.
• {"type": "any"}: the model must call some tool, but picks which one. Use when you have multiple extraction schemas (e.g. invoice vs. contract vs. resume) and the document type is unknown, the model routes to the right schema but can never fall back to prose.
• {"type": "tool", "name": "..."}: the model must call that specific tool. Use to force a prerequisite extraction step, or whenever exactly one output schema is acceptable.

4. Advanced Schema Design & Safety

Safe schemas prevent hallucinations and API errors in multi-step workflows.

4a. Hallucination Prevention with Nullable Fields

Use nullable fields, such as ["string", "null"], for information that may legitimately be missing from a source document. This lets the model return null instead of fabricating placeholder values to satisfy a required field.

{
  "type": "object",
  "properties": {
    "company_name": {"type": "string"},
    "annual_revenue": {"type": ["string", "null"]},
    "contact_email": {"type": ["string", "null"]},
    "ai_signal": {"type": ["string", "null"]},
    "budget_range": {"type": ["string", "null"]},
    "decision_date": {"type": ["string", "null"]}
  },
  "required": ["company_name", "annual_revenue", "contact_email", "ai_signal", "budget_range", "decision_date"],
  "additionalProperties": false
}

4b. Extensible Categorization with other

Enums are reliable, but production categories evolve. Use a bounded enum with an other escape hatch (the exam also uses the equivalent name "unclear" for this pattern) plus a required detail field so the system stays extensible without losing structure.

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["billing", "technical", "security", "other"]
    },
    "category_detail": {
      "type": ["string", "null"],
      "description": "Required when category is other; otherwise null."
    }
  },
  "required": ["category", "category_detail"],
  "additionalProperties": false
}

Validation rule: if category is "other", category_detail must explain the specific category. If category is a known enum value, category_detail should be null.

5. Validation-Retry Loops

Schema constraints guarantee shape, not truth. A response can be perfectly schema-valid and still semantically wrong, line items that don't sum to the stated total, a decision date before the document date. Catch these with code-level validation, and when validation fails, retry with feedback: send the original document, the failed extraction, and the specific validation errors appended, so the model can see exactly what to correct rather than guessing.

MAX_RETRIES = 2

def extract_with_retry(document_text: str):
    attempt = extract(document_text)                # structured-outputs extraction
    for _ in range(MAX_RETRIES):
        errors = validate(attempt)                  # returns [] when clean
        if not errors:
            return attempt
        if any(e.kind == "missing_from_source" for e in errors):
            return route_to_human_review(document_text, attempt, errors)
        feedback = (
            f"Document:\n{document_text}\n\n"
            f"Your previous extraction:\n{json.dumps(attempt)}\n\n"
            f"It failed these validation checks:\n"
            + "\n".join(f"- {e.message}" for e in errors)
            + "\nCall record_prospect again with a corrected extraction."
        )
        attempt = extract(feedback)
    return route_to_human_review(document_text, attempt, errors)

The key limit: retries fix format and structural errors, a miscopied number, a field mapped to the wrong key, an invalid enum value. Retries cannot fix information that is absent from the source document. If the contact email simply is not in the text, no amount of retrying will produce it, it will only burn tokens or, worse, pressure the model into fabricating one. Detect the absent-information case (a required semantic field that validation flags as missing and the model returns null for on retry) and route it to human review instead of retrying.

5a. Self-Correction Fields

You can also make the schema itself do validation work. Have the model extract calculated_total (the sum it computes from the line items) alongside stated_total (the total printed on the document) and flag any discrepancy, a mismatch is a built-in semantic check with no extra API call. Similarly, add a conflict_detected boolean the model must set when sources disagree (e.g. two documents state different revenue figures), so inconsistency surfaces as data instead of silently resolving to one arbitrary value.

Lab Exercise: Designing for Deterministic Extraction

Objective: master categorical criteria, union-type constraints, prerequisite forcing, and validation loops.

1. Explicit classification: create a Lead Quality classification tool using numeric thresholds such as Revenue > $10M; include one-line examples for each category.
2. Few-shot generalization: provide 2-4 extraction examples. Each example must include a Why: line explaining why specific data was mapped to a field.
3. Union-type constraints: design a Prospect Profile schema with 5 nullable fields and verify missing data returns null, not placeholder text.
4. Extensible categorization: design a schema where category is an enum that includes other; require a separate category_detail string when other is selected.
5. Prerequisite step: ensure metadata extraction runs before enrichment. On claude-sonnet-5-5, run it as its own structured-outputs request and only call enrichment once that JSON validates; do not rely on forced tool_choice, which returns 400 there.
6. Validation-retry loop: check for a semantic error, such as line items not summing to a total. If validation fails, send a follow-up request with the original document, the failed extraction, and the specific validation errors to guide self-correction.
7. Absent-information detection: run the loop on a document that is genuinely missing a required field (e.g. no contact email anywhere in the text). Verify your code recognizes that retries cannot recover absent information and routes the record to human review instead of retrying.