1. Exam at a Glance
| Item | Value |
|---|---|
| Exam code | CCAR-P |
| Number of items | 63 |
| Item format | Multiple-choice and multiple-response; each item states how many to select |
| Time limit | 120 minutes |
| Passing score | Scaled score of 720 on 100–1,000, set by a standard-setting study |
| Fee / validity | $175 USD · 12 months |
| Reporting | Pass/fail, scaled score, and percent-correct by domain |
| Prerequisites | None required. Recommended: 3+ yrs systems architecture, 6+ months hands-on Claude in production, end-to-end delivery experience. |
2. The Seven Domains
Item counts assume a 63-question exam. Integration is the heaviest single domain. Domains 6 and 7 have almost no API surface and are 21% of your score between them.
| Domain | Weight | Items | Lab |
|---|---|---|---|
| 1. Solution Design & Architecture | 17% | 11 | 1, 2 |
| 2. Claude Models, Prompting & Context Engineering | 13% | 8 | 3 |
| 3. Integration | 19% | 12 | 4, 5 |
| 4. Evaluation, Testing & Optimization | 16% | 10 | 6, 7 |
| 5. Governance, Safety & Risk Management | 14% | 9 | 8 |
| 6. Stakeholder Communication & Lifecycle | 14% | 9 | 9 |
| 7. Developer Productivity & Ops Enablement | 7% | 4 | 10 |
3. The Pattern Ladder
Climb only when the rung below cannot hold the requirement.
| Pattern | Who decides control flow | Use when |
|---|---|---|
| Augmented LLM | Nobody, there is no flow | Classification, extraction, summarization, Q&A over retrieved context |
| Workflow | You, in advance | The steps are enumerable and stable |
| Agentic | Claude, per turn | The path is unknown until the work begins |
"The steps never change" means workflow. Exposing fixed steps as tools does not make a workflow; it makes an agent you did not admit to. An agent needs all four of: complexity, value, viability, and a survivable cost of error. Irreversible actions with no detection is the classic disqualifier.
4. The Five Planes
| Plane | Contains | Answers |
|---|---|---|
| Ingress | Auth, rate limits, input validation, PII redaction, untrusted-content isolation | What is allowed in? |
| Context | Retrieval, caching, prompt assembly, token budget | What does the model see? |
| Reasoning | Model choice, effort, tools, orchestration | What does the model do? |
| Egress | Schema validation, guardrails, human gates, action execution | What is allowed out? |
| Feedback | Traces, evals, cost/latency telemetry, labelled failures | How do we know, and how do we improve? |
The feedback plane is the one candidates omit. Without it there is no eval set, no way to separate a retrieval regression from a prompt regression, and no evidence for the stakeholder conversation.
5. Control Strength: What Actually Enforces
| Control | Type | Strength |
|---|---|---|
| Remove the capability / tool | Preventive | Strong |
| Deterministic hook or allowlist before execution | Preventive | Strong |
| Output schema validation | Preventive | Strong |
| Human-in-the-loop gate | Preventive | Strongest, slowest |
| Confirmation prompt before the action | Compensating | Partial |
| Logging and audit | Detective | After the fact |
| Instruction in the system prompt | Advisory | Not a control |
| A bigger / more capable model | — | Not a control |
Anything enforced only by asking the model nicely is not a control. Least privilege removes a capability; it does not guard it. And the agent must act with the permissions of the requesting user, not of the application. A service account holding the union of everyone's permissions is a confused deputy. Authorization belongs inside the tool implementation, never in the prompt.
6. Prompt-Injection Defense
| Works | Does not work |
|---|---|
| Isolate untrusted content in a tool-less call or subagent | Telling the model in the system prompt to ignore embedded instructions |
| Remove privileged tools from any context that sees untrusted input | Switching to a larger, more capable model |
| Deterministic hooks and allowlists | Raising output_config.effort |
| Validate output against a schema before acting | Adjusting sampling parameters (they were removed and return 400) |
| Treat all retrieved and tool content as data, never instruction | Escaping or quoting the content alone |
7. Retrieval by Query Shape
| Query pattern | Use | Why |
|---|---|---|
| Exact key (policy ID, customer ID) | Database lookup / metadata filter | A vector search on a key is architecture theatre |
| Rare tokens: SKUs, error codes, proper nouns | Lexical / BM25 | Dense embeddings systematically miss high-entropy tokens |
| Paraphrase / semantic | Dense vector | Meaning, not surface form |
| Both (the common case) | Hybrid fusion + rerank | The strong default |
| Multi-hop / relational | Graph or query decomposition | One retrieval cannot span the join |
| Recency-sensitive | Metadata filter, then rank | Relevance is not recency |
Three RAG rules the exam leans on. Chunk boundaries destroy meaning: a table split across chunks answers nothing. Changing the embedding model invalidates the entire index; you must re-embed everything, or mixed embedding spaces silently return garbage. And ACL/tenant filtering happens at retrieval time as a metadata prefilter, never by asking the model to ignore documents it should not have seen.
8. Cost and Latency Levers
| Lever | Cost | Latency | Catch |
|---|---|---|---|
| Prompt caching | Down | Down | The rare lever that moves both. Needs a stable prefix. |
| Batches API | 50% off | Destroyed | 24h window, single-turn, results unordered |
| Streaming | Unchanged | TTFT down | Perceived latency only. Does not reduce cost. |
| Model routing (Haiku → Opus) | Down | Down | Quality risk if the task is misjudged |
Lower effort | Down | Down | Hard-task quality falls |
| Bigger model | Up | Up | Buys quality, nothing else |
max_tokens | — | — | A cap, not a cost control. It truncates. |
Caching rules. Prefix match only. Block order is
tools → system → messages. Max 4 breakpoints. Reads bill at roughly 0.1×, writes at 1.25× (5-minute TTL) or 2× (1-hour). Minimum cacheable prefix is 4,096 tokens on Opus 4.8. Anything that varies goes last. A timestamp at the top of the system prompt destroys every hit. Verify with usage.cache_read_input_tokens, and remember usage.input_tokens is the uncached remainder only, so naive cost math undercounts.
Cost per request is the wrong denominator. Track cost per resolved task. A change that halves cost per request while quality drops and users retry has raised your real cost and hidden it behind a green dashboard.
9. Symptom → Plane Differential
The thing that changed is the thing that broke.
| Symptom | Look at |
|---|---|
| Confidently wrong right after a data refresh; model and latency unchanged | Retrieval / indexing: a broken re-index or mismatched embeddings |
| Degrades only on long inputs | Context management: window, pruning, compaction |
| Output violates the schema | Egress / output contract |
| Correct on easy cases, fails on hard reasoning | Model mismatch, or effort too low |
| Output truncated mid-thought | stop_reason == "max_tokens" |
| Agent loops forever | Missing termination condition |
| Latency spike, prompt unchanged | Infrastructure or retrieval, not the model |
| Cache hit rate collapses | Someone put a timestamp at the front of the prompt |
Hallucination is a symptom, not a diagnosis. Decompose it: no grounding (retrieval), grounding present but ignored (force citations and validate the cited span exists), contradictory grounding (data quality), or genuinely out of distribution (model mismatch). The ablation ladder: rerun the failing case with a hand-built perfect context. If it passes, the defect is retrieval, not reasoning.
10. Evaluation
| Grader | Good for | Watch out |
|---|---|---|
| Programmatic | Schema, exact match, does the cited span exist | Narrow. Use it first because it catches most regressions. |
| LLM-as-judge | Rubric scoring at scale | Position, verbosity, and self-preference bias. An uncalibrated judge is a metric you invented. |
| Human | Ground truth, calibration, high-stakes cases | Expensive. Reserve it. |
Three rules. Average latency hides the tail, so report p95. Inter-annotator agreement caps your accuracy target: if humans agree 85% of the time, chasing 99% is overfitting one annotator. And pair every primary metric with a guardrail metric, or you will ship a win on one and a regression on the other.
Determinism does not come from sampling parameters.
temperature, top_p, and top_k were removed and return 400. Reproducibility comes from pinning the model ID and the prompt version, then reporting a pass rate over a set of cases rather than the verdict of a single trial.
11. SLA vs SLO vs SLI
| Term | Means |
|---|---|
| SLI | A measured signal: p95 latency, availability, citation-validity rate |
| SLO | An internal target on an SLI |
| SLA | A contractual commitment, with consequences |
| Error budget | The allowed shortfall, and it is spent on change |
Never promise an accuracy SLA on live traffic. Live traffic has no reference label, and you cannot commit above the ceiling at which your own experts agree. Accuracy is an SLO on a frozen, held-out set. Availability, latency, and the existence of a human gate are things you can actually commit to.
12. Compliance: Architectural Obligations
Architectural guidance, not legal advice.
| Regulation | What it forces into your design |
|---|---|
| GDPR | Data minimization (redact before the call); purpose limitation; right to erasure reaches your logs, traces, caches, and eval set; Art. 22 implies meaningful human review of automated decisions |
| HIPAA | A BAA with any processor; minimum necessary; audit controls; encryption in transit and at rest; de-identify where possible |
| FedRAMP | Authorized offerings and regions only; the authorized service boundary; control inheritance; continuous monitoring; change control |
| Cross-cutting | Residency drives where you deploy. Retention drives what you may log. Auditability drives structured traces with request IDs. |
An aggregate metric conceals a fairness failure. 92% overall accuracy hiding 71% on one segment is a governance problem, and stratifying your eval metrics is the only way to see it. Bias enters through the eval set, the retrieval corpus, and the prompt's examples, not only through "the model."
13. API Shapes That Now Return 400
Where this exam touches code, it touches the current API. These shapes are removed, not deprecated with a warning.
# WRONG - each of these returns HTTP 400 on current models
thinking={"type": "enabled", "budget_tokens": 8000}
temperature=0.7 top_p=0.9 top_k=40
output_format={...} # superseded by output_config.format
messages=[..., {"role": "assistant", "content": "{"}] # prefill: rejected
# CURRENT
thinking={"type": "adaptive", "display": "summarized"}
output_config={"effort": "high", "format": {...}}
model="claude-opus-4-8" # never date-suffixed
client.messages.count_tokens(...) # never tiktoken
| Model | ID | Context | $/1M in | $/1M out |
|---|---|---|---|---|
| Claude Opus 4.8 | claude-opus-4-8 | 1M | $5.00 | $25.00 |
| Claude Sonnet 5 | claude-sonnet-5 | 1M | $3.00 | $15.00 |
| Claude Haiku 4.5 | claude-haiku-4-5 | 200K | $1.00 | $5.00 |
| Claude Fable 5 | claude-fable-5 | 1M | $10.00 | $50.00 |
Batch results arrive in any order. Key them by
custom_id, never by position. Positional zipping is a silent data-corruption bug: every result attaches to the wrong input, nothing throws, and the pipeline looks green.
14. Harness vs Deployment
| Approach | Harness | Deployment | Tools |
|---|---|---|---|
| Manual loop | You | You | Yours only |
Tool Runner (anthropic SDK) | SDK | You | Yours only |
Claude Agent SDK (claude-agent-sdk) | SDK | You | Built-in file, bash, search + MCP |
| Managed Agents | Anthropic | Anthropic | Hosted sandbox + Skills/MCP |
The Tool Runner is not the Claude Agent SDK. Both supply a harness only; you still deploy. Managed Agents is the only one that supplies both.