Integration Protocols, Auth & Observability

1. Capability Bloat and Least Privilege

The exam guide's own Sample 1 is worth teaching verbatim, because it encodes the entire mental model for this domain in a single item:

The correct answer is to remove the refund and delete tools from the agent's configuration entirely. Not to log their use. Not to add a confirmation prompt. Not to move to a more capable model that "makes fewer mistakes." Sit with why each of those is wrong, because the distractors are the lesson:

Generalize the pattern: when one option removes a capability and the others monitor it, the removing option wins. The staff never needed refund or delete, so neither does the agent acting on their behalf. Grant is the exception that must be justified; the default is deny.

Bloat is a correctness problem, not only a security one

There is a second reason the exam keeps deleting tools, and candidates who think of this purely as a security concern miss it: more tools degrade tool-selection accuracy. Every tool you expose is another option the model weighs on every turn, another schema competing for attention, another chance to pick the plausible-but-wrong action. A twenty-tool agent is not just twenty tools' worth of attack surface; it is measurably worse at choosing among them than a five-tool agent. Pruning the surface makes the agent both safer and more correct. When an item asks how to improve a flailing agent's reliability, "give it fewer, sharper tools" is frequently the answer, not "give it more."

2. Authentication vs. Authorization: The Gap Analysis

The objective is phrased as "analyze authentication and authorization requirements to identify security gaps." The exam is testing whether you can tell the two apart and spot where a design conflates them. Authentication answers who is calling. Authorization answers what this caller is allowed to do. An agent that authenticates flawlessly and authorizes nothing is the most common failure in the domain.

The governing rule, and the sentence to carry into the exam:

The last row is where prompt injection and authorization meet. An MCP server returns content; that content can contain instructions; if your agent holds broad privilege and treats retrieved text as trusted, a malicious document becomes a command. The defense is the same least-privilege posture as Section 1: the agent should not hold the capability that the injected instruction wants to invoke.

Authorization belongs in the tool, not the prompt

Candidates reach for "add a system-prompt rule: only access records belonging to the current user." That is not an authorization control, it is a request. The model can be talked out of it, and a prompt injection routinely will be. The check must live in code the model cannot reason around, on the real arguments, using the real user's identity:

import anthropic
from anthropic import beta_tool

client = anthropic.Anthropic()

# The requesting user's identity is bound at request-assembly time,
# NOT supplied by the model. The model cannot forge or override it.
def make_tools(user):

    @beta_tool
    def get_invoice(invoice_id: str) -> str:
        """Fetch a single invoice by ID.

        Args:
            invoice_id: The invoice identifier to retrieve.
        """
        invoice = db.invoices.get(invoice_id)
        # Authorization is enforced HERE, in code, on the real arguments -
        # against the requesting user's grants, not the app's.
        if invoice is None or invoice.owner_id != user.id:
            # Do not leak existence. Same answer for "missing" and "forbidden".
            return "No invoice with that ID is available to you."
        return invoice.render()

    return [get_invoice]

# The agent authenticates and ACTS AS `user`, never as an omnipotent
# service account. Least privilege is enforced by what make_tools() grants.
runner = client.beta.messages.tool_runner(
    model="claude-opus-4-8",
    max_tokens=4096,
    tools=make_tools(current_user),
    messages=[{"role": "user", "content": user_question}],
)

The invariant: the model chooses which tool and what arguments; your code decides whether this user may have that result. Never the other way around.

3. Connection Protocols: MCP, API/CLI, Agent-to-Agent

The objective names three integration mechanisms and asks you to select among them. They are not interchangeable, and the exam rewards the architect who can say why one boundary fits and the others do not.

The selection criteria that discriminate exam answers: reuse across clients (many consumers → MCP), who owns the tool (you own it and only you call it → direct API/CLI), trust boundary (crossing one → agent-to-agent, and now you own an audit problem), and discoverability (need it → MCP; don't → the discovery machinery is overhead). A single bespoke integration wrapped in an MCP server is over-engineering; a capability that ten teams re-implement by hand should have been an MCP server.

The MCP connector: both halves are required

When you connect Claude directly to a remote MCP server from the Messages API, the classic bug is supplying only one of the two required parameters. You must declare the server in mcp_servers and reference it by name in a mcp_toolset entry inside tools. Supplying mcp_servers alone is rejected as a validation error; every declared server must be referenced by exactly one toolset. The beta header mcp-client-2025-11-20 is required.

import anthropic

client = anthropic.Anthropic()

response = client.beta.messages.create(
    model="claude-opus-4-8",
    max_tokens=2048,
    betas=["mcp-client-2025-11-20"],
    # HALF ONE: declare the server. Note there is NO credential here -
    # auth is injected at the transport layer, never in the prompt.
    mcp_servers=[
        {
            "type": "url",
            "name": "issue-tracker",
            "url": "https://mcp.internal.example.com/sse",
        }
    ],
    # HALF TWO: reference it by name. Omitting this - or letting the
    # names disagree - is the classic 400. Every declared server must
    # be referenced by exactly one mcp_toolset.
    tools=[
        {"type": "mcp_toolset", "mcp_server_name": "issue-tracker"}
    ],
    messages=[{"role": "user", "content": "Summarize open P1 issues."}],
)

4. Accuracy–Latency Trade-offs, and Justifying the Config

The objective is "evaluate accuracy-latency trade-offs and justify configuration decisions." The verb justify is the whole point: the exam does not want you to know that a lever exists, it wants you to state the SLO, name the lever, and name the cost. An architect who says "use caching" without saying which metric it moves and what it costs has not justified anything.

Two rows are exam magnets. Streaming moves perceived latency only. If an item asks to lower cost or total latency, streaming is a distractor. Prompt caching moves cost and latency together: if an item asks for both, it is almost always the answer. And batching is the trap that pairs a true fact ("50% cheaper") with the wrong requirement (anything a user is waiting on).

Justifying a config is a three-part sentence. The SLO is p95 < 3s and cost < $0.02/ticket. The lever is prompt caching on the 11k-token policy corpus. The cost is a stable prefix: the corpus must sit before anything that varies, or it caches nothing. That is a defensible decision. "Use caching" is not.

5. Observability at Scale

The objective is "analyze observability challenges and select monitoring strategies at scale." The word scale is doing the work: at ten requests a day you can log everything; at ten million you cannot, and the exam is testing whether you know what to keep and what to drop.

What to emit

Emitting the trace record is the concrete skill. The usage split is not optional detail, it is the difference between a cost dashboard that reconciles and one that does not:

import json, logging

logger = logging.getLogger("agent.trace")

def emit_trace(response, tenant_id, turn_id):
    u = response.usage
    record = {
        "turn_id": turn_id,
        "tenant_id": tenant_id,
        # Public despite the underscore - this is what Anthropic traces on.
        "request_id": response._request_id,
        "model": response.model,
        "stop_reason": response.stop_reason,
        # Split the usage out. input_tokens is the UNCACHED remainder only -
        # summing all three gives the true prompt size and the true cost.
        "input_tokens": u.input_tokens,
        "cache_read_input_tokens": u.cache_read_input_tokens,
        "cache_creation_input_tokens": u.cache_creation_input_tokens,
        "output_tokens": u.output_tokens,
    }
    # A metric (cheap, aggregate) - NOT the full prompt (PII + volume).
    logger.info(json.dumps(record))

The scale problem, and the three tiers of telemetry

You cannot log full prompts and responses for every request: that is PII you now have to protect and a volume you cannot afford to store or search. So you don't. You sample traces, redact, aggregate the metrics, and keep exemplars of the interesting cases. The discipline is knowing which tier each question belongs to:

Metrics tell you something broke. Traces tell you where. Evals tell you whether the fix actually improved correctness. Confusing the tiers (trying to answer "is it correct?" from production metrics, or "is it up?" from an offline eval set) is a recurring wrong answer in this domain.

6. The Three Questions Observability Must Answer

Strip everything above to its intent and observability exists to answer three questions, in order:

7. Design Exercise

Objective: harden a multi-tenant agent platform against the four failure modes this lab covered: bloat, broken authorization, wrong protocol, and blind observability. About 60 minutes.

Scenario. You run a multi-tenant agent platform. Each tenant's agent needs three integrations: read from the tenant's CRM, write to the tenant's calendar, and issue invoices against the tenant's billing account. Today every tenant's agent runs as one shared service account, holds all three capabilities plus a dozen more inherited from a template, authenticates once at session start, and logs nothing but HTTP status codes. Volume is growing past what full-prompt logging can sustain.

1. Run a capability-bloat audit. List every tool on the agent. For each, name the tenant task it serves. Delete the ones you cannot name, and justify each deletion as least privilege, not cleanup. Which of the twelve inherited tools survive? Write the sentence that kills the rest.
2. Design per-user authorization. The shared service account is a confused deputy: any tenant's agent can reach any tenant's data. Redesign so each agent acts with the requesting tenant's permissions, and each of the three tool calls is authorized against that tenant's grants on the real arguments, in code, not in the prompt. Where exactly does the check live for the invoice tool?
3. Choose a protocol per integration. CRM read, calendar write, invoice issuance: pick MCP, direct API/CLI, or agent-to-agent for each, and defend it with the four criteria (reuse, ownership, trust boundary, discoverability). Which one, if any, actually crosses a trust boundary? Which is a bespoke integration that an MCP server would over-engineer?
4. Reconcile the SLO. Invoice issuance must feel instant to the user but need not be cheap; the nightly CRM sync must be cheap but no one is waiting on it. Assign a lever to each, and name the one you would not use for the interactive path and why (hint: it is 50% cheaper and completely wrong here).
5. Specify the trace schema. Write the exact fields you emit per turn: include request_id, the three-way token split, stop reason, per-tenant cost, and tool error flag. State which of the three questions (up / correct / affordable) each field serves.
6. Specify the sampling policy. You cannot log full prompts at this volume. Define what you sample, what you redact, what you aggregate, and which exemplars you always keep (hint: keep every refusal and every tool error, sample the rest). State what you will be unable to answer as a result, and why that is an acceptable trade.

Step 2 is the one the exam tests hardest. Step 6 is the one that keeps the platform legal.

Key Takeaways

• Least privilege removes the capability; it does not guard it. When one option deletes a tool and the others log, prompt, or upgrade around it, deleting wins: the guide's own Sample 1 deletes two tools.
• Capability bloat is a correctness problem, not only a security one: more tools measurably degrade tool-selection accuracy. Fewer, sharper tools is often the reliability fix.
• The agent must act with the permissions of the requesting user, not the application. A shared omnipotent service account is a confused deputy; authorize each tool call in code, on the real arguments, against that user's grants.
• Authorization belongs in the tool implementation, never the prompt. A prompt rule is a request an injection overrides; a code check is a control it cannot.
• Protocol selection: MCP for reusable, discoverable, N-to-M surfaces; direct API/CLI for a single bespoke integration you own; agent-to-agent only when a real trust boundary is crossed, and now you own two audit trails. The MCP connector needs both mcp_servers and mcp_toolset.
• Justify configs as SLO + lever + cost. Streaming moves perceived latency only; caching moves cost and latency together; batching is 50% cheaper, latency-destroying, and returns results UNORDERED: key by custom_id.
• input_tokens is the uncached remainder only: sum it with cache-read and cache-creation tokens or your cost math is wrong. Always log response._request_id (public despite the underscore).
• At scale you cannot log everything: sample traces, redact, aggregate metrics, keep exemplars. Distinguish metrics (cheap, aggregate, alerting) from traces (expensive, sampled, debugging) from evals (offline, labelled, correctness). Most teams answer "is it up?" and are blind to "is it correct?"