{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Lab 9 Self-Driven Lab*\n",
    "\n",
    "# Stakeholder Communication & Lifecycle Management\n",
    "\n",
    "**Objective:** run the communication and lifecycle work for the hospital discharge-summary system: discovery, an ADR, SLIs and an SLO, a runbook, a demo plan, and the one sentence that wins the pilot.\n",
    "\n",
    "## Challenge Outline\n",
    "\n",
    "Build a complete notebook that demonstrates the following outcomes:\n",
    "\n",
    "- **Run the discovery interview:** write the six questions you would ask the Chief Medical Officer, drawn from the discovery framework, including at least one doomed-project question.\n",
    "- **Write one complete ADR for the model-tier decision:** choose among claude-opus-4-8, claude-sonnet-5, and claude-haiku-4-5, with at least two rejected alternatives, one accepted negative consequence, and a revisit trigger.\n",
    "- **Define three SLIs and one SLO with an error budget:** define three SLIs (at least one quality-facing), one SLO with an error budget, and state which number goes in an SLA and which you refuse to contract, and why.\n",
    "- **Draft the first three rows of the runbook:** write three runbook rows (symptom, first check, escalation), covering at least a latency breach and a quality-drop symptom.\n",
    "- **Write the demo plan:** one sentence on what data you demo on and why it is not curated.\n",
    "- **The sentence for the skeptical physician:** write the one sentence answering \"What happens when it's wrong?\" containing a number, a comparison to current practice, and the gate that protects the patient.\n",
    "\n",
    "Your solution should include enough code, output, or written observations to prove each outcome worked. Keep the notebook focused on final behavior and evidence rather than a guided walkthrough.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Student Workspace\n",
    "\n",
    "Use the sections below to build your solution. Each section maps to one required outcome from the challenge outline.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 1: Run the discovery interview\n",
    "\n",
    "write the six questions you would ask the Chief Medical Officer, drawn from the discovery framework, including at least one doomed-project question.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 1: Run the discovery interview\n",
    "# Add your implementation, outputs, or notes here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2: Write one complete ADR for the model-tier decision\n",
    "\n",
    "choose among claude-opus-4-8, claude-sonnet-5, and claude-haiku-4-5, with at least two rejected alternatives, one accepted negative consequence, and a revisit trigger.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 2: Write one complete ADR for the model-tier decision\n",
    "# Add your implementation, outputs, or notes here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 3: Define three SLIs and one SLO with an error budget\n",
    "\n",
    "define three SLIs (at least one quality-facing), one SLO with an error budget, and state which number goes in an SLA and which you refuse to contract, and why.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 3: Define three SLIs and one SLO with an error budget\n",
    "# Add your implementation, outputs, or notes here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 4: Draft the first three rows of the runbook\n",
    "\n",
    "write three runbook rows (symptom, first check, escalation), covering at least a latency breach and a quality-drop symptom.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 4: Draft the first three rows of the runbook\n",
    "# Add your implementation, outputs, or notes here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 5: Write the demo plan\n",
    "\n",
    "one sentence on what data you demo on and why it is not curated.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 5: Write the demo plan\n",
    "# Add your implementation, outputs, or notes here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 6: The sentence for the skeptical physician\n",
    "\n",
    "write the one sentence answering \"What happens when it's wrong?\" containing a number, a comparison to current practice, and the gate that protects the patient.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 6: The sentence for the skeptical physician\n",
    "# Add your implementation, outputs, or notes here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verification Notes\n",
    "\n",
    "Summarize the evidence that each part worked. Capture API signals, validation outcomes, errors, recovery behavior, cost observations, or comparisons required by this lab.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Verification notes\n",
    "# Record the evidence that proves each lab outcome worked.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Answer Key\n",
    "\n",
    "The cells below contain the completed reference implementation/content for this lab. Use this section only after attempting the self-driven lab.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Lab 9 Answer Key*\n",
    "\n",
    "# Stakeholder Communication & Lifecycle Management\n",
    "\n",
    "**Objective:** run the communication and lifecycle work for the hospital discharge-summary system: discovery, an ADR, SLIs and an SLO, a runbook, a demo plan, and the one sentence that wins the pilot.\n",
    "\n",
    "This is a communication domain, not a coding domain. The deliverables here are documents, not programs, so most cells in this notebook are worked artifacts in markdown. The one code cell is a machine-readable SLO definition: a document, not a program, exactly as the lab teaches.\n",
    "\n",
    "**Scenario recap:** 900 discharges/day across 11 facilities, PHI under HIPAA, physicians abandon any tool slower than 20 seconds, and every summary is signed by the attending physician. Compliance content is architectural guidance, not legal advice.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Run the discovery interview\n",
    "\n",
    "Discovery collects decisions and constraints, not feature lists. Each question should hunt for the answer that kills the project early, while it is cheap to kill.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The six questions for the CMO**, mapped to the framework themes:\n",
    "\n",
    "1. **Decision.** \"When a discharge summary is generated, what changes? Who acts on it, and what do they do differently than today?\" (If the answer is \"everyone benefits, generally,\" there is no decision, only a demo.)\n",
    "2. **Ground truth (doomed-project question).** \"How do you know today when a discharge summary written by a resident was wrong? Who arbitrates a dispute between the attending and the record?\" If the honest answer is \"we don't, really,\" there is no eval set and no way to defend quality later. That is a reason to stop, not a detail for the build phase.\n",
    "3. **Who loses (doomed-project question).** \"Who loses if this works? Whose hours, whose sign-off authority, whose budget line?\" The person who loses often owns the corpus or the approval you need. Find them now, not in month five.\n",
    "4. **Latency tolerance.** \"You said physicians abandon tools slower than 20 seconds. Is that measured or folklore? What happens at 21 seconds: retry, abandon, or route around?\"\n",
    "5. **Blast radius.** \"What is the cost of one wrong summary reaching a chart: a readmission, a lawsuit, a regulator? Who is accountable for it?\" (This sets the human gate; the attending signature is already that gate, so confirm it is real review, not rubber-stamping.)\n",
    "6. **Definition of done.** \"At 90 days, what number makes you renew: minutes saved per discharge, summary turnaround time, resident overtime? What number makes you cancel?\"\n",
    "\n",
    "The framework rows not asked directly (volume, regulatory regime, data residency) are already fixed by the brief: 900/day, HIPAA, PHI stays inside the covered entity's infrastructure under a BAA.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Write one complete ADR for the model-tier decision\n",
    "\n",
    "An ADR that lists no rejected alternative and no negative consequence is marketing. The 20-second latency bound is an architectural constraint on this choice, not a preference.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ADR-007: `claude-sonnet-5` for discharge-summary generation, not Opus and not Haiku**\n",
    "\n",
    "**Status:** Accepted. **Deciders:** Lead architect, CMIO, Compliance.\n",
    "\n",
    "**Context.** 900 discharges/day, each summary drafted from structured chart data plus unstructured notes. Physicians abandon any tool slower than 20 seconds end to end, and every summary is reviewed and signed by the attending, so the model output is a draft behind a mandatory human gate, never an unreviewed clinical record.\n",
    "\n",
    "**Decision.** Generate summaries with a single `claude-sonnet-5` call with structured output and required source citations back to the chart. Stream the response so the physician sees the draft forming well inside the 20-second bound.\n",
    "\n",
    "**Alternatives considered.**\n",
    "\n",
    "- **`claude-opus-4-8` (rejected).** Highest quality tier, but the task is grounded summarization of supplied chart data behind a mandatory physician gate, not open-ended reasoning. Opus buys latency and cost where the quality delta is absorbed by the human review that happens anyway. Kept as the escalation tier for the small set of complex, multi-admission histories flagged by length.\n",
    "- **`claude-haiku-4-5` (rejected).** Meets latency with room to spare, but groundedness on long, messy clinical notes is the SLI that protects trust, and shaving cost on the cheapest part of a physician-facing workflow is a false economy. The physician minute spent fixing a weak draft costs more than the token delta saves.\n",
    "\n",
    "**Consequences, positive.** p95 latency comfortably inside the bound with streaming; cost per discharge is predictable; attending edits become labelled data for the eval set.\n",
    "\n",
    "**Consequences, negative (accepted).** A mid-tier model will produce more drafts needing physician edits than Opus would; we accept a measurably higher edit rate in exchange for latency headroom and cost, because the signature gate exists by regulation regardless.\n",
    "\n",
    "**Revisit trigger.** Reopen if the attending edit rate exceeds 35% for two consecutive months (quality tier too low), or if p95 latency on the Sonnet tier drifts above 12 seconds (headroom eroded, re-evaluate the whole pipeline before upgrading tiers).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: Define three SLIs and one SLO with an error budget\n",
    "\n",
    "SLI is the measured number, SLO is your internal target for it, SLA is the external contract with penalties, and the error budget is the allowed shortfall you spend. Accuracy is an SLO on a frozen eval set, never an SLA on live traffic.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Three SLIs:**\n",
    "\n",
    "| SLI | Example objective | What it protects |\n",
    "|---|---|---|\n",
    "| p95 end-to-end draft latency | <= 15s at p95 (20s is the abandonment cliff; target with headroom) | Physicians keep using the tool |\n",
    "| Availability | 99.5% of requests succeed (non-5xx) monthly | The tool exists at discharge time |\n",
    "| Citation validity (quality-facing) | >= 98% of drafted statements carry a citation that resolves to a real chart entry | Trust, auditability, and the signature gate's workload |\n",
    "\n",
    "**The SLO with its error budget** is the latency objective, written as a machine-readable definition in the next cell.\n",
    "\n",
    "**What goes in the SLA and what does not.** Contract availability (99.5%) and p95 latency, because we control both. **Refuse to contract accuracy against physician edits.** The edit rate is not a clean correctness label: attendings edit for style, personal phrasing, and defensive documentation, and two attendings disagree with each other about what a good summary says. Signing an SLA on that number is contracting to overfit whichever physician reviewed last. Accuracy is tracked as an internal SLO on a frozen, adjudicated eval set.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "slo_definition = \"\"\"\n",
    "slo:\n",
    "  name: discharge-summary-latency\n",
    "  sli: end_to_end_draft_latency_seconds   # measured indicator\n",
    "  objective:\n",
    "    percentile: p95\n",
    "    threshold_seconds: 15.0                # 20s is the abandonment cliff; keep headroom\n",
    "    target: 0.99                           # met on 99% of eligible minutes\n",
    "  window: 30d_rolling\n",
    "  error_budget:\n",
    "    allowed_failure_ratio: 0.01            # 1% of the window may breach\n",
    "    on_exhaustion: freeze_feature_launches\n",
    "    owner: service-owner                   # decides how the budget is spent\n",
    "  note: >\n",
    "    Citation validity and edit-rate quality are tracked as SEPARATE objectives\n",
    "    on a frozen eval set. Accuracy is deliberately NOT an SLA on live traffic\n",
    "    (physician edits are disputed ground truth).\n",
    "\"\"\"\n",
    "\n",
    "with open(\"discharge_summary_slo.yaml\", \"w\") as f:\n",
    "    f.write(slo_definition)\n",
    "\n",
    "print(slo_definition)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: Draft the first three rows of the runbook\n",
    "\n",
    "The runbook is the artifact that decides whether the system survives your departure. Each row is a symptom, the first check that discriminates between causes, and the escalation path when the first check comes back clean.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Symptom | First check | Escalation |\n",
    "|---|---|---|\n",
    "| p95 draft latency breached the 15s SLO | Is the prompt-cache hit rate down, or did average chart size jump? Compare input token counts against the 30-day baseline | If cache and tokens are normal, check provider status and model tier health; page the service owner and spend error budget consciously |\n",
    "| Citation-validity SLI dropped below 98% | Did the chart-retrieval index rebuild fail, or did an EHR schema change break the source mapping? | If retrieval is healthy, suspect a prompt or model regression: freeze deploys, run the frozen eval suite, page the eval owner |\n",
    "| Attending edit rate spiking on one facility | Is it one facility (local template or specialty mix) or all eleven (system regression)? Compare per-facility edit rates | Single facility: engage that facility's clinical informatics lead. All facilities: treat as quality regression, freeze deploys, page the architect on record |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 5: Write the demo plan\n",
    "\n",
    "Never demo on curated data. A curated demo silently promises the system's best day as the baseline, a debt you spend two quarters repaying.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Demo plan:** We demo on a random sample of 20 de-identified discharges pulled from last week's live volume across at least 3 facilities, selected by a script in front of the audience, because an honest, slightly imperfect demo sets an expectation we can meet every week after, and a curated one sets an expectation we can only miss.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6: The sentence for the skeptical physician\n",
    "\n",
    "A number without a comparison is anxiety; a comparison without a lever is helplessness. This single sentence is the entire domain in miniature.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The sentence:**\n",
    "\n",
    "> \"About 2 drafts in 100 contain a statement the chart does not support, which is fewer unsupported statements than we measured in resident-written summaries on the same charts, and none of them reach the record because every summary still requires your review and signature before it is filed.\"\n",
    "\n",
    "Number: 2 in 100. Comparison: measured against current resident-written practice on the same charts. Gate: the attending signature that already exists, so the workflow protects the patient by construction rather than by promise.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}