Projects

Live demo

NPC Coach. Every new patient call graded against the playbook.

Every new-patient phone call gets transcribed and graded against the practice's documented playbook, producing a coaching report that quotes the exact moment a coordinator missed a non-negotiable, next to their own 90-day average.

Speech + LLM pipeline · Call coaching 6 steps · synthetic data

Walk through it, six steps

Step 1 of 6

Two ingestion paths, one call record

Outbound calls arrive as MP3 drops on SFTP; inbound calls arrive from the call platform with a transcript already attached, which the pipeline reuses instead of paying to transcribe twice. Both normalize into one record, claim an idempotency key, and create a visible run row before any filtering, so no call can silently disappear.

IngestSynthetic data
$ ingest SYN-4471.mp3 (inbound · transcript attached)
idempotency key claimed · run row created (visible pre-filter)
classify: new-patient call ✓ · not voicemail · not wrong-number
transcript reused from platform → $0 transcription cost
Step 1 of 6

Every name, number, transcript and document in this tour is synthetic. The mechanisms, thresholds and architecture are real. The production system and its data stay private.

The problem.

New-patient coordinators are the practice's first advocates, and their calls set the tone for everything after. But nobody could listen to a thousand calls a month, so coaching was anecdotal. The design constraint came from the top: correctness beats conversion, and booking rate is explicitly not the success metric.

  • First live day 285 → 40 recordings ingested, relevant calls graded
  • Volume 1k to 5k calls/mo 5 to 10 minutes each, EN + ES
  • Tests 1,000+ across ~19k lines of Python
  • Built in 16 days including the React dashboard

How it fits together.

4 layers, left to right
  1. Ingest

    • SFTP MP3 drops + call-platform webhooks
    • One normalized call record, idempotency claims
    • Run rows created before any gate: nothing disappears
  2. Understand

    • Speech-to-Text v2 with diarization (outbound)
    • Platform transcripts reused (inbound), cost deduped
    • Call classification: new patient vs everything else
  3. Score

    • Gemini structured output with response schema
    • 6 criteria, 3 hard gates, weighted buckets 40/40/20
    • Verbatim evidence quotes with a relevance gate
  4. Deliver

    • React dashboard: runs, steppers, logs, analytics
    • Email reports to coordinator + manager
    • Firestore state · BigQuery audit · Terraform · ports-and-adapters demo mode

Where it landed.

A thousand-plus monthly calls now get the same rubric a human coach would apply on their best day, with receipts. The ports-and-adapters design means the whole pipeline runs on synthetic data in demo mode, which is exactly what this tour is. This is a PHI workload in production: no real transcripts, recordings, or caller details appear here, and coordinators are anonymized as C-1 through C-8.

Concepts

Speech-to-text + diarizationStructured LLM scoringHard gates vs weighted scoresIdempotent ingestionPorts and adaptersEvidence-quoted evals

Stack

PythonFastAPICloud RunSpeech-to-Text v2Vertex AI GeminiFirestoreBigQueryReactTerraform