Projects

Live demo

Marketing Warehouse. Five platforms in one place a model can read.

A BigQuery warehouse that pulls five ad and CRM platforms into one place every morning and joins ad spend all the way through to booked patients, with a self-documenting catalog that lets an LLM answer marketing questions in plain English, and weekly graded evals to prove it keeps doing so.

Data platform · LLM-ready analytics 6 steps · synthetic data

Walk through it, six steps

Step 1 of 6

Five connectors, one morning sync

Cloud Scheduler fires the connector jobs at 6 AM: search ads, social ads, call tracking, and the CRM, with one platform arriving via BigQuery Data Transfer. All connectors share retry, load, and secret code, packaged as a single container image run as serverless jobs.

Nightly runsSynthetic data
SourceRowsStatusFinished
Search ads (Platform A)1,204ok06:04
Social ads (Platform B)912ok06:06
Call tracking388ok06:07
CRM boards2,730ok06:11
Search ads (transfer)·managed transfer06:15
Step 1 of 6

Every name, number, transcript and document in this tour is synthetic. The mechanisms, thresholds and architecture are real. The production system and its data stay private.

The problem.

Marketing questions like which channel actually produces patients, not clicks, required manually exporting from five platforms and stitching spreadsheets. And once an LLM was allowed to answer questions over the data, a harder problem appeared: how do you know its answers stay right as the schema evolves?

  • Sources unified 5 search ads, social ads, call tracking, CRM, and more
  • Funnel linkage 98.5% patients joined to leads; 99.5% agreement vs exports
  • LLM evals 22 golden Qs weekly graded run, fails under 80%
  • Running cost $5 to 15/mo serverless jobs + BigQuery

How it fits together.

4 layers, left to right
  1. Ingest

    • 5 connectors, one Docker image, Cloud Run jobs
    • Shared retry, load and secret modules
    • 6 AM schedule + managed transfer
  2. Warehouse

    • BigQuery: raw → staging → marts (+ evals dataset)
    • Inline descriptions on every view and column
    • data_guide catalog written for LLM readers
  3. Consume

    • LLM access via MCP Toolbox
    • Looker Studio dashboards
    • Plain-English answers with mart citations
  4. Trust

    • 6:45 freshness check + z-score drift sentinel
    • 22 golden questions graded weekly, 80% gate
    • Glass monitoring app with per-node health

Where it landed.

Marketing spend, calls, leads, and booked patients live in one queryable place with 98.5% linkage, validated at 99.5% agreement against the manual exports it replaced, for five to fifteen dollars a month. The part worth stealing: treating the LLM's access as a product surface with its own weekly, graded, gated eval suite instead of assuming yesterday's good answers persist. All dollar figures shown are synthetic.

Concepts

Dimensional modelingLLM-over-SQLGolden-question evalsDrift detectionSelf-documenting schemasFreshness monitoring

Stack

PythonBigQueryCloud Run JobsCloud SchedulerMCP ToolboxLooker StudioDocker