---
title: "Backend"
source: "backend/README.md"
---

> Part of [Backend](/backend/llms.txt). Every page: [llms.txt](/llms.txt).

# Shuke backend

Cloudflare Workers API for the Shuke spoken-expression trainer: auth, sessions, the audio → ASR → assessment
pipeline, settlement/rewards, growth, paywall/StoreKit and analytics. Implements `docs/api/openapi.yaml` (binding) and
ADR-001 (product rules, incl. D13 multilingual) with foundations for ADR-002 (content operations).

| Piece | Where |
|---|---|
| Router (Hono), middleware (request id, locale, auth, admin, idempotency, CORS, errors) | `src/app.ts`, `src/middleware.ts`, `src/routes/*` |
| Contract mirror (TS types + zod request schemas) | `src/contract/` |
| Product rules as pure functions (scores, bands, XP, streak, levels, ability EMA, comparison, validity, evidence) | `src/domain/` |
| Per-language tables (validity floor, reading rate, length unit, ASR hint, TTS voices, rubric notes) | `src/lang/` |
| API string catalogs (`en`, `zh-Hans`) | `src/i18n/` |
| Dynamic AI providers + rubric | `src/ai/` (`rubric/rubric-v1.md`) |
| Queue consumer (transcribe → validity → assess → comparison; exemplar TTS; purge jobs); shared pure steps in `steps.ts` | `src/pipeline/` |
| Dashboard Playground (several scoring models on one recording, issue #33) | `src/playground/`, `src/routes/admin-playground.ts` |
| Settlement, stats replay, deletion, StoreKit, cron | `src/services/` |
| Content operations (ADR-002): sources/fetcher/extractor, generation, QA, lifecycle, health, reports | `src/content/`, admin API `src/routes/admin*.ts` |
| D1 schema | `migrations/0001_init.sql` … `migrations/0004_playground.sql` |
| Seeds (topics, micro-skills, achievements, 14 zh-Hans + 9 en articles, pilot benchmarks) | `seeds/` |
| Assessment eval (5 labelled fixtures + runner) | `assessment-eval/` |

Bindings: D1 `DB`, R2 `AUDIO`, KV `CONFIG` (feature flags, AI provider chains) and `IDEMPOTENCY` (idempotent replays,
rate-limit counters), Queue `PIPELINE` (producer + consumer, DLQ `shuke-pipeline-dlq`), Workers AI `AI`, optional
Vectorize `CONTENT_INDEX` and KV `WAITLIST`, four crons (`0 18 * * *` cohort benchmarks, `30 18 * * *` retention purge +
deletion jobs, `5 * * * *` content sources + inventory, `0 19 * * *` article health).

## Setup

```bash
cd backend
pnpm install
cp .dev.vars.example .dev.vars      # JWT_SECRET, ADMIN_KEY, optional provider keys
pnpm seed                           # local D1: migrations + seeds (.wrangler/seed.sql)
pnpm dev                            # wrangler dev on http://localhost:8787
pnpm smoke                          # end-to-end curl run (starts wrangler dev if needed)
```

Without provider keys (and `ENVIRONMENT != production`) the default chains use the deterministic **mock** provider:
every result is labelled `mock/<model>` in `model_ref`/`asr_ref`/`tts_ref`. The mock ASR reads an optional
`MOCKASR{"transcript": "...", "language": "en"}` marker embedded in the uploaded file (`scripts/make-audio.mjs`,
`src/lib/mp4.ts#buildMp4`), so tests and the smoke script can drive any scenario. The mock is disabled in production.

Checks (also run in CI, `.github/workflows/backend.yml`):

```bash
pnpm typecheck   # Workers + Node tsconfigs (regenerates the embedded rubric first)
pnpm lint        # eslint, incl. a rule forbidding `=== "zh-Hans"` branches outside src/lang
pnpm test        # vitest in the Workers runtime (@cloudflare/vitest-plugin, Miniflare D1/R2/KV/Queues)
```

Tests validate real responses against the OpenAPI component schemas of both specs (`test/openapi.ts`) and assert that
the registered routes equal the paths in `docs/api/openapi.yaml` ∪ `docs/api/openapi-admin.yaml` (plus the documented
non-contract routes in `src/app.ts`). Content tests use recorded fixtures (`test/fixtures/content/`), never the network.

## Deploy

```bash
wrangler d1 create shuke                       # put the id into wrangler.jsonc
wrangler kv namespace create CONFIG            # and IDEMPOTENCY
wrangler r2 bucket create shuke-audio
wrangler queues create shuke-pipeline && wrangler queues create shuke-pipeline-dlq
wrangler d1 migrations apply shuke --remote
pnpm tsx scripts/build-seed.ts && wrangler d1 execute shuke --remote --file=.wrangler/seed.sql
wrangler secret put JWT_SECRET                 # and ADMIN_KEY, APPLE_*, *_API_KEY, R2_* as needed
pnpm deploy
```

### Secrets / vars

| Name | Purpose |
|---|---|
| `JWT_SECRET` | HS256 access tokens (1 h), HMAC for signed media URLs, AES key for stored Apple refresh tokens |
| `ADMIN_KEY` | `X-Admin-Key` for `/v1/admin/*` (Cloudflare Access hook in `src/middleware.ts#verifyAccessJwt`) |
| `APPLE_BUNDLE_ID`, `APPLE_TEAM_ID`, `APPLE_KEY_ID`, `APPLE_PRIVATE_KEY` | Sign in with Apple (identity token audience; code exchange + token revocation on account deletion) |
| `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`, `DEEPGRAM_API_KEY`, `ELEVENLABS_API_KEY` | AI providers (entries without a key are skipped) |
| `R2_ACCOUNT_ID`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET_NAME` | Optional: presigned R2 S3 URLs. Without them uploads/playback go through HMAC-signed worker routes (`PUT /v1/uploads/:id`, `GET /v1/media`) |
| `ENVIRONMENT`, `MIN_APP_VERSION`, `CORS_ORIGINS`, `TERMS_URL`, `PRIVACY_URL`, `PUBLIC_BASE_URL` | vars |
| `AI_DEFAULT_CHAIN=mock` | force the mock chain as the code default |
| `CONTENT_FETCH_ENABLED` | `"true"` lets content sources download pages (var, default `"false"`) |
| `ACCESS_TEAM_DOMAIN`, `ACCESS_AUD` | Cloudflare Access for `/v1/admin/*` (team domain, application AUD tag) |
| `RATE_LIMIT_REPORT` | `POST /v1/reports` per user, default `20/86400` |

## Dynamic AI providers (ADR-001 D11, ADR-002)

Capabilities: `asr`, `llm` (assessment), `content_llm`, `qa_llm`, `tts`, `embeddings`. The live config is KV
`ai:providers` (V2 shape of `docs/api/openapi-admin.yaml`), edited via `GET/PUT /v1/admin/ai-providers`; every PUT is
written to `ai_provider_config` history and `audit_log`. When KV is empty the code default applies
(`src/ai/config.ts`). Each chain is tried in order; the runner falls back on transport errors, 5xx, timeouts, 429 and
unconfigured entries, and **stops** on content-policy, client (4xx) and invalid-output errors. Every attempt is
recorded in `provider_calls` (capability, provider, model, latency, tokens, error kind, fallback flag).

```bash
curl -X PUT localhost:8787/v1/admin/ai-providers -H "X-Admin-Key: $ADMIN_KEY" -H 'Content-Type: application/json' -d '{
  "asr": { "chain": [
    { "provider": "workers-ai", "model": "@cf/openai/whisper-large-v3-turbo", "enabled": true, "timeout_ms": 90000 },
    { "provider": "deepgram", "model": "nova-3", "enabled": true } ] },
  "llm": { "chain": [
    { "provider": "anthropic", "model": "claude-sonnet-5-5", "enabled": true, "options": { "effort": "medium" } },
    { "provider": "openai", "model": "gpt-4.1", "enabled": true } ] },
  "tts": { "chain": [
    { "provider": "openai", "model": "gpt-4o-mini-tts", "enabled": true, "options": { "voices": { "en": "alloy", "zh-Hans": "alloy" } } },
    { "provider": "workers-ai", "model": "@cf/myshell-ai/melotts", "enabled": true, "options": { "voices": { "en": "en", "zh-Hans": "zh" } } } ] },
  "price_table": [{ "provider": "anthropic", "model": "claude-sonnet-5-5", "input_per_1k": 0.002, "output_per_1k": 0.01 }]
}'
```

Adapters: Workers AI (Whisper large-v3-turbo, Llama 3.3 JSON-schema mode, MeloTTS), OpenAI (`whisper-1` verbose JSON
with word timestamps / `gpt-4o-transcribe`, Responses API with strict JSON schema, speech), Anthropic (official SDK,
strict tool for structured output, `tool_choice: auto`, optional server-side refusal fallback), Google Gemini
(`responseJsonSchema`), Deepgram (`nova-3`), ElevenLabs. Every result carries `model_ref = provider/model`.

### Adding a provider

1. Implement `AsrProvider` / `LlmProvider` / `TtsProvider` (`src/ai/types.ts`) in `src/ai/providers/<name>.ts`;
   map failures with `providerFetch` / `ProviderError` kinds so fallback works.
2. Register it in `src/ai/registry.ts` and list it under the interface family in `SUPPORTED` (`src/ai/config.ts`).
3. Add the id to `PROVIDER_IDS` (`src/contract/types.ts`) and to the `provider` enum in both OpenAPI files.
4. A new *capability* only needs an entry in `CAPABILITY_FAMILY` plus a default chain.

## Assessment (rubric-v1)

`src/ai/rubric/rubric-v1.md` is the language-neutral system prompt; `src/lang` appends per-language notes (fillers,
excerpt length, paraphrase rules). It is embedded as `rubric-v1.generated.ts` by `scripts/gen-rubric.mjs` (a test
checks they match). The server, not the model, enforces ADR-001 D3: overall = 0.35/0.30/0.20/0.15, bands, evidence
bounds `[0, duration]` + fuzzy excerpt match (character-bigram Dice ≥ 0.6) — invalid evidence is dropped and the
dimension shown as "insufficient evidence" without lowering its score — every key point present in
`key_point_coverage`, `achieved_previous_action` null without a previous action, confidence lowered (never raised)
when ASR confidence is poor or evidence was dropped. Fluency input is a pause map from ASR timestamps. Round 2's
`comparison` is computed server-side; `shorter_but_lost_points` wins over `improved` whenever a covered key point
or key_info was lost.

```bash
pnpm eval                                                   # mock, offline
ANTHROPIC_API_KEY=… pnpm eval --provider anthropic --model claude-sonnet-5-5 --json
```

prints each fixture's scores next to its human labels (coverage, band, acceptable one-actions) for manual review.

## Languages (ADR-001 D13)

UI strings are resolved from `Accept-Language` → the user's `ui_locale` → device locale captured at sign-in → `en`.
Content strings (assessment notes, one action, exemplar) are in the article's language. Practice languages filter the
daily quest, quick picks, library and topic counts. Adding a language = an entry in `src/lang`, a catalog in
`src/i18n`, localized seed fields and articles; ESLint rejects `=== "zh-Hans"` elsewhere.

## Product rules implemented (ADR-001)

- **Validity (D2)**: ≥ 20 s, ≥ 40 CJK chars (zh-Hans) / ≥ 30 words (en), speech ratio ≥ 0.35, detected language =
  article language, ASR confidence ≥ 0.35 → else `unreliable` with a reason; no score, no XP, free re-record.
- **Settlement (D4)**: one D1 batch; the ledger row `{session_id}:complete` (UNIQUE) is written first, so repeats
  return the stored settlement with `replayed: true` (UI strings re-rendered for the request locale). XP 15/20/10 +10
  improvement, 50 % (floored) for an article completed in the last 7 days, daily cap 165 on session XP in the user's tz.
  Weekly quests pay 30/30/40 once per week (`{user}:{week_start}:{quest_id}`), outside the cap. Diagnostic = round 1 +
  completion (25 XP).
- **Streak**: +1 on the first completed session of a local day; missed days consume banked freezes (1 per 7-day
  milestone, max 2); `GET /me` shows 0 once a streak is broken. *Interpretation of the 20 h guard*: a second increment
  within 20 h is refused only when the user's time zone changed since the previous increment (the anti-gaming case the
  ADR describes); otherwise a 23:00 → 08:00 next-day session would break honest streaks. The day is still recorded.
- **Ability**: EMA α = 0.3 over the per-session best attempt (all four dimensions + overall), `ability-v1`, shown from
  3 samples. Growth with `?language=` replays the EMA over that language's sessions only; a weekly point needs ≥ 2
  sessions (`min_sessions_per_point`), the running week is `is_partial`.
- **Diagnostic sessions** stay in stage `round1` once round 1 is assessed (no round 2); `POST /complete` settles them,
  generates the 14-day plan (target = diagnostic one-action, baseline = diagnostic overall) and sets onboarding to
  `plan_ready` (`complete` once StoreKit reports trial/active).
- **Deletion (D6)**: `DELETE /sessions/{id}` is synchronous; `/me/training-data` and `/me` delete D1 rows immediately
  and purge R2 through a `deletion_jobs` row processed on the queue (cron retries). Ledger rows of deleted sessions are
  tombstoned; all stats are recomputed by replaying the remaining sessions.
- **Retention**: nightly purge of audio older than the user's 7/30/365-day setting (`audio_purged_at`, playback 410).
- **Hard paywall (D8)**: `requireEntitlement` (src/middleware.ts) returns 402 `entitlement_required` on home, library,
  session create/list, growth, achievements, weekly quests and plan once onboarding is `plan_ready`/`complete` and the
  entitlement is not trial/active/grace; `/me*`, paywall, entitlements, subscriptions, config, events, onboarding and the
  user's own diagnostic session/attempts/article stay open. Locked difficulty tiers return 403 `article_locked`.
- **Paywall benchmarks (D8)**: cohort table per (weekly_days × baseline bucket) from the nightly job; any cell with < 30 samples
  makes the whole table `pilot_benchmark` with values from `seeds/pilot_benchmarks.json`. Social proof is null below 100.
- **RevenueCat** (purchase layer, `src/services/revenuecat.ts`): `POST /subscriptions/revenuecat/sync` and the webhook
  (`Authorization` = `REVENUECAT_WEBHOOK_AUTH`, idempotent on `event.id`) both re-read `GET /v1/subscribers/{user_id}` with
  `REVENUECAT_API_KEY` and map entitlement `pro` to trial/active/grace/expired/revoked. A TRANSFER re-syncs both users.
- **StoreKit 2** (deprecated path for pre-RevenueCat builds): JWS x5c chain verified to the pinned Apple Root CA – G3 (SHA-256
  `63343ABFB89A6A03EBB57E9B3F5FA7BE7C4F5C756F3017B3A8C488C3653E9179`), Apple marker OIDs on intermediate/leaf, validity
  windows, then the ES256 signature. Notifications V2 are idempotent on `notificationUUID`.

## Content operations (ADR-002)

Goal: every (language × topic × difficulty) cell stays stocked with legally safe, retell-worthy articles with almost no
human in the loop. Everything is driven by `content_jobs` rows (the dashboard's job log) processed on the `PIPELINE`
queue (`{ stage: "content", job_id }`); `PIPELINE_INLINE=true` (tests) runs them inline. Admin API =
`docs/api/openapi-admin.yaml` (binding), implemented in `src/routes/admin.ts` + `src/routes/admin-ops.ts`.

```
cron :05 hourly ─┬─ sources whose schedule_cron fired since last_run_at ─→ fetch job ─→ source_items (new|skipped|failed)
                 └─ fill_gaps: per cell target − unseen_available − in_pipeline (≤ batch_max) ─→ generate jobs
generate job ─→ content_llm draft (structured output, prompt library) ─→ article version (status qa) ─→ QA
QA ─→ deterministic checks + embeddings duplicate check + qa_llm rubric ─→ quality_reports
   ─→ auto_publish (+ audit sample) | needs_review (review item) | reject (reason kept for tuning)
cron 19:00 UTC ─→ health job: article_health from sessions/assessments/votes/reports ─→ retire < threshold (≥ 20 sessions) + replacement job
POST /v1/reports (rights|harmful on an opened article) ─→ unpublish now + `reported` review item
```

**Pieces** (`src/content/`): `cron.ts` (5-field UTC cron matcher; `isDue` = fired in (last_run, now]), `feeds.ts`
(RSS/Atom, sitemap + index, HTML list page via HTMLRewriter CSS selector, JSON API via a jq-like `selector` such as
`.data.results[]`), `extract.ts` (DOM-free Readability-style extractor), `fetcher.ts` (URL-hash + simhash dedupe, size
bounds 150–4000 words / 250–8000 chars by the source language's unit, license inherited from the source),
`inventory.ts` (cells + pure `planFillGaps`), `generate.ts`, `prompts/` (versioned prompt library), `qa.ts`,
`vectors.ts` (Vectorize `CONTENT_INDEX` when bound, else D1 `content_vectors` + cosine in JS; un-indexed published
articles are back-filled during QA), `articles.ts` (versions + lifecycle table), `review.ts`, `health.ts`,
`reports.ts`, `aggregates.ts` (overview/usage/quality), `runner.ts` (queue consumer + cron entry points).

**Article versions.** Versions stay in `articles` (sessions pin `article_id + version`). `is_head` marks the latest
version (what operators see; its status is the lifecycle status), `is_current` the version the app serves when
`published`. Editing a live article creates a new head that is *not* served until it is published, so the app keeps
the old version during QA/review. `article_versions` stores a full snapshot + provenance per version. Content edits
(paragraphs / key points) re-run QA; `publish_now` skips QA and is audited as `article.publish_override`.

**Lifecycle** (`TRANSITIONS` in `src/content/articles.ts`, mirrors `dashboard/src/lib/lifecycle.ts`): publish ←
needs_review|rejected; reject ← needs_review|qa|draft (reason required); retire ← published|unpublished; unpublish ←
published; republish ← unpublished; regenerate ← needs_review|rejected|unpublished|retired (new version of the same
article; the operator note is fed to the prompt); request_review ← published|rejected. Anything else → 409
`illegal_transition`. Review decisions per kind follow `DECISIONS` (409 `illegal_decision`, `already_decided`).

**QA checks** (`QualityCheck` ids): deterministic `length_in_tier` (ADR-001 D9 tiers per language, `src/lang`
`lengthTiers`), `key_points_count` (3–5), `language_match` (script detection), `no_contact_info`,
`no_private_names` (honorific + name heuristic per language), `attribution_present` (licensed only); embedding
`duplicate_check` (cosine > 0.92 ⇒ reject); LLM `argument_clarity`, `retell_worthiness`, `grounding`, `readability`
(each passes at ≥ the language's auto_publish threshold), `safety` (hard gate) and a soft `sensitive_politics` flag.
`qa_score = min(4 rubric dims)`. Decision: safety fail, duplicate or `qa_score < review_floor` → reject;
`qa_score ≥ auto_publish` with no soft flag → auto-publish; otherwise needs_review. Audit sampling is deterministic per
(article, version) at `audit_sample_rate`.

**Thresholds and targets** live in `generation_config` (`GET/PUT /v1/admin/generation/config`): per-language
`auto_publish` (80) / `review_floor` (60), `default_target` (20) + per-cell `targets`, `batch_max` (5 per cell per
run), `audit_sample_rate` (0.05), `retire_health_below` (40), `mix.ai_original` (0.5), `pair_generation`, `paused`,
`prompt_version`, `daily_cost_cap_usd` (20), `max_jobs_per_run` (25), `paused_reason`. Threshold changes show up in
`/admin/quality` `threshold_history` (from the audit log).

**Spend guard.** An inventory run enqueues at most `max_jobs_per_run` generate jobs across cells (highest shortfall
first). Before a generate/qa job is enqueued or run, today's (UTC) spend on `content_llm` + `qa_llm` + `embeddings`
(`provider_calls.cost_usd`, priced from the price table) is compared with `daily_cost_cap_usd`; at or above it the
pipeline pauses with `paused_reason = "cost_cap"` (remaining jobs of a run are not enqueued, queued ones fail with
"generation paused" and can be retried) and resumes by itself on the next UTC day. Three consecutive `content_llm`
chain failures pause with `provider_errors`. Both automatic pauses are audited as actor `system`; a manual pause is
`manual`; `PUT /generation/config` with `paused: false` clears the reason. The overview shows `paused_reason`,
`spend_today_usd` and `daily_cost_cap_usd`.

**Health score** (nightly, null below 20 sessions): `round(100 × (0.35·completion + 0.35·avg_key_info/100 +
0.30·helpful_rate)) − 5 × reports`, `helpful_rate` defaults to 0.7 without votes. Weekly points are kept in `trend`.

### Adding a source / license / prompt

- **Source**: `POST /v1/admin/sources` (or the dashboard) with `type` rss|sitemap|html|api, `url`, `license` (must be an
  allowed `licenses.id`; else 422 `license_not_allowed`; `OWNER-CLEARED` needs `owner_cleared: true`), `language`,
  `attribution_name`, `topic_hints` (topic ids used to match generation cells), `schedule_cron` (UTC), `selector` (CSS
  selector for html list pages, jq-like path for api). Starter sources (seeded): Wikinews en (CC BY 2.5), Wikipedia
  featured article feed (CC BY-SA 4.0), NASA news releases (US federal public domain). Fetching only happens when
  `CONTENT_FETCH_ENABLED="true"`.
- **License**: add a row to `licenses` in a new migration (`allowed`, `requires_attribution`, `share_alike`,
  `attribution_template` with `{title} {author} {source_name} {license_name} {license_url} {url} {retrieved_date}`).
  Disallowed licenses (ARR, NC, ND) stay listed with `allowed = 0` so the dashboard can explain refusals.
- **Prompt**: copy `src/content/prompts/v1.ts` to `v2.ts`, change `PROMPT_VERSION`, register it in
  `prompts/index.ts`, then set `prompt_version` in the generation config. System prompts are authored natively per
  content language (`DRAFT_SYSTEM[en|zh-Hans]`); adding a practice language means adding an entry there and in
  `src/lang` (`lengthTiers`, `bylines`, `privateNameRe`). Every article records `prompt_version` in its provenance.

### Cloudflare Access

Protect the dashboard and `https://<api>/v1/admin/*` with one Access application. Set `ACCESS_TEAM_DOMAIN`
(`<team>.cloudflareaccess.com`) and `ACCESS_AUD` (the application's AUD tag). The API verifies
`Cf-Access-Jwt-Assertion` (RS256 against `https://<team>/cdn-cgi/access/certs`, cached 1 h and refetched for an unknown
`kid`; `aud`, `iss`, `exp` checked) and records the `email` claim as the actor in `audit_log`. In production with Access
configured `X-Admin-Key` is refused; elsewhere it remains the dev path (actor `admin-key`).

### Costs

Each provider call is priced from the AI config `price_table` (`input_per_1k`, `output_per_1k`, `per_minute`;
`model: "*"` = provider default) into `provider_calls.cost_usd` with the `job_id` that made it; `content_jobs.cost_usd`
is the job's sum and `provenance.cost_usd` the generating job's cost. `/admin/overview` and `/admin/ai-providers/usage`
report cost per published article (content_llm + qa_llm + embeddings ÷ articles published in the window). Budget
levers: `daily_cost_cap_usd`, `max_jobs_per_run`, `batch_max`, targets, `paused`, `max_items_per_run`, cheaper chains
for `qa_llm`/`embeddings`. Without a price table entry calls are unpriced (cost null) and do not count toward the cap.

### Local runs

```bash
pnpm dev                                        # mock providers without API keys
pnpm content:run -- --mode fill_gaps --count 5  # enqueue + wait + summary (decisions, cost, review queue)
pnpm content:run -- --mode brief --brief brf_seed_en_career_1 --pair
pnpm smoke                                      # includes the admin section (fixture source → fetch → generate → review → app)
```

Mock `content_llm` drafts are deterministic per input (brief/source/cell) with sentence variety per job; markers in a
brief or operator note force a path: `[mock:strong]`, `[mock:weak]` (→ needs_review), `[mock:low]` (→ reject),
`[mock:unsafe]` (→ safety gate), `[mock:numbers]` (grounding flag), `[mock:short]` (outside the tier). Mock `qa_llm`
scores come from the draft's features; mock embeddings are hashed 3-shingles (256 dims).

Optional bindings: Vectorize `CONTENT_INDEX` (`wrangler vectorize create shuke-content --dimensions=1024
--metric=cosine` for bge-m3, plus a `language` metadata index) and the landing page's KV namespace as `WAITLIST`
(keys `email:<address>` → `{createdAt, locale}`) for `GET /v1/admin/waitlist/export`; without it the export is an empty
CSV with an `X-Shuke-Note` header.

## Playground (ADR-002 D4, issue #33)

Operators run the app's loop on a published article and compare 2–6 scoring models side by side. Admin API =
`docs/api/openapi-admin.yaml`, tag `playground` (binding), implemented in `src/routes/admin-playground.ts` with the
queue stages in `src/playground/runner.ts` and the response shape in `src/playground/store.ts`.

```
POST /admin/playground/runs ─→ playground_runs + 2 playground_rounds (awaiting_audio) + 2–6 playground_lanes (random labels)
PUT  …/rounds/{n}/audio ─→ R2 playground/<run_id>/r<n>.<ext> ─→ { stage: "playground_transcribe", run_id, round }
playground_transcribe ─→ the run's pinned ASR entry once ─→ processTranscript (same validity as the app)
   ├─ unreliable ─→ every result `skipped` (POST …/assess-anyway opens them)
   └─ one playground_results row per lane ─→ one { stage: "playground_assess", lane_id, round } message per lane
playground_assess ─→ assess() with the lane's pinned entry ─→ same post-processing as the app; round 2 compares with the lane's round 1
POST …/lanes/{id}/exemplar ─→ { stage: "playground_exemplar", lane_id, round } ─→ text by the lane's model, audio by the tts chain
```

- **Pinned models.** `runEntry` (`src/ai/chain.ts`) is one call to one entry with the chain's timeout, error mapping,
  `provider_calls` row and cost; `runChain` calls it per entry. `transcribe`, `generateJson`, `assess` and
  `generateExemplarText` take `opts.entry` to call exactly that entry (no fallback); a failure is still a `ChainError`.
- **Same scoring as the app.** `src/pipeline/steps.ts` holds the pure steps both pipelines call: `processTranscript`
  (segments, text, pause map, validity), `assessContext` and `roundComparison`.
- **Isolation.** The `playground_*` tables have no foreign keys into user tables, and runs never write users, sessions,
  attempts, assessments, XP, ability or stats. One lane's failure is stored on its result; the other lanes go on.
- **Recovery.** A delivery claims a pending result, and a running one only after its lease (lane timeout + 60 s), so
  overlapping deliveries never both call the model and only the claim owner's write lands. A transcribe redelivery
  re-drives lanes still pending. When a queue send fails after its commit, the route reverts (round `failed`, result
  `failed`, retryable) and answers 500. The hourly cron sweeps work left waiting over 15 min (`sweepPlayground`).
- **Blind.** Until a verdict or reveal, the API returns `model_ref: null` and `options: null` per lane,
  `assessment.model_ref = "hidden"`, no latency, usage, cost or `finished_at`, and errors with `details: { kind }` only.
  A round's results are released together: while any lane of the round is pending or running, every lane reports
  `running` with nothing in it. Lane and lane-assessment ids are fully random (`randomId`), and the audit trail records
  lane counts, never the lane ↔ label mapping.
- **Verdicts.** The first verdict given while blind is stored as `blind_verdict` and never changes; `verdict` is the
  latest one and may be replaced after the reveal.
- **Cost.** Every call is recorded with `provider_calls.job_id = <run id>` (`pg_…`); the run's `cost_usd` is that SUM.
  `NOT_PLAYGROUND_CALL` (`src/lib/provider-calls.ts`) leaves these rows out of every aggregate: overview cost and
  provider error rates and fallbacks, `/ai-providers/usage`, cost per published article and the spend guard.
- **Audio.** `audio/mp4`, `audio/webm`, `audio/mpeg`, `audio/wav`, at most 5 MB; caps 180 s / 120 s (ADR-001 D2).
  Duration: MP4 from the container, WAV from the RIFF header (`src/lib/wav.ts`), the header ignored. webm / mp3 use
  `X-Client-Duration-Ms` at upload, then after ASR the larger of that and the last segment's end
  (`client_checked`); over the cap there, the round fails with `audio_too_long` (stage transcribe).
- **Re-upload.** Replaces the round's audio, transcript, results and model answers; round 1 also resets round 2.
  Once the run has a verdict, a round that already has audio can't be uploaded again (409 `verdict_given`); the first
  upload of round 2 is still allowed. Uploads never clear verdicts or blind a revealed run again: a new attempt is a new run.
  409 `round_busy` while the round (for round 1, also round 2) is transcribing, assessing or has a model answer pending.
  Lane retry takes a failed result, or a round-2 skip once that lane's round 1 is done (nothing is re-queued by itself).
- **Retention.** The cron deletes playground recordings and model-answer audio after 30 days
  (`purgePlaygroundAudio`); playback then answers 410 `audio_purged`. Runs and results stay.
- **WebM and ASR.** OpenAI transcription and Deepgram document WebM support (the OpenAI adapter names the uploaded file
  after its type, since the API reads the format from the extension). Cloudflare's Whisper model page names no
  formats, so WebM is not yet confirmed on Workers AI: record `audio/mp4` where the browser supports it.
- **Mock mode.** `mockLlm` adds a deterministic per-model score bias (`mockModelBias`) and token estimates, so lanes
  differ and costs show without keys.

## Non-contract routes

`PUT /v1/uploads/:attempt_id` and `GET /v1/media` (HMAC-signed, back the URLs returned by the contract when R2 S3
keys are absent) and `GET /health`.