Shukellms.txt

Backend

Shuke backend

Cloudflare Workers API for the Shuke spoken-expression trainer: auth, sessions, the audio → ASR → assessment pipeline, settlement/rewards, growth, paywall/StoreKit and analytics. Implements docs/api/openapi.yaml (binding) and ADR-001 (product rules, incl. D13 multilingual) with foundations for ADR-002 (content operations).

Piece Where
Router (Hono), middleware (request id, locale, auth, admin, idempotency, CORS, errors) src/app.ts, src/middleware.ts, src/routes/*
Contract mirror (TS types + zod request schemas) src/contract/
Product rules as pure functions (scores, bands, XP, streak, levels, ability EMA, comparison, validity, evidence) src/domain/
Per-language tables (validity floor, reading rate, length unit, ASR hint, TTS voices, rubric notes) src/lang/
API string catalogs (en, zh-Hans) src/i18n/
Dynamic AI providers + rubric src/ai/ (rubric/rubric-v1.md)
Queue consumer (transcribe → validity → assess → comparison; exemplar TTS; purge jobs); shared pure steps in steps.ts src/pipeline/
Dashboard Playground (several scoring models on one recording, issue #33) src/playground/, src/routes/admin-playground.ts
Settlement, stats replay, deletion, StoreKit, cron src/services/
Content operations (ADR-002): sources/fetcher/extractor, generation, QA, lifecycle, health, reports src/content/, admin API src/routes/admin*.ts
D1 schema migrations/0001_init.sql … migrations/0004_playground.sql
Seeds (topics, micro-skills, achievements, 14 zh-Hans + 9 en articles, pilot benchmarks) seeds/
Assessment eval (5 labelled fixtures + runner) assessment-eval/

Bindings: D1 DB, R2 AUDIO, KV CONFIG (feature flags, AI provider chains) and IDEMPOTENCY (idempotent replays, rate-limit counters), Queue PIPELINE (producer + consumer, DLQ shuke-pipeline-dlq), Workers AI AI, optional Vectorize CONTENT_INDEX and KV WAITLIST, four crons (0 18 * * * cohort benchmarks, 30 18 * * * retention purge + deletion jobs, 5 * * * * content sources + inventory, 0 19 * * * article health).

Setup

cd backend
pnpm install
cp .dev.vars.example .dev.vars      # JWT_SECRET, ADMIN_KEY, optional provider keys
pnpm seed                           # local D1: migrations + seeds (.wrangler/seed.sql)
pnpm dev                            # wrangler dev on http://localhost:8787
pnpm smoke                          # end-to-end curl run (starts wrangler dev if needed)

Without provider keys (and ENVIRONMENT != production) the default chains use the deterministic mock provider: every result is labelled mock/<model> in model_ref/asr_ref/tts_ref. The mock ASR reads an optional MOCKASR{"transcript": "...", "language": "en"} marker embedded in the uploaded file (scripts/make-audio.mjs, src/lib/mp4.ts#buildMp4), so tests and the smoke script can drive any scenario. The mock is disabled in production.

Checks (also run in CI, .github/workflows/backend.yml):

pnpm typecheck   # Workers + Node tsconfigs (regenerates the embedded rubric first)
pnpm lint        # eslint, incl. a rule forbidding `=== "zh-Hans"` branches outside src/lang
pnpm test        # vitest in the Workers runtime (@cloudflare/vitest-plugin, Miniflare D1/R2/KV/Queues)

Tests validate real responses against the OpenAPI component schemas of both specs (test/openapi.ts) and assert that the registered routes equal the paths in docs/api/openapi.yaml ∪ docs/api/openapi-admin.yaml (plus the documented non-contract routes in src/app.ts). Content tests use recorded fixtures (test/fixtures/content/), never the network.

Deploy

wrangler d1 create shuke                       # put the id into wrangler.jsonc
wrangler kv namespace create CONFIG            # and IDEMPOTENCY
wrangler r2 bucket create shuke-audio
wrangler queues create shuke-pipeline && wrangler queues create shuke-pipeline-dlq
wrangler d1 migrations apply shuke --remote
pnpm tsx scripts/build-seed.ts && wrangler d1 execute shuke --remote --file=.wrangler/seed.sql
wrangler secret put JWT_SECRET                 # and ADMIN_KEY, APPLE_*, *_API_KEY, R2_* as needed
pnpm deploy

Secrets / vars

Name Purpose
JWT_SECRET HS256 access tokens (1 h), HMAC for signed media URLs, AES key for stored Apple refresh tokens
ADMIN_KEY X-Admin-Key for /v1/admin/* (Cloudflare Access hook in src/middleware.ts#verifyAccessJwt)
APPLE_BUNDLE_ID, APPLE_TEAM_ID, APPLE_KEY_ID, APPLE_PRIVATE_KEY Sign in with Apple (identity token audience; code exchange + token revocation on account deletion)
OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, DEEPGRAM_API_KEY, ELEVENLABS_API_KEY AI providers (entries without a key are skipped)
R2_ACCOUNT_ID, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET_NAME Optional: presigned R2 S3 URLs. Without them uploads/playback go through HMAC-signed worker routes (PUT /v1/uploads/:id, GET /v1/media)
ENVIRONMENT, MIN_APP_VERSION, CORS_ORIGINS, TERMS_URL, PRIVACY_URL, PUBLIC_BASE_URL vars
AI_DEFAULT_CHAIN=mock force the mock chain as the code default
CONTENT_FETCH_ENABLED "true" lets content sources download pages (var, default "false")
ACCESS_TEAM_DOMAIN, ACCESS_AUD Cloudflare Access for /v1/admin/* (team domain, application AUD tag)
RATE_LIMIT_REPORT POST /v1/reports per user, default 20/86400

Dynamic AI providers (ADR-001 D11, ADR-002)

Capabilities: asr, llm (assessment), content_llm, qa_llm, tts, embeddings. The live config is KV ai:providers (V2 shape of docs/api/openapi-admin.yaml), edited via GET/PUT /v1/admin/ai-providers; every PUT is written to ai_provider_config history and audit_log. When KV is empty the code default applies (src/ai/config.ts). Each chain is tried in order; the runner falls back on transport errors, 5xx, timeouts, 429 and unconfigured entries, and stops on content-policy, client (4xx) and invalid-output errors. Every attempt is recorded in provider_calls (capability, provider, model, latency, tokens, error kind, fallback flag).

curl -X PUT localhost:8787/v1/admin/ai-providers -H "X-Admin-Key: $ADMIN_KEY" -H 'Content-Type: application/json' -d '{
  "asr": { "chain": [
    { "provider": "workers-ai", "model": "@cf/openai/whisper-large-v3-turbo", "enabled": true, "timeout_ms": 90000 },
    { "provider": "deepgram", "model": "nova-3", "enabled": true } ] },
  "llm": { "chain": [
    { "provider": "anthropic", "model": "claude-sonnet-5-5", "enabled": true, "options": { "effort": "medium" } },
    { "provider": "openai", "model": "gpt-4.1", "enabled": true } ] },
  "tts": { "chain": [
    { "provider": "openai", "model": "gpt-4o-mini-tts", "enabled": true, "options": { "voices": { "en": "alloy", "zh-Hans": "alloy" } } },
    { "provider": "workers-ai", "model": "@cf/myshell-ai/melotts", "enabled": true, "options": { "voices": { "en": "en", "zh-Hans": "zh" } } } ] },
  "price_table": [{ "provider": "anthropic", "model": "claude-sonnet-5-5", "input_per_1k": 0.002, "output_per_1k": 0.01 }]
}'

Adapters: Workers AI (Whisper large-v3-turbo, Llama 3.3 JSON-schema mode, MeloTTS), OpenAI (whisper-1 verbose JSON with word timestamps / gpt-4o-transcribe, Responses API with strict JSON schema, speech), Anthropic (official SDK, strict tool for structured output, tool_choice: auto, optional server-side refusal fallback), Google Gemini (responseJsonSchema), Deepgram (nova-3), ElevenLabs. Every result carries model_ref = provider/model.

Adding a provider

  1. Implement AsrProvider / LlmProvider / TtsProvider (src/ai/types.ts) in src/ai/providers/<name>.ts; map failures with providerFetch / ProviderError kinds so fallback works.
  2. Register it in src/ai/registry.ts and list it under the interface family in SUPPORTED (src/ai/config.ts).
  3. Add the id to PROVIDER_IDS (src/contract/types.ts) and to the provider enum in both OpenAPI files.
  4. A new capability only needs an entry in CAPABILITY_FAMILY plus a default chain.

Assessment (rubric-v1)

src/ai/rubric/rubric-v1.md is the language-neutral system prompt; src/lang appends per-language notes (fillers, excerpt length, paraphrase rules). It is embedded as rubric-v1.generated.ts by scripts/gen-rubric.mjs (a test checks they match). The server, not the model, enforces ADR-001 D3: overall = 0.35/0.30/0.20/0.15, bands, evidence bounds [0, duration] + fuzzy excerpt match (character-bigram Dice ≥ 0.6) — invalid evidence is dropped and the dimension shown as "insufficient evidence" without lowering its score — every key point present in key_point_coverage, achieved_previous_action null without a previous action, confidence lowered (never raised) when ASR confidence is poor or evidence was dropped. Fluency input is a pause map from ASR timestamps. Round 2's comparison is computed server-side; shorter_but_lost_points wins over improved whenever a covered key point or key_info was lost.

pnpm eval                                                   # mock, offline
ANTHROPIC_API_KEY=… pnpm eval --provider anthropic --model claude-sonnet-5-5 --json

prints each fixture's scores next to its human labels (coverage, band, acceptable one-actions) for manual review.

Languages (ADR-001 D13)

UI strings are resolved from Accept-Language → the user's ui_locale → device locale captured at sign-in → en. Content strings (assessment notes, one action, exemplar) are in the article's language. Practice languages filter the daily quest, quick picks, library and topic counts. Adding a language = an entry in src/lang, a catalog in src/i18n, localized seed fields and articles; ESLint rejects === "zh-Hans" elsewhere.

Product rules implemented (ADR-001)

  • Validity (D2): ≥ 20 s, ≥ 40 CJK chars (zh-Hans) / ≥ 30 words (en), speech ratio ≥ 0.35, detected language = article language, ASR confidence ≥ 0.35 → else unreliable with a reason; no score, no XP, free re-record.
  • Settlement (D4): one D1 batch; the ledger row {session_id}:complete (UNIQUE) is written first, so repeats return the stored settlement with replayed: true (UI strings re-rendered for the request locale). XP 15/20/10 +10 improvement, 50 % (floored) for an article completed in the last 7 days, daily cap 165 on session XP in the user's tz. Weekly quests pay 30/30/40 once per week ({user}:{week_start}:{quest_id}), outside the cap. Diagnostic = round 1 + completion (25 XP).
  • Streak: +1 on the first completed session of a local day; missed days consume banked freezes (1 per 7-day milestone, max 2); GET /me shows 0 once a streak is broken. Interpretation of the 20 h guard: a second increment within 20 h is refused only when the user's time zone changed since the previous increment (the anti-gaming case the ADR describes); otherwise a 23:00 → 08:00 next-day session would break honest streaks. The day is still recorded.
  • Ability: EMA α = 0.3 over the per-session best attempt (all four dimensions + overall), ability-v1, shown from 3 samples. Growth with ?language= replays the EMA over that language's sessions only; a weekly point needs ≥ 2 sessions (min_sessions_per_point), the running week is is_partial.
  • Diagnostic sessions stay in stage round1 once round 1 is assessed (no round 2); POST /complete settles them, generates the 14-day plan (target = diagnostic one-action, baseline = diagnostic overall) and sets onboarding to plan_ready (complete once StoreKit reports trial/active).
  • Deletion (D6): DELETE /sessions/{id} is synchronous; /me/training-data and /me delete D1 rows immediately and purge R2 through a deletion_jobs row processed on the queue (cron retries). Ledger rows of deleted sessions are tombstoned; all stats are recomputed by replaying the remaining sessions.
  • Retention: nightly purge of audio older than the user's 7/30/365-day setting (audio_purged_at, playback 410).
  • Hard paywall (D8): requireEntitlement (src/middleware.ts) returns 402 entitlement_required on home, library, session create/list, growth, achievements, weekly quests and plan once onboarding is plan_ready/complete and the entitlement is not trial/active/grace; /me*, paywall, entitlements, subscriptions, config, events, onboarding and the user's own diagnostic session/attempts/article stay open. Locked difficulty tiers return 403 article_locked.
  • Paywall benchmarks (D8): cohort table per (weekly_days × baseline bucket) from the nightly job; any cell with < 30 samples makes the whole table pilot_benchmark with values from seeds/pilot_benchmarks.json. Social proof is null below 100.
  • RevenueCat (purchase layer, src/services/revenuecat.ts): POST /subscriptions/revenuecat/sync and the webhook (Authorization = REVENUECAT_WEBHOOK_AUTH, idempotent on event.id) both re-read GET /v1/subscribers/{user_id} with REVENUECAT_API_KEY and map entitlement pro to trial/active/grace/expired/revoked. A TRANSFER re-syncs both users.
  • StoreKit 2 (deprecated path for pre-RevenueCat builds): JWS x5c chain verified to the pinned Apple Root CA – G3 (SHA-256 63343ABFB89A6A03EBB57E9B3F5FA7BE7C4F5C756F3017B3A8C488C3653E9179), Apple marker OIDs on intermediate/leaf, validity windows, then the ES256 signature. Notifications V2 are idempotent on notificationUUID.

Content operations (ADR-002)

Goal: every (language × topic × difficulty) cell stays stocked with legally safe, retell-worthy articles with almost no human in the loop. Everything is driven by content_jobs rows (the dashboard's job log) processed on the PIPELINE queue ({ stage: "content", job_id }); PIPELINE_INLINE=true (tests) runs them inline. Admin API = docs/api/openapi-admin.yaml (binding), implemented in src/routes/admin.ts + src/routes/admin-ops.ts.

cron :05 hourly ─┬─ sources whose schedule_cron fired since last_run_at ─→ fetch job ─→ source_items (new|skipped|failed)
                 └─ fill_gaps: per cell target − unseen_available − in_pipeline (≤ batch_max) ─→ generate jobs
generate job ─→ content_llm draft (structured output, prompt library) ─→ article version (status qa) ─→ QA
QA ─→ deterministic checks + embeddings duplicate check + qa_llm rubric ─→ quality_reports
   ─→ auto_publish (+ audit sample) | needs_review (review item) | reject (reason kept for tuning)
cron 19:00 UTC ─→ health job: article_health from sessions/assessments/votes/reports ─→ retire < threshold (≥ 20 sessions) + replacement job
POST /v1/reports (rights|harmful on an opened article) ─→ unpublish now + `reported` review item

Pieces (src/content/): cron.ts (5-field UTC cron matcher; isDue = fired in (last_run, now]), feeds.ts (RSS/Atom, sitemap + index, HTML list page via HTMLRewriter CSS selector, JSON API via a jq-like selector such as .data.results[]), extract.ts (DOM-free Readability-style extractor), fetcher.ts (URL-hash + simhash dedupe, size bounds 150–4000 words / 250–8000 chars by the source language's unit, license inherited from the source), inventory.ts (cells + pure planFillGaps), generate.ts, prompts/ (versioned prompt library), qa.ts, vectors.ts (Vectorize CONTENT_INDEX when bound, else D1 content_vectors + cosine in JS; un-indexed published articles are back-filled during QA), articles.ts (versions + lifecycle table), review.ts, health.ts, reports.ts, aggregates.ts (overview/usage/quality), runner.ts (queue consumer + cron entry points).

Article versions. Versions stay in articles (sessions pin article_id + version). is_head marks the latest version (what operators see; its status is the lifecycle status), is_current the version the app serves when published. Editing a live article creates a new head that is not served until it is published, so the app keeps the old version during QA/review. article_versions stores a full snapshot + provenance per version. Content edits (paragraphs / key points) re-run QA; publish_now skips QA and is audited as article.publish_override.

Lifecycle (TRANSITIONS in src/content/articles.ts, mirrors dashboard/src/lib/lifecycle.ts): publish ← needs_review|rejected; reject ← needs_review|qa|draft (reason required); retire ← published|unpublished; unpublish ← published; republish ← unpublished; regenerate ← needs_review|rejected|unpublished|retired (new version of the same article; the operator note is fed to the prompt); request_review ← published|rejected. Anything else → 409 illegal_transition. Review decisions per kind follow DECISIONS (409 illegal_decision, already_decided).

QA checks (QualityCheck ids): deterministic length_in_tier (ADR-001 D9 tiers per language, src/lang lengthTiers), key_points_count (3–5), language_match (script detection), no_contact_info, no_private_names (honorific + name heuristic per language), attribution_present (licensed only); embedding duplicate_check (cosine > 0.92 ⇒ reject); LLM argument_clarity, retell_worthiness, grounding, readability (each passes at ≥ the language's auto_publish threshold), safety (hard gate) and a soft sensitive_politics flag. qa_score = min(4 rubric dims). Decision: safety fail, duplicate or qa_score < review_floor → reject; qa_score ≥ auto_publish with no soft flag → auto-publish; otherwise needs_review. Audit sampling is deterministic per (article, version) at audit_sample_rate.

Thresholds and targets live in generation_config (GET/PUT /v1/admin/generation/config): per-language auto_publish (80) / review_floor (60), default_target (20) + per-cell targets, batch_max (5 per cell per run), audit_sample_rate (0.05), retire_health_below (40), mix.ai_original (0.5), pair_generation, paused, prompt_version, daily_cost_cap_usd (20), max_jobs_per_run (25), paused_reason. Threshold changes show up in /admin/quality threshold_history (from the audit log).

Spend guard. An inventory run enqueues at most max_jobs_per_run generate jobs across cells (highest shortfall first). Before a generate/qa job is enqueued or run, today's (UTC) spend on content_llm + qa_llm + embeddings (provider_calls.cost_usd, priced from the price table) is compared with daily_cost_cap_usd; at or above it the pipeline pauses with paused_reason = "cost_cap" (remaining jobs of a run are not enqueued, queued ones fail with "generation paused" and can be retried) and resumes by itself on the next UTC day. Three consecutive content_llm chain failures pause with provider_errors. Both automatic pauses are audited as actor system; a manual pause is manual; PUT /generation/config with paused: false clears the reason. The overview shows paused_reason, spend_today_usd and daily_cost_cap_usd.

Health score (nightly, null below 20 sessions): round(100 × (0.35·completion + 0.35·avg_key_info/100 + 0.30·helpful_rate)) − 5 × reports, helpful_rate defaults to 0.7 without votes. Weekly points are kept in trend.

Adding a source / license / prompt

  • Source: POST /v1/admin/sources (or the dashboard) with type rss|sitemap|html|api, url, license (must be an allowed licenses.id; else 422 license_not_allowed; OWNER-CLEARED needs owner_cleared: true), language, attribution_name, topic_hints (topic ids used to match generation cells), schedule_cron (UTC), selector (CSS selector for html list pages, jq-like path for api). Starter sources (seeded): Wikinews en (CC BY 2.5), Wikipedia featured article feed (CC BY-SA 4.0), NASA news releases (US federal public domain). Fetching only happens when CONTENT_FETCH_ENABLED="true".
  • License: add a row to licenses in a new migration (allowed, requires_attribution, share_alike, attribution_template with {title} {author} {source_name} {license_name} {license_url} {url} {retrieved_date}). Disallowed licenses (ARR, NC, ND) stay listed with allowed = 0 so the dashboard can explain refusals.
  • Prompt: copy src/content/prompts/v1.ts to v2.ts, change PROMPT_VERSION, register it in prompts/index.ts, then set prompt_version in the generation config. System prompts are authored natively per content language (DRAFT_SYSTEM[en|zh-Hans]); adding a practice language means adding an entry there and in src/lang (lengthTiers, bylines, privateNameRe). Every article records prompt_version in its provenance.

Cloudflare Access

Protect the dashboard and https://<api>/v1/admin/* with one Access application. Set ACCESS_TEAM_DOMAIN (<team>.cloudflareaccess.com) and ACCESS_AUD (the application's AUD tag). The API verifies Cf-Access-Jwt-Assertion (RS256 against https://<team>/cdn-cgi/access/certs, cached 1 h and refetched for an unknown kid; aud, iss, exp checked) and records the email claim as the actor in audit_log. In production with Access configured X-Admin-Key is refused; elsewhere it remains the dev path (actor admin-key).

Costs

Each provider call is priced from the AI config price_table (input_per_1k, output_per_1k, per_minute; model: "*" = provider default) into provider_calls.cost_usd with the job_id that made it; content_jobs.cost_usd is the job's sum and provenance.cost_usd the generating job's cost. /admin/overview and /admin/ai-providers/usage report cost per published article (content_llm + qa_llm + embeddings ÷ articles published in the window). Budget levers: daily_cost_cap_usd, max_jobs_per_run, batch_max, targets, paused, max_items_per_run, cheaper chains for qa_llm/embeddings. Without a price table entry calls are unpriced (cost null) and do not count toward the cap.

Local runs

pnpm dev                                        # mock providers without API keys
pnpm content:run -- --mode fill_gaps --count 5  # enqueue + wait + summary (decisions, cost, review queue)
pnpm content:run -- --mode brief --brief brf_seed_en_career_1 --pair
pnpm smoke                                      # includes the admin section (fixture source → fetch → generate → review → app)

Mock content_llm drafts are deterministic per input (brief/source/cell) with sentence variety per job; markers in a brief or operator note force a path: [mock:strong], [mock:weak] (→ needs_review), [mock:low] (→ reject), [mock:unsafe] (→ safety gate), [mock:numbers] (grounding flag), [mock:short] (outside the tier). Mock qa_llm scores come from the draft's features; mock embeddings are hashed 3-shingles (256 dims).

Optional bindings: Vectorize CONTENT_INDEX (wrangler vectorize create shuke-content --dimensions=1024 --metric=cosine for bge-m3, plus a language metadata index) and the landing page's KV namespace as WAITLIST (keys email:<address> → {createdAt, locale}) for GET /v1/admin/waitlist/export; without it the export is an empty CSV with an X-Shuke-Note header.

Playground (ADR-002 D4, issue #33)

Operators run the app's loop on a published article and compare 2–6 scoring models side by side. Admin API = docs/api/openapi-admin.yaml, tag playground (binding), implemented in src/routes/admin-playground.ts with the queue stages in src/playground/runner.ts and the response shape in src/playground/store.ts.

POST /admin/playground/runs ─→ playground_runs + 2 playground_rounds (awaiting_audio) + 2–6 playground_lanes (random labels)
PUT  …/rounds/{n}/audio ─→ R2 playground/<run_id>/r<n>.<ext> ─→ { stage: "playground_transcribe", run_id, round }
playground_transcribe ─→ the run's pinned ASR entry once ─→ processTranscript (same validity as the app)
   ├─ unreliable ─→ every result `skipped` (POST …/assess-anyway opens them)
   └─ one playground_results row per lane ─→ one { stage: "playground_assess", lane_id, round } message per lane
playground_assess ─→ assess() with the lane's pinned entry ─→ same post-processing as the app; round 2 compares with the lane's round 1
POST …/lanes/{id}/exemplar ─→ { stage: "playground_exemplar", lane_id, round } ─→ text by the lane's model, audio by the tts chain
  • Pinned models. runEntry (src/ai/chain.ts) is one call to one entry with the chain's timeout, error mapping, provider_calls row and cost; runChain calls it per entry. transcribe, generateJson, assess and generateExemplarText take opts.entry to call exactly that entry (no fallback); a failure is still a ChainError.
  • Same scoring as the app. src/pipeline/steps.ts holds the pure steps both pipelines call: processTranscript (segments, text, pause map, validity), assessContext and roundComparison.
  • Isolation. The playground_* tables have no foreign keys into user tables, and runs never write users, sessions, attempts, assessments, XP, ability or stats. One lane's failure is stored on its result; the other lanes go on.
  • Recovery. A delivery claims a pending result, and a running one only after its lease (lane timeout + 60 s), so overlapping deliveries never both call the model and only the claim owner's write lands. A transcribe redelivery re-drives lanes still pending. When a queue send fails after its commit, the route reverts (round failed, result failed, retryable) and answers 500. The hourly cron sweeps work left waiting over 15 min (sweepPlayground).
  • Blind. Until a verdict or reveal, the API returns model_ref: null and options: null per lane, assessment.model_ref = "hidden", no latency, usage, cost or finished_at, and errors with details: { kind } only. A round's results are released together: while any lane of the round is pending or running, every lane reports running with nothing in it. Lane and lane-assessment ids are fully random (randomId), and the audit trail records lane counts, never the lane ↔ label mapping.
  • Verdicts. The first verdict given while blind is stored as blind_verdict and never changes; verdict is the latest one and may be replaced after the reveal.
  • Cost. Every call is recorded with provider_calls.job_id = <run id> (pg_…); the run's cost_usd is that SUM. NOT_PLAYGROUND_CALL (src/lib/provider-calls.ts) leaves these rows out of every aggregate: overview cost and provider error rates and fallbacks, /ai-providers/usage, cost per published article and the spend guard.
  • Audio. audio/mp4, audio/webm, audio/mpeg, audio/wav, at most 5 MB; caps 180 s / 120 s (ADR-001 D2). Duration: MP4 from the container, WAV from the RIFF header (src/lib/wav.ts), the header ignored. webm / mp3 use X-Client-Duration-Ms at upload, then after ASR the larger of that and the last segment's end (client_checked); over the cap there, the round fails with audio_too_long (stage transcribe).
  • Re-upload. Replaces the round's audio, transcript, results and model answers; round 1 also resets round 2. Once the run has a verdict, a round that already has audio can't be uploaded again (409 verdict_given); the first upload of round 2 is still allowed. Uploads never clear verdicts or blind a revealed run again: a new attempt is a new run. 409 round_busy while the round (for round 1, also round 2) is transcribing, assessing or has a model answer pending. Lane retry takes a failed result, or a round-2 skip once that lane's round 1 is done (nothing is re-queued by itself).
  • Retention. The cron deletes playground recordings and model-answer audio after 30 days (purgePlaygroundAudio); playback then answers 410 audio_purged. Runs and results stay.
  • WebM and ASR. OpenAI transcription and Deepgram document WebM support (the OpenAI adapter names the uploaded file after its type, since the API reads the format from the extension). Cloudflare's Whisper model page names no formats, so WebM is not yet confirmed on Workers AI: record audio/mp4 where the browser supports it.
  • Mock mode. mockLlm adds a deterministic per-model score bias (mockModelBias) and token estimates, so lanes differ and costs show without keys.

Non-contract routes

PUT /v1/uploads/:attempt_id and GET /v1/media (HMAC-signed, back the URLs returned by the contract when R2 S3 keys are absent) and GET /health.