# System Architecture

The engineer's map of CoreReflex: how the codebase is laid out, how a request flows from
the wire to a database row, why there is no web framework, and how the three deployables
(app, editor, render worker) fit together around MinIO and Postgres.

CoreReflex is a governed, self-hosted, white-label AI creative studio. Video is the
flagship — you brief a director, it boards a continuity-linked shot plan, generates each
shot, narrates, scores, grades, and cuts it on a real timeline — but the same platform
carries image, audio, brand, slides, write, and workflow studios, plus the client-ops
layer an agency runs its business on. Every asset lands on your own infrastructure. The
platform is **multi-tenant**: nearly everything is scoped to `user.workspace.id`.

The UI is **white-label themed** — the default look is the Cobalt preset, and every page
resolves its colors from CSS custom properties (`/css/themes.css` presets + per-workspace
overrides). Don't hardcode brand colors into new UI; use the theme tokens. See
[White-label theming](#white-label-theming) below.

> New to the product side? The end-user walkthrough lives in [`../user/README.md`](../user/README.md).
> For the AI deep-dives this doc only summarizes, see [`../MODEL-ROUTER.md`](../MODEL-ROUTER.md),
> [`../PROMPTING.md`](../PROMPTING.md), [`../AGENTIC-ADK.md`](../AGENTIC-ADK.md), and
> [`../ASSET-SERVING.md`](../ASSET-SERVING.md).

## On this page

- [The three deployables](#the-three-deployables)
- [Repository layout](#repository-layout)
- [The app server: `server.js` → `api.js` → domain modules](#the-app-server-serverjs--apijs--domain-modules)
- [Request flow, end to end](#request-flow-end-to-end)
- [Real-time: the SSE bus](#real-time-the-sse-bus)
- [Asset + render data flow](#asset--render-data-flow)
- [Subsystem overviews](#subsystem-overviews)
- [Data layer](#data-layer)
- [Identity, tenancy, and admin](#identity-tenancy-and-admin)
- [Client-operations layer](#client-operations-layer-pro-gated-off-by-default)
- [Billing & metering](#billing--metering)
- [The editor SPA, briefly](#the-editor-spa-briefly)
- [Quick reference: build & run](#quick-reference-build--run)

---

## The three deployables

CoreReflex ships as **three independent Coolify apps** on a private fleet over WireGuard.
They share one Postgres database and one MinIO object store, but each is built and deployed
on its own.

| Deployable | Source dir | What it is | Public domain |
|------------|------------|------------|---------------|
| **API / app** | [`src/`](../../src) + [`public/`](../../public) | The custom Node HTTP server: marketing site, studio dashboard, and the whole `/api/*` surface. | `corereflex.com` |
| **Editor studio** | [`editor/`](../../editor) | A separate Remotion 4.0.483 + React Router v8 SPA (SSR disabled). The full timeline workspace plus the image/audio/brand/diagram/workflow studios. | `editor.corereflex.com` |
| **Render worker** | [`render-worker/`](../../render-worker) | A headless container that drains `render_jobs`: it bundles the **same** `editor/src/remotion` composition the editor previews and renders in-container with `@remotion/renderer`. Also hosts the DSP/media sidecar HTTP services (audio FX, upscale, remove-bg, vectorize, clip proxy). | none (worker) |

How they relate:

- The **editor** is a pure SPA. All of its `/api/*` calls proxy (in dev) or go cross-origin
  (in prod) to the **app**. Auth is the shared `corereflex_session` cookie, scoped to the
  apex domain so it works across `corereflex.com` and `editor.corereflex.com`.
- The **app** owns all state: it persists the editor's timeline as `compositions.manifest`
  (a JSON blob), enqueues renders into `render_jobs`, and serves every private asset through
  a signed proxy.
- The **render worker** has no public domain. It is a database-driven loop: it claims a
  queued `render_jobs` row with `FOR UPDATE SKIP LOCKED`, loads the saved manifest, bundles
  and renders the composition, uploads the video to MinIO, inserts an `assets` row, and
  flips the job to `succeeded`/`failed`. Because it bundles `editor/src/remotion` itself,
  **any composition change in the editor requires a worker redeploy** or renders won't
  match preview. Remotion versions must stay in lockstep (both pin `4.0.483`).

```
                          corereflex.com                       editor.corereflex.com
                       ┌──────────────────────┐               ┌──────────────────────┐
   browser ──────────► │  API / app (src/)    │ ◄──/api/*───── │  Editor SPA (editor/)│
   (SSE streams too)   │  Node http server    │   (cookie)     │  Remotion <Player>   │
                       └──────────┬───────────┘               └──────────────────────┘
                                  │
                  ┌───────────────┼────────────────┐
                  ▼               ▼                ▼
          ┌─────────────┐  ┌─────────────┐  ┌──────────────┐
          │ Postgres    │  │ MinIO / R2  │  │ Google Vertex│
          │ (pgvector)  │  │ object store│  │ AI (Veo etc.)│
          └──────┬──────┘  └──────┬──────┘  └──────────────┘
                 │                │
                 │ render_jobs    │ MP4 upload + assets row
                 │ (claim/lease)  │
                 ▼                ▲
          ┌────────────────────────────────┐
          │  render-worker/ container       │  bundles editor/src/remotion in-container
          │  (+ audio-fx / upscale /        │  @remotion/bundler + @remotion/renderer
          │   remove-bg / vectorize /       │
          │   clip-proxy HTTP sidecars)     │
          └────────────────────────────────┘
```

---

## Repository layout

| Path | What lives here |
|------|-----------------|
| [`src/`](../../src) | The API/app server. Entry is `server.js`; all routing is in `api.js`; everything else is a domain module (~130 focused files). |
| [`src/prompts/`](../../src/prompts) | The portable prompt-spec registry (`registry.js`) and the versioned JSON specs (`specs/`). |
| [`src/content/`](../../src/content) | The Write pillar's built-in template catalog + quality analyzer. |
| [`src/voice-agent/`](../../src/voice-agent) | The real-time voice bot (browser/Twilio WebSocket ⇄ Gemini Live bridge). |
| [`editor/`](../../editor) | The Remotion + React Router v8 SPA. `editor/src/remotion/` holds the compositions; `editor/src/editor/` the timeline, canvas, inspector, and the sibling studios (image, audio, brand, diagram, workflow/CrFlow, agent panel, comments). |
| [`render-worker/`](../../render-worker) | The production render container: `worker.js` (queue loop), `props-from-manifest.js` (manifest → inputProps), its own dependency-free SigV4 `storage.js`, `audio-fx-chain.js`/`audio-fx-service.js`, `upscale-service.js`, `remove-bg-service.js`, `vectorize-service.js`, `clip-proxy-service.js`. |
| [`extension/`](../../extension) | The Chrome MV3 extension for tab-to-timeline screen recording. |
| [`public/`](../../public) | Static HTML the server serves directly — see [Static pages](#static-pages-and-friendly-paths). |
| [`migrations/`](../../migrations) | Hand-written, additive SQL migrations `001`–`033`. Source of truth for the schema. |
| [`scripts/`](../../scripts) | Operational scripts: `migrate.mjs`, `admin-set-password.mjs`, `check-syntax.mjs` (the lint step), `verify-backend.mjs`, `batch-embed.mjs`, `ingest-chat-kb.mjs`, `db-tunnel.sh`, `vertex-smoke.mjs`. |
| [`test/`](../../test) | Node built-in test runner suites (`node --test`), one file per domain module. |
| [`planning/`](../../planning) | Bootspring planning docs — the task-level source of truth (`MASTER_PLAN.md`, `TODO.md`, per-pillar plans). |
| [`docs/`](..) | Engineering strategy + architecture (this folder's parent), plus `docs/user/` end-user guides. |
| [`ops/`](../../ops) | Coolify/database runbook + compose for the self-hosted Postgres. |

---

## The app server: `server.js` → `api.js` → domain modules

### Entry point — `src/server.js`

The app is a single `node:http` `createServer`. There is no Express, Fastify, or Koa.
[`src/server.js`](../../src/server.js) does the following on every request:

1. **Sets `Alt-Svc: clear`.** HTTP/3 (QUIC) is disabled at the edge because QUIC to this
   VPS stalls (~15s), which turns video range requests into black clips. Browsers that
   cached an `h3` advertisement are told to forget it and fall back to HTTP/2 over TCP.
   `setHeader` persists through the downstream API and static `writeHead` calls, so this
   lands on everything.
2. **Mints a per-response CSP nonce and sets the security headers.** The CSP is
   **nonce-based**: `script-src 'self' 'nonce-<random>'` — there is **no `'unsafe-inline'`
   for scripts** anymore. Every inline `<script>` in served HTML is stamped with the nonce
   on the way out (see step 5), so an injected inline script without the unguessable
   per-request nonce is refused by the browser. `style-src` keeps `'unsafe-inline'`
   (the pages rely on inline `style=""` attributes, which nonces cannot authorize).
   When `GA_MEASUREMENT_ID` is set, Google's tag domains are appended to `script-src`
   (GA4 is fully opt-in). Also sent on every response: `X-Content-Type-Options: nosniff`,
   `X-Frame-Options: SAMEORIGIN`, `Referrer-Policy: strict-origin-when-cross-origin`,
   `Strict-Transport-Security: max-age=31536000; includeSubDomains` (no `preload` — that is
   an effectively irreversible commitment for every subdomain), and a `Permissions-Policy`
   that keeps camera/microphone `self` (the editor records via `getUserMedia`) and denies
   the rest. `connect-src 'self' https: wss:` lets the voice widget reach the live bridge.
3. **Handles infrastructure routes inline**: `GET /healthz` (trivial liveness probe for the
   Docker `HEALTHCHECK`, distinct from the rich `GET /api/health`), `robots.txt`
   (disallows `/app`, `/admin`, `/dashboard`, `/proof`, `/api/`), a generated
   `sitemap.xml` over the public marketing routes + every published journal post,
   category hub, and case study (canonical host from `SITE_URL`), and `/js/analytics.js`
   (the first-party analytics beacon, with `window.CR_GA_ID` injected only when GA is
   configured).
4. **Server-renders the Journal and case studies.** `/blog/:slug`, `/blog/category/:slug`,
   and `/case-studies[/:slug]` are rendered server-side ([`blog.js`](../../src/blog.js) +
   [`blog-render.js`](../../src/blog-render.js), [`case-studies.js`](../../src/case-studies.js) +
   [`case-study-render.js`](../../src/case-study-render.js)) with full per-page
   `<title>`/meta/canonical/OG + JSON-LD, so the content library is crawlable without JS.
   The rendered HTML is nonce-stamped and sent `cache-control: no-store` like all HTML.
5. **Delegates to `handleApiRequest`**, otherwise **serves static files** — with two
   HTML-specific transforms applied to every `.html` file it serves:
   - **Nonce stamping + no-store.** Every inline `<script` is rewritten to
     `<script nonce="...">`, and the response carries `cache-control: no-store` —
     HTML holds a per-request nonce, so a shared proxy must never cache it (a stale
     body's nonce would not match this response's CSP header).
   - **Theme + flash-of-white control**, injected at three points:
     - *Server-side (zero-flash):* the visitor's saved theme is read from the
       **`cr-theme` cookie** ([`theme-settings.js`](../../src/theme-settings.js)
       `themeFromCookieHeader`), sanitized against a strict allowlist grammar
       (preset name + up to 12 CSS custom-property overrides; no `;{}<>` can survive),
       and stamped straight onto the markup — `data-theme` on `<html>` plus a `:root`
       override `<style>` — so the very first paint is already the right look with no
       script race, even for a cross-origin first visit.
     - *First in `<head>` (dark early-paint):* `<meta name="color-scheme" content="dark">`,
       `html{color-scheme:dark;background:var(--bg,#0a0e16)}`, and a tiny nonce-carrying
       inline script that applies the `localStorage['cr-theme']` fallback for same-origin
       visitors without the cookie. This kills the flash-of-white before any external CSS
       or font loads.
     - *Last in `<head>`:* the shared token stylesheet (`/css/themes.css`) and the theme
       switcher (`/js/theme.js`, deferred, nonce-stamped).
   - Non-HTML static assets get `cache-control: public, max-age=31536000, immutable` and
     are streamed. `resolveSafePath` guards against path traversal by normalizing and
     confining everything under the repo root.

It also installs process-level `uncaughtException`/`unhandledRejection` handlers
(log-and-keep-serving, so a stray socket error can't take down a single-process server),
Slowloris guards (`headersTimeout` default 60s, `requestTimeout` default 300s — generous
so large uploads complete), **attaches the WebSocket voice bridge** (`attachVoiceBridge`)
before `listen`, and starts the **scheduled-automation tick loop** (`startScheduler` from
[`automation.js`](../../src/automation.js); on by default, disable with
`AUTOMATION_SCHEDULER=false`).

#### Static pages and friendly paths

| Path | File | What it is |
|------|------|------------|
| `/` | `public/home.html` | Marketing homepage. |
| `/app`, `/dashboard`, `/admin`, `/sign-in`, `/login` | `public/dashboard.html` | **The canonical studio hub** (God Console built in; `/admin` deep-links its route). |
| `/book`, `/book/:slug` | `public/book.html` | White-label public scheduler (reads the slug client-side, calls the unauth `/api/calendars/:slug/*` routes). |
| `/proof`, `/proof/:token` | `public/proof.html` | Public client-review surface (calls the unauth `/api/proofs/review/*` routes). |
| `/voice` | `public/voice.html` | Voice-bot widget (WebSocket to `/api/voice/live`). |
| `/auto-clips` | `public/auto-clips.html` | Auto-Clips studio — a self-contained client-side pipeline over `/api/highlights/*` + `/api/auto-clips`. |
| `/pricing`, `/docs`, `/terms`, `/privacy` | matching `public/*.html` | Marketing, docs hub, and legal pages. |
| `/blog` (bare index) | `public/blog.html` | Client-rendered journal index (individual posts and category hubs are server-rendered, see above). |
| `/case-studies[/:slug]` | server-rendered | Case-study index + pages (honest empty state until real stories exist). |
| `/og-image.png`, `/favicon.png`, `/favicon.ico`, `/js/*`, `/css/*` | `public/…` | Clean root paths mapped into `public/`. |

### The route ladder — `src/api.js`

[`src/api.js`](../../src/api.js) is the heart of the API (~2,500 lines): a single
`handleApiRequest` function that is one long **if / pathname ladder**. It returns `false`
for anything not under `/api/` (so the server falls through to static), and otherwise
matches routes in order. There is no router object and no middleware stack — auth, CORS,
and rate limiting are explicit function calls inside each branch.

Structural facts worth internalizing before you add a route:

- **Order matters.** After `/api/health`, the **chat surface** is delegated first
  (`handleChatApi` from [`chat-routes.js`](../../src/chat-routes.js) — it owns the raw
  response object for SSE streaming), then the Twilio voice webhook and the **Stripe
  webhook** — both read the **raw request body** (`readRaw`) for signature verification,
  so they must run before anything JSON-parses the request. Static-string routes generally
  precede regex/`:id` matchers.
- **Auth is a call, not a decorator.** `requireUser(request)` (from
  [`src/session.js`](../../src/session.js)) throws a 401 the outer `try/catch` converts
  into a JSON error. `optionalSession(request)` is the soft variant (public gallery).
  `requireApiUser(request)` ([`api-keys.js`](../../src/api-keys.js)) additionally accepts a
  **platform API key** (`crk_…` bearer token, sha256-hashed at rest, migration 018) so
  SDKs/CI/MCP can act as a workspace without a browser session — used across the files,
  collections, compositions-engine, and version-history routes.
- **`:id` segments are parsed by small regex helpers** at the bottom of the file
  (`compositionIdFromPath`, `recipeRunActionFromPath`, `personaRoute`, `crmContactRoute`, …).
- **Errors are normalized.** `httpError(status, message)` attaches `statusCode`; the outer
  `catch` masks 5xx detail unless the error is flagged `expose`.
- **CORS is allow-listed.** `corsHeaders` reflects the request origin only if it appears in
  `CORS_ALLOWED_ORIGINS` (default `https://editor.corereflex.com`) and always sends
  `access-control-allow-credentials: true` so the editor's cross-origin XHR carries the
  session cookie.
- **Admin routes are delegated** to `routeAdminApi` ([`admin-api.js`](../../src/admin-api.js)),
  which also mounts the admin live-agent chat surface
  ([`admin-chat-api.js`](../../src/admin-chat-api.js)).
- **`GET /api/health`** reports DB/storage/auth status plus a `version` field
  (`BUILD_VERSION` env, falling back to a hardcoded stamp) — set `BUILD_VERSION` per
  deploy in Coolify so you can prove which build is live.

The ladder then calls into **domain modules**, one per concern. Each module owns its SQL
and its workspace-scoping; `api.js` is mostly glue + validation.

| Domain module | Responsibility |
|---------------|----------------|
| [`generation.js`](../../src/generation.js) / [`generation-jobs.js`](../../src/generation-jobs.js) | Async video-generation jobs (Veo/Kling) via `providers.js`/`vertex.js`, with durable status rows in `generation_jobs` (015). |
| [`director.js`](../../src/director.js) / [`producer.js`](../../src/producer.js) | Agentic storyboard planning (`planFilm`) and the PRODUCE → CRITIQUE → ASSEMBLE loop (`produceFilm`). |
| [`agent.js`](../../src/agent.js) + [`agent-tools.js`](../../src/agent-tools.js), [`agent-runs.js`](../../src/agent-runs.js), [`agent-ask.js`](../../src/agent-ask.js), [`agent-recipe.js`](../../src/agent-recipe.js), [`agent-context.js`](../../src/agent-context.js) | The suite-wide conversational **AGENT kernel** (see [below](#the-agent-kernel)). |
| [`render.js`](../../src/render.js) | Enqueues `render_jobs`; reports status/progress/`outputUrl`. Gated by `RENDER_ENABLED`. |
| [`shorts.js`](../../src/shorts.js) / [`auto-clips.js`](../../src/auto-clips.js) / [`highlights.js`](../../src/highlights.js) / [`story-narrator.js`](../../src/story-narrator.js) | Vertical-short reframing and the long-form → shorts **Auto-Clips** pipeline (see [below](#auto-clips-long-form--shorts)). |
| [`recipes.js`](../../src/recipes.js) / [`recipe-runner.js`](../../src/recipe-runner.js) / [`recipe-starters.js`](../../src/recipe-starters.js) / [`budget.js`](../../src/budget.js) / [`checkpoints.js`](../../src/checkpoints.js) | Programmable creative workflows with budget governance and human checkpoints (see [below](#recipes-checkpoints-and-budget-governance)). |
| [`captions.js`](../../src/captions.js) / [`caption-generate.js`](../../src/caption-generate.js) / [`caption-styles.js`](../../src/caption-styles.js) | Speech-to-text transcription, caption translation, and styled caption generation (9 personas). |
| [`voice.js`](../../src/voice.js) / [`voice-design.js`](../../src/voice-design.js) / [`music.js`](../../src/music.js) / [`imagen.js`](../../src/imagen.js) / [`soundscape.js`](../../src/soundscape.js) | TTS + voice cloning seam + voice design, music (Lyria), image + logo generation (Imagen), soundscapes. |
| [`image-edit.js`](../../src/image-edit.js) / [`image-assist.js`](../../src/image-assist.js) / [`reframe.js`](../../src/reframe.js) / [`enhance.js`](../../src/enhance.js) | AI image editing, edit-assist, subject-box detection (auto-reframe), prompt/brand-voice enhancement. |
| [`upscale.js`](../../src/upscale.js) + `*-engine.js` (audio-fx, upscale, remove-bg, vectorize) | Thin adapters that POST media jobs to the render-worker sidecar services (deploy-gated by `*_PORT`/`*_URL` env). |
| [`json-api.js`](../../src/json-api.js) / [`json-engine.js`](../../src/json-engine.js) / [`catalog.js`](../../src/catalog.js) | The dependency-free programmable JSON engine over compositions + the API catalog — see [json-engine.md](json-engine.md) and [platform-api.md](platform-api.md). |
| [`gallery.js`](../../src/gallery.js) / [`collections.js`](../../src/collections.js) / [`assets.js`](../../src/assets.js) / [`content-credentials.js`](../../src/content-credentials.js) | The Files pillar: workspace file repository, DAM collections, asset metadata/deletion, signed provenance credentials (see [below](#files-pillar-and-content-credentials)). |
| [`comments.js`](../../src/comments.js) / [`composition-versions.js`](../../src/composition-versions.js) | In-editor comment threads + composition version history (see [below](#editor-comments--composition-versions)). |
| [`proofing.js`](../../src/proofing.js) | Tokenized client review links: versions, comments, approve/request-changes (Pro-gated). |
| [`social.js`](../../src/social.js) / [`ghl.js`](../../src/ghl.js) / [`crypto-box.js`](../../src/crypto-box.js) | Social publishing relay (see [below](#social-publishing)). |
| [`brands.js`](../../src/brands.js) / [`brand-generate.js`](../../src/brand-generate.js) / [`brand-kit.js`](../../src/brand-kit.js) | The account-owned Brand layer + cascade (see [below](#brand-cascade)). |
| [`chat.js`](../../src/chat.js) / [`chat-routes.js`](../../src/chat-routes.js) / [`chat-kb.js`](../../src/chat-kb.js) / [`chat-handoff.js`](../../src/chat-handoff.js) / [`chat-safety.js`](../../src/chat-safety.js) / [`support.js`](../../src/support.js) / [`self-heal.js`](../../src/self-heal.js) | The chat + support + self-healing loop (see [below](#chat-support-and-self-healing)). |
| [`content.js`](../../src/content.js) / [`content-store.js`](../../src/content-store.js) / [`src/content/`](../../src/content) | The Write pillar: templated copywriting, documents, template quality scoring (020). |
| [`meeting.js`](../../src/meeting.js) | Transcript → brief (summary, action items, decisions, chapters). |
| [`crm.js`](../../src/crm.js) · [`tasks.js`](../../src/tasks.js) · [`booking.js`](../../src/booking.js) · [`automation.js`](../../src/automation.js) | The Pro-gated client-operations layer (see [client-ops.md](client-ops.md)). |
| [`personas.js`](../../src/personas.js) / [`scripts.js`](../../src/scripts.js) | Consent-gated video-clone personas and natural-voice script projects (both have live studio UIs in the dashboard). |
| [`account.js`](../../src/account.js) / [`password-reset.js`](../../src/password-reset.js) / [`email-verification.js`](../../src/email-verification.js) / [`team.js`](../../src/team.js) | Profile, change-password, anti-enumeration reset, email verification (014), team invites/members. |
| [`billing.js`](../../src/billing.js) | Stripe checkout, customer portal, webhooks, usage metering, credit ledger, storage add-on grants. |
| [`votes.js`](../../src/votes.js) / [`preferences.js`](../../src/preferences.js) / [`embeddings.js`](../../src/embeddings.js) | Vote-learning + pgvector semantic recall feeding the Director. |
| [`analytics.js`](../../src/analytics.js) / [`leads.js`](../../src/leads.js) / [`blog.js`](../../src/blog.js) / [`case-studies.js`](../../src/case-studies.js) / [`testimonials.js`](../../src/testimonials.js) | First-party analytics events (023), marketing lead capture (024), the file-based journal + case studies + testimonials. |
| [`admin-api.js`](../../src/admin-api.js) / [`god-console.js`](../../src/god-console.js) / [`admin-chat-api.js`](../../src/admin-chat-api.js) | God Mode: metrics, impersonation, admin CRUD, audit logging, support takeover. |
| [`theme-settings.js`](../../src/theme-settings.js) | White-label theme sanitization + workspace persistence (033). |
| [`voice-agent/`](../../src/voice-agent) | The real-time voice bot — browser/Twilio WebSocket ⇄ Gemini Live bridge (see [below](#real-time-voice-agent)). |

Cross-cutting infrastructure modules:

| Module | Responsibility |
|--------|----------------|
| [`database.js`](../../src/database.js) | The `pg` connection pool + `query()`, `withClient()`, and `withTransaction()` (real `begin`/`commit`/`rollback`). |
| [`storage.js`](../../src/storage.js) | S3-compatible (MinIO / R2) signing, presign, fetch/put/delete, and the HMAC capability-token scheme. |
| [`vertex.js`](../../src/vertex.js) / [`gemini-cache.js`](../../src/gemini-cache.js) / [`inference.js`](../../src/inference.js) | Google Vertex AI access (service-account auth), context caching, inference tracing. |
| [`realtime-bus.js`](../../src/realtime-bus.js) | The in-process **SSE realtime bus** — see [below](#real-time-the-sse-bus). |
| [`crypto-box.js`](../../src/crypto-box.js) | AES-256-GCM secrets-at-rest (scrypt key from `SECRET_ENCRYPTION_KEY`/`JWT_SECRET`) + Ed25519 credential signing. |
| [`net-guard.js`](../../src/net-guard.js) | **SSRF guard** (`safePublicUrl`) for user-supplied media URLs the server fetches — requires HTTPS, rejects loopback/private/link-local/cloud-metadata hosts. |
| [`auth.js`](../../src/auth.js) / [`session.js`](../../src/session.js) | scrypt password hashing, HS256 JWTs, session/cookie lifecycle. |
| [`api-keys.js`](../../src/api-keys.js) | Platform API keys (hash-at-rest, workspace-scoped) + `requireApiUser`. |
| [`email.js`](../../src/email.js) | Outbound email — **provider wired: usermails.com**. Set `EMAIL_PROVIDER=usermails`, `USERMAILS_API_KEY`, and `EMAIL_FROM` (a verified sender) to go live; unset, sends are logged (dev) and report `delivered:false` so flows like password reset degrade safely instead of leaking a token into an HTTP response. Also exposes `verifyEmailAddress` (deliverability check at signup; fails open to `unknown`). |
| [`rate-limit.js`](../../src/rate-limit.js) | In-memory per-process rate limiter (`rateLimited`, `clientIp`) used by every abuse-prone route. |
| [`permissions.js`](../../src/permissions.js) / [`modes.js`](../../src/modes.js) | Admin privilege matrix + release-mode (production/beta/god) UI gating. |
| [`http-error.js`](../../src/http-error.js) / [`env.js`](../../src/env.js) | `httpError`/`badRequest`/`notFound` helpers; `.env` loading + typed env helpers. |

### Why no web framework

This is a deliberate choice, not an oversight:

- **One production dependency.** `package.json` lists exactly one runtime dependency:
  `pg`. Everything else — HTTP, routing, crypto, JWT signing, S3 SigV4, SSE, AES-GCM —
  is built on Node built-ins. That keeps the supply-chain attack surface and the Docker
  image small, and makes the whole request path readable in two files
  (`server.js` + `api.js`). This is the repo-wide **no-new-deps** rule.
- **The behavior is unusual enough that a framework would fight it.** The signed asset
  proxy streams Range requests and rewrites `Content-Range`; the Stripe webhook needs the
  raw body before any parse; HTML is nonce-stamped per response; several routes hold the
  response open as an SSE stream. These are easier to express as plain handlers than as
  middleware exceptions.
- **It is genuinely small.** The route ladder is long but flat; there is no hidden control
  flow. You can read top-to-bottom and see every route, its auth check, and its handler.

The trade-offs to be aware of: routing is linear (`O(routes)` string/regex checks — fine at
this scale), and both the rate limiter and the SSE bus are **in-memory per process** — they
reset on restart and do not coordinate across replicas. Scaling the app horizontally would
need a shared limiter and a cross-replica pub/sub before realtime fan-out works everywhere.

---

## Request flow, end to end

A representative authenticated write — saving the editor timeline — traces like this:

```
PUT /api/compositions/:id   (from editor.corereflex.com, with session cookie)
  │
  ▼  src/server.js   sets Alt-Svc/CSP-nonce headers, calls handleApiRequest
  ▼  src/api.js      OPTIONS? no. matches compositionIdFromPath(); method PUT
  │                  requireUser(request)  ──► src/session.js verifies the JWT/cookie
  ▼  saveComposition(user, id, body)
  │                  splitSessionsFromManifest(manifest)   ← audio-session normalization
  │                  UPDATE compositions ... WHERE workspace-scoped  (src/database.js)
  │                  snapshotComposition(...)              ← version history, best-effort
  ▼  json(response, 200, { id })           ──► CORS headers reflected back to the editor
```

Note the two side-cars on the save path:

- **Audio-session normalization** (`splitSessionsFromManifest` — the
  [invariant](#audio-sessions-the-manifest-split-invariant) below) runs on **every**
  manifest write, so no client can persist an illegal shape.
- **Version snapshot** rides the save **best-effort** — a version write (or the
  `updated_at` touch) must never fail the save itself, because a deploy always lands
  before its migration is applied in prod.

Patterns that recur across the surface:

- **Async jobs are polled.** `POST /api/generate` and `POST /api/render` return
  `202 Accepted` with a `jobId`; the client polls `GET /api/generate/:id` /
  `GET /api/render/:id`. Recipes and agent runs follow the same pattern.
- **Live streams are SSE, not WebSockets.** Chat answers, admin live-agent, in-editor
  comments, and agent progress all stream over Server-Sent Events (see the next section).
  The only WebSockets in the system are the voice bridge's audio streams.
- **Workspace scoping is enforced in the query, in the app layer** — not by Postgres
  row-level security. Every read/write joins back to `workspaces` / `campaigns` and filters
  on `user.workspace.id`. A raw connection (like the render worker) **bypasses** this, so it
  must include `workspace_id` in its own `WHERE` clauses.

---

## Real-time: the SSE bus

CoreReflex has real-time surfaces, and they all run over **Server-Sent Events** on the
in-process bus in [`src/realtime-bus.js`](../../src/realtime-bus.js) — no realtime SDK,
staying inside the no-new-deps rule. (The old "there are no WebSockets" claim is only true
for the API surface; the voice agent's audio streams are genuine WebSockets, attached
separately via the `upgrade` handler.)

The bus is a frozen `{ publish, subscribe, subscriberCount, emitter }` built on an
`EventEmitter` + a `Map<channel, Set<subscriber>>`:

- `subscribe(channel, response, { cors })` writes the SSE headers
  (`text/event-stream`, `cache-control: no-store, no-transform`,
  `x-accel-buffering: no` to defeat nginx proxy buffering), sends a `ready` event, and
  holds the response open with a 25s `: keepalive` heartbeat. Cleanup is automatic on
  `close`/`error`.
- `publish(channel, event, payload)` fan-outs to current subscribers and returns the
  delivered count. Event names are validated (`/^[A-Za-z][A-Za-z0-9_-]{0,63}$/`).
- **Channels are strictly shaped** — `conv:<uuid>` (a chat conversation),
  `comp:<uuid>` (a composition), and `admin:support` (the staff inbox). Channel *shape*
  is validated here; channel *authorization* happens at the subscribing route (workspace
  ownership / admin permission is verified **before** the socket is held open).

Consumers:

| Stream | Route | Publisher |
|--------|-------|-----------|
| Chat answer tokens | `POST /api/chat` (SSE response, streamed directly by [`chat-routes.js`](../../src/chat-routes.js)) | the Vertex streaming answer loop |
| Conversation live events (handoff, live messages, tickets) | `GET /api/chat/conversations/:id/stream` | [`chat-handoff.js`](../../src/chat-handoff.js) publishes to `conv:<id>` |
| Admin live-agent inbox | admin chat stream in [`admin-chat-api.js`](../../src/admin-chat-api.js) | `chat-handoff.js` publishes `handoff_requested` / `admin_joined` / `live_message` / … to `admin:support` |
| In-editor comments | `GET /api/compositions/:id/comments/stream` | [`comments.js`](../../src/comments.js) publishes `comment` add/resolve/reopen to `comp:<id>` |
| Agent run progress | `GET /api/agent/compositions/:id/events` | the agent executor publishes plan → step → done to `comp:<id>` |

Two design rules to preserve:

1. **Realtime is an enhancement, never a dependency.** Every publish site is wrapped in
   try/catch; every panel is fetch-correct without the stream.
2. **The bus is in-process, single-replica.** Events published in one app process are
   invisible to subscribers held by another. Fine today (one web instance); a multi-replica
   deploy needs a shared pub/sub behind the same `publish/subscribe` seam.

SSE routes cap concurrent opens per user via the rate limiter (each stream holds a socket,
a subscriber, and a heartbeat timer, and the bus disables the listener cap).

---

## Asset + render data flow

Assets (generated clips, uploads, reference media, rendered masters) live in **MinIO**
(in-house, S3-compatible) or **Cloudflare R2** — selected by `ASSET_PROVIDER` in
[`storage.js`](../../src/storage.js). The metadata row lives in the `assets` table,
workspace-scoped, with a `provenance` JSONB, an optional user `metadata` JSONB (031), and a
`VECTOR(1536)` embedding column for semantic search (`GET /api/media/search`).

### Serving private assets

In-house MinIO is **not browser-reachable** (it's on the fleet's internal network). So the
app serves every private object through a **signed proxy** at `GET /api/assets/:key`
(`streamAsset` in `api.js`):

- Authorization is either a **capability token** (`?t=...`, an HMAC over the key,
  verified by `verifyAssetToken`) **or** a live session.
- `Range` is forwarded so `<video>` seeking works; `Content-Range` / `Accept-Ranges` are
  passed back.
- `HEAD` is special-cased for the editor's `checkFileExists()`.
- `PUT /api/assets/:key` is the **upload proxy**: the browser can't write to private MinIO,
  so the editor PUTs here and the app streams the body in server-side.
- `GET /api/assets/:key/download` and `DELETE /api/assets/:key` are **ownership-gated**:
  they require a real session whose workspace owns the asset — capability tokens are for
  playback/upload only. For R2, download 302s to a short-lived presigned GET.
- `GET /api/assets/:key/credentials` is deliberately **public** — anyone can verify an
  asset's signed provenance record (see
  [Content Credentials](#files-pillar-and-content-credentials)).

Full design rationale is in [`../ASSET-SERVING.md`](../ASSET-SERVING.md).

### The render pipeline

Rendering is the clearest example of the database-as-message-queue pattern that ties the
deployables together. Enqueue is unchanged; **the production renderer is the
[`render-worker/`](../../render-worker) container**, and the queue has bounded
retry/lease/DLQ semantics from migration 017.

```
editor ──POST /api/render──► app: enqueueRender()          render-worker/ container
        { compositionId,      │  workspace-scope the comp        │
          codec }             │  INSERT render_jobs              │  loop (worker.js):
                              │  (status='queued',               │   1. claim: UPDATE ... FOR UPDATE
        ◄── 202 { jobId } ────┘   codec on the error jsonb)      │      SKIP LOCKED → 'running',
                                                                 │      attempts+1, locked_at=now()
editor ──GET /api/render/:id──► app: getRenderJob()              │      (also reclaims 'running' rows
        (polls)                 │  read render_jobs + assets      │       whose 15-min lease lapsed,
        ◄── { status,           │  outputUrl = signedAssetUrl()   │       while attempts < max)
             progress,          │  progress from error->progress  │   2. load compositions.manifest
             outputUrl } ───────┘                                 │   3. bundle editor/src/remotion
                                                                  │      (once per process, reused)
                                                                  │      → selectComposition → renderMedia
                                                                  │   4. ffmpeg audio-FX pass (best-effort)
                                                                  │   5. upload video to MinIO, INSERT assets
                                                                  │   6. UPDATE render_jobs → 'succeeded'
                                                                  │      (or requeue / 'failed' DLQ)
```

**Production mode — in-container rendering (`render-worker/`).** The worker is its own
Docker image (repo-root build context, `render-worker/Dockerfile`: node:20 + Chromium libs
+ ffmpeg), deployed as a Coolify worker resource — no exposed port, no domain. Facts from
[`render-worker/worker.js`](../../render-worker/worker.js):

- It bundles **`editor/src/remotion/index.ts`** — the *same* entry the browser editor uses
  (`registerRoot(Root)`, composition id `Main`) — with `@remotion/bundler`, once per
  process, then `selectComposition()` (which runs `Root.tsx` `calculateMetadata` to resolve
  width/height/fps/duration/fontInfos from the manifest) and `renderMedia()`. The
  `inputProps` mapping lives in one auditable file,
  [`props-from-manifest.js`](../../render-worker/props-from-manifest.js) — update it in
  lockstep if the `UndoableState` manifest shape changes.
- **Remotion versions must match the editor exactly.** Both the editor and the worker pin
  `4.0.483`; a mismatch has broken video bundling before.
- The same container hosts the **media sidecar HTTP services** the app's `*-engine.js`
  adapters call: audio FX (`audio-fx-service.js` — real ffmpeg DSP), upscale, remove-bg,
  vectorize, and the Auto-Clips **clip proxy** (`clip-proxy-service.js` — a
  time-preserving 1-fps low-res proxy so hour-long sources fit Gemini inline). Each is
  deploy-gated by its `*_PORT` env on the worker and `*_URL` on the app. (In the current
  prod fleet the clip proxy also runs as a dedicated zero-dep container; folding it into
  the worker is a known follow-up.)
- If you change anything under `editor/src/remotion` (or the `editor/src/editor/*` it
  imports), **redeploy the worker** or output won't match preview.

**Alternate mode — the injected-engine worker (`src/render-worker.js`).** The app codebase
carries a second, zero-dependency queue worker (`npm run worker`) with the frame render
**injected** as `renderFn`. The default engine,
[`src/render-engine.js`](../../src/render-engine.js), POSTs the job
(`{compositionId, manifest, hyperframesHtml, codec}`) to **`RENDER_WORKER_URL`** — an HTTP
render service — and streams back the video bytes; an absent URL is a clear error, never a
silent hang (`RENDER_ENGINE` can point at a different engine module). This split exists so
the queue/ledger logic is fully unit-testable offline and so a GPU render service can live
behind an HTTP seam; the in-container worker above is what production runs.

Shared queue semantics (both workers, migration 017):

- **Atomic claim + lease + bounded retry.** Claims use `FOR UPDATE SKIP LOCKED` (safe to
  scale replicas), stamp `locked_at`, and increment `attempts`. A job whose worker died
  sits in `running` until its 15-minute lease lapses, then is reclaimed — bounded by
  `RENDER_MAX_ATTEMPTS` (default 3). Early failures **requeue** (preserving the requested
  codec, which rides the otherwise-free `error` jsonb); at the cap the job is left
  `failed` — the DLQ — instead of looping forever.
- **Progress + heartbeat.** Fine-grained progress persists under `error->progress`
  (`render.js` reads it; otherwise progress derives 0/0.5/1 from status), and each progress
  write re-stamps `locked_at` so a long render is never reclaimed and double-rendered
  mid-flight.
- **Watermarking is plan-gated server-side**, default-deny: the worker looks the plan up in
  the DB and bakes `watermark` into the manifest it renders — only `pro`/`enterprise` get a
  clean export. Never trust the (editor-controlled) manifest for this.
- **Enqueue is gated by `RENDER_ENABLED`.** `enqueueRender` refuses with a clear error
  unless `RENDER_ENABLED=true`, so the product never silently accepts jobs no worker will
  claim. Valid codecs: `h264` (mp4) and `vp8` (webm).
- **On success** the worker inserts an idempotent `assets` row
  (`on conflict (bucket, object_key)`), records a **render QA report**
  ([`render-qa.js`](../../src/render-qa.js) + [`render-qa-analyzer.js`](../../src/render-qa-analyzer.js))
  under the job's jsonb, mints a **Content Credential** for the rendered bytes
  (best-effort — a missing `CREDENTIAL_SIGNING_KEY` never fails a render), and fires the
  `render_succeeded` **automation trigger** with a durable HMAC-signed `assetUrl` — which
  is how a scheduled recipe's output flows straight into social publishing.
- **Output is served through the same signed proxy** as generated clips
  (`outputUrl = signedAssetUrl(output_key)`).

Deep dive (bundling, Docker context gotchas, ffmpeg audio-FX chain, GPU notes):
[render-worker.md](render-worker.md).

### Native shorts

`POST /api/shorts` ([`shorts.js`](../../src/shorts.js)) repurposes a finished film into a
vertical short. It is a **pure manifest transform** — no ffmpeg, no transcription, no
probing on the API box. It workspace-scope-loads the source `compositions.manifest`, applies
one uniform center-cover affine scale to `compositionWidth/Height` and every item's geometry
(`reframeManifest`, which also scales absolute-px style fields like caption `fontSize`),
copies every other field verbatim into a **new** campaign + composition, and enqueues a
normal render. A short is therefore exactly as render-safe as the film it came from.
Targets: `9:16` (default), `1:1`, `16:9`.

---

## Subsystem overviews

Each of these is a self-contained slice with its own deep doc or planning file; this
section is the map.

### The agentic Director — PRODUCE → CRITIQUE → ASSEMBLE

The self-directing film loop, split across two modules and two routes:

- **Plan.** `POST /api/director/plan` calls `planFilm` ([`director.js`](../../src/director.js)),
  enriched by **vote-learning** (`getPreferenceBrief`) and **pgvector semantic recall**
  (`findSimilarPrompts`) so the director learns from what worked for *similar* ideas.
- **Produce.** `POST /api/director/produce` runs `produceFilm`
  ([`producer.js`](../../src/producer.js)): PRODUCE each shot (shots `2..n` chain the
  previous shot's last frame as `startImageUrl` for continuity), CRITIQUE each against the
  QA bar (`scoreVideoReadiness` in [`qa.js`](../../src/qa.js)) and selectively regenerate
  misses, then ASSEMBLE the editor composition IR. The heavy lifting sits behind injectable
  seams (`generate` / `extractLastFrame` / `score`) so the loop unit-tests offline.
  **No browser surface should call `/api/director/produce` directly** — it is long-running
  and 502s behind the edge; the editor and dashboard both run the client-side
  plan → generate → poll → assemble flow instead.

Deeper orchestration/eval design: [`../AGENTIC-ADK.md`](../AGENTIC-ADK.md).

### The AGENT kernel

[`src/agent.js`](../../src/agent.js) is the suite-wide conversational agent: it turns a
natural-language request into a **validated, cost-estimated, multi-step plan** over the
real JSON command engine, then executes it atomically behind an approval gate.

- **Two-phase by construction.** `POST /api/agent/plan` → `interpret()` PLANS with zero
  side effects; `POST /api/agent/execute` → `runAgentPlan()` EXECUTES only the plan the
  caller approved. Every step is classified `read` / `mutate` / `generate` — reads
  auto-run, writes and generates gate.
- **Engine steps are one atomic batch** (they compile into a single JSON-engine
  `EngineRequest` batch that rolls back wholesale on failure). Generative tools the
  executor runs inline: `generate.image` / `generate.voice` / `generate.music` / `recipe`.
  Cross-studio verbs (`director`, `upscale`, `publish`, `create.short`, `auto.reframe`,
  `cut.silences`, `add.captions`) are **planned + priced but deferred** to their durable
  gated paths — the dispatcher reports them deferred with no inline spend.
- **Cost governance before spend.** `estimatePlanCost` produces a low/expected/high credit
  range for the approval card (director shot count and recipe fan-out are the uncertainty);
  a pre-flight budget check throws 402 when billing is enabled and the worst case exceeds
  remaining credits. `committedCredits` computes the honest post-run figure.
- **Every run is a ledger row** (`agent_runs`, migration 032, via
  [`agent-runs.js`](../../src/agent-runs.js)): intent → plan JSON → per-step status/result →
  credits → verdict. Ledger writes are **best-effort** — an audit failure never fails the
  execution. Read back via `GET /api/agent/runs[/:id]`.
- **Tooling surface.** `GET /api/agent/tools` returns the declarative tool registry
  ([`agent-tools.js`](../../src/agent-tools.js)); `?format=functions` emits the
  function-declaration shape for a model tool-use loop. `POST /api/agent/ask` answers
  read-only questions about a project; `POST /api/agent/plan-to-recipe` converts an
  approved plan into a reusable recipe. Live progress streams over SSE at
  `GET /api/agent/compositions/:id/events`.
- Input safety rides [`chat-safety.js`](../../src/chat-safety.js) (`redactSecrets`,
  `moderateInput`); the reasoner is injectable with a deterministic mock so the whole
  kernel is testable key-free offline.

### Auto-Clips: long-form → shorts

A white-label long-form → vertical-shorts workflow on the Vertex/Remotion stack, zero new
deps. The studio page is `/auto-clips` (self-contained, client-side pipeline:
prepare → detect → select → captions → generate → poll). Stages:

1. **Prepare (long sources).** `POST /api/highlights/proxy`
   ([`autoclips-proxy.js`](../../src/autoclips-proxy.js)) builds a time-preserving 1-fps
   low-res proxy via the render-worker clip-proxy service (`CLIP_PROXY_PORT`/`CLIP_PROXY_URL`
   deploy-gated) so hour-class videos fit Gemini inline.
2. **Detect.** `POST /api/highlights/detect` → `detectHighlightsFromVideo`
   ([`highlights.js`](../../src/highlights.js)) — video-native Gemini analysis replaces a
   GPU heuristic stack; returns ranked `{startMs, endMs}` windows.
3. **Cut + render.** `POST /api/auto-clips` → `createAutoClips`
   ([`auto-clips.js`](../../src/auto-clips.js)): for each selected window, build a
   single-clip composition trimmed to the window, reframe with the **same** tested
   center-cover affine `createShort` uses, insert a new composition, and enqueue an
   ordinary render — so every output lands in storage + the Media browser automatically.
   Pure manifest transform + enqueue; batch capped at `AUTO_CLIPS_MAX` (default 10).
4. **Captions / voiceover / story.** `POST /api/captions/generate` produces styled captions
   (9 personas, [`caption-styles.js`](../../src/caption-styles.js)); clips can burn them in
   on their own track. `POST /api/auto-clips/voiceover` synthesizes a per-persona voiceover
   ([`voice-presets.js`](../../src/voice-presets.js)) and ducks the source audio (-10.5 dB).
   `POST /api/auto-clips/story` ([`story-narrator.js`](../../src/story-narrator.js)) writes
   one narrative arc across N clips.
5. **Publish.** `POST /api/auto-clips/publish` schedules the finished clips to social via
   the relay below.

Infra deep-dive: [auto-clips-infra.md](auto-clips-infra.md); plan:
`planning/AUTO_CLIPS_PILLAR.md`.

### Recipes, checkpoints, and budget governance

Programmable creative workflows (the "intent wizard" + autopilot engine), all pure JSON:

- **A recipe** ([`recipes.js`](../../src/recipes.js), migration 016) is either a flat
  `steps[]` list or a **staged** pipeline (`stages[]` with `requires`/`produces` artifact
  wiring, explicit unique stage ids, ≤20 stages / ≤50 steps). Every step kind is validated
  against the runner's whitelisted `STEP_KINDS`
  ([`recipe-runner.js`](../../src/recipe-runner.js)); a parity test locks the editor's
  workflow-graph templates to the server's step kinds.
- **Runs are detached.** A run records `running` immediately and executes in the
  background; callers poll `GET /api/recipes/:id/runs/:runId`. Scheduling rides
  `/api/automations` (a `schedule` trigger wrapping a `run_recipe` action, migration 014).
- **Budget governance** ([`budget.js`](../../src/budget.js)): a pre-flight estimator
  (`estimateRecipe` — credit range + vendor-cents range + a 2-scene sample cost) and a
  run-scoped reserve → reconcile → refund ledger with three modes — `observe` (never
  blocks), `warn` (flags over-budget), `cap` (hard 402). Video reserves the conservative
  high end because the real shot count isn't known until the plan exists.
- **Checkpoints** ([`checkpoints.js`](../../src/checkpoints.js), migration 019): a staged
  run can pause `awaiting_human` at creative decision points under a policy
  (`guided`/`manual`/`auto`); approve resumes past the stage, send-back re-runs it with the
  reviewer's note as audit trail, and completed stages' artifacts seed the resume so
  upstream work is never re-paid. The whole path is **soft-gated by a `to_regclass`
  probe** — pre-migration deploys simply run straight through instead of 500ing.

### Chat, support, and self-healing

In-house chat infrastructure (migration 027), ported clean-room onto the pg-only
SSE/Vertex stack:

- **Front-of-house widget.** [`chat-routes.js`](../../src/chat-routes.js) owns the public
  surface: `POST /api/chat` streams the answer over SSE, grounded on the **global
  `chat_kb` RAG corpus** (blog/docs/faq/pricing — deliberately not workspace-scoped; the
  pre-sales bot answers anonymous visitors; ingest with `npm run kb:chat`).
- **Handoff + live agent.** A visitor can request a human; `chat-handoff.js` publishes
  `handoff_requested`/`live_message`/… to the `admin:support` channel, and the God-Console
  live-agent surface ([`admin-chat-api.js`](../../src/admin-chat-api.js)) subscribes and
  takes over the conversation in real time.
- **Self-healing loop.** A bug-category support ticket automatically gets an AI-fix brief
  ([`self-heal.js`](../../src/self-heal.js)): title/severity/repro/expected/actual/suspected
  areas/acceptance, stored in `self_heal_briefs` with a status lifecycle
  (`queued → in_progress → pr_open → done`).

Deep dives: [chat-system.md](chat-system.md), [self-healing.md](self-healing.md).

### Editor comments + composition versions

- **Comments** ([`comments.js`](../../src/comments.js), migration 030) are
  element/timecode/region-anchored threads on a composition with @mentions and resolve.
  They ride the proofing comment table but are deliberately **ungated** (team comments are
  core collaboration, not a Pro feature) and must never import the proofing gate. Adds and
  resolves publish to the composition's SSE channel as an enhancement. The migration's
  safety invariant: every proofing query filters by `proof_id`, so editor comments
  (`proof_id` null) can never leak into a tokenized `/proof/:token` view.
- **Version history** ([`composition-versions.js`](../../src/composition-versions.js),
  migration 028): every save snapshots the manifest (skipping byte-identical re-saves via a
  sha256 hash), pruned to the most recent 20. Restore snapshots the *current* state first
  ("Before restore"), so restore itself is always reversible.
  `GET /api/compositions/:id/versions`, `POST .../versions/:n/restore`.

### Audio sessions: the manifest split invariant

The Audio studio's session tracks live in `UndoableState.tracks/items` at **runtime**
(tagged `track.sessionId`), but **at rest** they are split into
`manifest.audioSessions[id].{tracks,items}` — a persisted `manifest.tracks` must **never**
contain a session track. [`src/audio-sessions.js`](../../src/audio-sessions.js) is the
server mirror of the editor's split, kept in lockstep:

- `splitSessionsFromManifest(manifest)` normalizes an illegal tagged-root-tracks shape into
  the at-rest split — **applied on every save** in `saveComposition` (`api.js`). It is
  idempotent and returns the *same reference* for already-clean manifests, so old projects
  persist byte-identically.
- Read paths (render/QA/query) filter with `videoTracks()` / `sessionItemIds()`; the
  session predicate is always **track-based** (items carry no session marker). Filters are
  for reads only — never write a filtered tracks array back into a manifest.
- Orphaned session tracks are recovered into a reconstructed session entry rather than
  dropped.

The only bridge from a session back to the video timeline is the **bounce** (flattened
asset), with stale-derivative invalidation. Plan: `planning/AUDIO_PILLAR.md`.

### Files pillar and Content Credentials

One Files section is the repository for every uploaded + generated file:

- **`GET /api/files`** ([`gallery.js`](../../src/gallery.js) `listWorkspaceFiles`) lists
  the workspace's full asset repository with `type`/`kind` filters, sort, collection
  filter, and semantic search via `?q=`. `GET /api/media/search` is pure NL → cosine
  ranking over the caller's own embeddings.
- **Collections** ([`collections.js`](../../src/collections.js), migration 031) are named
  workspace-scoped folders; membership is a pure join (no `byte_size` — the storage meter
  sums `assets.byte_size` directly, so a collection can never double-count). The security
  boundary is the cross-tenant attach: an asset joins a collection only when **both**
  belong to the caller's workspace. Custom fields land in a dedicated `assets.metadata`
  jsonb (`PATCH /api/files/:id/metadata`), kept separate from `provenance`.
- **Deletion** ([`assets.js`](../../src/assets.js)) removes the object and the row,
  ownership-gated.
- **Content Credentials** ([`content-credentials.js`](../../src/content-credentials.js)):
  a zero-dependency, CoreReflex-signed (Ed25519 via `crypto-box.js`) provenance record for
  rendered/generated assets, including an explicit `aiGenerated` flag. Verification
  (`GET /api/assets/:key/credentials`, public by design) re-hashes the **live** object
  bytes so a tampered object fails; the signing public key is at
  `GET /api/credentials/pubkey`. **Honest-messaging contract (load-bearing): this is NOT
  C2PA** — no JUMBF box, no X.509 chain, will not verify in CAI tools; never claim
  otherwise.

Plan: `planning/FILES_PILLAR.md`.

### Social publishing

[`social.js`](../../src/social.js) (migration 029) is a **relay through GoHighLevel**: a
workspace connects its GHL account once — the API key is validated live before storing and
encrypted at rest with [`crypto-box.js`](../../src/crypto-box.js) (AES-256-GCM, scrypt key
derived from `SECRET_ENCRYPTION_KEY`/`JWT_SECRET`, per-record salt) — then pushes a
render/asset to its connected platforms (publish now or schedule) and pulls status back.
Every relayed post is recorded in `social_posts` (a failed relay is auditable, not lost),
and one `usage:publish` credit is metered **before** egress. Routes:
`/api/social/connection` (connect/status/disconnect), `/api/social/accounts`,
`/api/social/posts` (publish/list). The `render_succeeded` automation trigger's
`{{assetUrl}}` template is what lets a scheduled recipe auto-publish its render. A native
per-platform OAuth path is the v2 upgrade behind the same service functions.

### Brand cascade

[`brands.js`](../../src/brands.js) (migrations 025 + 026) is the account-owned brand layer
(Account → Brand → Project → service). A Brand is a **living container of
individually-approved assets** — each logo/palette/typography/voice/tagline/photo is its
own row with its own draft → approved lifecycle, and the brand's **resolved kit is exactly
its approved assets**, so nothing half-baked ever cascades. Workspaces (Projects) point at
a brand via `workspaces.brand_id` and inherit it; `getWorkspaceBrandKit`
([`brand-kit.js`](../../src/brand-kit.js)) resolves through the brand and falls back to the
legacy per-workspace `brand_kits` row. In-platform generation of brand assets is
[`brand-generate.js`](../../src/brand-generate.js). Gated to Pro+
(`BRAND_STUDIO_ENABLED=true` force-enables, matching `CLIENT_OPS_ENABLED`). Storage add-on
grants (026) raise the account's effective storage limit idempotently by Stripe session id.

### White-label theming

Per-workspace UI theming (migration 033): a theme is `{preset, overrides}` — a
`themes.css` preset name plus up to 12 CSS custom-property overrides.
[`theme-settings.js`](../../src/theme-settings.js) sanitizes **everything** that leaves the
module against a strict allowlist grammar (the server interpolates the result into HTML).
Three cooperating pieces:

- The **server-side injection** in the static handler (see
  [Entry point](#entry-point--srcserverjs)) reads the `cr-theme` cookie for a zero-flash
  first paint.
- **`public/js/theme.js`** is the client bridge: it persists to `localStorage` **and**
  mirrors to a `.corereflex.com`-scoped cookie (so the server and the editor subdomain see
  it), pulls/pushes the workspace default via `GET/PUT /api/theme` when signed in, and
  mounts the switcher FAB on app surfaces only. Presets: Cobalt (default), Quantum Teal,
  Indigo Signal, plus full custom colors.
- **Workspace persistence** (`workspaces.ui_theme`) degrades silently pre-migration
  (reads return null; writes 503 with a clear message).

### Real-time voice agent

Browser or phone audio ⇄ **Gemini Live**, brokered by the WebSocket bridge in
[`src/voice-agent/`](../../src/voice-agent) — attached to the same HTTP server
(`attachVoiceBridge`), so there is still only one public app deployable.

- **Transports.** The browser widget (`/voice`) opens `wss://<host>/api/voice/live`.
  Twilio Voice hits `GET/POST /api/voice/twilio` (TwiML), then streams audio to
  `wss://<host>/api/voice/twilio/media`. The `upgrade` handler routes **only** those two
  WS paths and destroys any other upgrade. `telephony.js` verifies `X-Twilio-Signature`
  when `TWILIO_AUTH_TOKEN` is set and converts G.711 mulaw ⇄ PCM16.
- **Bridge.** `bridge.js` mints a Vertex token and opens the Gemini Live
  `BidiGenerateContent` stream against the **global** endpoint (regional hosts 404 the
  live model slugs). Default model `gemini-live-2.5-flash`; 16 kHz PCM16 in / 24 kHz out.
- **Persona + tools.** `?persona=` selects a system persona (appointment-setter,
  lead-qualifier, sales-assistant, support, assistant); `?calendar=` binds a booking
  calendar so the bot can check slots/book, capture CRM leads, request handoffs, or search
  the workspace KB (`tools.js`).
- **Abuse/cost guards** (unauthenticated + billed, so bounded *before* any handshake):
  WS Origin allow-list (`VOICE_ALLOWED_ORIGINS`), per-IP new-session limit
  (`VOICE_MAX_NEW_PER_IP`), global concurrency cap (`VOICE_MAX_CONCURRENT`), hard
  per-session lifetime + idle timeout.

The plain, non-streaming TTS path (`POST /api/voice`), voice design, and voice cloning are
separate — see [own-the-stack.md](own-the-stack.md).

---

## Data layer

- **Raw Postgres, no ORM.** [`database.js`](../../src/database.js) exposes a `pg` pool plus
  `query()`, `withClient()`, and `withTransaction()` (explicit `begin`/`commit`/`rollback` —
  `register`/`saveComposition`/`createShort` use a transaction so a partial write rolls
  back). The pool strips `sslmode` from the URL and configures TLS itself; size is
  `POSTGRES_POOL_MAX` (app default 10, worker default 4).
- **Migrations are hand-written, additive SQL** in `migrations/*.sql` (`001`–`033`),
  applied once by `scripts/migrate.mjs` and recorded in `schema_migrations`. They run as a
  separate `corereflex_migrator` role (via `DATABASE_MIGRATION_URL`); the app connects as
  `corereflex_app`.
- **pgcrypto + pgvector** are expected extensions (the health check reports `has_vector` /
  `has_pgcrypto`). Embeddings power director vote-learning, semantic prompt recall, media
  search, the workspace KB, and the global chat KB.

Migration map:

| File | Adds |
|------|------|
| `001_core_platform.sql` | accounts, app_users, workspaces, memberships, auth_sessions, brand_kits, assets, campaigns, compositions, render_jobs, storyboards, qa_*, review_events, … |
| `002_app_admins.sql` | app_admins, admin_audit_events, impersonation_sessions |
| `003_grant_runtime_role.sql` | re-grants + DEFAULT PRIVILEGES so the app role sees all tables (fixes a grant gap from 002) |
| `004_votes_visibility_plan.sql` | asset_votes, asset visibility, plan column |
| `005_personas.sql` | personas, reference_media, script_projects |
| `006_billing.sql` | stripe/subscription columns, credits, credit_ledger |
| `007_crm.sql` | crm_contacts, crm_activity, crm_tags, crm_contact_tags, meeting_briefs |
| `008_tasks.sql` | tasks |
| `009_booking.sql` | booking_calendars, member_availability, calendar_events, bookings |
| `010_automation.sql` | automations, automation_runs |
| `011_billing_credit_idempotency.sql` | partial unique index making a duplicate credit grant impossible (Stripe webhook-retry race) |
| `012_webhook_idempotency.sql` | whole-event Stripe webhook dedupe by event id (exactly-once handler) |
| `013_knowledge.sql` | per-workspace RAG knowledge base (knowledge_docs, chunked + embedded, HNSW cosine) |
| `014_automation_schedule.sql` | `automations.next_run_at` — the clock the scheduler tick reads for `schedule` triggers |
| `014_email_verification.sql` | `app_users.email_verified_at` + email_verification_tokens (trust gates on password change / checkout) — **note: two files share the 014 number** |
| `015_generation_jobs.sql` | durable generation_jobs (moves `/api/generate` state out of a process-local Map) |
| `016_recipes.sql` | recipes + recipe_runs (programmable workflows) |
| `017_render_job_retry.sql` | render_jobs `attempts` + `locked_at` — retry / lease reclaim / DLQ |
| `018_api_keys.sql` | platform API keys (sha256 hash at rest, visible prefix, workspace-scoped) |
| `019_workflow_checkpoints.sql` | recipe_run_stages + the `awaiting_human` run status (staged-workflow checkpoints) |
| `020_content.sql` | content_documents + content_templates (the Write pillar) |
| `021_brand_kit_unique_workspace.sql` | unique index per workspace so the brand-kit upsert is atomic |
| `022_proofing.sql` | proofs, proof_versions, proof_comments, review tokens/decisions (client review) |
| `023_analytics_events.sql` | first-party analytics_events (privacy-first: salted IP hash, nullable workspace) |
| `024_leads.sql` | marketing leads (public capture, deduped by email, admin-only read) |
| `025_brands.sql` | brands + brand_assets (per-asset draft→approved lifecycle) + `workspaces.brand_id` |
| `026_storage_grants.sql` | storage add-on grants (idempotent by Stripe session id) |
| `027_chat_selfheal.sql` | chat_kb (global corpus), chat conversations/messages, support tickets, self_heal_briefs |
| `028_composition_versions.sql` | composition_versions + `compositions.updated_at` |
| `029_social_publishing.sql` | social_connections (encrypted GHL key) + social_posts |
| `030_editor_comments.sql` | proof_comments extended to anchor to a composition (in-editor comments) |
| `031_collections_metadata.sql` | collections + collection_assets (membership join) + `assets.metadata` jsonb |
| `032_agent_runs.sql` | agent_runs — the auditable agent-session ledger |
| `033_workspace_theme.sql` | `workspaces.ui_theme` jsonb (white-label theme) |

The full table-by-table reference is in [data-model.md](data-model.md). Ops note: prod's
`schema_migrations` history had a gap at 003 — reconcile before applying new migrations
(see [deployment.md](deployment.md)).

---

## Identity, tenancy, and admin

- **Auth.** Passwords are hashed with scrypt ([`auth.js`](../../src/auth.js)). Sessions are
  HS256 JWTs (`JWT_SECRET`, ≥32 chars), default 12h TTL, stored in the HttpOnly
  `corereflex_session` cookie (apex-scoped, SameSite=Lax) and tracked in `auth_sessions`
  (revocable). `Authorization: Bearer <token>` is a fallback for external clients, and
  **platform API keys** (`crk_…`, migration 018) authenticate a workspace for
  server-to-server use on the routes that accept `requireApiUser`.
- **Email verification** (migration 014) gates sensitive actions (password change, billing
  checkout/portal) until `email_verified_at` is set; signup optionally checks
  deliverability via the usermails verifier (fails open).
- **Tenancy.** `accounts` → `workspaces` → `memberships` (roles: `owner` / `producer` /
  `reviewer` / `client`). Registration creates account + user + workspace + owner membership
  in one transaction. Workspaces may point at an account-level Brand (`brand_id`).
- **God Mode.** `app_admins` is a separate principal table with a privilege model
  ([`permissions.js`](../../src/permissions.js)) — `god_mode`, `impersonate_users`,
  `view_metrics`, `manage_admins`, etc. Admin routes live in
  [`admin-api.js`](../../src/admin-api.js); every privileged action writes an append-only
  `admin_audit_events` row, and impersonation stamps the actor admin into the session so
  it's **never invisible** to the impersonated user. Release modes (production/beta/god) in
  [`modes.js`](../../src/modes.js) gate which UI surfaces are visible. The owner account
  resolves to the top tier via `resolveEffectivePlan` so every Pro gate opens, with an
  owner-only view-as-tier preview.

---

## Client-operations layer (Pro-gated, off by default)

The CRM, tasks, booking, and automation modules ([`crm.js`](../../src/crm.js),
[`tasks.js`](../../src/tasks.js), [`booking.js`](../../src/booking.js),
[`automation.js`](../../src/automation.js)) form a native, white-label client-ops suite.
It is **Pro-gated** — `clientOpsEnabled(user)` requires `workspace.plan = 'pro'` (or
`CLIENT_OPS_ENABLED=true` to force-enable for testing).

Notable design points: public booking endpoints (`/api/calendars/:slug/slots`,
`/api/calendars/:slug/book`, `/api/bookings/:token/ics`) are unauthenticated and
rate-limited; double-booking is prevented with a transactional `pg_advisory_xact_lock`
per assignee; automation triggers (`meeting_briefed`, `render_succeeded`,
`payment_completed`, `schedule`, …) fire **best-effort and fire-and-forget** so they never
break the domain event that triggered them; and the `schedule` trigger is driven by the
in-process scheduler tick started from `server.js`. Full doc: [client-ops.md](client-ops.md).

---

## Billing & metering

[`billing.js`](../../src/billing.js) integrates Stripe via the **raw REST API** (no SDK),
with HMAC-verified webhooks reading the raw request body and whole-event idempotency
(migration 012). The whole system is **inert by default**: `meterUsage()` is a no-op unless
`BILLING_ENABLED=true`, and checkout 503s without `STRIPE_SECRET_KEY`. Expensive AI is
metered against a prepaid **credit ledger** (`credit_ledger`, append-only; running balance
on `accounts.credits`): `usage:veo`, `usage:kling`, `usage:render`, `usage:lyria`,
`usage:imagen`, `usage:voice`, `usage:persona`, `usage:upscale`, `usage:content`,
`usage:captions`, `usage:publish`, and friends (`USAGE_COSTS` is the single price table —
the budget/agent estimators read it too). `meterUsage` throws **402** *before* spend when
credits are insufficient. Plans (`free`/`pro`/`enterprise`) gate watermark removal,
private-mode generation, the client-ops layer, brand studio, and proofing; storage add-on
grants (026) extend the storage cap on any plan. Full doc: [billing.md](billing.md).

---

## The editor SPA, briefly

[`editor/`](../../editor) is a **React Router v8** SPA (SSR disabled, React 19, Vite 8,
TypeScript 6, **Remotion pinned at 4.0.483** — keep the render worker in lockstep) that
shells the Remotion `<Player>` with a multi-track timeline, canvas, and inspector — plus
the sibling studios that share the same project store: the **image studio**, the **audio
studio** (session tracks — see the [manifest split invariant](#audio-sessions-the-manifest-split-invariant)),
the **brand studio**, the **diagram studio** (branded Excalidraw — the one whitelisted
dependency besides Remotion's own stack), and the **workflow studio**, whose node graph
runs on the in-house **CrFlow** canvas engine
(`editor/src/editor/workflow-studio/graph/flow/` — pan/zoom, bezier edges, drag, connect,
minimap; React Flow was removed to stay fully white-label). The Agent panel and the
comments panel ride the SSE streams described above.

State is the `UndoableState` IR, persisted as `compositions.manifest` on the app. In dev,
`vite.config.ts` proxies `/api/*` to the app (`CORE_API_TARGET`) and rewrites the session
cookie first-party. Asset uploads always go through the app's upload proxy; keyframe
animation, markers, the SMPTE/BPM grid, per-clip audio FX, and export/import live under
`editor/src/editor/` — see [editor.md](editor.md). The user-facing tour is in
[`../user/editing-and-mastering.md`](../user/editing-and-mastering.md).

---

## Quick reference: build & run

| Command | What it does |
|---------|--------------|
| `npm start` | Run the app server (`src/server.js`, port `PORT` / 4173). |
| `npm run worker` | Run the injected-engine queue worker (`src/render-worker.js`; needs `RENDER_WORKER_URL` or `RENDER_ENGINE`). Production rendering normally runs the `render-worker/` container instead. |
| `npm run lint` | Syntax check via `scripts/check-syntax.mjs` (also the `build` step). |
| `npm test` | `node --test` over [`test/`](../../test). |
| `npm run db:migrate` | Apply pending `migrations/*.sql` (uses `DATABASE_MIGRATION_URL`). |
| `npm run verify:backend` | End-to-end backend smoke check. |
| `npm run kb:embed` / `npm run kb:chat` | Batch-embed the workspace KB / ingest the global chat KB corpus. |
| `npm run admin:set-password` | Provision/rotate an app-admin password. |

The app's Docker image ([`Dockerfile`](../../Dockerfile)) is `node:20-alpine`, installs only
production deps (`npm ci --omit=dev`), and health-checks `/healthz`. The editor and render
worker have their own Dockerfiles under [`editor/`](../../editor) and
[`render-worker/`](../../render-worker) (the worker **must** build with the repo root as
context so `editor/src` is reachable — see [render-worker.md](render-worker.md)). The
`/api/health` `version` field proves which build is live — set `BUILD_VERSION` in Coolify
per deploy. Deployment runbook: [deployment.md](deployment.md).