# Vertex scale & ops runbook — Agent Engine and Provisioned Throughput

This is an **operational** runbook, not app code. corereflex deliberately self-hosts
its agentic stack (direct Veo + Gemini via Vertex, the model router, MinIO
footage-residency, a near-zero-dep Node service). The two capabilities below are
**managed Google infrastructure** you reach for only when self-hosting stops paying
off — so they live here as playbooks with the trigger, the commands, and the
cost trade-off, rather than as code we ship by default.

For the agent **features** already implemented in-process, see the bottom of this
doc and [AGENTIC-ADK.md](../AGENTIC-ADK.md).

---

## 1. Vertex AI Agent Engine (managed agent runtime)

**What it is.** A fully managed runtime for deploying an agent (ADK / LangGraph /
custom) with autoscaling, request tracing, and a managed **Memory Bank** for
cross-session state — the things our single Node service does not get for free.

**When to adopt it (the trigger).** Defer until at least one is true:
- planning/eval traffic outgrows the Node service's vertical headroom and you want
  autoscaling without running the fleet's container orchestration yourself;
- you need durable **cross-session agent memory** (character cards, brand canon,
  continuity) beyond what pgvector + `knowledge_docs` already give us;
- you need built-in distributed tracing of multi-step agent runs.

Until then, the in-process pipeline (`director.js` → `producer.js`) on the
self-hosted fleet is cheaper and keeps the near-zero-dep posture honest.

**Deploy (when the trigger fires).** Agent Engine deployment is driven from the
ADK, not gcloud CLI. With `@google/adk` (TS, GA — see AGENTIC-ADK.md) the shape is:

```bash
# One-time: enable the API + grant the runtime SA.
gcloud services enable aiplatform.googleapis.com --project girard-media
gcloud projects add-iam-policy-binding girard-media \
  --member="serviceAccount:corereflex-vertex@girard-media.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"
```

```ts
// Deploy the agent graph (SequentialAgent(plan, produce, assemble) + LoopAgent critic)
// behind src/agents/ — adapters wrapping our existing tools, no logic moved.
import { AgentEngine } from '@google/adk';
const engine = await AgentEngine.create({
  project: 'girard-media',
  location: 'us-central1',
  agent: rootAgent,                 // the ADK graph
  requirements: ['@google/adk'],
});
// engine.resourceName -> store as AGENT_ENGINE_RESOURCE; call via engine.query(...).
```

Wire `AGENT_ENGINE_RESOURCE` into the service and route planning to it behind a
flag (mirror the `DIRECTOR_EVAL` / `GEMINI_CONTEXT_CACHE` opt-in pattern) so the
in-process path stays the default and the offline `mockReason` path stays testable.

**Cost trade-off.** Agent Engine bills for the managed runtime (vCPU/GB-hours) **on
top of** per-token model calls — you pay for always-on managed capacity. Worth it
when autoscaling + Memory Bank + tracing replace ops work; wasteful while one Node
box serves planning comfortably.

**Teardown.** `engine.delete()` (or the console) — the runtime keeps billing until
removed.

---

## 2. Provisioned Throughput (reserved model capacity)

**What it is.** A reservation that guarantees a fixed throughput (GSU —
Generative AI Scale Units) for a specific model at a fixed monthly/weekly price,
instead of pay-as-you-go (PAYG) with shared-pool rate limits.

**When to adopt it (the trigger).** Switch from PAYG only when:
- sustained Gemini/Veo volume is high and **predictable** (not spiky), so a
  reservation is cheaper than metered tokens; **and/or**
- you're hitting PAYG quota/429s at peak and need **guaranteed** capacity (e.g. a
  launch, a large client batch) rather than best-effort shared throughput.

For our current single-tenant planning + occasional render bursts, PAYG is the
right default. Re-evaluate when the God-Console profitability view shows steady,
high vendor token spend (see `src/admin-api.js` `profitabilitySummary`).

**Estimate + reserve.**
```bash
# 1. Estimate GSU need for a model from your peak QPS + token sizes
#    (use the console's Provisioned Throughput estimator, or Google's GSU calculator).
# 2. Create the order (commitment term + GSU count) for the target model:
gcloud ai provisioned-throughput create \
  --project=girard-media --region=us-central1 \
  --model=publishers/google/models/gemini-3.5-flash \
  --gsu-count=1 --commitment-duration=WEEKLY
# 3. List / verify:
gcloud ai provisioned-throughput list --project=girard-media --region=us-central1
```

> Note: Gemini 3.x serves on the **global** endpoint (see `geminiEndpoint` in
> `director.js`); confirm the model+region combination supports Provisioned
> Throughput before ordering — not every model/region pair does.

**Use it.** Once active, requests to that model+region draw from the reservation
automatically. To **force** dedicated-only (fail instead of spilling to PAYG), send
the header so overflow doesn't silently bill at PAYG rates:

```
X-Vertex-AI-LLM-Request-Type: dedicated   # dedicated-only; omit for spillover to PAYG
```

This maps cleanly onto the director/eval/embeddings fetch calls — add the header in
`vertex.js` request building behind a `VERTEX_DEDICATED=true` flag if/when reserved.

**Cost trade-off.** A reservation is a **fixed commitment** billed whether or not
you use it — cheaper per token only above the break-even utilization. Below it,
PAYG wins. Never reserve "to be safe"; reserve against measured, sustained demand.

**Teardown.** Reservations bill for the full commitment term; don't over-commit.
Prefer the shortest term (weekly) until demand is proven, then move to longer terms
for the better rate.

---

## Already implemented in-process (no managed infra needed)

These agent-platform capabilities ship in the codebase today and need **no** Agent
Engine / Provisioned Throughput:

| Capability | Where | Notes |
|---|---|---|
| Structured output (controlled generation) | `director.js` `directorGenConfig` (PLAN_SCHEMA), `eval.js` (EVAL_SCHEMA) | responseSchema JSON, no fragile parsing |
| Google Search grounding | `director.js` `directorGenConfig` | `DIRECTOR_GROUNDING=true` |
| Thinking budget | `director.js` `directorGenConfig` | `GEMINI_THINKING_BUDGET` |
| RAG grounding (knowledge base) | `src/kb.js`, migration `013_knowledge.sql`, voice `kb_lookup` tool | pgvector HNSW; per-workspace; grounds Support/Sales agents |
| LLM-as-judge eval | `src/eval.js`; `planFilm` attaches `plan.eval` | `DIRECTOR_EVAL=true`; degrades to offline `mockJudge` |
| Context caching | `src/gemini-cache.js`; used by `director.js` | `GEMINI_CONTEXT_CACHE=true`; token-fit guarded, no-ops below the model minimum |
| Batch (re-)embedding | `scripts/batch-embed.mjs` (`npm run kb:embed`) | backfill / model-swap for `knowledge_docs`; `--all`, `--dry` |

All follow the same discipline: **opt-in via env, degrade-safe, offline-testable**
(a key-free mock path), so enabling any of them can save cost/quality but never
breaks a request.