🧠

SE Intel

A role-gated multi-agent revenue intelligence platform — pre-call research and sales coaching, grounded in a 102-chunk RAG knowledge base with streaming responses and cross-session memory.

View the full Build Log (14 study chapters, weekly theory insights) →

Live

Workers Durable Objects Workers AI Vectorize KV D1 Hono

Cycle 1 — Production Hardening (June–July 2026)

Everything below "What I Built" was the original prototype. Since then I've been running a weekly hardening cycle against this same system — each week answers one specific production question and ships a deterministic proof, not just a demo.

Week 1 — Multi-tenancy. Added a real orgId boundary across RAG (Vectorize metadata filter), memory (Durable Object keys scoped to orgId:userId, KV keys scoped to ltm:{orgId}:{userId}:{factId}), and audit (org-scoped, role-split reads). Three deterministic admin probes prove zero cross-org leakage — no LLM in the test path. Read the writeup →

Week 2 — Evals as a CI gate. The LLM-as-judge harness was scoring "groundedness" but missed a real bug: the KB chunk said a 35% discount, the agent answered 25%. Added a deterministic faithfulness check that compares what was actually retrieved against what the response says — separate from the LLM judge, so it can't be masked by the same hallucination it's hunting. The harness caught a second real, flaky bug on its first run. Read the writeup →

Week 3 — Observability + SLOs. Every agent request now writes a metric (latency, org, status) via waitUntil(), feeding a per-org health scorecard with p50/p95 latency, error rate, and error budget remaining. The interesting part: the initial p95 target (8000ms) was a guess — real 70B-model-with-RAG traffic measured 11-13 seconds. Recalibrated to 15000ms with the before/after numbers documented and a concrete improvement roadmap (KB answer caching, smaller-model routing for simple queries), instead of shipping a permanently-red dashboard.

Week 4 — Failure under load. Two fixes: (1) found a concurrency bug in the KV-based rate limiter — 25 concurrent requests only counted ~10. Moved the counter into a Durable Object where all requests are serialized (actor model, no locks needed). Verified: 20 concurrent requests counted exactly 20. (2) Built a 3-tier fallback chain: primary 70B model → fallback 8B model → canned degraded response. An X-Debug-Fallback header lets you demo degraded mode on demand. Full technical details in the build log →

The Problem

Enterprise sales teams spend 30-60 minutes on manual pre-call research for every prospect. Objection handling is inconsistent — different reps give different answers to the same customer question. And the technical knowledge that lives in internal playbooks never reaches the people who need it in front of a customer.

The standard AI assistant approach — one model, one chat box, same answer for everyone — ignores the fact that different roles should be asking different questions and getting different depths of answer.

What I Built

Two AI agents — AccountIntelAgent and EnablementAgent — each running as a Durable Object with its own embedded SQLite conversation history. Five roles (account_executive, solutions_engineer, csm, tam, manager) with JWT-gated access to three knowledge base namespaces: public (80 product chunks), technical (12 deep-dives), and leadership (10 deal strategy and pricing chunks).

Responses stream token-by-token via Server-Sent Events — tool badges appear before the first word, so the user sees what the agent retrieved before reading the answer. After every response, a lightweight LLM call extracts personal facts ("working on Stripe deal, 300 engineers") and stores them in KV — injected into the system prompt on every future conversation.

The system includes a Python evaluation harness: 15 test cases covering both agents and all five roles, an LLM-as-judge scorer (4 dimensions, 12-point scale), and a regression report that diffs against the previous run.

SE Intel dual-panel chat UI showing Account Intel and Enablement agents with streaming responses and tool badges

Architecture

Browser
  │
  ├─ POST /api/v1/account/stream
  │    Authorization: Bearer <HS256 JWT>
  │
  ▼
Cloudflare Worker (Hono router)
  ├─ extractUserContext()  → verify JWT → UserContext{userId, role, orgId}
  ├─ checkRateLimit()      → KV sliding window → 429 if exceeded
  └─ stub.fetch("/stream") → Durable Object for this userId
       │
       ▼
  AccountIntelAgent (Durable Object — one per user)
  ├─ memory.getHistory()   → SQLite: last 20 turns this threadId
  ├─ ltm.formatForPrompt() → KV: stored facts about this user
  ├─ buildSystemPrompt()   → role-specific + LTM injection
  ├─ dispatchTools()
  │    ├─ fetchNews()      → NewsAPI (se/tam/manager only)
  │    ├─ kbSearch()       → Vectorize RAG (namespace-filtered by role)
  │    └─ webSearch()      → DuckDuckGo → KB fallback
  ├─ AI.run(stream: true)  → Llama 3.3 70B → ReadableStream
  ├─ SSE: tools → tokens → done
  └─ state.waitUntil()
       ├─ memory.append()  → SQLite
       ├─ writeAudit()     → D1
       └─ extractFacts()   → Workers AI → KV

Technical Deep Dive

Why Durable Objects for agent state

A Worker is stateless — every request might hit a different isolate. If a user sends two messages quickly, both requests could read the same conversation history in parallel and write back conflicting state. Durable Objects solve this: each user gets exactly one DO instance, single-threaded, with serialized request handling. Alice's agent can never run concurrently with itself. Each DO has its own embedded SQLite (10GB), so Alice's history is physically isolated from Bob's — not just logically separated by a key prefix.

Three-tier RAG with role gating

Queries are embedded with BGE-base-en-v1.5 (768 dims), searched against Vectorize with cosine similarity, and filtered by the user's allowed namespaces at query time. Access is enforced at three layers:

1. ROLE_KB_ACCESS map in types/index.ts
2. Re-check inside the kbSearch tool at execution time
3. Vectorize filter: { namespace: ns } at the query layer

Defense in depth: if any one layer has a bug, the other two still enforce the constraint. Multiple namespaces are queried in parallel with Promise.all, results merged and sorted by score.

Streaming SSE

AI.run({ stream: true }) returns a ReadableStream. The DO's handler parses the Workers AI SSE format and re-emits three event types: tools (fires before first token, shows what was retrieved), token (one per token), done (latency + tools used). The Worker passes the DO's stream straight through to the browser with no buffering — TTFB under 100ms.

Long-term memory extraction

After every response, a short LLM call extracts personal facts from the exchange ("prefers bullet points", "working on Stripe deal") and writes them to KV via state.waitUntil(). Without waitUntil, the DO isolate terminates before async work completes — a silent failure. On subsequent requests, ltm.formatForPrompt() injects the stored facts into the system prompt automatically.

Evaluation harness

15 test cases across both agents and all roles. Notably: paired test cases ask the same question as two different roles — the judge's role_appropriateness dimension penalizes over-technical responses to non-technical roles and vice versa. LLM-as-judge scores 4 dimensions (0-3 each), pass threshold 8/12, regression report diffs against previous run.

Cloudflare Products Used

Workers — Hono router, JWT auth middleware, rate limiting, SSE stream passthrough, admin routes. Zero cold starts means streaming TTFB under 100ms.

Durable Objects (SQLite) — One DO per user per agent. Embedded SQLite for conversation history. Single-threaded execution eliminates race conditions on concurrent writes.

Workers AI — Llama 3.3 70B for chat generation (streaming), BGE-base-en-v1.5 for embeddings, Llama 3.3 70B again for memory extraction and eval judging. All free on the paid plan.

Vectorize — 102 vectors across 3 namespaces. Cosine similarity search with metadata namespace filtering enforces RBAC at the retrieval layer.

KV — Two namespaces: rate limit counters (TTL-keyed sliding window) and long-term user memory (fact store, evicts oldest beyond 50 facts).

D1 — Two tables: audit_log (userId, role, agent, tools used, org-scoped reads for compliance) and request_metrics (latency, status, agent type — feeds the per-org SLO health scorecard). Kept separate because one carries PII and one doesn't.

What I Learned

• state.waitUntil() is not optional. Fire-and-forget async work inside a DO dies when the Response is returned. Without it, memory extraction silently produces zero results — no errors thrown.
• Workers AI returns objects, not strings. When the model responds with JSON, result.response is sometimes already a parsed object. Always check typeof before calling JSON.parse().
• Defense in depth beats DRY for access control. Enforcing RBAC at three layers (type map, tool execution, Vectorize filter) means any single bug doesn't break security.
• Paired eval cases are the most valuable. Running the same question as two different roles catches both over-permissive KB access and mis-calibrated system prompts in a single test.

Why I Built This

• I wanted to understand what it actually takes to go from a demo to a production-ready agent — two real agents, three tools, role-gated access, deployed and live.
• Evaluation was the part I cared most about. Building the LLM-as-judge harness with regression tracking and RBAC test cases taught me more about agent reliability than any tutorial.
• RAG with namespace filtering, streaming SSE, and cross-session memory are the patterns that separate a real system from a weekend project. I wanted all three working together in one codebase.