Compoze

2025 · sole founder · sold to a client

I built Compoze on evenings and weekends while working full time at TikTok. It sold document-grounded assistants to businesses. At the end of 2025 it was sold to a client, terms private.

The pitch was narrow. A company has a few thousand documents nobody reads and staff who ask the same handful of questions about them every week. Compoze answered those questions with the documents attached, so the answer could be checked, not believed.

An answered question in Compoze: a knowledge base search over three documents, a structured answer containing a table, and cited sources listed with match scores. — fig. 1 — the tool call is shown, the answer is structured, and the sources sit under it with their match scores.

Working alone

I did all six stages of selling and delivering it, discovery through deployment and training. Knowledge engineering was the slow one. Deciding what belongs in a knowledge base, and what must never go near it, is not a technical question, and you cannot do it for the customer. Running it beside a full-time job shaped the product more than any opinion I had about architecture. Anything that needed me awake did not get built.

Figure 02 · The whole system

One request path, one store, ingest off to the side

A question walks agent, retrieval, pgvector, gateway and cited answer inside one request. Ingest runs on QStash workers and lands in the same Postgres.

Request
Response
Ingest

fig. 2 — the whole system. A question walks ① agent → ② retrieval → ③ pgvector → ④ gateway → ⑤ cited answer inside one request. Ingest runs elsewhere: Ⓐ download, Ⓑ extract, Ⓒ chunk, Ⓓ embed, Ⓔ store. One database, every row under a tenant id and row-level security.

Compoze chat in its empty state, showing the selected agent, four example prompts and the list of tools it can use. — fig. 3 — the empty state. Each agent arrives with its own example prompts and a visible list of what it can reach.

An empty chat box gets you empty questions, so nothing ships blank: example prompts, a visible tool list, past threads in the sidebar. Cheapest fix I made to how much people used it.

One codebase, configured per customer

Next.js 15 and React 19 on the front, Postgres with pgvector behind Drizzle. Every core table carries a tenant id. Tenant checks in the application and Postgres row-level security provide two layers of isolation. One shared database keeps migrations and billing in one place, while tenant scoping has to stay consistent across both layers. On top: three roles (Admin, Manager, User), an admin portal per tenant, and nine models from OpenAI, Anthropic and Google behind one gateway, picked per environment in configuration.

Retrieval is the part you get judged on

Documents are chunked at about a thousand characters. Each chunk gets a one-sentence header placing it in its document and is embedded with OpenAI at 1536 dimensions. A question runs across up to five knowledge bases in parallel. Each lane is hybrid: cosine search over an HNSW index, keyword search beside it because dense search misses exact tokens, the two ranked lists merged with reciprocal rank fusion. Chunks under a 0.35 similarity floor are dropped. An LLM scores the survivors per database, and a cross-encoder reading question and chunk together sets the final order.

Figure 04 · Query

One question, five knowledge bases, every citation checked

A question fans out across five knowledge bases with hybrid search. Only chunks over the 0.35 floor survive, a cross-encoder orders them, and each citation is checked against its chunk before the answer ships.

Question
Retrieval
Answer

fig. 4 — one question, five knowledge bases at once. Only chunks over the 0.35 floor survive, and a cross-encoder orders them.

Citations are stored with the message, checked for entailment against the sentence they support, and rendered with their match scores. A visible score gets people to open two or three citations and check. Nobody trusts the answer before they have.

The prompt is fixed blocks (role, rules, context tagged with chunk ids, history, question) so the static prefix caches per tenant. The rules: answer only from the context, name what is missing, and refuse when the best chunk sits under the floor.

How I knew it worked

A fixed set of question and answer pairs: logged questions, every question a tester asked, and adversarial questions the corpus cannot answer, where the right reply is a refusal. Retrieval and generation are scored apart: context precision for whether the good chunks ranked high, faithfulness for whether each claim is entailed by what came back. The set runs in CI. A prompt, chunker or model change fails on a regression against main. Deltas, not absolute thresholds.

Getting the documents in

Files come from Lark and Google Drive over OAuth, with an MCP server for Lark docs, wiki, sheets and bitable. Ingest is async through QStash: download, extract with LlamaParse, chunk, batch-embed, store. The failure paths took more of my time than the happy one, because a run that dies after the download leaves a document that looks uploaded and answers nothing.

Figure 05 · Ingest

Hash, redact, chunk, embed, store, or fail into a queue you can see

A content hash drops files already seen. PII comes out before chunking. The dashed region runs later on QStash, and every stage can drop out to a dead-letter queue the admin UI shows.

Document
Failure

fig. 5 — a content hash drops files already seen, PII comes out before chunking, and the dashed region runs later on QStash. Each stage is a checkpoint, so a dead run can be retried from the step that broke.

The Compoze admin knowledge base page: a document table with type, size, source connector and ingest status, status filter counts above it, and a banner saying processing continues in the background. — fig. 6 — the admin view of ingest. Every document shows where it came from and where it stopped; the counts across the top are how an admin finds the failures.

Jobs are keyed by content hash, so a re-upload is a no-op; embeddings are cached by chunk hash and model, so editing one paragraph re-embeds one chunk. PII is redacted at ingest, which keeps the embeddings clean. Anything that exhausts its retries lands in a dead-letter queue with the payload and the error attached. A silent drop is worse than a loud failure.

Nine tools: contextual RAG search, retrieve, summarise, read, create and update a document, web search, extract and crawl.

The Compoze agent picker showing four domain agents, each with a short description. — fig. 7 — four domain agents, each with its own prompt, tools and examples. Which ones a tenant sees is configuration.

Safety

Retrieved text is untrusted input. An instruction planted in a synced document can influence the model, so context arrives in a delimited block the prompt declares as data. Tools are scoped per agent and tenant connector. These controls limit the available tool surface. They do not guarantee that the model will ignore a planted instruction.

Implementation inventory

domain agents: 4
agent tools: 9
models / providers: 9 / 3
knowledge bases per search: 5, in parallel
similarity floor: 0.35 cosine
test specs: 27
api routes: 53
database tables: 23
typescript: ~76k lines

Optimisations and operating details

Chat streams by default. If the client disconnected mid-answer and the stream had not completed, an abort listener handed the job to a QStash workflow. Each phase (init, load context, RAG query, generation, finalise) was a durable step, so a retry could resume after the last completed phase. When QStash was unreachable the route fell back to plain streaming.

Edge middleware replaced any client-supplied identity header with the user and tenant id from the verified JWT, so routes trusted it and skipped a database round trip. Indexes led with the tenant id, and chat creation was an atomic insert-on-conflict.

cursor pagination: fetch limit + 1 to learn whether a next page exists, no COUNT
adaptive polling: 3s while processing, 5s while pending, 30s idle, previous data kept so the table never flashes
connector caches: per-connector LRU with per-action TTLs for Drive and Lark, plus a sliding-window rate limit per tenant
sse anti-buffering: no-transform cache headers, X-Accel-Buffering off, framework compression disabled, per-route duration budgets on fluid compute
tool surface: only the tools the agent and the tenant connectors allow reach the model, with a tighter step budget on background runs

With no reviewer, CI is the reviewer. Route contracts are typed, every payload is validated at runtime with Zod, and contract tests run on each push. That is what let me keep changing 53 routes and 23 tables alone. Each request is traced as one span tree with tokens, model and tenant on every span, so pricing is built on cost per tenant per answer. Time to first token is the latency I watched.