Kyle

The problem

Assembling a briefing document is two jobs wearing one hat. Finding sources is the visible one and the easy one. The work that actually takes the time is deciding what the sources collectively support, in what order it should be said, and which claims need attribution. Tools that automate the first job leave you with a folder of tabs and the whole problem still in front of you. Kyle automates the second: it plans a structure, decides whether the topic needs live sources at all, fetches them, writes against them, and audits its own output before anything is delivered. The constraint that shaped every decision below is that a research tool which is confidently wrong is worse than no tool, because a document that looks sourced gets trusted like one.

Architecture

Four agents run in sequence behind a Go orchestrator. The planner receives the topic and must call one of two tools: web_search, or skip_search with a written justification. It cannot answer directly, which is what stops it deciding on its own that research is optional. The researcher scrapes the top results and returns both the text the writer will read and structured source metadata for attribution. The planner then runs a second turn with the results in context and emits the outline. The writer synthesises the document as a typed component library rather than prose, and cites sources by numeric id. The verifier audits the result against the schema before anything is rendered. Upstash Redis holds identity, per-user quota, task state and the run rollups behind the public dashboard. Documents render to DOCX or PDF and upload to Cloudinary. Every agent boundary emits a telemetry span carrying tokens, model, outcome, and a cost computed at that moment from a versioned pricing table.

Decisions and what they cost

1. The writer emits source ids, never URLs. The pipeline resolves each id against what the researcher actually fetched and materialises the reference list itself; any model-supplied source metadata is discarded.

   Alternative considered

   Let the writer emit full citations inline, the way a person would, and trust the prompt to keep it honest.

   Why rejected

   A model that can write a URL can write a plausible URL. Constraining the output to an integer index makes a fabricated citation structurally impossible rather than merely discouraged: the worst a confused model can do is cite a number, and a number outside the fetched range gets stripped before the reader sees it.

   What it cost

   The writer can no longer cite anything it was not handed, including things it knows perfectly well. It also cost a schema change and rendering work in three places, since DOCX, PDF and the web preview all had to learn the new block types. Adding the schema without the renderers would have dropped citations at export, which is the same bug one layer down.

2. Cost is computed at the moment an event is emitted, from a pricing table checked into the repo, with the table version recorded on every run.

   Alternative considered

   Store token counts and multiply by current prices when the dashboard renders.

   Why rejected

   Provider prices change. Computing at query time means every historical figure rewrites itself the next time a rate moves, so last month's cost per document silently becomes whatever this month's prices imply it was. There would be no way to notice.

   What it cost

   The pricing table is now a thing that has to be maintained, and a model with no row in it reports zero. That failure is loud rather than quiet: a test asserts every configurable model has a row, and it caught a model swap within hours of being written.

3. A degraded outcome, distinct from both success and failure. A run that delivers a document but was unresearched, uncited, or cited something never fetched is counted separately.

   Alternative considered

   Two states. Either the request succeeded or it did not.

   Why rejected

   Both existing states were wrong for it. Calling it success hides the exact failure the project exists to prevent, and the dashboard would have shown 100% while shipping ungrounded documents. Calling it failure would refund the user's quota and imply they got nothing, when they got a usable document.

   What it cost

   A third state that every consumer has to handle: the aggregator, the failure taxonomy, the rate limiter, the dashboard. Cost-per-document had to start counting degraded runs, since they burn real tokens, while success rate had to stop counting them.

4. Architecture diagrams are static SVG committed to the repo, not a client-side renderer.

   Alternative considered

   Mermaid, rendered in the browser from a fenced code block.

   Why rejected

   A Mermaid bundle is roughly forty times the page's entire JavaScript budget, it produces nothing with scripting disabled, and it cannot be used in an Open Graph image. A diagram that fails to render is worse than a diagram nobody can edit quickly.

   What it cost

   Every diagram change is now hand-editing SVG rather than editing two lines of text, so diagrams get updated less often than they should.

What broke

• Documents presented as grounded in live sources were written entirely from model priors. The trace showed a researcher stage running for eight seconds and completing successfully. No error, no alert, no elevated failure rate.

  Root cause

  DuckDuckGo began routing every outbound result through its own redirector (//duckduckgo.com/l/?uddg=…). A filter written to skip DuckDuckGo's internal links therefore discarded all ten results on every query. ExecuteSearch returned "No results found." with a nil error, and the writer fell back to internal knowledge exactly as designed for the case where research genuinely finds nothing.

  Fix

  Unwrap the redirector and resolve the real target. More importantly, an empty result set is now recorded as a scraper failure rather than a success, so the next breakage lands on the failure taxonomy instead of nowhere. A unit test pins the redirector shapes seen in a live response.

• Every topic that triggered a web search failed at the planner with HTTP 400: "Function call is missing a thought_signature in functionCall parts."

  Root cause

  Gemini 3.x attaches an opaque thought_signature to tool calls and requires it echoed back when the assistant turn is replayed. The ToolCall struct did not model the field, so it was dropped during deserialisation and the follow-up request was rejected. The entire web-search path was broken, since that is the only flow that replays a tool call.

  Fix

  Round-trip the provider's extra_content as raw JSON rather than modelling it. A narrower type would be a narrower thing to break the next time a provider adds a field. Verified by A/B against the live API: 400 without, 200 with.

• The demo routes returned 404 in production while building correctly on a developer machine.

  Root cause

  A .gitignore rule of "data/", written for the ephemeral telemetry log, also matched frontend/data/demo-runs. All three recorded artifacts were untracked. The local build generated three pages from files on disk; the deployed build generated none.

  Fix

  Negate the rule for the artifact directory, and make a production build with zero artifacts throw. The loader had treated an empty directory as a normal state and let the call-to-action hide itself, which is what made a packaging bug invisible.

• On a topic explicitly comparing two architectures, the planner declared research unnecessary and produced a confident document with no sources. An earlier identical call had chosen to search.

  Root cause

  The routing turn ran at temperature 0.3, a reasonable setting for prose and the wrong one for a classification. The decision was being sampled. The skip_search tool also described when skipping was appropriate rather than enumerating when it was permitted, and the model reasoned its way in with "conceptual architecture and trade-off analysis, requiring no external temporal facts".

  Fix

  Temperature 0 for forced-tool calls only. The tool description became a three-case allowlist. A deterministic backstop withholds skip_search entirely for comparison, superlative, temporal or quantitative topics, since a tool that is not offered cannot be chosen. Verified over ten runs on each of seven topics: every topic that returned a decision returned the same one every time.

Measured results