Ningen

The problem

Built for the Data Science Nigeria x Bluechip Tech LLM Agent Challenge, with a hard deadline of 24 May 2026 and nine days to reach it (README:1-4, first commit 15 May). The challenge asked for two capabilities that pull in opposite directions: predicting how a specific user would rate and review something they have not seen, and holding a conversation that arrives at a recommendation. The first needs a model of one person. The second needs retrieval over a corpus large enough that the answers are not obvious. Doing either convincingly on a synthetic dataset is easy and proves nothing, so the constraint we set was that everything had to run against real review data at a scale where retrieval quality actually matters.

Architecture

Four containers under one compose file: Postgres 16 with pgvector, a Python ONNX embedding sidecar, a one-shot Go ETL worker, and a long-running Go API (README:105-110). The ETL streams reviews directly from source URLs without writing large files to disk, embeds each one, bulk-inserts with dedup, builds an HNSW index, and exits. Recommendation runs a five-stage pipeline: a corpus pre-search that grounds the model in what the database actually contains, an LLM signal extractor, multi-vector pgvector retrieval with a full-text fallback, an LLM quality gate that can accept, refine or ask a clarifying question, and a psychographic reranker (README:56-98). Each stage has a 25 second timeout and a defined failure behaviour: a gate that fails to parse defaults to accept, a reranker that fails to parse falls back to retrieval order (README:98). Review generation is a separate four-node loop of profiler, rater, drafter and critic, capped at two draft iterations (README:565-571).

Decisions and what they cost

1. Item ids are UUID v5 derived from domain plus the full review text, computed before the text is truncated for embedding (README:637-639).

   Alternative considered

   A surrogate key, or a hash of the stored search_text.

   Why rejected

   Deterministic ids make reingestion free: the same review always produces the same id, so ON CONFLICT DO NOTHING handles dedup with no separate uniqueness check and a partial ETL run can resume without tracking where it stopped (README:316, 324).

   What it cost

   The id cannot be recomputed from anything in the database. search_text is truncated to 1,000 characters because long reviews degrade retrieval, while the id comes from the untruncated original (README:639). Any later job that needs to match rows by id has to re-stream the source file rather than query Postgres, which is exactly what the name backfill ended up doing.

2. Resolve Amazon product names in a separate one-shot backfill that re-streams the SNAP reviews file, recomputes item ids, joins metadata on ASIN, and updates only rows where the name is null (README:31, 326-346).

   Alternative considered

   Resolve names during the original ETL, or re-run the whole ETL with metadata joined in.

   Why rejected

   Re-running ETL means re-embedding every item, and the embedder is the bottleneck at roughly 820 items per minute on the deployed VM (README:649). The backfill touches no vectors at all: it is a pure metadata update, safe to run while the API serves traffic (cmd/backfill_amazon_names/main.go:8).

   What it cost

   Streaming the full reviews files again, plus a 2 GB metadata file for Books that is in Python-literal format and has to be parsed by regex, which misses titles containing apostrophes (README:648). It resolved names for roughly 499,000 of 770,000 items and could do nothing for the remaining Yelp rows.

3. Both tasks are served by one Go binary and one API surface: POST /recommend and POST /generate-review on the same mux (cmd/api/main.go:61-62).

   Alternative considered

   Two services, since the two tasks share no pipeline code.

   Why rejected

   One deployment. Two services during a nine-day build means two containers to keep healthy, two sets of provider credentials, and two things that can be broken at submission time, in exchange for a separation neither task was asking for.

   What it cost

   Very little, and the one difference that looks like an inconsistency is not one. Task A never runs the post-hoc humanizer that Task B ends with, because it does not need to: the Drafter localizes the product context and injects Nigerian vernacular while it writes (internal/pipeline/nodes/drafter.go:20-24, localization.go:7), and the Critic fails any draft that reads like a generic American AI (critic.go:96). Task B has no drafting step to bake voice into, so it humanizes what retrieval assembled. Each task applies the voice at the only point where it can. The real cost is that Task A keeps its request and response types in the handler and pipeline packages rather than in internal/models/schemas.go alongside Task B (README:584), so the two halves of one API are not documented in one place.

4. The ONNX sidecar runs the embedding model on CPU instead of pulling Ollama (README:113, 627).

   Alternative considered

   Ollama, which is the default choice for local model serving.

   Why rejected

   Ollama needs a 4 GB image pull and a GPU-optimised runtime. The ONNX sidecar pulls roughly 90 MB of weights, starts in under ten seconds, and caches the model in a Docker volume after the first run.

   What it cost

   CPU embedding is the throughput ceiling for the whole pipeline. The container is capped at one CPU deliberately, because letting the embedder saturate a 2 vCPU VM triggered Azure deallocation, and that cap holds ingest to about 820 items per minute (README:649).

What broke

• The Yelp dataset carries no business names, so a third of the corpus can only be described by its review text. /recommend omits the name field entirely for those items (README:33, 213).

  Root cause

  SetFit/yelp_review_full deliberately strips business metadata and ships only star labels and review text. The full Yelp Open Dataset includes names but requires an academic licence (README:647).

  Fix

  Not fixed. Written into the README as a named limitation with the reason and the blocker, rather than left for a reviewer to discover by finding an item with no name. The Amazon half of the corpus was backfilled to 100% named coverage, so the gap is specific and explainable rather than general.

• The ETL worker ran the VM out of memory after deploy, and Yelp ingest failed on the weak instance (commits "stop etl_worker after deploy to prevent OOM crashes" and "throttle yelp ETL to prevent OOM and timeout failures on weak VM", both 24 May 2026).

  Root cause

  Not recorded, and I no longer remember it. The commits name the symptom and the fix and nothing in between. What the record supports is the shape of it: a 485 MB JSONL file streaming over HTTP while the embedder works through the same items, on a VM with 2 vCPU (README:649-650). Which of those was the binding constraint on the day, I cannot now say.

  Fix

  Throttled Yelp ingest, capped the embedder at one CPU, and stopped the ETL worker once the corpus was populated. Production ended at roughly 770,000 items against a configured target of 1,000,000 (README:22-29, 318), so the corpus is the size the infrastructure allowed rather than the size that was asked for.

• Cross-domain requests came back dominated by Yelp items, and the reranker pool was not balanced across domains (commits "balance cross-domain retrieval to prevent Yelp embedding bias" and "interleave reranker pool by domain when cross_domain=true", 24 May 2026).

  Root cause

  Yelp is the largest single source at roughly 270,000 of 770,000 items (README:24-29), so an unweighted nearest-neighbour search over the union returns Yelp disproportionately regardless of what the user asked for.

  Fix

  Balance retrieval across domains before the union, then interleave the reranker pool by domain when cross_domain is set. Both landed on deadline day.

• A backfill that computed item ids with a mismatched formula would write correct-looking product names onto the wrong items, and nothing downstream would flag it.

  Root cause

  The backfill recomputes ids from a re-streamed file. If the id formula, the namespace, or the source file drifted from what the ETL used, every join would still succeed and every write would be wrong.

  Fix

  The script streams 500 reviews before writing anything, computes their ids, and requires at least 10 of 20 to already exist in the database, aborting otherwise (README:346, cmd/backfill_amazon_names/main.go:92-100). It also refuses to run if the metadata file yields zero titles. A dry-run mode prints matches without writing.

Measured results