When the LLM Isn't the Product

One morning in late June, the pipeline that builds the Citizen’s Daily Brief generated an item calling the mayor of New York City “Zoe Mamdani.” A few words later it called him “herself,” with the same easy confidence it brings to everything.

The mayor of New York is Zohran Mamdani. He had not, in the intervening hours, changed his first name or his gender. The model had invented both and written them down without a flicker of doubt, in clean declarative prose, sitting inches from a structured data field that had his name spelled right the whole time.

I could tell you this is a story about an AI making a mistake. Yet another “don’t trust the insane hallucinating machine” take. Most AI writing this year is, one way or another, a story about the AI: here’s what ours can do, watch it generate and analyze and predict, look how smart it is. The model plays the lead.

I want to tell you the opposite story. The interesting thing about the Zoe incident is the field three inches away that was correct, and the unglamorous machinery that now compares the two before anything reaches you. That machinery is the product. The model is a part I could swap out.

The Citizen’s Daily Brief uses Claude for every analytical step in its pipeline: clustering news sources, scoring significance, synthesizing assessments, generating weekly analyses. LLM inference is literally the most expensive line item in the budget, which is admittedly not saying much when everything else runs on free tiers. And yet if you ask me what the product actually is, the answer has nothing to do with AI.

The product is the format.

The Format Is the Innovation

The Citizen’s Daily Brief is a finite daily artifact with a fixed structure. Five to nine items, each with a headline, a “what changed” section, a “why it matters” assessment, a “what’s next” forward look, and an evidence panel showing sources, confidence, agreement, and the points where the sources disagree. Published once per day. Same brief for everyone. No personalization, no infinite scroll, no engagement optimization. I don’t want your attention, I want you informed.

A single item, stripped to its bones, reads something like this:

EU fines AliExpress €550 million for counterfeit and illegal goods under the Digital Services Act What changed: The European Commission issued a record €550 million fine against AliExpress, the Alibaba-owned platform, for failing to keep illegal, unsafe, and counterfeit goods off its marketplace. Why it matters: It sets a concrete enforcement benchmark for the Digital Services Act: a signal that the EU will levy heavy penalties on large non-European platforms that don’t police their own marketplaces. What’s next: Whether Alibaba appeals through the EU courts, and whether other big platforms tighten their moderation preemptively. Evidence: 5 sources · confidence: moderate · sources broadly agree.

That’s the product. Not “AI-powered news analysis.” Not “intelligent news curation.” A briefing document. A specific format borrowed from the intelligence community and adapted for public consumption.

The AI enables the format. It doesn’t constitute the format. Swap out Claude for a team of human analysts and the product would be identical: same structure, same trust signals, same editorial voice. It would just cost a thousand times more, take longer, and apply the methodology less consistently. A human analyst has bad days. The pipeline runs the same rubric every morning at the same hour, in the same mood, which is no mood at all.

This framing — AI as infrastructure rather than product — inverts the typical narrative, and I think it’s the more honest way to think about most AI applications. Once the novelty wears off, and the AI equivalents of all the early iOS fart apps have gone away, I suspect that this is what we’ll be left with.

Infrastructure Means Invisible

When you use an app backed by PostgreSQL, nobody tells you about the database. You don’t see “Powered by Postgres” in the footer. The product team doesn’t write blog posts about their query optimizer. The database is infrastructure: essential, invisible, valued for staying up rather than for being clever.

That’s how I think about the LLM in the CDB pipeline. It’s a reasoning engine that sits between structured input and structured output, transforming data according to well-defined constraints. It’s called via API, returns JSON, and is validated before its output reaches anyone. If it goes down, the pipeline retries once and then skips the day. If it produces garbage or inadvertently reassigns the mayor of New York a new first name and gender, the validator catches it.

The LLM doesn’t talk to users. It doesn’t have a chat interface. Readers never interact with it, never prompt it, never see its raw output. They see the formatted result of its work, wrapped in a structure that I designed and validated against rules that I wrote. The LLM provides the analytical reasoning. Everything else — what questions to ask, what format to require, what counts as valid output, how to present it — is product design.

How the System Learned People’s Names

Back in March, that same pipeline referred to President Trump as President Biden.

My first fix was the obvious one. I handed the model a block of current officeholders — the president, the cabinet, a handful of world leaders — and injected it into every prompt. Here is who currently holds which office; do not get this wrong. It worked. It was also a hack, and I knew it while I was writing it. A hardcoded who’s-who goes stale the moment a secretary resigns (or more likely in this case, is fired) and it only covers the names I thought to list.

So I tore it out. Instead of telling the model who’s who, the pipeline now reads the day’s own sources and extracts the (role → name) pairings itself, trusting one only when at least two independent sources corroborate it. I added a rule for a specific failure I kept seeing: a current and former holder of the same title getting swapped inside a single item, Pope Leo XIV becoming Pope Francis halfway through a paragraph. Two sources, grounded in the corpus, disambiguated by office. Cleaner. More general. I figured names were solved.

Then June, and Zoe.

What made the Zoe miss different is the part I keep coming back to. The model got the name right in the structured timeline field and wrong in the prose, in the same item, seconds apart. No amount of “here is who currently holds office” catches that, because the system already knew who he was. It just contradicted itself on the way out the door. It felt less like hallucination than like the machine forgetting, mid-sentence, something it had known a field ago.

The fix wasn’t a smarter prompt. It was dumber than that, on purpose: a deterministic check, plain code, no model call, that reads every field of an item and flags any surname that appears with two different first names. If “Mamdani” shows up as both “Zohran” and “Zoe,” the item never ships. I also let the prose-writing step see the structured timeline it was supposed to match, and added a weekly pass that re-checks every name against the week’s own items.

That check, in its first version, was too eager. It started flagging ordinary words that happened to look like colliding names, and I spent a following evening teaching it the difference. The fix needed a fix. This is what maintaining infrastructure actually feels like — not a triumphant demo, but a series of small, specific, slightly embarrassing corrections that make the thing quietly harder to break.

What This Means for Trust

Here’s where the infrastructure framing earns its keep.

When the AI is the product, users have to trust the AI. They have to believe the model is smart enough, unbiased enough, reliable enough. Every hallucination story and every “AI confidently wrong” headline chips at that faith, and the product is only ever as credible as the model behind it.

When the AI is infrastructure, trust shifts to the system around it. You don’t have to believe the model never invents a name. From experience, it does. You have to believe that when it does, something boring and deterministic is standing between the mistake and your screen. The confidence levels, the agreement signals, the cited sources sitting right there under each item: those are all versions of the same promise. Not trust the model, but check the work — here’s the work.

This is the intelligence community’s oldest move. The President’s Daily Brief doesn’t say “trust us because we’re smart.” It says here’s what we assessed, here’s the evidence, here’s our confidence, here’s what we still don’t know. Credibility comes from the shown reasoning. Zoe is the reason I believe that, and not just quote it.

Someone Already Read the Spread

There’s a class of tools I admire that comes at this from the other end. Take one story, line up how outlets across the spectrum covered it, tag each source for bias and reliability, and surface the gaps — the piece the left led with and the right ignored. The promise is media literacy: here’s the whole spread, now you can see the spin and decide for yourself.

The Brief points the same distrust of the single feed at the opposite workflow. Instead of handing you the spread, it reads the spread and hands you the assessment: one item, what changed, why it matters, how sure it is, where the sources fight, what’s still unknown. The spectrum still does the work — significance scoring literally weights a story by how many distinct editorial perspectives are covering it — but that happens upstream, in the pipeline, so it never lands on your plate as twelve headlines to reconcile before breakfast.

These are peers aimed at the same disease from different directions. One says see everything and trust yourself; the other says here’s the call, and here’s the work behind it. I built the second because most mornings I don’t want to run a media-bias seminar. I want the brief, and I want to be able to check why it says what it says.

Boring on Purpose

A product built to show off the model has a shelf life. It’s thrilling in the demo and thinner every month after, because the novelty was doing the work, and novelty is the one ingredient guaranteed to spoil. When the trick stops being surprising, there has to be something left.

So the Daily Brief is boring on purpose. Nobody opens it and thinks “wow, an AI wrote this.” They open it and think “what happened yesterday.” The format is decades old, the content is just the world, and the whole thing is dressed to be trusted rather than to impress. There’s no reveal to wear off, because there was never a reveal. If I’ve done my job right, the AI part of this project recedes to the background just like the Postgres database you don’t care about.

This means the product doesn’t need AI to get better over time for users to keep coming back. And it doesn’t need to write flowery prose. All it really needs is for current events to keep happening, for the world to keep spinning with us on it, which is a reasonably safe bet. The AI really just needs to keep doing its job reliably — clustering, scoring, synthesizing, validating — the same way a database just needs to keep storing and retrieving data reliably.

So What IS the Product?

If the AI is infrastructure, the product is the answer to a design question: what format would make a busy person feel genuinely informed in ten minutes? Every decision in the app is downstream of that one sentence.

Five to nine items, because fewer than five misses something and more than nine feels like homework. Significance scoring, so the day sorts itself by weight instead of by whatever broke last. Confidence and agreement on every item, because a busy person deserves to know how sure the assessment is before they act on it. A “what’s next” on every item, because knowing where a story is heading is half of being informed about it. One brief a day, and then nothing, because the second scroll of the morning has never once made anyone wiser. The AI executes all of this at a scale I couldn’t touch by hand. The judgment underneath it — what to ask, what to require, what counts as good enough to ship — is mine, and it’s the part that took the years.

So here’s the test I’d hand any AI founder. Describe your product without saying “AI,” and see if it still sounds like something a person would want.

Mine survives it: a daily briefing that tells you the handful of things that actually mattered yesterday, shows you how confident it is and what its sources fight about, and then leaves you alone until tomorrow. A think tank could produce that. A very dedicated journalist could produce that. The AI is how it gets produced cheaply enough to give away, which is the only reason it exists, but it’s not why you’d read it. If your product only holds up while the AI is still a novelty, you don’t have a product yet. You have a fart app with a subscription page.

That’s not a modest role for the model to play. Databases don’t get enough credit either. But it is an honest one.

The Citizen’s Daily Brief is a free daily intelligence briefing from Stalefish Labs.