AI Systems

Most AI projects in operator-scale businesses fail for the same reason: the team built a demo, got a positive reaction in a meeting, and assumed the path from demo to production was a matter of more prompt engineering. It almost never is. The path from demo to production is a matter of <em>systems engineering</em> — the same discipline that turns a clever shell script into a piece of infrastructure other people can rely on. The model is the easy part. The interesting work is everything around it.<br><br>This pillar is what we have actually shipped, in production, against real workloads, across the last eighteen months. It includes the architectural choices that worked, the ones that did not, and the playbook we run when a client asks us to build something AI-native and durable rather than impressive-once-and-fragile-thereafter. None of it depends on any single AI provider. All of it depends on a handful of disciplines that compound.

The router is the architecture

The single most important architectural decision when building AI into an operator-grade stack is to put a router in front of every model call. Not later, not when you can afford it, not when you outgrow your first provider — on the first workflow.

A router is a thin layer that takes a request, classifies the task, picks the cheapest model that can credibly do the job, tries it, and only escalates to a more expensive model if the cheaper one fails a defined quality bar. The classifier itself can be a smaller model. The escalation logic is a few rules that any operator can read. The point is the abstraction.

With a router in place: switching providers is a configuration change, not a rewrite. Cost is predictable per workflow, observable, and recoverable if any single provider has a bad week. The cheapest tier — typically a local open-weight model — handles the long tail of routine work for free. The frontier-class model is reserved for the small percentage of requests that genuinely need it. The cumulative AI cost of an entire operator-scale stack comes out a fraction of what most teams expect, because the router has eaten the easy work before it ever reached a paid endpoint.

Without a router: every workflow is hardwired to one provider, every price change is an emergency, every outage cascades, and the architecture is one vendor decision away from being repriced.

Local models, locally run

For an enormous range of routine tasks — classification, sentiment, extraction, structured output, light summarisation, the boring backbone of most production AI — open-weight models in the seven-to-thirteen-billion parameter range, quantised to fit on consumer hardware, are indistinguishable in quality from frontier-class cloud models. Llama variants. Mistral variants. Domain-tuned descendants. The GGUF format is the lingua franca for distributing them.

We run them locally on Apple Silicon. The marginal cost of inference is electricity. The latency is faster than most cloud round-trips because the request never leaves the building. The privacy profile is total: nothing about the prompt or the response touches a third-party server. For client work where the data is sensitive — and most institutional client work is — this is non-negotiable.

Where the local model is genuinely insufficient, we route up. The classification of what counts as genuinely insufficient is itself an engineering exercise that pays back many times over. The honest answer in our experience: maybe 10–15% of production AI requests actually need a frontier-class model. The rest is a router exercise.

Tool use as the new API

The most consequential development in AI infrastructure in the last two years is the standardisation of tool-use protocols. The most public version is the Model Context Protocol. The pattern, regardless of which specific protocol you adopt, is the same: instead of cobbling together one-off integrations between language models and the systems they should be acting on, you publish tools that any compliant model can discover, call, and reason about.

The architectural payoff is enormous. We expose every internal capability — the customer database, the orchestration layer, the financial dashboards, the design library, the document store, the analytics — as a protocol-addressable tool. A model can read from one tool, reason about the result, write to a second tool, notify a third, and answer the user, without any of the glue having been written for that specific request.

The implications for operators are large. The cost of adding a new capability to the AI surface area of the business drops to the cost of writing one tool definition. The ROI on every existing tool grows every time a new model or a new workflow can use it. The practical agency of the system compounds, rather than scaling linearly with engineering hours. That compounding is what an AI-native business actually looks like in production.

Generative engine optimisation

Search is not what it was. The major search engines have integrated AI answers, the major AI assistants have integrated search and citation, and a growing share of the people who would have been your inbound traffic now read an AI-generated synthesis instead of clicking through to a result. The discipline of optimising for that environment is generative engine optimisation, or GEO, and it is materially different from classical SEO.

The mechanics that matter:

• Schema everywhere. Every page that wants to be cited is marked up with the appropriate JSON-LD types — Person, Article, FAQ, HowTo, Speakable. Models prefer well-typed answers because they reduce hallucination. Schema discipline is the single highest-leverage GEO practice.
• Entity-first content design. Pages are organised around named entities, with explicit relationships between them. The entity graph an AI can extract from your site is the asset; the prose is the dressing.
• Citation-friendly structure. Tight definitions near the top, concrete numbers and specific claims throughout, the author's identity attached to every long-form piece. Models prefer to cite specific people making specific claims over anonymous content libraries.
• Cross-domain entity binding. The author's identity is consistent across every domain in their network, every public profile, every byline. Models build a single person model and route their citations to whichever surface is most authoritative on the specific question.

Done properly, GEO is more durable than classical SEO because the citation surface is not subject to the same attention-economy attacks as the click surface.

Production patterns we keep using

A handful of patterns appear in almost every production AI workflow we ship:

Critic and refiner. The first model attempts the task. A second model — often a different family, often a smaller one — critiques the result against an explicit checklist. A third pass refines based on the critique. The two-step costs more tokens and produces measurably better output for any task with a quality bar, particularly content generation, structured analysis, and code synthesis.

Schema-first prompting. The output shape is declared first, as JSON Schema, and the model is constrained to fill it. Free-form output is tolerated only at the field level. The downstream parsing is deterministic, the failure modes are explicit, and the integration into the rest of the stack is trivial.

Retrieval before reasoning. Any task that depends on internal knowledge runs a vector retrieval first, surfaces the relevant context, and only then asks the model to reason. The retrieval quality dominates the answer quality. Time spent on the retrieval pipeline pays back many times over what the same time spent on prompt-tuning would.

Anti-hallucination guards. Numerical claims are post-checked against the source. Named entities are confirmed against an authoritative table before being committed to output. Confidence is asked for explicitly and propagated. Hallucination is a cost-of-doing-business in production AI, and the only viable defence is layered.

Cost-and-latency observability. Every model call writes a measurement to the time-series store: tokens in, tokens out, latency, cost in currency, model and provider. The dashboards make uneconomic workflows visible long before the bill does.

Voice and multimodal

Speech-to-text on local hardware is now genuinely good. We run a Whisper-class model on the workstation, accept arbitrary audio in, and get accurate transcripts back at faster than real-time. That single capability has changed how we ingest raw material from clients, how we process voice notes, and how we operate during driving or walking time. Voice in is the highest-leverage interface for an operator who refuses to be tied to a keyboard.

Voice synthesis is moving faster than text-to-speech ever did. We use cloned voices for long-form listening of our own published work — there is a button on every long-form piece we write that says listen to this in the author's voice, and it actually does. The technical lift is a one-time training run against a few minutes of clean reference audio. The ongoing cost is approximately zero. The signal it sends about the operator behind the brand is disproportionate.

Photo and video understanding is similar — local vision models can extract structure from screenshots, identify products in photos, and parse documents without ever touching a cloud endpoint. The combination of local speech, vision, and language models means the gateway between the physical world and the operator's stack is broader and cheaper than it has ever been.

Where AI fails the operator test

We will not put AI in front of:

• Decisions with irreversible legal, financial, or reputational consequences without an explicit human-in-the-loop review.
• Long-form content published under a real author's name without that author actually reviewing every paragraph.
• Financial calculations whose correctness can be tested deterministically. Spreadsheets and code do this better, more cheaply, and with fewer surprises.
• Real-time customer-facing dialogue without a clear escalation path to a human, the moment the model is uncertain.

The discipline is to use AI for what it is uniquely good at — pattern recognition, structured extraction, generation under constraint, working at volume — and to refuse to use it as a general-purpose substitute for judgement. The operators who have got this wrong are visible. The ones who have got it right are not, because their AI is doing infrastructure work the customer never sees.

The roadmap, briefly

The interesting near-term frontier is autonomous task chains: a model that is given a high-level goal, decomposes it into steps, executes each step against the available tools, observes the result, and replans. We run early versions of this for research, content production, and lead enrichment. The honest assessment is that the technology is genuinely capable of running ten-to-twenty-step chains today with high reliability when the tools are well-defined and the goals are well-specified. Anything beyond that is still research, but the rate of progress is fast enough that we plan for the twenty-step horizon to become the hundred-step horizon within the next year. The infrastructure we build today should be ready to host that capability without a rewrite.

The architectural commitments that survive that transition are: protocol-based tool exposure, persistent memory in a vector store the model can read and write across sessions, structured logging of every tool call so the behaviour of the system can be audited after the fact, and a governance layer that defines which tools any given model is allowed to call without human review. The teams that have those four in place will be ready for agent-grade workloads as the underlying model capability catches up. The teams that do not will rebuild their stack within twelve months of the capability becoming useful.

The honest cost of waiting is opportunity. The honest cost of moving early is operational drift on a moving capability frontier. We split the difference by building the infrastructure now and using it conservatively against well-defined production tasks today, with the deliberate intention of lengthening the chain depth as the underlying capability matures. The cost of being right about the architectural pattern compounds. The cost of being wrong about the timing of any specific capability is a small slice of the total.

Related pillars