Same AI, same recipe. Why do the results still depend on who uses it?

## The short answer
Picture a sushi chain. Every shop gets the same fish, the same rice and the same recipe card. Still, one shop can taste better than another. Often the difference is in the last cut with the knife.

AI at work looks similar. A clever answer is often not the hard part. The hard part is getting a reliable result every time. That repeatable part is best written as a program. What still differs from person to person is knowing where AI stops being reliable, and which of those places really matter.

## What "symbolic" really means
You may have heard of "neuro-symbolic AI." It joins two parts:
- Neural: learns patterns from data. Like learning what a cat looks like from many photos.
- Symbolic: works with rules, logic and facts. Like a checklist.

It is often described as adding human-like reasoning to AI.

The symbolic part is made of things that can be written down and run again in the same way each time. In that sense it is closer to a program than to a person thinking on the spot.

For everyday work, a more useful question might be whether a program can fix what changes each time.

## The real wall: a reliable result every time
AI models have a setting called "temperature." It controls how much randomness goes into an answer. Even at zero, the same question can get a different answer. One model maker's own documentation says the same output is not guaranteed.

In the 2026 survey of 1,340 people building agents that we used last time, quality was the top barrier to going live. About one in three named it. This does not prove that repeatability is the cause. But an answer that changes each time is hard to trust, even when it is often right.

Many teams split the work in two:
- Workflow: code decides the steps, and AI fills in some of them. Like a train on tracks.
- Agent: AI decides its own steps. Like a taxi driver choosing the route.

A widely read guide on building agents says workflows "offer predictability and consistency for well-defined tasks."

Code does not make the AI give the same answer. What it can keep the same is the pass rule: a fixed check that decides what counts as a pass. The answers may vary, but the rule does not.

## Same tool, different results
In 2023, researchers from Harvard and BCG tested AI with 758 BCG consultants on 18 realistic tasks. They called the set of tasks AI does well the "frontier."
- Inside the frontier, consultants with AI finished 12.2% more tasks.
- They also worked 25.1% faster, with over 40% higher quality.
- Inside the frontier, weaker performers gained more. Their performance rose 43%, against 17% for stronger ones.
- Outside the frontier, on a task picked to be hard for AI, consultants using AI were 19 percentage points less likely to be right. That is about 19 fewer right answers in every 100.

People who did well worked in different styles. Some split the work between themselves and AI. Others worked with AI all the way through.

This is one study, with 2023 AI, and the frontier moves as models change. So we hold it loosely. To us, it hints that the difference is moving: from "How well can you do the task?" to "Can you tell when a task is outside what AI does well?" That still takes real knowledge of the task.

## What we see in our own work
We wrap agent work in automatic checks written as scripts. A result must pass them before it counts as done. The checks do not make the agent consistent. They make the pass-or-fail answer consistent. If the check itself does not change, the same output gets the same answer.

A person still decides where the checks go and what they test. For us, the most important places are the points of no return from the last episode: submit, pay, send.

The checks have a limit. They only catch what someone thought of. So a very useful moment can be when every check passed and the job still went wrong. Someone has to notice that, and turn it into a new check.

So which matters more here, the program or the person?

## The other side
Before publishing, we collected the strongest arguments against this note:
- A chain exists to remove differences. If the cut differs by shop, the chain treats it as a defect and trains it away.
- What one person finds can be written down. Once someone knows where AI fails, that becomes a check. Then the person's edge becomes part of the program.
- Fix the slow spot. A well-known factory idea, the "Theory of Constraints," says a line can only produce as much as its slowest station. So the usual advice is to fix that station.
- Some failures appear only later. They show up at the customer, after a real decision. A team can be good at known weak spots and still miss the costly new ones.
- More checks are not always better. A fast team can add many checks and also add false alarms and extra upkeep.

These are fair points. They suggest that "program or person?" may not be the right question.

## A better picture: choosing which failures matter
Think about the sushi chain again. Each time a master's trick goes into the recipe card, every shop gets better. But the master still decides which tricks are worth writing down.

AI work may be similar. Call the "edge" the place where AI stops being reliable. There are many edges. Most are small. A few sit right before a point of no return, where a mistake cannot be undone.

So the lasting skill may be three decisions:
1. Which failures matter? Usually the ones that can pass a point of no return.
2. How do we catch them? With a check at the right place, or with a person when no fixed check can judge it well.
3. When can we trust the check? Only when it catches real failures without too many false alarms.

Each good decision becomes part of the program, and the whole line gets better. If agents keep becoming more alike, these decisions may be one of the few things that still differ between teams.

## Open question
In your organization, who decides which AI failures really matter, and how is that work valued today?