A rota agent covers a shift by picking whoever's available and qualified. Over a few months, one nurse ends up covering more weekend shifts than anyone else on the team, not because a person chose to lean on her, but because she happened to satisfy the agent's rules most consistently. Nobody decided this was fair. Nobody decided it was unfair either. Nobody decided anything, and that's exactly the problem.
That gap is the design. Everything else is plumbing.
#What the ethical question actually is
Most agentic AI conversations are about the person running the system. Can she trust the output. Can she check the reasoning. That's real, and it matters.
It leaves out someone else entirely: the person the decision is actually about, who never opens a dashboard, never sees a citation, and often doesn't know an agent was involved at all.
An agent optimising for "available and qualified" isn't malicious. It's also not fair by default. Fairness isn't a byproduct of accuracy, it has to be designed in on purpose, the same way a hand-off rule or a citation requirement does.
#Fairness is the product, not a side effect
We build agents that make repeated decisions about the same group of people, staff cover, supplier approval, case flagging, with a check built into the rules themselves, not bolted on afterward. If an agent is deciding about the same people over and over, someone reviews the pattern, not just the individual calls, on a schedule, not only when a complaint arrives.
A properly built agent doesn't quietly reinforce a pattern nobody intended. It surfaces the pattern so a person can look at it.
If a decision-maker wouldn't be comfortable explaining a pattern to the person it affects, that's the signal to fix the rule, not the output.
#Three questions before an agent decides about a person
We won't let an agent make repeated decisions about people, not just processes, until we can answer these three, in plain English, to the person the decisions land on:
- Who reviews the pattern, not just the individual decision? If nobody's looking at the shape of outcomes over weeks or months, a bias nobody intended can run for a long time before anyone notices.
- Does the person affected know an agent was involved? Finding out after the fact turns an accurate decision into a trust problem, even when nothing was actually wrong.
- What happens when the pattern looks off? Say so and adjust the rule, or wait for a complaint, that decision needs to be written down before the agent goes live, not discovered the first time it happens.
#When the ethics question doesn't apply
Not every agent needs this. If an agent is only ever deciding about a process, a supplier quote against a fixed spec, an invoice against a purchase order, there's no person on the other end whose pattern of treatment needs watching. The fairness question only bites once an agent is repeatedly deciding about people, not things.
Building a fairness review for a process that never touches a person is effort spent on a problem that doesn't exist there. Save it for the agents that actually decide about staff, clients, or patients.
#Where to start
Pick one agent already making repeated decisions about the same group of people. Write down who, if anyone, currently reviews the pattern across those decisions, not the individual calls. That's usually where the ethical work needs to start, and that's usually the review that was never going to happen on its own.
If you can't name who reviews the pattern, the agent isn't the thing to fix first. The review is.
Recognise any of this in your own business? Tell us about it and we’ll say whether it’s worth automating.
Talk through your process
