A working glossary for agent operations

1. Decision and authority

What is actually being decided, who is allowed to decide it, and what happens at the edge of that permission.

Decision. A choice between options that changes something in the business. A prediction is not one. Neither is an insight, which changes nothing at all until somebody acts on it.

Decision loop. The full path from signal to effect and back: notice the deviation, form options, choose, authorise, execute, observe what happened. Investment concentrates on the first two stages, which is one reason so much analytics output ends as a report and not as a change.

Decision operating system°. A system that governs a decision across its whole life, holding the contract, the evidence, the permission and the record, and delegating the authorised action to whatever executes it. Distinct from a model that answers questions and from a dashboard that displays them. A useful test: ask what the thing is accountable for after the answer appears.

Decision readiness°. Whether a specific decision class has what it needs to be taken under governance: the evidence available at the moment of deciding, a stated set of options, someone or something permitted to choose, a path to execution, and a way to observe the outcome. A different question from data readiness, and the one that determines whether better data changes anything.

Decision contract. The written statement of a recurring decision before any automation touches it: which decision this is, what goes into it, what the options are, who may approve, what counts as an acceptable outcome, and what makes it unacceptable. Automating a decision nobody has written down automates whatever the last person happened to do.

Decision class. A family of decisions handled as one unit for the purposes of permission. Autonomy is granted to a class, not to a system: markdowns under a threshold on non seasonal lines is a class, and so is replenishment for a store cluster with stable demand.

Governed autonomy. The arrangement in which a system acts on its own inside an explicit permission and stops at its edge. The opposite of both extremes: neither a tool that only ever suggests, nor a system that acts wherever it can reach.

Autonomy contract. The permission attached to a decision class, stated in terms an operator can check: what the system may do alone, what needs a second signature, what it may never do, and what conditions revoke the permission.

Permission mode. The rule set deciding which actions a system may take without asking and which stop for a person. The operational form of an autonomy contract, and worth asking to see, because a contract nobody encoded is a document rather than a control.

Authority boundary. The line between proposing and authorising. A system on the proposing side changes nothing until a person moves, no matter how good its analysis is, and a system on the authorising side needs everything else in this glossary.

Escalation trigger. The predefined condition that moves a case from automatic to human. An unfamiliar situation, evidence that disagrees with itself, a value above a threshold, a run that already failed twice. Written in advance, this is governance. Decided in the moment, it is improvisation.

Decision owner. The named person accountable for the outcome of a decision class. Automation moves the work. It does not move the accountability, and any design that appears to is hiding it, not removing it.

Blast radius. How much damage one misbehaving agent can do before something stops it, set by the permissions and the systems it can reach, not by how well it usually behaves. The relevant question when granting access, and rarely the one asked.

Copilot and autopilot. The two autonomy modes. In copilot the system proposes and a person approves every action. In autopilot the system acts inside stated guardrails and a person handles the exceptions. Both are legitimate, they suit different decision classes, and the mistake is running an entire portfolio in one of them by default.

Copilot plateau°. The ceiling a deployment can settle against when every decision still terminates at a human approval and that approval becomes the binding constraint. Assistance keeps improving, and the process stops getting faster, because the queue is no longer waiting on the analysis. Moving past it means granting a decision class. Better suggestions will not do it.

2. Evidence and traceability

What a decision stands on, and what it leaves behind for the person who has to defend it later.

Evidence pack. The bundle a decision stands on, assembled at the time it is made: the data used, the options considered, the options rejected and the reason each was rejected, the evidence that pointed the other way, and the limits of confidence. Not a report generated afterwards to justify a choice already taken.

Dissent requirement. The rule that an evidence pack has to show disagreement and not only support. A system that returns only the case for its recommendation is optimising for the appearance of a good decision.

Decision trace. The record of how one specific decision was produced, complete enough that someone else can reconstruct it months later: inputs, versions, rules applied, who authorised, what the system did next.

Acquisition trace°. The companion record of what was asked and checked before deciding: which questions were put, what came back, what was still missing when the decision was taken anyway. Systems that keep the first trace commonly keep nothing of this one, which is what leaves a post mortem unable to separate a good decision from a lucky one.

Tamper evident record. A record in which any later alteration leaves a visible mark. The difference between a log and evidence, and it matters at exactly one moment: when somebody has an incentive to change what it says.

Evidence boundary. Where support for a statement ends. Stated in four parts: the source, its date, its owner, and its status, meaning built, tested, demonstrated, planned or proposed. Useful because most disagreements about a claim are actually disagreements about which of those five it is.

Grounding. Bringing verifiable external facts into the model’s input so that the answer rests on retrievable material rather than on the model’s weights alone. The related discipline for decisions is constraining output against explicit rules and known quantities instead of letting the model roam.

Hallucination. Output that is confidently wrong. Worth splitting in two, because the remedies differ: the model not having the knowledge, and the model having the material in front of it and not using it. The second survives better retrieval, and it is the one that reaches production.

Provenance. The lineage of a piece of evidence: where it originated, what was done to it on the way, and whether the same inputs would produce it again tomorrow. A figure that cannot be re derived is an opinion with a decimal point.

Primary and secondary source. The thing itself against an account of the thing: the contract against the summary of the contract, the log against the report assembled from it. Every step away from the primary loses detail that somebody later needs, and systems built on summaries summarise eagerly.

Reconstructability. The property of being able to show why a decision came out the way it did, after the fact, to someone who was not there. Regulatory frameworks are converging on this, and it is hard to supply after the fact: either something recorded it at the moment of the decision or it did not.

Information gap. What is missing from the picture at the moment of deciding. The interesting question about an agent is not how much it knows but whether it identifies what it is missing and goes to get it before committing, which is what separates a competent professional from a fluent one.

3. Execution and effect

How permission becomes a real change in a real system, and what proves that it did.

Harness. The layer around the model that turns an answer into a process: what the model may call, how failures are handled, what happens on a retry, when it stops, what gets recorded. The model supplies capability. The harness is where reliability comes from, which is why two teams on the same model get results that do not resemble each other.

Durable execution. Long running work that survives crashes, restarts and infrastructure failures, picking up where it stopped instead of starting again. The difference between a workflow that completes overnight and one that has to be watched.

Idempotency. The property that performing an action twice leaves the same result as performing it once. Without it, a retry after a timeout is a second purchase order. Worth asking a supplier directly, because every reliability mechanism above it is unsafe to use until the answer is yes.

Receipt. Confirmation returned from the target system to whoever authorised an action, carrying enough to reconcile it: which system, which action, the resulting state, and when. An agent reporting success is a claim about itself. A receipt is evidence from the system that was supposed to change, which is a materially different thing to be holding when something goes wrong.

Execution adapter. The component that performs the action in the system of record, holding the credentials, the format and the failure semantics of that specific system. Usually the least glamorous part of the build, and a frequent reason a pilot that demonstrated well never reaches production.

Tool access. The specific set of actions a system is able to perform: what it can read, what it can write to, what it can run. In an evaluation this is a better question than how capable the model is, because capability is shared across the market and tool access is what your particular deployment was handed.

Sandbox. An isolated environment where an agent can act without reaching anything that matters. It is how blast radius stops being a hope and becomes a boundary.

System of record. The system whose state is authoritative for a given fact: the ERP for stock, the CRM for the account, the pricing system for the price. An authorised action that never reaches one has not taken effect, whatever the dashboard says.

Closed loop. An architecture in which the observed result of an executed action returns as an input to the next decision. Without it a system can be confidently wrong indefinitely, because nothing ever contradicts it.

Resume versus restart. Whether a failure part way through a chain costs you the failed step or the whole chain. Decided in the harness long before anyone chooses a model, and capable of changing the running cost of a process by multiples. A question to ask before the model question, not after.

Open loop deployment. Automation that acts without observing the effect of its own actions. Common, cheap to build, and the reason error tends to surface when a person notices something odd, not when the system does.

Model agnostic. Built so the underlying model can be replaced without rebuilding the layer above it. Worth insisting on for a simple reason: the model you would choose today is not the one you will be running in eighteen months, and the cost of that swap is decided by where your logic lives.

4. Context and memory

What the model is holding while it works, what it forgets, and what survives the run.

Context window. What the model sees in a single call. It is finite, everything in it competes for attention, and what is not in it does not exist for that call regardless of what the system knows elsewhere.

Statelessness. The model itself remembers nothing between calls. Every appearance of memory is something the surrounding system chose to put back in front of it, which is why memory is an architectural decision and not a model feature you can buy.

Parametric and contextual knowledge. What the model absorbed in training against what you placed in its input. Only the second is under your control, and only the second can be dated, sourced or audited. Effort spent on the first is mostly wishing.

Knowledge cutoff. The date beyond which a model has no training knowledge. It matters less for general facts than for anything that moved recently: prices, regulation, your own last quarter.

Context assembly°. Deliberately choosing what sits in front of the model on each turn instead of letting a conversation accumulate. The working image is a desk, cleared and re set before every turn. Not a bucket that fills. It has two halves and teams practise only the first: putting the right slice in, and taking the stale material out.

Window rot°. Our label for what happens when the input degrades and the model does not. The four modes come from a vocabulary credited to Drew Breunig in a 2026 practitioner talk: poisoning, where a wrong fact enters and keeps being cited; distraction, where the window grows so large the model leans on it over what it knows; confusion, where irrelevant material degrades the answer; and clash, where two sources in the window contradict each other.

Attention degradation. Quality falling as the window fills, which is a different problem from the window overflowing. Matt Pocock’s dictionary of AI coding terms frames it as a fixed budget of attention per token that gets spread thinner as context grows, with an early sharp period that decays. Worth knowing because it explains a complaint that otherwise sounds like superstition: same agent, same task, worse answers later in the session.

Window reproducibility. The ability to show exactly what the model was looking at when it failed. The gate worth putting in front of every “the agent got worse” escalation, because without it the team is debugging blind and the model is the easiest thing to blame.

Memory kinds. Four, because they need different handling. Working notes: what this task cannot afford to lose. Records: what was done and why, with a receipt, the file an auditor will ask for. Knowledge: durable facts about the business that the next case inherits. Procedures: how things are done here, the standing rules nobody should re derive.

History. The default memory, and the form that rots fastest: it grows without limit and files a superseded decision next to a standing rule as though they weighed the same. Collapsing all four kinds into history is the most common memory design, and it is the one that fails quietly.

Memory policy. Who owns the rules for what enters the window, what leaves it, and what is remembered in which kind. If nobody owns it, all three happen by accident.

Compaction. Summarising the working history so a run can continue in a smaller window. Compression is loss, so the question is who chose what to drop. When the surrounding system compacts automatically at the moment the window fills, nobody chose.

5. Reliability and cost

How you find out whether it works every time, and what a usable result actually costs.

Non-determinism. The same input can produce different output, because generation samples rather than computes. Not a defect, and the reason every metric in this group is about consistency rather than capability.

pass@k. Whether at least one of k attempts succeeded. The natural metric for a demonstration, for research, and for anything where a human picks the good output from several.

pass^k. Whether all k attempts succeeded. The natural metric for an operation, where the process runs unattended and each failure has to be caught by something. The gap between the two is where most disappointment with agent pilots lives.

Attempts until a usable result. How many runs it takes to get one output you can act on. Where attempts are independent and each succeeds with probability p, the expected number of attempts is 1/p. Chains are a separate matter and behave worse, because the success probabilities of the steps multiply: five steps at 0.9 each give about 0.59 for the chain as a whole.

Verification. The work of establishing that a result is usable, which is separate from producing it and is where organisations most often discover unplanned cost.

Judge model. A model used to check another model’s output. It converts review hours into tokens, which is frequently the right trade, and it brings its own price and its own error rate. A judge reading the same context as the worker shares its blind spots, so it catches different failures from the ones a person catches, not the same ones more cheaply.

Review regime. The chosen mix of automatic checks, sampling and full human review for a given process. A cost decision as much as a safety one. Usually inherited, seldom chosen.

Escape. An error that passed verification and reached the system of record. The term is borrowed from quality engineering, where an escaped defect is one that got past inspection. Two numbers describe it and they are not the same: what one escape costs when it happens, and the expected loss over a period, which is that cost multiplied by how often escapes occur. The second is what belongs in a cost model, and the shape it takes, rare and expensive, is the one spreadsheets handle worst.

Process TCO°. Our accounting frame for what arriving at something actionable costs, as opposed to what thinking once costs. At minimum five entries: the amortised harness, and then, per usable result, the cost per attempt multiplied by the attempts it takes, plus verification, plus the expected loss from escapes. A frame for counting, not a closed formula, and your own list may well be longer. Only the first entry appears on a price per task comparison, which is why that comparison is a poor guide to what a process will cost you.

Input and output tokens. The two halves of what a request costs. Everything sent counts as input, everything generated counts as output, and output is the expensive half by a wide margin. It is why a verbose system costs more than a concise one doing identical work.

Prefix cache. Provider-side reuse of the unchanged opening portion of a request, billed at a fraction of the normal rate. It is why two requests that look identical from the outside can differ severalfold in price, and why a stable prompt is cheaper to run than a clever one.

Reasoning effort. A dial on how much internal work the model does before answering. It moves quality and cost in the same direction, which makes it one of the few levers here that is honest about the trade.

Benchmark. A fixed task set with reference answers, used to compare models under laboratory conditions. It measures capability, not readiness, and a high score is not evidence that a process will hold.

Eval. The check of whether a system does a specific job reliably enough on cases close to the real ones. Closer to a trial than to an exam, and it is the eval rather than the model that builds the bridge to production.

Promotion gate. The check a change has to pass before it runs on live work: a defined set of scenarios, a threshold, and a decision that is allowed to be no.

Graduation pathway. The route by which an agent earns wider scope: recorded results on progressively more consequential work, not a convincing demonstration and an enthusiastic sponsor. It gives the organisation a way to say yes slowly, which is the only kind of yes worth giving here.

Reliability plateau. The observation that capability has improved considerably faster than reliability. Evaluating fifteen models across two benchmarks, the Princeton study Towards a Science of AI Agent Reliability (2026) found that twenty four months of capability gains produced only small reliability improvement, and that the pattern held across major providers instead of being specific to one. Within the scope of that evaluation, the practical reading is that a smarter model reads the same degraded window and runs on the same harness.

6. Human control

Where the person stays in the loop, and why that control is more perishable than it looks.

Human in the loop. Any arrangement that keeps a person inside the working process instead of outside it: approving before an action takes effect, handling the exceptions the system refuses, intervening mid run, correcting after the fact. The standard answer to the question of who is accountable, and a real answer only for as long as the involvement is real.

Approval gate. The specific pattern most people mean by the term above: a named person must sign before the action executes. The strongest form of control in this group and the most expensive, Spend it on the decision classes that need it instead of applying it everywhere by default.

Perishable control°. The property that makes human in the loop harder than it appears. Review quality depends on judgement the reviewer keeps sharp by making the underlying decisions themselves, and a reviewer who has spent a year approving instead of deciding is not the reviewer the design assumed.

Automation complacency. The drift from checking to approving. It shows up as approval rates rising over time while review time per case falls, which is measurable and rarely measured.

Sycophancy. The tendency to agree. Training rewards answers that land well, agreement lands well, and the result is a system that leans toward confirming whatever premise it was handed. This is the mechanism underneath the illusion described below.

Automated check. A deterministic pass or fail: the test, the schema, the reconciliation against the system of record. Cheap, unambiguous, and limited to what somebody thought to encode in advance.

Automated review. A judgement by a second model on the first one’s work. Not the same thing as a check, because it returns an opinion rather than a result, and it can be wrong in the same direction as the work it is reviewing.

Unattended run. Work started and left to finish with nobody watching. Legitimate for the right decision classes, and the point at which every control in this glossary stops being theoretical.

Vibe coding. Andrej Karpathy’s term for accepting generated output because it appears to work, without reading it. It names a real practice and, at any scale, a real liability: automation complacency with a friendlier name.

Agent estate. The whole population of agents an organisation is running, taken as one thing to be seen and governed. Plenty of organisations can name their pilots. Far fewer can name their estate.

Agent sprawl. What the estate becomes when business teams build agents faster than anyone counts them: uncounted, unowned, and outside every control the organisation believes it has. It is not a technology failure, it is the absence of an operating model.

Agent registry. The inventory that makes the estate visible. What exists, who owns it, what it can reach, what it may do. Unglamorous work, and every other control in this group depends on it.

Span of control. How many parallel agent workstreams one person can genuinely supervise. Past that number, oversight quality falls faster than output rises, and the organisation is buying throughput with control it no longer has.

Cognitive surrender. The moment a person stops forming their own position before consulting the system. What follows is not a review of a proposal, because there is nothing to compare it against. The term is Wharton’s, from work by Steven Shaw and Gideon Nave defining it as accepting AI output with minimal scrutiny, overriding both intuition and deliberation.

Illusion of independent confirmation°. Agreement from a system that has echoed your own premise back to you, read as a second opinion. Costly out of proportion to how obvious it looks in hindsight, because it feels exactly like having checked.

The opponent. A named and rotated role whose job is to argue the other side of a decision, held by a person or constructed deliberately in the system. The correction for the two entries above, and assigning it matters more than it sounds: volunteering to disagree carries a social cost that being handed the role removes.

How to use this in a vendor conversation

Take four questions from this list into the next evaluation. Which decision classes can this take on its own, and what revokes the permission. What comes back after it acts, and does that constitute a receipt or a claim. What happens when a step fails half way through a chain. What does one usable result cost when the attempts, the verification and the errors that got through are all counted.

Answers to those four separate systems that are ready to run a process from systems that are ready to be demonstrated. They are also, not coincidentally, the questions we ask ourselves.

Across the portfolio the division of labour follows the same path this glossary does. DecidRA is designed to notice a deviation and propose candidate actions. BoardRA is designed for the design time question of how a class of decisions should be made at all. SHEPORD is designed to decide and to authorise, holding the evidence pack, the autonomy contract and the trace. Operstead is our execution harness, designed to carry out exactly what was authorised and return a receipt, with nothing about it minting its own permission. Diagnosis is not authorisation, design is not activation, and execution is not deciding.