Before you let an AI agent act in production: 7 questions to answer first

Introduction

Getting an AI agent to give you a good answer has become fairly straightforward. Getting one to safely take an action, like changing a price, launching a promotion or unpublishing a product, is a different problem entirely. I think of it as a completely new paradigm.

The day an agent can change a price in your system, your risk, legal and audit teams start asking what exactly it’s allowed to do. Those questions tend to arrive sooner than anyone expects, and the answers decide whether an autonomous commerce initiative ever leaves the test environment.

This post is a checklist for those governance questions, so autonomy can start to feel more comfortable because it stops meaning losing control. It draws on what my teams have learned building commerce agents at commercetools, and on what Roman Tejada, AI Innovation Lead at Google Cloud, sees when he helps enterprises move AI from experiment to production. 

Note: This post draws on a conversation I had with Roman Tejada, AI Innovation Lead at Google Cloud. Watch the full conversation on demand.

What changes when agents act instead of suggest?

For about twenty years, commerce software, or any software, really, has been organized around tasks. Someone decides what should happen, files a request and it waits in a backlog for a person or a system to act. Autonomous commerce changes that model. AI agents run work like pricing, inventory, promotions, order intake, quoting and approvals continuously, inside guardrails you set, and keep a record of everything they did.

That shift from tasks to outcomes is what makes the questions below urgent. An agent that only drafts a recommendation can be wrong cheaply. An agent that changes your catalog or your prices can’t. 

But first, we need to answer key questions. 

1. Who is the agent?

Every agent needs its own identity. When something goes sideways, you need to know which agent did what, and that’s very hard if actions get buried in a tangle of background processes. This applies whether the agent was built by you, by a vendor or by a third party.

In practice, this means a single registry where every agent in your commerce environment is registered, with what it is, what it’s allowed to do and what it’s allowed to touch.

2. What can it touch, and what should it never touch?

Scope is the guardrail. An agent should only be able to act on the things you have decided it can act on, and the data it can see should be a factor in its decisions, too. Which agents can use which tools, which data can flow to those tools and which people can use which agents are all separate decisions, and they multiply quickly when you go from one agent to a hundred.

The anti-pattern is the “god mode” service account that can do anything. Define the scope per agent, and, where possible, write a policy once and have it apply to every agent.

3. Who approves?

Human sign-off is where many teams stop short. The agent can use data, call tools and reason, so it’s tempting to let it act. But Roman’s view is firm, and so is mine: With anything that costs money, a human in the loop is imperative. 

The useful question is which moments need a person. Approval gates can be set by value, by blast radius and by whether the action can be undone. In one demo of our operational agents, a scheduled inventory check found two sold-out products that were still published, allowing customers to order them. The agent didn’t quietly fix it, but it showed exactly what it proposed to change and waited for a person to approve.

4. Can I see what it did?

For every action, you should be able to see the data the agent read, the sequence of tool calls it made, and the reasoning behind the outcome. Roman compares it to a student showing their work. You also need the full record: What it did, on whose authority, and when.

This isn’t only good engineering hygiene. Auditors and regulators will ask for it, and the teams that built observability and audit trails in from the start, rather than retrofitting them, are the ones that get agents into production.

5. Can I undo it? Can I just pull the plug?

Guardrails are one thing. A reverse button is another. You need the ability to revoke an agent’s access and halt it mid-flight, before it does any damage, in an instant rather than after a meeting. Reversibility also shapes which work you hand over first. 

“It was one thing to have guardrails, but it’s another thing to have a stop button that can immediately stop loss.”

Roman Tejada, AI Innovation Lead, Google Cloud

6. How much damage can one bad action do?

This is the blast-radius question, and Roman’s advice is blunt: If the honest answer is “we’ll find out,” start somewhere else. You want the worst case to be a paper cut.

Consider a promotion that fires without checking stock. It oversells. Each agent can do its own job perfectly and the business still takes the hit, which is why orchestration across pricing, promotions and inventory agents matters as much as each agent’s individual behavior, and why scope and approval limits should be set with the worst case in mind.

7. Will it still behave well over time?

This one tends to surface later. An agent can be well-behaved and pass every test on launch day. Four or five months on, the data has shifted, the underlying models have changed, and behavior can drift. That’s why we at commercetools run continuous evaluations to catch that drift. Governance isn’t something you set up once at go-live; it’s ongoing.

The checklist at a glance

Before an agent takes action in production, you should be able to answer:

• Identity: Who is the agent, and is it registered?
• Scope: What can it touch, and what data should it never touch?
• Approval: Who signs off, and at what thresholds?
• Visibility: Can I see the data it used, the steps it took and on whose authority?
• Reversibility: Can I stop it instantly and undo what it did?
• Blast radius: What’s the worst one bad action can do?
• Drift: How will I know it’s still behaving as expected months from now?

How to start without boiling the ocean

None of this means autonomy is a long way off. It means starting in the right place.

Pick something boring. Choose a repetitive, reversible process with a small blast radius. Your first use case shouldn’t be pricing your top-revenue SKU at peak. Roman’s example is a store that keeps missing its morning numbers because prep starts after the rush begins. The fix is a schedule change, and if it doesn’t work, you change it back the next day. It’s low drama and real money.

Build in the human, the observability and the audit from day one. Don’t retrofit them.

Measure outcomes and let confidence build. Move deliberately from AI-assisted humans to human-assisted AI, as a decision you make on evidence. Trust builds in small steps, and autonomy should grow at the same pace.

What autonomous commerce looks like in practice

In commercetools AgentSphere, the agent registry is where each agent is registered with its scope. From there, a merchandiser can use the promotions agent to describe a promotion in plain language, such as buy one, get one half price on furniture. The agent asks a clarifying question (should this be global or limited to specific stores?), returns the promotion structure for review, and saves it as a draft. Nothing goes live until a person activates it.

The same pattern applies to scheduled tasks, from a daily low-stock check to the catalog fix described above: Plain-language setup, a clear record and human approval where it matters. It’s a journey from guided, to scheduled, to autonomous, and each step is something you can see and control.

Missed the live session? Watch our webinar, Autonomous Commerce in Practice: From Tasks to Outcomes, on demand. 

FAQs

What is autonomous commerce? 

Autonomous commerce is the use of AI agents to run commerce operations such as pricing, promotions, inventory, order intake, quoting and approvals continuously, inside guardrails set by the business, with a full record of what the agents did.

What is AI agent governance? 

It’s the set of controls that determine what an agent is allowed to do: identity, scope, approval gates, audit trails, the ability to stop it, and ongoing evaluation.

Why do AI agents need a human in the loop? 

Because once agents can act, not just advise, mistakes have real costs. Human approval at high-value or hard-to-reverse moments keeps autonomy accountable, and you can loosen it as trust is earned.

What’s a good first use case for an autonomous agent? 

Something repetitive, measurable and reversible, with a small blast radius, where the answer already sits in your data but nobody has the time to act on it.

How do I know an agent stayed within its scope? 

Register every agent with a defined scope, record every action and the data behind it, and run continuous evaluations to catch drift over time.