Building with AI: Best practices for guardrails from real-world development

The most damaging failures rarely announce themselves

Over the past several months, we built storefronts and commerce agents on commercetools using agentic coding tools and the commercetools for Builders skills. Those builds were either prompt-first or spec-driven. We analyzed data from every run to understand what worked and what didn’t, then turned those insights into guardrails you can use for any AI build. 

Almost nothing crashed, but we noticed real problems that cost us time and money, such as these: 

• The cost grows with the length of the session. A model has no memory between turns, so every turn re-sends the whole conversation.
• Silent failures reach shoppers. A misconfigured search filter returns zero results with no error, which a shopper reads as “we don’t stock that.”
• Completed work hides broken releases. A deployment reports success while serving nothing, and the agent marks the task as completed.
• Every run relearns the same lesson. The model works out something the skill got wrong, routes around it, and then loses the discovery when the session ends.

It’s worth noting that the best practices for guardrails we developed, which you can find in the next sections, aren’t about trusting the agentic model less. 

The idea here is that by moving trust outside the model, removing capabilities instead of relying on restraint, verifying effects and capturing what the agent learns before the session ends.

Best practice #1: Give the agent less access, not more instructions

An AI agent building on your commerce platform needs credentials. The instinct is to pair them with instructions, like “never delete products” or “don't touch live prices.” But an instruction only holds while the model is reading it and choosing to comply, so any real limit on a coding agent has to be enforced outside the model’s control.

Where written rules are needed, put them in top-level system rules so they apply universally, rather than relying on one part of the system to remember them. Where a capability should never be used, don’t grant it at all. For instance,  our write-capable API client was provisioned without scopes for resources that must never change, such as products and prices in shared commercetools environments. As a result, those writes fail on the platform, regardless of whether the model reads the rules.

Scopes on a commercetools API client are immutable after creation, so it’s worth deciding them upfront.

Best practice #2: Plan in one session, build in many

Our first run was one long conversation that spent most of its time re-reading its own history rather than adding to the work. A later run gave each task its own fresh session, delivered far more scope, and cost a fraction as much.

So, our take is that it’s best to use one long session for the spec, architecture and task list. That’s where the thinking goes, and it ends up being your most expensive phase. Then use one scoped session per task group, each starting from the written plan rather than the last conversation.

Best practice #3: Decide which skill does what before building

With all of the commercetools for Builders skills installed, our storefront build leaned hard on two skills we hadn’t expected. At the same time, the two whose names matched the actual work were never loaded. The model also reached for its general knowledge instead, because it knew how. 

The lesson here is that simply having all the relevant skills installed in the solution space does not guarantee that the model will load and use the right skills when executing a task.

When you’re planning what to build, map each architectural area to a named skill or explicitly to none, and annotate every task on the list with the skill that should handle it. Set up this way, the planning phase ensured that the previously unused skill was explicitly assigned to the relevant steps, making it the most-used skill in the run.

Best practice #4: Check the result, not the confirmation

Across every run, the most damaging defects were never explicit crashes. They were silent failures that appeared successful, while doing nothing in reality. Models and engineers alike read a confirmation as proof of execution. So, assume that silent failures will happen. To identify and fix them, enforce three rules:

• Verify every write through a different path than the write used, so the check shares none of the code under test.
• Never mark a control as tested until a check that should pass and a check that should fail have both run the same way.
• Deliberately break your own code to prove the tests can fail. We introduced a fault into the working code, but some tests still passed when they should have flagged the code as faulty. This highlights why it’s important to test the tests themselves by deliberately triggering the conditions they are meant to catch.

Best practice #5: Register what you learn, every single time

A skill is a vendor’s description of how a platform behaves and should behave. It will be incomplete, and the gaps will surface while you implement. Without somewhere to put the correction the moment it’s found, the next run rediscovers it.

So, keep a corrections file for each skill, alongside the skill: What the skill says, what’s actually true, how it was confirmed, and the date. Keep the session data in a queryable store, too: Most of our insights into cost and practice came from that rather than from recollection.

Set the guardrails before your first session

None of this necessarily made the model better. But it made errors along the way cheaper, and the lessons learned from them more durable.

Before your next build, spend an afternoon on four things: 

1. Grant the agent only the permissions you would defend in an audit.
2. Write the plan before opening an implementation session. 
3. Resolve which skill covers which work.
4. Keep the session data somewhere you can query it. 

It may take you a little bit longer to start building, yes, but each step is cheaper to do first than to retrofit after.

Explore what you can build with commercetools for Builders in our documentation — or contact our team for more information.