Good Enough to Ship, Safe Enough to Sleep
Why pilot success proves capability - but production rollout proves institutional readiness
June, 2026
There is a question that often gets skipped after a successful AI pilot: Who approved the knowledge the assistant is using, and who keeps it current when the pilot team moves on?
During a pilot, most things remain close enough to manage. The knowledge base is limited, the scenarios are familiar and the project team is still around to notice an awkward answer before it becomes awkward in public.
Production is less forgiving.
Policies change. Service definitions shift. Customers arrive with disputes, hardship issues, accessibility needs, language barriers and circumstances that have shown very little interest in the pilot plan.
Pilot success shows that the capability can work. Production rollout shows whether the organisation is ready to stand behind it.
Those are different tests.
What changes in production
The specific capability here is not a general chatbot drawing answers from the open internet.
It is a grounded AI assistant: a system that generates responses from approved organisational sources such as policy libraries, service guides, hardship procedures, complaint pathways, internal knowledge articles and customer-service scripts.
In practice, it might support a contact-centre employee, help a chatbot answer more consistently, guide a customer through an after-hours issue or summarise the next best step from approved material.
For high-volume service environments, the appeal is obvious. Responses can be faster and more consistent, with less dependence on every frontline employee remembering every policy variation on a difficult Tuesday.
But grounding an assistant in approved content does not settle the leadership question. It brings the question into sharper focus.
Once the assistant begins turning organisational knowledge into customer-facing answers, the organisation is no longer simply managing documents. It is operationalising its policies and promises.
Where the operating model gets exposed
A grounded assistant is only as dependable as the knowledge it is allowed to use.
That sounds straightforward until a hardship policy changes and the source article is not updated. Or the website reflects a new complaint pathway while the internal knowledge base still carries the old one. Or a customer asks a question that is technically answerable, but sensitive enough that a human should be involved.
In each case, the assistant may be doing exactly what it was designed to do.
The model can be grounded while the organisation around it is not.
Customers do not see the retrieval process, confidence score or internal debate about whether the pilot was ready for production. They experience the organisation answering them.
If the response is wrong, outdated or poorly escalated, the issue is no longer confined to model performance. It becomes a service, governance and accountability problem.
It can also create very ordinary operating costs: repeat contact, complaints, manual correction, employee workarounds and senior attention being pulled into issues that should have been resolved in the service design.
This is one reason AI initiatives often slow down between pilot and rollout. The technology may have worked, but production forces the organisation to decide what it is prepared to own.
Knowledge becomes part of the service
A grounded assistant can retrieve, draft, summarise and respond. It can make language more consistent and reduce the time employees spend searching for information.
It can also expose gaps in policy, content ownership and channel governance that have been sitting quietly in the background because no system had tried to apply them at scale.
That makes it useful in another way: it is diagnostic.
Once organisational knowledge becomes response logic, content governance is no longer administrative housekeeping. It becomes part of the operating model.
An answer about hardship eligibility may shape what a customer believes they can access. Guidance in a portal may influence whether someone escalates a matter, accepts an outcome, complains, waits or gives up.
The production question therefore sits one level above whether the answer looks correct.
It is whether the organisation can continue to own that answer when the policy, customer circumstances or service context changes.
The ownership test
Before moving a grounded AI assistant from pilot to production, I would want three things made explicit.
Who owns the knowledge?
Not the vendor, the project team or “the business” in the abstract.
The actual person or function accountable for the policy, service rule or customer commitment being expressed.
That owner also needs a workable process for reviewing and updating the source. A named owner without time, authority or a maintenance rhythm is really just a name in a governance document.
Where must the system stop?
Some answers can safely go directly to a customer. Others should support an employee rather than replace their judgement.
Hardship, vulnerability, complaints, disputed charges, accessibility needs and unclear policy are clear examples where escalation needs to be designed before launch.
The assistant should not discover its boundaries one customer at a time.
Can the answer be traced later?
If a customer challenges a response six months from now, the organisation should be able to establish what source the assistant used, what it produced, whether the source was current and what happened next.
Not every interaction requires the same audit trail. But where an answer affects money, access, rights, safety or a formal decision, traceability should not depend on someone finding an old screenshot.
Before approving rollout
This does not require a 47-page governance framework that everyone politely agrees is important and no one can quite apply.
A practical production-readiness check is more useful.
Take the highest-consequence scenarios the assistant is expected to handle and work through them with the people who own the policy, service, risk and frontline response.
Confirm which source is authoritative, who keeps it current, when the assistant can answer directly, when it must hand over to a person and what needs to be recorded when the issue carries material consequence.
If those answers remain vague, the organisation may not yet have a production system.
It may have a pilot wearing a production badge.
There is nothing wrong with extending the pilot while those questions are resolved. The greater risk is moving into production because the demonstration was successful and discovering later that the organisation cannot support the service it has created.
Confidence is not about assuming nothing will go wrong. It comes from knowing what the organisation is prepared to own, where judgement sits and who acts when the context changes.
That is the difference between good enough to ship and safe enough to sleep.
Executive note for leaders
Before a grounded AI assistant moves from pilot to production, ask:
Who owns the knowledge, where must the system stop, and can the answer be traced later?
If those questions do not have clear owners and workable processes behind them, the organisation is still carrying pilot risk into production.
The practical test is not simply whether the assistant can answer.
It is whether the organisation is ready to stand behind the answer when the situation changes.
Subscribe
You can subscribe for free. Every edition is delivered directly to your inbox and published here on Substack.
If a piece raises a question, surfaces a pattern, or helps you think more clearly about a decision, I’d value the conversation.
Thanks for reading,
Stuart Gonsal MAICD
With occasional help from Springsteen, my Border Collie, who reminds me that clarity comes from movement 🐾.
Connect
LinkedIn – Follow for practical leadership on AI-era opportunity. ↗
Disclaimer
Everything shared in The Ripple Effect reflects my personal views and does not reflect those of my current or past employers, clients or partners. Any examples are illustrative, drawn from publicly known patterns or anonymised experience.