Executive Resources · for UK SME leaders
How to Start an AI Pilot Properly
Most AI pilots fail before they have a chance to prove anything, and the failure is rarely technical. It is structural: wrong scope, no baseline measurement, no executive authority to remove blockers, and staff who were not involved in the design and do not trust the output. An eight-week pilot in three phases - design, execute, evaluate - with one executive sponsor, one specific use case, a measured baseline, shadow-mode validation, a graduated ramp and explicit go/no-go criteria, prevents all of those failures.
Why most AI pilots fail before they begin
The post-mortem on a failed AI pilot almost never lands on the model. It lands on scope that was too broad, a measurement baseline that was never taken, a sponsor who was not senior enough to clear blockers, and an end-user team that found out about the project the week before go-live.
Eight weeks of structure prevents all of that. The design phase commits the organisation to a single use case with a measured baseline and a real executive sponsor. The execute phase de-risks the AI by running it in shadow mode before it touches live work. The evaluate phase produces a defensible go or no-go decision, not a vibes-based extension into month four.
Phase 1: Design (weeks 1–2) - get executive sponsorship first
An AI pilot without an executive sponsor is a project waiting to stall. When a blocker appears - a budget approval, a scope dispute, a systems-access issue - someone needs the authority to resolve it in days, not months. That person must be a C-suite or SVP-level owner who meets with the pilot team at least monthly, has budget authority and will report results to the board.
Without this, the pilot will lose momentum the moment things get hard. And things always get hard.
Phase 1: Define one specific use case and measure the baseline
Good pilot use cases are narrow and well-defined: a single process, a clear input and output, a measurable outcome. Bad pilots try to explore what AI can do across a broad domain. You do not learn from exploration - you learn from a specific question with a specific answer.
A well-defined pilot reads like this: 'Automatically categorise incoming customer support tickets as urgent, high, medium or low to reduce manual sorting time from two hours to fifteen minutes daily.' Everything else is out of scope.
Before touching any AI tool, document the current state of the process: volume per day or week, time per item, error rate, cost per transaction and where the bottlenecks actually are. That is your before measurement. Without it, the after measurement means nothing.
Phase 2: Execute (weeks 3–6) - run in shadow mode first
Do not replace the human process on day one. Run the AI in parallel for the first two weeks. Staff perform the process as normal. The AI also processes the same inputs. Compare the outputs side by side.
- It measures real accuracy on your actual data before anything is at risk.
- It builds staff confidence as they see the AI perform accurately rather than just being told it is accurate.
- It surfaces edge cases and failure modes before they affect live work.
Phase 2: Ramp gradually and track daily
Once shadow mode validates accuracy, increase AI-handled volume in stages: 20% in week three, 50% in week four, 80% in week five and 100% in week six with a manual fallback still available. Monitor accuracy and gather staff feedback at each stage. Do not accelerate the ramp if accuracy is below target or staff confidence is low.
A daily dashboard does not need to be sophisticated. A shared spreadsheet updated each morning showing volume processed, AI accuracy, manual overrides, time saved and any error patterns is enough. The point is that you see trends as they develop, not after they have become problems.
Phase 3: Evaluate (weeks 7–8) and run the go/no-go
Compare the post-pilot state to your baseline on every metric you measured at the start. Calculate actual ROI: time saved per week multiplied by staff cost, minus the tool cost per year. Get staff to complete a short satisfaction survey covering confidence in the AI, what still needs improvement and whether they would recommend extending it.
The criteria for a Go decision are explicit: accuracy at or above 80%, measurable time or cost savings, staff confidence at 7 out of 10 or higher, no unresolved governance concerns, and the sponsor wants to scale. If any of those criteria are not met, the right answer is either to fix the specific problem and rerun the pilot, or to wind it down and apply the learning to a different use case.
Eight weeks is enough. A pilot that drags on for six months is not a pilot - it is an unmanaged project.
Common pilot pitfalls
Six failure patterns recur across the pilots that do not deliver. Each has a single, cheap prevention. None of the preventions are clever; all of them are skipped under time pressure.
| Pitfall | Impact | Prevention |
|---|---|---|
| No baseline measurement | Cannot prove ROI | Measure current state before AI |
| Multiple use cases at once | Diluted resources, no clear learning | One process per pilot |
| No executive sponsor | Pilot stalls when blocked | Confirm sponsor commitment before starting |
| End users not involved | Low adoption and trust | 20% of pilot team are operational staff |
| No fallback procedure | One failure breaks everything | Manual backup always ready |
| Wrong success metrics | Cannot demonstrate value | Define metrics before pilot, not during |
Take the next step
Want help applying this to your organisation? Use the resource below or book a 30 minute strategy call with Simon — no pitch, just practical advice.
Frequently asked questions
For a single, well-scoped process, yes. The constraint is the point. Eight weeks forces you to choose one use case rather than a vague exploration, to measure the baseline before you start, and to commit to a go/no-go at the end rather than rolling into month four. If a pilot genuinely needs longer, that is usually a sign the use case is too broad or the integration work was not budgeted as a separate project. Cut the scope, do the eight weeks, and use the result to fund the next phase.
Then they are not the sponsor; they are a name on a slide. The role of the sponsor is to remove blockers in days rather than months - a budget approval, a systems-access dispute, a scope decision when something unexpected surfaces in shadow mode. If the most senior person willing to give that time is a head of department rather than a C-suite owner, scope the pilot down until it fits the authority you actually have. Do not start a pilot whose blockers need a sponsor who is not really there.
Vendor demonstrations run on curated data. Shadow mode runs on yours. The two weeks of parallel operation does three jobs the demo cannot: it measures real accuracy on your actual inputs before anything is at risk, it lets the team that will use the system see it perform rather than be told about it, and it surfaces the edge cases that only your data contains. Skipping shadow mode is the single most common reason a pilot looks fine in week six and falls apart in week ten.
Eighty per cent is a sensible default for low-risk operational tasks like ticket categorisation, where a manual review catches the misses cheaply. For higher-stakes use cases - anything affecting a person's access to a service, a financial decision or a regulated outcome - the threshold needs to be higher and the human-in-the-loop step needs to be designed into the process rather than bolted on. Set the threshold against the real cost of an error before the pilot starts; do not negotiate it down once the results are in.
A clean no-go is a documented decision that the use case did not meet the agreed criteria, why it did not, and what the organisation has learned that informs the next pilot. It is not a failure to be hidden; it is a £20,000 tuition fee that just saved a £200,000 rollout. The sponsor reports the result honestly to the board, the tool is wound down with the manual fallback still in place, and the next use case is chosen on the basis of what was learned. Most leadership teams that handle one no-go well get markedly better at choosing the second pilot.
Related resources
Executive Resources
AI Engagement Framework
The structured methodology AI-Si.com uses with every client: strategic assessment first, then implementation, then optimisation as internal capability builds. Named deliverables, accountable timelines, no open-ended discovery phase.
Executive Resources
30-Day AI Quick Wins
Four weeks. One working AI automation inside your own business. The framework breaks it into weekly stages - use case selection, manual proof, build, and live launch with ROI measurement from day one.
Executive Resources
Why AI Implementations Fail
73% of AI projects fail to reach production. Five preventable failure patterns and the success markers that distinguish projects that actually deliver.
Executive Resources
AI Business Case Template
Boards reject AI cases built on vendor productivity claims. This template builds the case on your own baseline: a financial model, a risk register, and a governance section the audit committee can scrutinise.
Find Out Where AI Can Save or Generate Money in Your Organisation
Book a free 30-minute call with Simon. Bring a real problem - staff time, governance worry, vendor proposal, failing pilot - and leave with a concrete first step you can take next week.