In this guide 6 sections
Compare what real users did with the assumption you wrote down before the build. If the main behaviour happened at the level you set, build the next phase. If users reached it by a different route, change the scope. If the core action never happened, stop and keep the lessons.
Bring the assumption, not just the dashboard
Start the read out with the sentence written before development began. It usually reads like this: a dispatcher will book a pickup in under two minutes, or a clinic receptionist will stop using the paper diary. That sentence is the test. Everything else is context only.
Put the agreed pass line beside it. No line, no verdict. Without a line set in advance, every number looks encouraging to the person who paid for the build. Write it on the first slide.
If nobody wrote the assumption down, reconstruct it from the original brief and get the sponsor to sign the wording before you show any data. Do that first.
- The assumption in one sentence
- The pass line and the date it was agreed
- The pilot group, its size and how long it ran
- Anything that changed mid-pilot, such as a new tester or a fixed bug
Separate what users did from what they said
Event logs show the sign-up that never reached a first booking, the form abandoned at the address field and the screen opened nine times and never used. Interview quotes explain why. Bring both, and label which is which.
Behaviour wins when the two disagree. Always. A tester who calls the app lovely but opened it twice in three weeks has told you something quieter and more useful. Quote them anyway, with the date and role, so the room hears the contradiction.
Pull four items for the read out: completion rate of the core journey, drop-off step, repeat use per tester and the manual workarounds staff had to perform. Workarounds reveal what the product is missing.
- Funnel from invitation to first completed core action
- The single step where most testers left
- Repeat use by week, per named tester
- Support messages and WhatsApp requests grouped by theme
- Three direct quotes, each tied to a logged event
Build when the core action happened
Build means the main behaviour reached the pass line and the complaints are about missing extras, not about the core. Testers ask for exports, roles or a reminder email. They do not ask to go back to the old way.
Turn each request into a ticket, then rank by how many testers hit it and whether it blocks the core journey. Keep the scope tight. The next phase should be one funded block with a demo at the end, not an open list.
Check the plumbing too. A pilot can pass on a shortcut such as a hard-coded price list or a single shared login. Name each shortcut and decide whether the next phase replaces it.
Change when users succeeded by another route
Sometimes the evidence says the idea is right and the shape is wrong. Testers skipped the onboarding wizard and used the search box. Buyers ignored the dashboard and forwarded the weekly PDF. That pattern points to a change.
Write the new assumption in the same format as the old one, with a new pass line and a short second pilot. Keep it to weeks, not months. Reuse the working parts of the build and drop the screens nobody opened.
Tell the team plainly what is being dropped. People who built a feature will defend it unless the usage figures sit next to it.
- Name the route users actually took
- State the revised assumption and its pass line
- List screens to cut and screens to keep
- Set a date for the second read out
Stop cleanly if the core action never happened
Stopping is a result. It is not a failure. Testers were willing and the product worked. Yet the core action never happened. So the demand is not there at this price, in this channel or for this role. That finding saved a full build.
Close it properly. Export the usage data, the interview notes and the code repository into one folder. Write a one-page memo covering what was tested, what happened and which untested ideas remain. Revoke tester accounts and tell them the pilot has ended.
Then ask one more question: is there a cheaper test of a different assumption? If yes, scope it before the team disperses. If no, release the budget.
Pilot read-out sheet
A worksheet of one page with three blocks. The first holds the assumption, pass line, pilot dates and tester count. The second is a table with columns for metric, target, result, and a note saying whether it came from the event log or an interview. The third lists build, change and stop in three columns, each with a tick box, the evidence for it and the owner of the next step.
Open a blank worksheet to printQuestions owners ask
How many pilot users are enough to decide?
There is no fixed number. Enough that the same pattern repeats across several people, not one loud tester. For a business tool, a dozen active users in the same role can be enough to see the drop-off step.
What if the results are mixed?
Split the pilot by tester type. One group often passes while another fails. Build for the group that passed, or run a short change phase aimed at the group that did not.
Who should attend the read out?
The sponsor, whoever runs the day-to-day process and someone who spoke to testers directly. Leave out anyone who will be defending a feature.
Should we keep the pilot code if we stop?
Archive it with the notes. Parts such as login, data models or integrations often serve the next idea. Do not keep the live environment running without an owner.