User Testing and the Feedback Loop
User testing pairs a few observed sessions with people from your market against the events the product records. Usage numbers show where people stopped, and the sessions show why.
The test sessions
Sessions use people from the trade the product is built for. The rules are simple.
What the product records
- An event at each step of the journey, so drop-off has a location
- How many people came back a second time, which says more than signups
- Feedback raised inside the product, attached to the record on screen
- Steps that nobody uses, the most useful list of all
Why small rounds work
Observed sessions are a qualitative method. After the first few people, the same problems keep repeating, so more watching adds cost and little news. Nielsen Norman Group advises testing no more than five users per study, and running as many small studies as you can afford. Several small rounds, each aimed at a different question, beat one large study. If your users fall into highly distinct groups, such as staff and customers, a few people from each group are needed. Questions about how many, how often or how much need counts from event tracking, not from sessions.
Planning a round
Closing the loop after each test round
Findings matter only if something changes. Each round ends with a short list that sorts every finding into one of three groups: fix now, change later, or leave alone. The person who reported a problem hears what happened to it. The next round then runs against the same journey, to check that the fix helped.
What a feedback round looks like in practice
A researcher, a founder and a developer each have a part. Here is one round. It runs from the first invite to the written decision.
What to settle before a round begins
A round that starts without agreed questions produces opinions rather than evidence. The founder and researcher settle a few choices first, in one short meeting. Each choice guards against the usual ways rounds go wrong. Keep them simple and write them down.
Questions to settle before rollout.
Use a representative scenario to decide what this part of the system must show and where review is still needed.
User Testing and the Feedback Loop
Can the test fail for the right reason when run on its own?
For “User Testing and the Feedback Loop”, agree the expected visible result and the starting records. Run the scenario independently, so another test cannot supply the state that makes it pass.
Background reading: Playwright: testing visible behaviour
Frequently asked questions
How many people does useful user testing need?
A handful of the right people, watched properly, surfaces the serious problems, because large samples matter for measuring and not for finding. Measuring is the job of event tracking.
Can we test with our own staff?
For obvious breakages, yes. For anything else, no. Staff know too much about the idea, so the confusion a new user meets does not show up.
Users are asking for features. Should we build them?
Treat each request as a description of a problem and not a specification. Ask what the user was trying to do. Building every request as stated turns a focused first version into a crowded one.
Should sessions be in person or remote?
Both work. In person lets you see the phone and the setting. Remote reaches people in other cities, and a screen share still shows hesitation. Choose the one your real users would find easiest.
Do we pay people to take part?
A small thank-you is fair, such as a voucher or a free month. Keep it modest so people come to help, not only for the reward. Agree it before the session.
What if testers disagree with each other?
Look for the pattern across the group and weigh it against the event data. One person’s taste is an opinion. Three people stalling at the same step is a finding.
Tell us what you need.
Tell us what your team does today, where work gets stuck and what you want to change.
What happens after you share your requirements
- We review your brief and the systems you already use.
- We clarify the workflow, missing information and the work to consider first.
- The proposed scope sets out the deliverables, responsibilities and dependencies.
- Review the scope and commercial terms before deciding whether to proceed.
Start with your requirements
Describe your current process, the people involved and the change you need. Start with the details you know; we can clarify the rest together.
Share your requirements