Skip to main content
GullySystem

User Testing and the Feedback Loop

User testing pairs a few observed sessions with people from your market against the events the product records. Usage numbers show where people stopped, and the sessions show why.

The test sessions

Sessions use people from the trade the product is built for. The rules are simple.

Give a task, not a tour

Ask the tester to book the slot or place the order, then stop talking. Silence is the hard part of the method.

Record the hesitation

Where the thumb pauses matters more than the comment afterwards, since people are polite when asked about software.

Note the words they use

If a tester says booking and the screen says appointment, the screen changes. The user’s word wins.

Test on their own phone

An older Android handset on a weak network is the real condition, and a new device on office wifi hides problems that real users will meet.

What the product records

  • An event at each step of the journey, so drop-off has a location
  • How many people came back a second time, which says more than signups
  • Feedback raised inside the product, attached to the record on screen
  • Steps that nobody uses, the most useful list of all

Why small rounds work

Observed sessions are a qualitative method. After the first few people, the same problems keep repeating, so more watching adds cost and little news. Nielsen Norman Group advises testing no more than five users per study, and running as many small studies as you can afford. Several small rounds, each aimed at a different question, beat one large study. If your users fall into highly distinct groups, such as staff and customers, a few people from each group are needed. Questions about how many, how often or how much need counts from event tracking, not from sessions.

Planning a round

One question per round

Decide what you want to learn, such as whether people find the payment step. A round with five goals produces five half answers.

Recruit from the real market

Use customers, a waiting list or people the trade knows. Friends and family are kind, and kind feedback is poor evidence.

Agree who watches

A developer in the room hears the confusion first hand. One person leads the session and another takes notes.

Compare with the numbers

After a round, check the event data for the same step. If sessions say the button is missed, the drop-off figure should agree.

Closing the loop after each test round

Findings matter only if something changes. Each round ends with a short list that sorts every finding into one of three groups: fix now, change later, or leave alone. The person who reported a problem hears what happened to it. The next round then runs against the same journey, to check that the fix helped.

What a feedback round looks like in practice

A researcher, a founder and a developer each have a part. Here is one round. It runs from the first invite to the written decision.

Researcher books the people

The researcher invites five people from the target market and sets short slots. Each person gets the same task. A quiet room and a recorded screen are arranged the day before. Small rooms work well.

Founder watches without helping

The founder sits in on a session and holds back. When the participant hesitates, the founder writes down the screen and the time. Later, those times are matched with the recorded events. Silence is the skill.

Developer matches events

The developer compares the written notes with the product’s event log for each session. Where a person said they were lost, the log shows the screen they left. That match makes a finding firm. Evidence beats impressions.

Team picks the changes

After the round, the team lists what was seen and chooses the top three changes. The rest wait in the backlog, with the evidence attached for the next round. Everything else waits. Each waiting item keeps a note saying why.

What to settle before a round begins

A round that starts without agreed questions produces opinions rather than evidence. The founder and researcher settle a few choices first, in one short meeting. Each choice guards against the usual ways rounds go wrong. Keep them simple and write them down.

Which task is tested

Choose one task that matters to the business, such as completing a first booking. Everyone in the round attempts it. Mixed tasks make results hard to compare and decisions harder to defend. Choose before inviting anyone.

Who is excluded

Decide who must not take part, such as colleagues, relatives and anyone who helped build the product. Their familiarity hides problems. Strangers from the market find the real trouble. Find strangers who fit the market.

What the product records

Agree which events the product logs, such as screen visits and failed steps. Keep personal details out. A simple event list is easier to match against what people say. Event names should be plain words.

Who decides after the round

Name the person who chooses the changes. Committees stall. One owner, with the notes in front of them, picks the top items and states why the others wait. Write the decision and the date next to it.

Make an informed decision

Questions to settle before rollout.

Use a representative scenario to decide what this part of the system must show and where review is still needed.

User Testing and the Feedback Loop

Can the test fail for the right reason when run on its own?

For “User Testing and the Feedback Loop”, agree the expected visible result and the starting records. Run the scenario independently, so another test cannot supply the state that makes it pass.

Background reading: Playwright: testing visible behaviour

FAQ

Frequently asked questions

How many people does useful user testing need?

A handful of the right people, watched properly, surfaces the serious problems, because large samples matter for measuring and not for finding. Measuring is the job of event tracking.

Can we test with our own staff?

For obvious breakages, yes. For anything else, no. Staff know too much about the idea, so the confusion a new user meets does not show up.

Users are asking for features. Should we build them?

Treat each request as a description of a problem and not a specification. Ask what the user was trying to do. Building every request as stated turns a focused first version into a crowded one.

Should sessions be in person or remote?

Both work. In person lets you see the phone and the setting. Remote reaches people in other cities, and a screen share still shows hesitation. Choose the one your real users would find easiest.

Do we pay people to take part?

A small thank-you is fair, such as a voucher or a free month. Keep it modest so people come to help, not only for the reward. Agree it before the session.

What if testers disagree with each other?

Look for the pattern across the group and weigh it against the event data. One person’s taste is an opinion. Three people stalling at the same step is a finding.

Talk to us

Tell us what you need.

Tell us what your team does today, where work gets stuck and what you want to change.

What happens after you share your requirements

  1. We review your brief and the systems you already use.
  2. We clarify the workflow, missing information and the work to consider first.
  3. The proposed scope sets out the deliverables, responsibilities and dependencies.
  4. Review the scope and commercial terms before deciding whether to proceed.

Start with your requirements

Describe your current process, the people involved and the change you need. Start with the details you know; we can clarify the rest together.

Share your requirements