During the research you can start to group the observations into themes.
These themes might be common topics that are coming up or they might be observations and actions that map to specific interface elements - printing out the interface to help group the observations can be very useful.
After the research, expect to allow at least an hour of analysis for every two hours of research conducted
Affinity sorting is a group activity - encourage people who observed the research to participate in the analysis. This way you can ensure that the research findings are widely accepted and understood by the team.
After the research, use affinity sorting to do your research analysis. You do this by making groups of the post it note observations into groups with similar themes and then labeling that group with a title. You will usually have to do this a few times until you end up with a label that represents the insight you’ve gained from the research.
Once you have got the insights you can then (use orange post its) to make actions, additional questions and design hypothesis in response to each insight. (although some insights will not require an orange post it)
Get the entire team to prioritise the actions/hypotheses from the research as a joint activity so that the most important things are done first.
Generally, issues that affect many participants are a higher priority than those that affect just one or two.
Generally, issues that would cause a participant to fail to complete their task are a higher priority than preference expressed by participants that would impact completion.
There will usually be two different kinds of insights that you will gain from your research, firstly the more propositional/strategic insights and secondly usability/tactical insights.
It is important that both are captured and analysed.
Propositional/strategic insights are very important to creating personas, mental models, concept maps etc. and feeding back to policy people and service managers.
Usability insights are vital to improving the interaction design of the service.
these initial ideas are taken from a draft for a blog post (yet to be published) written by Jay Spanton
Friendly, yet neutral
User research is artificial, we can’t change that. You want your user to behave as normally as they can. Don’t rush them, offer them a drink, keep the setting relaxed and and the tone amicable.
Make sure your test plan provides some simple scene setting and a reminder that you’re testing the system and not the user. A few simple questions about their computer use gets them talking without any challenge and hopefully it lowers any barriers.
You want build rapport with the participant but avoid being too friendly. As the test progresses you can give encouragement with nods, but too much encouragement. Repeatedly saying “great”, “well done” can encourage the user to seek your affirmation and not do what was natural to them.
Often users will be unable to complete a task. Don’t jump right in with help or clues, wait and see if they find the right path by themselves. This is exactly why user testing is done.
If the user says they don’t know what to do, ask them what they’d do if you weren’t there? Or ask what are they expecting to see at that point?
Ask questions carefully
Hopefully you have a great test script packed with prompts and well-written questions. But your user will do or say something you didn’t anticipate and you’ll need to ask them for more information.
Make sure your questioning is open and encourages more than one word responses.
Can you tell me more about that?
Why did you do that?
Focus your questions on the user’s behaviour, what they have done in the session or in the recent past, not on their opinions and what they might do
Tell me about how you applied for a TV licence?
Avoid leading questions and the bias they create
How easy did you find this section of the site?
Why didn’t you click that button, did you not see it?
Avoid using trigger words that are on the page
Can you add something to your basket?
Where do you find the contact us section?
Keep aware of when usability issues are being encountered. Sometimes this means that you make a note, mark a recording or maybe re-question the participant to make it very clear to observers and/ or a good quote. You’ll be glad of the sign-post when you come to the analysis.
Be sure to listen
The test is not about you, make sure you’re hearing from the user most of the time, and think about how much you’re speaking.
When the user tells you something pause for a few seconds, they may well add to their comments. A strategic gulp of water or glance at notes will help make the time.
Sometimes you’ll want the participant to expand on what they’ve said, or you need to be sure you’ve understood things. This is a good time to replay to them what you’ve heard, reframed a little and encouraging them to add to it: “What I’m hearing is that you….”
Be ready to move away from the script. Prototypes go wrong, users take unexpected paths, new questions come from observers. Some users complete tasks at light speed, others deliberate far more carefully than you ever expected.
Fall back tasks and questions should be at the facilitators finger-tips. Back-up test stimuli are crucial and should be dealt with in the planning stage.
Good facilitation depends on good preparation. Recruitment,test facilities and the test scripts are all covered elsewhere but a good facilitator makes sure she’s all over the planning details before something goes awry.
In the end the only thing better than a good facilitator is two good facilitators and user research is ateam sport. If you have the luxury of choice consider rotating who note takes, this can even out any bias from the questions and keep someone fresh for the next session.