Hackpads are smart collaborative documents. .

Leisa Reichelt

1013 days ago
Unfiled. Edited by Leisa Reichelt 1013 days ago
Analysing research
 
Leisa R Analysing data captured in qualitative research
 
  • Do it.
  • Don’t do it alone.
  • Analysis is sorting through the data generated in research sessions so that you can make sense of what you’ve learned and what actions you should take in response.
  • You can start doing the analysis during the research sessions but you need to allow time after the research sessions have finished to ensure you get the most from the research you’ve done.
 
During the research
 
During the research you can start to group the observations into themes. 
These themes might be common topics that are coming up or they might be observations and actions that map to specific interface elements - printing out the interface to help group the observations can be very useful. 
Leisa R
  • (single case/multiple case instead?)
 
After the research
 
After the research, expect to allow at least an hour of analysis for every two hours of research conducted
 
Affinity sorting is a group activity - encourage people who observed the research to participate in the analysis. This way you can ensure that the research findings are widely accepted and understood by the team.
 
After the research, use affinity sorting to do your research analysis. You do this by making groups of the post it note observations into groups with similar themes and then labeling that group with a title. You will usually have to do this a few times until you end up with a label that represents the insight you’ve gained from the research.
 
Once you have got the insights you can then (use orange post its) to make actions, additional questions and design hypothesis in response to each insight. (although some insights will not require an orange post it)
 
Get the entire team to prioritise the actions/hypotheses from the research as a joint activity so that the most important things are done first.
 
Generally, issues that affect many participants are a higher priority than those that affect just one or two.
 
Generally, issues that would cause a participant to fail to complete their task are a higher priority than preference expressed by participants that would impact completion.
 
There will usually be two different kinds of insights that you will gain from your research, firstly the more propositional/strategic insights and secondly usability/tactical insights. 
 
It is important that both are captured and analysed. 
 
Propositional/strategic insights are very important to creating personas, mental models, concept maps etc. and feeding back to policy people and service managers. 
 
Usability insights are vital to improving the interaction design of the service.
 
 
Related reading:
 
 
 
1013 days ago
Unfiled. Edited by Leisa Reichelt 1013 days ago
Leisa R Writing a recruitment brief or screener
Write a recruit brief
 
 
 
includes:
  • how many participants - max 6
  • allow 1-2 reserve participants?
  • screener
  • instruct recruiter not to tell participants recruiting for GOV.UK/Dept or Agency - 
 
796 days ago
Unfiled. Edited by Leisa Reichelt 796 days ago
At the end of the discovery phase, your team should be ready to start putting together design approaches to meet the user needs you’ve discovered and begun to understand. 
 
Leisa R Related reading:
 
1013 days ago
Unfiled. Edited by Leisa Reichelt 1013 days ago
Leisa R Moderation Techniques
 
Leisa R
  • these initial ideas are taken from a draft for a blog post (yet to be published) written by Jay Spanton
  •  
Friendly, yet neutral
 
User research is artificial, we can’t change that.  You want your user to behave as normally as they can.  Don’t rush them, offer them a drink, keep the setting relaxed and and the tone amicable.  
 
Make sure your test plan provides some simple scene setting and a reminder that you’re testing the system and not the user.  A few simple questions about their computer use gets them talking without any challenge and hopefully it lowers any barriers.
 
You want build rapport with the participant but avoid being too friendly.  As the test progresses you can give encouragement with nods, but too much encouragement.  Repeatedly saying “great”, “well done” can encourage the user to seek your affirmation and not do what was natural to them.
 
Often users will be unable to complete a task.  Don’t jump right in with help or clues, wait and see if they find the right path by themselves.  This is exactly why user testing is done.
 
If the user says they don’t know what to do, ask them what they’d do if you weren’t there?  Or ask what are they expecting to see at that point?
 
Ask questions carefully
 
Hopefully you have a great test script packed with prompts and well-written questions.  But your user will do or say something you didn’t anticipate and you’ll need to ask them for more information.
 
Make sure your questioning is open and encourages more than one word responses.
 
  • Can you tell me more about that?
  • Why did you do that?
 
Focus your questions on the user’s behaviour, what they have done in the session or in the recent past, not on their opinions and what they might do
 
  • Tell me about how you applied for a TV licence?
 
Avoid leading questions and the bias they create
 
  • How easy did you find this section of the site?
  • Why didn’t you click that button, did you not see it?
 
Avoid using trigger words that are on the page
 
  • Can you add something to your basket?
  • Where do you find the contact us section?
 
Keep aware of when usability issues are being encountered.  Sometimes this means that you make a note, mark a recording or maybe re-question the participant to make it very clear to observers and/ or a good quote.  You’ll be glad of the sign-post when you come to the analysis.
 
Be sure to listen
 
The test is not about you, make sure you’re hearing from the user most of the time, and think about how much you’re speaking.
 
When the user tells you something pause for a few seconds, they may well add to their comments.  A strategic gulp of water or glance at notes will help make the time.
 
Sometimes you’ll want the participant to expand on what they’ve said, or you need to be sure you’ve understood things.  This is a good time to replay to them what you’ve heard, reframed a little and encouraging them to add to it:  “What I’m hearing is that you….”
 
Stay flexible
 
Be ready to move away from the script.  Prototypes go wrong, users take unexpected paths, new questions come from observers.  Some users complete tasks at light speed, others deliberate far more carefully than you ever expected.
 
Fall back tasks and questions should be at the facilitators finger-tips.  Back-up test stimuli are crucial and should be dealt with in the planning stage.
 
Planning
Good facilitation depends on good preparation.  Recruitment, test facilities and the test scripts are all covered elsewhere but a good facilitator makes sure she’s all over the planning details before something goes awry. 
 
In the end the only thing better than a good facilitator is two good facilitators and user research is a team sport.  If you have the luxury of choice consider rotating who note takes, this can even out any bias from the questions and keep someone fresh for the next session.
 
Further reading
 
670 days ago
Unfiled. Edited by Leisa Reichelt 670 days ago
  • You can choose to have your video files on the day or the next day through to your email
Leisa R
  • I'm not sure whether/how we should be sending video files via email... suggest, get them on the day is easier and more secure.
  • Tech support 
 

Contact Support



Please check out our How-to Guide and FAQ first to see if your question is already answered! :)

If you have a feature request, please add it to this pad. Thanks!


Log in / Sign up