UXBristol Special Workshop 1: Advanced User Testing with Rolf Molich

Most usability professionals learned their skills by reading books on usability testing or by watching someone else prepare and conduct tests. But how do we know that the particular way we learned to do usability testing is still the best way? Until the Comparative Usability Evaluation (CUE) project, there was little opportunity for usability professionals to objectively compare their various approaches. As it turns out, there are some striking differences in effectiveness.

This course gives rare insights into the practical doings of usability professionals, normally shrouded behind walls of confidentiality. This course is based on the practical accumulated experience - both good and bad - of the almost 100 professional usability teams who participated in the CUE projects, conducting controlled usability tests in realistic, industrial settings.

If you're a usability professional who has conducted a number of tests, here is an opportunity to assess and improve your skills in the key areas of task creation, usability moderation, creating great recommendations, and communicating results. It's the best tune-up you'll ever get.

What you'll learn

• How to run reliable usability tests and avoid common problems
• Characteristics of good test tasks
• How to moderate tests skillfully
• Writing useful and usable recommendations
• How to communicate your results well

Course Outline

• Introduction
  • Basic dogmas about useful and usable usability evaluations
  • The CUE-studies and how you can benefit from their results
  • Resources for dedicated usability testers
• Creating good test tasks
  • Warm-up exercise: Find the key task
  • Self-assessment: Can you spot the frequently occurring problems in a task set?
  • Hidden clues
  • Open ended versus closed tasks
  • Tasks versus scenarios
  • Humorous tasks
• Moderating tests skillfully
  • Effective and efficient moderation
  • How to follow the basic dogmas in usability testing
  • The facilitator's role
  • Ethical rules
  • Exercise: Watch videos from usability test sessions. Make notes about exemplary facilitation and episodes that have room for improvement
• Recommendations on recommendations
  • Pros and cons of including recommendations
  • Exercise: Test your skills in providing great recommendations. Suggest improvements to recommendations for usability issues described in the handouts
  • Ten rules for creating great recommendations based on the examples from the exercise.
• Usable communication techniques
  • Various means of communicating results (other than reports).
  • The Politics of Usability ("Cowboy programmers don't need no stinkin' usability")
  • The KJ method for effective communication of usability problems
  • Exercise: Review a professional usability test report - Can you spot the communication problems?
  • Key elements of a usable usability test report
  • Describing a finding
  • Including positive findings
  • Good and bad examples from publicly available usability test reports
• Trends in usability evaluation

Format

One-day workshop-like course encompassing 60% lectures and 40% quizzes, self-assessments, exercises and discussions.

About Rolf Molich

Rolf owns and manages DialogDesign, a small Danish usability consultancy that he founded in 1993.

Rolf conceived and coordinated the Comparative Usability Evaluation studies CUE-1 through CUE-9, in which more than 100 professional usability teams tested or reviewed the same applications.

Rolf was a principal investigator in the Nielsen Norman Group's large-scale usability test of 20 US e-commerce websites, involving more than 60 users.

He has worked with usability since 1984 and wrote the best-selling Danish bookUser Friendly Computer Systems of which roughly 30,000 copies have been sold. The book is now available in English, with the title Usable Web Design. Rolf is also the co-inventor of the heuristic evaluation method (with Jakob Nielsen).