„Usability evaluations are at best less than scientific.“

Schon wieder ein provokantes Zitat. Diesmal aus einem Artikel von dem Kollegen Robert Hoekman Jr.

In seinem Artikel diskutiert er die Ergebnisse zweier Versuche, bei denen die Trefferquote von Usability-Evaluationen untersucht wurden. Resultate der Versuche waren, dass Usability-Teams (bei dem Einsatz der Methode ihrer Wahl) nur einen Bruchteil der vorhandenen Usability-Fehler fanden. Der Versuchsleiter und „Mit-Erfinder“ der Heuristischen Evaluation, Rolf Molich, wird wie folgt zitiert:

Asked how development teams could be confident they are addressing the right problems on their websites, Molich concluded, “It’s very simple: They can’t be sure!

Jetzt zu den Punkten, die der Kollege an Usablity-Evaluationen auszusetzen hat:

Usability evaluations are good for a lot of things, but determining what a team’s priorities should be is not one of them. Fortunately, there is an explanation for these counterintuitive outcomes that can help us choose a more appropriate evaluation course.

First, different teams get different results because tests and research are often performed poorly: teams either ask the right questions of the wrong people or ask the wrong questions of the right people. […]
Usability teams also have wildly differing experience levels, skill sets, degrees of talent, and knowledge, and although some research and testing methods have been homogenized to the point that anyone should be able to perform them proficiently, a team’s savvy (or lack thereof) can affect the results it gets. […]

Next, while usability testing is perhaps no more reliable a prioritization method than an expert-level, qualitative evaluation performed by a lone reviewer or a small group of reviewers, testing is like any other evaluation or discovery method: It must be, but frequently is not, put in context. Page views and time-spent-per-page metrics, while often foolishly considered standard measures of site effectiveness, are meaningless until they are considered in context of the goals of the pages being visited.

Anschließend beschreibt er, wofür Usability-Tests gut seien:

A good usability professional must be able to identify high-priority problems and make appropriate recommendations […] but a good designer must also be able to design well in the first place. This is one area in which usability testing has real power. It can hone designers’ instincts so they can spot potential usability problems and improve the designs without the cost of formal testing on every project.

Sammelt ein [Produkt|Interaction|sonstwie] Designer also hinreichend Testerfahrung, hat er Usability irgendwann im Urin – um eine schlechte Redewendung in ein noch schieferes Bild umzugestalten. Na ja, das überzeugt mich nicht wirklich.

Abschließend empfiehlt Hoekman Usability-Tests in die Design Ideation Phase zu verschieben:

[T]o adapt to and harness the power of usability testing, current users should be brought in to test new ideas—ideas that surface from expert evaluation and collaboration with designers to create new solutions.

Lieber noch mal zurück zu der Ausgangsstudie. Hier die wichtigsten Punkte aus Sicht von Rolf Molich:

  • The number of usability problems in a typical website is often so large that you can’t hope to find more than a fraction of the problems in an ordinary usability test.
  • There’s no measurable difference in the quality of the results produced by usability tests and expert reviews.
  • Six – or even 15 – test participants are nowhere near enough to find 80% of the usability problems. Six test participants will, however, provide sufficient information to drive a useful iterative development process.
  • Even professional usability evaluators make many mistakes in usability test task construction, problem reporting, and recommendations.

Ich fände es ja interessant, ob ein Team mit einem Methoden-Mix mehr Usability-Probleme finden würde. Vielleicht ließe sich so die Trefferquote steigern.

Was meint ihr?

The Myth of Usability Testing
Usability Testing Best Practices: An Interview with Rolf Molich

Update: Wer sich genauer mit den Unterschieden der beiden Methoden befassen will, dem kann ich folgenden Artikel sehr ans Herz legen: Usability Testing Versus Expert Reviews.

Similar Posts:

Kommentar verfassen