Who cares about finding new usability problems?
I interact with a lot of UXers in my company and elsewhere. Based on those interactions I have to ask, "Why do we spend so much time talking about the best way to evaluate usability or find usability problems?" It's clear that the biggest obstacles to overcome in order to deliver a usable product have very little to do with knowing what the usability problems are, and everything to do with figuring out a way to fix the usability problems you already know about.
I'm not saying that running usability tests is a worthless activity. I'm not saying we shouldn't think about running tests in new ways (particularly more efficient ways). But I think most products very quickly reach the point where they know about more usability problems than they have resources to fix. At this point the return on investment for discovering new problems is very low.
Obviously, most usability testing is done on NEW design going into a product (or website or whatever), so one might argue that in this case usability testing is still obviously needed. But again, in my experience this isn't true -- by the time the usability test is run, so many compromises have been made to meet schedules that the usability tester already knows what the user is going to complain about. It's just busywork at that point.
In my career, I've run precious few usability tests where I honestly did not know going in what the best design was, and the usability testing provided exactly the right insight that I needed to make the right design decision. I love those moments. But in my opinion they are the exception to the rule.
(Note: This opinion does not apply to user research, which I find is almost always valuable and worthwhile)
4 comments:
I agree. Far too many usability practicioners place far too much emphasis on usability testing and its twin, "design validation". Both stem from the practice of
"worshiping at the temple of experimental psychology". Or worse yet, "worshiping at the temple of radical empiricism". Don't get me wrong, I think usability testing has its place as does experimental psychology. (I'm not so sure about radical empiricism - it sent the science of psychology off into the weeds for about 50 years.)
The problem is that usability engineering is exactly what the name says it is -
"engineering". It should be based on whatever accumulated scientific knowledge lends itself to the problem at hand. However, the attempt to apply scientific methods is like swatting a warm of flies with a hammer. The problem is that technology is fast while science is slow. Usability testing does flatten a few bugs but it doesn't do much about the swarm.
If usability engineering is to have a big impact we need to think big - more like brick and mortar architects. And we need to remind our selves that we do know a few things, after all.
"Usability testing does flatten a few bugs..." Oooh, I like that one. Gonna have to steal it.
And I completely agree that we need to move more towards being knowledge-based than testing-based. Technical architects don't say, "I think we need to make this piece extensible. Let's go poll 100 developers to see if I'm right."
My guess is a lot of this discussion is put out there by consultants, who many not know the solution to a problem that's outside their domain. Even if they did, the client would want evidence.
I've been running some enterprise usability tests recently, and they serve two extra purposes. One, the customers feel special and listened to, and two, the product team, whom I work with rarely, hears the customers and comes away fired up to make change on their behalf.
Good point about testing giving users the impression that you care about them. I agree that that is a valid reason for doing testing, though I would argue that ethnographic, contextual inquiry-type, user researchy studies are even better for accomplishing that... and they put the designer in the position of being able to create better "knowledge-based" designs in the future without expensive usability testing.
Post a Comment