Shades of the famous Star Trek IV movie moment when Scotty tried to talk to his mouse, here is a sobering anecdote about how one woman went badly (but understandably) awry in usability testing:
Over the last several months we at Novell have sent a team of people around the world with a portable usability testing lab: two video cameras — one on the face, one on the hands — and a frame grabber, recording everything the user does. We ask our subjects to perform five or six simple tasks with GNOME and burn the result to a DVD.
…For example, we asked a lady to send mail to a friend. Against all odds, she started Evolution (nothing in the menus indicates that it’s a mail program; something we hadn’t realized before but which was immediately obvious after watching her stalk one-by-one through the menu items muttering to herself along the way).
The correct next step would have been for her to click on the “New” button that’s in the upper-left-hand corner of the window. This button didn’t even register for her, however. Instead, because she wanted to “send” a mail, she clicked repeatedly on the “Send” part of the “Send / Receive” button just to the right. For about a minute.