A blog post that ends with “Microsoft UI has officially entered the realm of self-parody” is going to get quite a lot of mileage. I lost count of how many times I saw it go by on my twitter stream. Laurie Voss posted a response to Steven Sinofsky’s MSDN blog post about the improvements to Windows Explorer that are coming in Windows 8.
Voss takes a look at the data that Sinofsky posted about the usage of various commands in Windows Explorer, and is less than impressed at how they’ve applied this data to the new design of Windows Explorer. He has two major complaints. First, he complains that even by Microsoft’s own data, many of the commands that are elevated in the new Windows Explorer design are ones that aren’t commonly used. Second, he complains that Microsoft’s data says that the menu bar within the Windows Explorer is very infrequently used, so what’s the point of doing it at all?
Both of these complaints are a very common misuse of usage data. Usage data only tells you what happened in the past. It doesn’t tell you why it happened, nor does it tell you what will happen in the future. Furthermore, usage data can often not be broken down very far, so we don’t know what types of users and usages it represents. I’ve written about this before in my blog post about the usage fallacy:
Usage data is directional. It doesn’t tell you what action to take, it tells you that there might be an action to take.
This certainly applies here. For example, Voss is upset that few users use the menu bar, instead using contextual menus. Do we know why they don’t use the menu bar? Are there commands that they’re more likely to use in the menu bar? Is there a discoverability problem (that is, are there commands that users would like to use but can’t find them)? None of these are questions that can be answered by usage data. To answer these questions, you need to use other research methodologies.
The true irony of Voss’s lack of understanding of how to appropriately apply usage data is found in another recent blog post of his about the need for statisticians, in which he says the following:
The whole web industry is accumulating vast quantities of data and storing it, magpie-like, as if it has intrinsic value, aided by ever-falling prices for storage. But the data isn’t valuable. It doesn’t mean anything until somebody who knows what they’re doing looks at it, sifts through it, and produces a tool that lets others use it to draw valid and useful conclusions.
He’s right: data isn’t valuable until someone who knows what they’re doing looks at it and helps draw valid and useful conclusions. It’s always amusing when bloggers officially enter the realm of self-parody.