Category Archives: user experience

back in the usability lab

This week, I’m getting back to my roots.  It’s been some time since I’ve done a standard discount usability study.  I often use other research methods and let newer researchers carry on with a standard usability study.  I’m in the lab to learn more about Zimbra.

As of this writing, I’m about halfway through my study (12 participants scheduled, 1.5 hours each).  I walked into this with some thoughts about issues that I might observe.  As ever, I found new issues that I didn’t guess in advance.  Which is, of course, the point of running the usability study, and is one of the reasons that I love being a researcher.

Q&A: a day in the life

I’m coming up on six months with VMware (!!!), and it seems that life is settling down into routines.  I recently got asked what life is like as a researcher at VMware.

I’m currently actively working on three different projects:

  • Zimbra web client usability study
  • vCloud Director contextual inquiry
  • vSphere/vCloud/more longitudinal research

Today, I’ll have a bit of each of these.  As of this writing (10am), I’m anticipating that most of my day will be spent split between vCD and Zimbra, but we’ll see what happens.

I take a combination of Caltrain and VTA to get to work.  VMware pays for all of my public transit costs, it only takes a few minutes longer to get to work than it does if I drive, and it’s a great excuse to buy a cup of coffee at Red Rock on my two-block walk to the train station.  What’s not to like?

I usually don’t have meetings on Monday mornings, and today is one of those Mondays.  The morning starts off with email.  I’m one of those Inbox Zero sorts, so the first order of the day is getting as close to that as possible.  A few of the items in my inbox are things that will get handled as I go through my to-do list for the day, so I leave them there for now.

First order of the day is to get ready for a Zimbra usability study that I’m running next week.  Most importantly, I need to start to recruiting participants, so I spend the morning putting together a survey to find participants that meet my criteria (in short, they regularly use email, calendar, and address book for business purposes).  Once that’s sent out to some potential participants, it’s time for lunch.

After a VMware cafeteria lunch of aloo gobi and a samosa, it’s time for another email check and a couple of steps closer to Inbox Zero.  A couple of the mails that I received were from Bugzilla.  I submitted some fit’n’finish bugs for one of our applications late last week, and I got some questions about those today.  I spent a half-hour doing a bit more research and getting screenshots to further illustrate the issues that I observed.  I’m pleased to note that a couple of the bugs that I had submitted were pulled into an earlier release than I had previously requested.

Just as I wrapped that up, one of the interaction designers on my team popped by my office to ask a question about the survey that I’m using to screen participants for the Zimbra study.  We had a quick chat about the study, my goals for the study, and what kind of participants I’m interested in.  He was especially interested in the difference between two questions that I asked and how that differentiates between potential participants.  I opened up the survey results to date and quickly showed him some of the anonymous results to help him understand what distinction I’m drawing and why it makes a difference.  (Sorry, I can’t say more about this without disclosing too much about the study!)  I love having such an extensive user experience team who are highly invested in what’s going on, even when they’re not involved with the project that I’m currently working on.  It’s such a great environment.

Then it was part of the non-sexy part of being a user researcher: doing all of the work necessary to run the usability study.  I checked out the usability lab to make sure that I’ve got everything I need there (and grabbed a picture of the VMware turtles, who were out sunning themselves when I walked past their pond).  I also spent some time getting a couple of test accounts set up, sending mails to those test accounts, and populating the calendar.  I even pressed a few members of my team into service and got them to send mails to my test account, so that not everything is coming from me.

With that done, I spent some time prepping for the vCloud Director research that I’m kicking off.  I worked on the discussion guide for the study, and sent off a flurry of mails about getting participants scheduled.

I wrapped up my day with my weekly 1:1 meeting with my manager.  We talked about my upcoming projects, the open positions that we’re hiring for, and some ideas for future research directions.  Then I came back, made it all the way to Inbox Zero (yay!), finished up this post, and caught the bus home.

load it with details

Last week, I complained about rejecting feedback out-of-hand because it’s not delivered in a way that you like reading.  A developer said that they ignored feedback that called their application “useless”, and that really bothered me.

Here are two pieces of feedback that are on the opposite ends of the spectrum:

  • Your app is awesome!
  • Your app is useless!

As a developer, I have an emotional reaction to both of them.  The first one gives a nice warm fuzzy.  The second one causes a scowl.  Neither of them tell me anything useful about my application.  It’s human nature to want to ignore the latter because it’s negative, but I have to ignore the former as well.  Ignore them not because the feedback has a message that I don’t want to hear, but because it doesn’t have a message at all.

The best kind of feedback helps you make decisions about how to proceed.  I would “never, ever, ever” tell someone that they shouldn’t say something when giving me feedback.  Instead, I tell them what they should do: they should be verbose.  I don’t care about loaded words.  What I care about is feedback that is loaded with details.

If you want me to consider making a change to my application, phrase it like an elevator pitch.  You’ve got to tell me why your request is something that I should consider.  Tell me why you want it and how you want to use it.  Tell me what you’re doing to work around not having it today.  Tell me how not having your request impacts your opinion of my application.

For example, I went looking through the VMware Community for the new vSphere Client for iPad.  In there, I found this feature request:

Since hosts can be seen, it would be a nice feature to enable vmotion/storage vmotions from the iPad client.

Let’s assume that adding such a feature is a non-trivial amount of development time.  If you want a significant development to be undertaken, you’ve got to justify it.  I can come up with a few different use cases where being able to kick off of a vMotion activity from your iPad would be nice, but I don’t know if any of these use cases are the ones that you have in mind.  Tell me why you want to do this, and tell me how it would make your life better if you were able to do this with our spiffy new iPad client.  Tell me how important it is to you: would it be “nice to have”, or is it “useless” without it?

Adding in details doesn’t guarantee that your request will be met, but it gives me a lot more information to use as I make decisions about my application going forward.

loaded words matter, so listen up

John Welch pointed out an article from Justin Williams, an independent Mac/iOS developer, about how ‘Useless’ Is a Loaded Word.  Williams opens up with this:

Here is a tip for all the non-developers out there. When you email your favorite developer with a feature request or bug report never, ever, ever use the word useless to describe their product. Useless is kryptonite to developers and puts us on the defensive instantly.

I cannot disagree enough with this assertion.  Feedback is love.  Your user, or at least a potential user, took the time out of their day to write to you to tell you something that you could do that might help you win their time, their loyalty, and their money.  Feedback is a gift, a priceless gift.  The wrapping paper might be an unattractive shade of brown, but the gift inside is one that you should never ignore.

People use hyperbole.  They’re prone to using it when excited or upset.  People are most likely to offer feedback when they’re excited or upset.  As a result, you’re likely to that hyperbole in action when you receive unsolicited feedback.  Yes, they’re probably using loaded words.  But those loaded words matter.

When you get any piece of feedback, you’ve got to figure out what to do about it.  Marco Arment said this:

If you call my app “useless”, I stop reading right there and either hit Delete or keep scrolling.

Well, I suppose that’s one way to make your decision.  Making decisions about your applications based solely on an emotional response might not be the right way to go about it, but everyone gets to do their own thing.  Instead, I think you should step back a bit and try to determine what’s caused such a response.  It behooves you to do this for both positive hyperbole (“OMG YOUR APP IS THE BEST APP EVAR!!!!!!”) and negative hyperbole (“OMG YOUR APP KILLS USELESS KITTENS DEAD!!!!!!!”).  This isn’t to say that you’ll necessarily act on the feedback, but you do need to at least understand it.

Determining what gift is hidden behind that ugly brown wrapping paper is a hard-won skill.  It’s not about “developing a thicker skin”.  It’s about learning how to hear what someone is really saying, not just the words that are coming out of their mouth.  In fact, Williams himself says nearly the same thing in another post about edge cases:

Any developer worth his salt hears about [edge case] issues like this and their skin starts to crawl.

Yes!  That’s it, right there!  You hear someone describe an issue, and your skin crawls because you know that there’s something more going on than what they’re saying.  So you take the time to consider what their feedback indicates about your application, and you decide what to do about it.  Of course, Williams had said earlier in that post that his “nerd ego” had been boosted a bit, which makes it psychologically easier to try to tease apart the feedback to determine if there’s something to be done.

It’s tempting to only listen to the feedback that tells you what you want to hear, or is offered without any kind of (real or perceived) judgement.  Don’t just pay attention to what your nerd ego wants to hear.  Listen to it all.  Accept the love, accept the hate, and continue striving to make your apps better based on what you learn from both the lovers and the haters.

this is why I tweet

I was asked recently why I’m on Twitter.  It’s all about serendipity.

The metaphor that I use to describe Twitter is that it’s a neverending cocktail party that’s full of people I like.  Just like a large cocktail party, there’s lots of different conversations going on at once.  You can participate in them, or not, as the mood strikes.  And, just like a cocktail party, the topics of the conversations vary widely.  There’s always someone talking politics, there’s always someone sharing something banal about their life, and there’s always someone talking shop.

Likewise, as at a cocktail party, it’s okay to leave to go get some fresh air.  I don’t think that anyone expects that you’ll read every single tweet.  I certainly don’t expect it, nor do I read everything.  I ignore Twitter with aplomb and have no guilt whatsoever.  After I’ve gotten my fresh air, I can come back into the cocktail party, and it will have continued on just fine without me.

Twitter is great for serendipity.  I’ve randomly learned that a friend is nearby, so we’ve taken advantage of the proximity to grab a coffee and catch up.  I’ve helped answer questions that I’ve noticed, helping someone else out.  I’ve had someone find me at a conference to thank me for my assistance.  It’s all good.

I had another example of that kind of serendipity late last week.  One of my VMware colleagues, who I haven’t yet met, tweeted that he was out in the field helping out with a vCloud Director installation.  This is perfect timing: I’m in the midst of some longitudinal research, and wanted to add a vCloud Director customer to that effort.  I sent off an email, and received a lightning-fast response from him.  He’s happy that someone in the company saw his tweet and reached out to him, I’m happy that it looks like I’m going to fill a need (and improve my research too!).

And this, this is why I tweet.  Yes, I admit, I’ve tweeted about cats, cocktails, and the coast.  I’ve also tweeted about projects I’m working on, like Horizon.  It all comes together in one big tweetstream that represents who I am, and hopefully will continue to create serendipity.

how the user experience of Angry Birds contributes to its success

I stumbled across an interesting article recently: Why Angry Birds is so successful and popular: a cognitive teardown of the user experience.  It’s a great discussion how all of its user experience components together have made it such a highly-successful game.

I especially find the discussion of response time to be relevant, not just for game design.  We often assume that response time should be as short as possible, but that’s not really true.  Response time in Angry Birds is used to help you learn how to play the game and correct errors.  I think that this line is the one that every software engineer should take to heart:

The bottom line on how Angry Birds manages response time: fast is good, clever is better.

Q&A: moving from software engineering to user experience

Via email, I received the following question:

I am interested to know how you made the transition from being a software engineer to UX researcher?

My background is a technical one.  I have a BS in mathematics and another in computer science.  I’ve been a developer, including some VAX assembler.  I’m reasonably comfortable in Xcode, even though I don’t really code these days.

I don’t think that moving from a development role to a user experience role is a difficult change to make.  As with any kind of job hunting, it’s about finding the right team that will value the skillset that you have.  Not all teams will find such a background useful, but there are many that will.  I think that there are several unique skills that someone who is currently a developer can bring to the table.

Technical skills are quite useful when you’re considering the user experience and brainstorming potential solutions to issues.  Your potential solutions will consider what’s possible and thus have an increased probability of having an impact on the product.

Having technical skills and user experience skills can help bridge a communication gap.  UX professionals sometimes don’t have a technical background, and developers sometimes don’t have a UX background.  Being able to speak both languages is a positive asset.  I’ve seen several cases where the UX team and the development team were both talking about the same thing, but getting frustrated with each other because they didn’t realise they were doing so.

There are a lot of UX problems to be solved on applications that are complex and deeply technical.  They might not be as sexy as working on the new social media hotness1, but they do have an impact on a lot of people and a lot of multi-billion-dollar corporations.  In my opinion, technical skills help in getting up-to-speed on deeply technical applications.  For example, I worked on DB2.  Understanding databases and knowing SQL helped me immeasurably in that position, and meant that I could hit the ground running.

Another potential positive aspect of being a developer is having experience in shipping applications and continuing to support them after their release.  Understanding the software development lifecycle from deep in the trenches means that you have a great understanding of when various types of research will have the greatest impact on the application.  This helps you formulate the right research plan to answer the right questions at the right time.  A development background isn’t the only way to reach this understanding, but it’s a great way to get there.

It goes without saying that having development skills isn’t sufficient to move to a UX position.  You need to be familiar with UX methodologies, and ideally you would have examples of applying such methodologies in your development work.  You also need to display excellent communications skills, since so much of UX isn’t so much about applying the right methodologies as it is about communicating with the application teams and influencing them to make changes based on your research and recommendations.

As a developer, you can set yourself apart from the competition if you can show product impact, and discuss how your unique skillset of both CS+UX helped you have a significant impact on your product.  Changing roles within the software industry happens frequently; I think it’s to be expected that people in many disciplines within software engineering will want to try out new things.  As I was thinking about this question, I came across an article about moving from engineering to product management; a fair amount of the advice in there is applicable to a move from engineering to UX too.

In other words, if you want to make a change, get out there and find the right position for you.  I don’t think that finding the right position in this case is any harder than finding the right position for anyone who is looking for a software engineering position that is outside the norm.  After all, there are many more web developer jobs than there are UX jobs.  You’ll also find that, within UX, there are more design jobs than researcher jobs.  It’s rewarding once you find the right job, but getting there can be frustrating and time-consuming.

  1. Nothing against social media.  After all, I’m active on twitter!

the silent installation of Growl

I know plenty of other Mac geeks love Growl, but I’ve never liked it.  To be more accurate, I dislike any kind of notification system; it’s just that Growl is the most visible example of it.  I find notifications to be disruptive.  I don’t mind an Adium window appearing when I get a new instant message.  Beyond that, though, I prefer not to be interrupted from whatever I’m doing.  This isn’t to say that other people shouldn’t like Growl.  Everyone works differently.  If it works for you, I’m perfectly happy for you.  It’s just not welcome on my Macs.

Today, Macworld published an article about the mystery of spontaneously installed Growl.  This is one of the things that drives me crazy: other applications which install Growl without notifying me.  Adium gives me the option of installing it (and I thank them for that option), but Adobe CS5 doesn’t.  This is an especially frustrating user experience, given that I did a custom installation of CS5 and so have an expectation that it shouldn’t install anything other than what I selected during that custom installation.

I really appreciate that one of the developers for Growl said this in their interview with Macworld:

We hate it when people install software—any software, including ours—on other people’s systems without permission.

That’s pretty classy.  An official comment from Growl is a great thing, even though Growl has no way to enforce it.  It’s something that I hope that software developers take to heart.  If your software is going to install something else that’s not advertised as part of your software on my system, then you’ve got to both tell me about it and let me opt out of it.