Category Archives: user experience

Q&A, user experience

Q&A: Good websites for user researchers looking for a job

On Quora, someone anonymously asked me to answer this question:

What are the best job boards or professional organizations with job postings, specifically for User Researchers?

I’m not sure that there is a best job board for User Researchers. There are many fewer jobs for researchers than there are designers, which makes finding positions more challenging.

For positions in the San Francisco Bay Area, BayCHI is an excellent resource. You have to pay for a annual membership to get access to its Job Bank, which is well worth it.  BayCHI also gets some job postings for companies who are headquartered in the Bay Area (or have a significant presence here) but are looking for people in other locations. BayCHI covers all of user experience, not just research; my unscientific glance over the past few weekly emails says that 10-20% of the job ads posted there in any given week are for researchers. Also, the monthly BayCHI meetings are good places to network, which might be an even better way of finding out about a position.

On Twitter, there are a few UX job aggregators. The one that I’ve found with the most researcher jobs is @UXdesignjobs.  It, like BayCHI, does not focus on UX research jobs, so you will have to filter out the design jobs that aren’t relevant to you. They do have a good mix of jobs at US companies.

A good Glassdoor search can also reveal appropriate user researcher positions, and they will automatically email you with new ads posted that meet your search terms. Glassdoor shows more jobs outside of the Bay Area. A good search is more likely to result in user researcher jobs; a few non-research jobs still sneak in. In general, the jobs that show up on Glassdoor are at larger companies, and their daily email can be repetitive.

Since you asked, I will note that my team at VMware is hiring user researchers.  You can ping me directly if you’re interested in this role.

I’m not aware of a great job posting site for UX positions outside of the US. Perhaps someone else can share insight there.

I’ve generally found that having a robust LinkedIn profile is an excellent method for getting approached by recruiters. This might be a Bay Area bias showing, I’m not sure how pervasive LinkedIn is outside of the Bay Area. I do regularly get contacted by recruiters on LinkedIn from outside the Bay Area. I sometimes hear from recruiters who don’t get the difference between UX design and UX research, and I occasionally get recruiters who see “researcher” and think that I’m conducting scientific research. Overall, though, I think that my LinkedIn profile is one of the most useful tools when I am looking for a job.

Nadyne, user experience

on letting go

When I first started at VMware, I noticed that there was my User Experience team (about 25 people at that time, working mostly on vSphere and related applications), and that there were several individual interaction designers sprinkled throughout the company.  I proposed that we all get together, and eventually, a two-day internal UX conference was born.  I chaired it the first year and the second year.

Then I had to let it go.  I explicitly said this to my conference committee, as well as my management, as I was working on it for the second year.  This conference couldn’t just be my thing.  If it was to have any legs, it had to become a VMware thing.  So, vUE 2014 planning kicked off, and I am not part of its conference committee.  I provide whatever assistance they ask me for, but I’m explicitly not part of the conference committee.

I did a lot to try to set up the future iterations of the conference for success.  The first year, I did pretty much everything, with assistance from a small conference committee.  The second year, I made some changes to the conference committee: I broke up the work into Conference Chair and Technical Chair (following the model used by many academic and industry conferences), and I made sure to choose my successor so that she could learn as much as possible from me.  I was also aware that I was setting precedents, so I was very careful about what I chose to do (or not do).

Letting go is hard.  I have a big investment in this conference.  But it can’t be mine.  It has to become something that VMware’s UX community owns.  From the outside, they seem to be doing a fine job.  It’s a bit odd to see the announcements coming out about the theme and how to submit papers and not be the one who sent them.  I’m sure it will be even more strange to attend the conference as an attendee and not as its conference chair.

I’m so glad to see it continuing forward without my involvement.  This tells me, more than anything, that it was truly something that VMware needed and continues to need.  It tells me that it’s something that the VMware UX community values.  It tells me that UX at VMware is vibrant and growing.  It tells me that it was the right thing to do.  I know some of the evolution that it has already undergone with new leadership (some that I anticipated and tried to make sure things were set up so that it could evolve in that direction, some that I never anticipated), and I’m sure that I’ll see more evolution when the event itself arrives.

user experience

when user interviews go bad

The Cranky Product Manager asks what to do when you’re stymied when interviewing a user.  In her example, she asks a broad question, and the user responds with a list of details that they want fixed in the application.

I think that scenario is familiar to everyone who has ever tried to interview users.  I usually handle this situation by reviewing the list and trying to understand what brought us to the point where this is the interaction that the user wants to have with me.  Are there items on their list that are so important to them that these items block them from being able to consider anything else?  Are there categories of problems on their list that tell me that there’s a bigger problem to solve here?  Are they trying to accomplish a workflow that we didn’t consider or didn’t get right?  Is the overall problem one of death by a thousand papercuts?  If I can identify a trend in their list, I can make more headway in making user experience improvements that go beyond filing 23 bugs.

Another option in handling this is “tell me more”.  “Tell me more about item #3 here.  Can you show me what happens?”  This gets them talking about the context of item #3, and puts it in perspective of what they’re trying to accomplish when they run into the problem that they’ve identified that they want you to fix.  It can give you the insight that you need to address the underlying problem.

When you receive a list like this, you must be open to the feedback that you get, analyze it carefully to understand what the problems are, and make informed decisions about the next steps to take (which might or might not be doing exactly what is outlined by the list).

user experience, vSphere

“why I like the vSphere web client a lot!”

I missed this post over the holidays: “why I like the vSphere web client a lot!”  There’s a couple of things in it that are near and dear to my heart, such as Mac support1, Works in Progress2, and tags3.

  1. I’ve been advocating for this ever since I joined ↩
  2. Internally, we still call it by its original name, TIWO (tee-whoa): Things I’m Working On. ↩
  3. My first major research project at VMware was about tags. ↩
Apple, user experience

consistency matters

2 Comments

There is an inconsistency in two things that I use frequently, and it trips me up all the time.

In OS X Mavericks (and previous versions1), when using the Finder, there’s one keyboard shortcut that I use all the time.  If you have a long list of folders and files, you can go to the parent folder and type the first letter of the file that you want.  It will jump to the first file that starts with that letter.  I have lots of folders with lots of files in them, and I use this all the time so that I don’t have to scroll.2  For example, if you have a folder with a lot of files in it, and you want to jump to “nadyne.txt”, you can select the parent folder (say, Documents) and then type an N.  You’ll be taken to the first item in the list with an N, and have much less scrolling to do.

However, in iTunes 11, if you want to jump to the first item in the list that begins with an N, the focus has to be on the list itself.  For example, if you want to jump to “Neko Case” in your artist listing, you have to have focus there, and then click N.  To be consistent with the Finder, you would need to have focus on the parent object of “Music”.

I don’t really care which behavior is the winning behavior, I just wish that the two were consistent.  I trip over this all the time, and it drives me utterly batty.  (And yes, I’ve submitted the Radar: 15750167.)

  1. I can’t say how far back.  Muscle memory says that it’s far back indeed. I can’t remember when it didn’t work this way on OS X. ↩
  2. I use this even more frequently now that the arrow buttons have disappeared from scrollbars, but that’s another post for another time. ↩
user experience

how to benchmark UX

One of the LinkedIn groups that I’m subscribed to, Mobile UX, asked a question about how to benchmark UX.  This is something that we do at VMware, and that I did during my time at Microsoft too.  The methods at both VMware and Microsoft were reasonably similar.

My team has two different methods that we use to benchmark UX: scorecards and baseline studies.

Scorecards are during the product cycle, and rate the user experience of the top use cases. Scorecard studies can be repeated at any time during the development cycle, and allow the team to see progress being made (or not!). We use different methodologies for scorecard studies, depending where we are in the development process and what the resources available are. Scorecards are generally qualitative in nature. The goal is to provide an at-a-glance view of our current UX, with additional details about how to address any UX issues that might exist or what is currently being done about it (say, a link to a bug that has been opened to track a particular issue).

Baseline studies are conducted on the completed product that ships. Baseline studies are quantitative. Participants have a task list which they are asked to complete without thinking out loud. Our important metrics here are time-on-task and success rates. The task list for the baseline usability study is based on the most important use cases for the product, and is intended to be repeated on each subsequent product release.  A baseline study can help you determine whether you met your UX goals for this release.  It can also identify places where you unintentionally had a positive or negative impact on your UX.  Trend data over multiple releases helps others understand the ROI of UX.

I’ve also used baseline studies to help teams understand the usage of their application better.  When an application or team has never had involvement from UX before and isn’t sure how to best use this new UX resource, a baseline usability study of the top use cases often makes it clear where UX should focus their time and attention.  If you learn in your baseline study that your users have trouble completing one of the most important workflows in your application, you know what you’ve got to fix.

I like benchmarking UX to be able to show where we came from, how we got there, and what we learned along the way.  I especially like being able to show where we are today, since it usually illuminates at least part of where we need to go tomorrow.

user experience, VMware

requirements for being a UX researcher

A major requirement for being a UX researcher is flexibility.  User research has many moving pieces.  Any of them can fail, and a researcher has to be able to handle it with aplomb and with minimal disruption.

If a user study involves a prototype or working code, there is always a chance for error.  There can be bugs in the prototype or unexpected technology issues.  The user could also come up with a different solution than the one intended to be studied by the researcher, which might not work at all.  The user researcher has to try to minimize the chances of such issues in advance, as well as handle them discreetly during the study in such a way that the research participant doesn’t feel interrupted.

Research that involves participants at scheduled times comes with its own set of challenges.  Participants can be late (or early!), or need to reschedule, or not show up.  There is also the chance that a carefully-selected participant might not be a good participant, in which case the researcher has to gently handle the situation to ensure that the participant doesn’t have a bad experience and also that anyone who is observing the research doesn’t draw invalid conclusions.

There are several characteristics that I look for in researchers to indicate how good they are.  Researchers tend to be scrupulously on-time (often early), because they never want to keep a participant waiting.  They tend to keep their calendar updated so that they can accurately schedule participants.  They build in time for unforeseen issues.  They are excellent communicators, and are also very good at following up.  They take excellent notes, and they are able to get to the heart of the matter quickly.  They can get up-to-speed on something new very quickly, and ask excellent questions along the way.

This is all a very long way of saying that my team is currently hiring for a researcher.  Ping me if you’re interested.

user experience

user research fallacies

Wired posted an interesting opinion piece about user research, although its title is about startups and innovation.  The central point of the article is that startups often skip user research on the theory that they can fail fast and pivot to the next thing, and that this works but has a massive opportunity cost associated with it1.  This paragraph quite nicely sums up many of the arguments against doing user research:

When the research focuses on what people actually do (watch cat videos) rather than what they wish they did (produce cinema-quality home movies) it actually expands possibilities. But a common concern and excuse for not doing research is that it will limit creative possibilities to only those articulated by the target users, leaving designers devising a faster horse (lame) rather than a flying car (rad).

Research isn’t transcription.  There’s a lot of other things that research is and isn’t, and the Wired piece is a great starting point to learning more about it.

  1. I’d argue that this theory isn’t limited to startups. ↩
user experience

context matters

Of all of the methods of doing user research, contextual inquiry and ethnography are two of my favorites.  They help you understand your the context that your user is operating in.  Working for VMware, I often think about this in terms of what is happening in a system administrator’s datacenter, what their workspace looks like, what kind of mobile devices they use and why, and so on.

These methods are important for consumer products as well.  Take, for example, Procter and Gamble’s newest men’s razor for the emerging Indian market.  They knew from talking to Indians living in America about some characteristics of Indian men that they would have to take into account in designing their razor.  They even tested a new razor on Indian students at MIT, and that razor failed.  It took a trip to India to understand the context of men shaving there for them to fully understand the user need.  Indian students at MIT had running water to rinse their razor in.  Men in India were using a cup of water to shave, doing it in the dark without a mirror, and the process would take around a half-hour.  They also learned that not cutting themselves was a much larger concern for men in India.

The whole article is a really interesting read about what happens when you think you know your user, or when you think you’ve got a pretty good stand-in for your user.  P&G didn’t know their user, and they didn’t know their context.  Once they understood their user, their concerns, and their context, they were able to develop a razor that captured 9% of the market in India, bringing P&G’s overall market share to 49.1%.

(Thanks to @JakeHercules for the pointer to this article!)

user experience

when research results go rogue

When I interview candidates for user research roles, one of the questions that I am always looking to answer is how the candidate has ensured that their research results were acted upon.  For researchers who work inside a company, I want them to have ownership of their research results and recommendations based on the research.  I don’t want them to just throw the report over the wall and hope that someone on the other side of the wall will do something.  I want them to own those results, to work with teams to ensure that they understand the results and the recommendations, to help brainstorm ideas if the recommendations can’t be contained in this release.  It doesn’t help anyone if a researcher conducts fantastic research if the product doesn’t become better as a result of that research.  The best research is the research that impacts the product.

I recently had a conversation with another researcher who was frustrated.  He had done some great research, and had shared the results and recommendations with the team, and had been working with the team to ensure that they understood everything and would be able to take action on them, and was even tracking bugs o ensure that things were getting into the product.  He was doing everything right.  And then he finds out about a meeting after it was already 90% done, in which someone else was talking about the area that he had researched.  The presenter was sharing the researcher’s results and recommendations, and was discussing next steps.  The research results had gone rogue: they weren’t accompanied by the researcher who had a deep understanding of them, and they were being used by someone else who mostly (but didn’t completely) understand them for a different purpose and with a different audience than originally intended.

The researcher was upset: he had put a lot of work into creating, conducting, and disseminating that research.  It barely got acknowledged that the user insights that the presenter discussed were insights from the researcher, let alone that some of the slides were actually taken from the researcher’s presentation.  The researcher felt like his work wasn’t being acknowledged, and that he was being cut out of discussions about this area where he could continue to contribute.

His feeling was valid.  He should have been acknowledged for the work that he had done, and how his work was forming the basis for what the presenter was discussing as future work to be done.  His frustration is completely understandable.  The presenter should have contacted him to ask permission to re-use his slides, as well as get him involved so that he could help address follow-up questions about his work.

I reminded him that there was something really positive in all of this.  He had done all of the right things.  He had done them so well that his research is now just part of that team’s DNA.  It’s part of what they use to make decisions, and now it’s an important part of what they’re using to go forward.  This is one of the best possible outcomes for user research: not only does the team understand it and are taking action on it, it has a continuing impact as they think about what they should do next.  We talked about strategies to get him involved with the ongoing conversation so that he can contribute other things that he learned in conducting that research, as well as help with additional research as they move forward.

Research results can go rogue.  This can be good, this can be bad.  It’s frustrating either way, but I’m so glad that these rogue results are being used for good, and that there are ways that it can be managed to help rope them back in and grow those research results into an even better understanding of our users, their needs, and the challenges that they face.  He did a great job with the research, so good that his research is something that the team has forgotten was something that they didn’t know and couldn’t make headway on until he did that work.  We often talk about how a good UI should fade into the background.  Maybe that’s true for good research as well: it’s so good, and the results and recommendations are so much a part of what we do and so important to our understanding of our product and our user, that we forget that research happened at all.