Stop Bullshit Research in Five Easy Steps

Anyone in the UX field who’s worked for a few companies will recognize a type of moderated research that gives off a reek of inauthenticity. Tell me if this sounds familiar: one moderator and six users sit around a table in a converted meeting room. The moderator tells the users, each of whom have been prescheduled and screened through a recruiting agency, to go to a prototype website and pretend they’re looking for a 20 GB googlydooter, or whatever. The users go into their cubicles, where the prototype is brought up on six identical, factory-default computers. Some of the users finish in five minutes, some don’t finish at all, but everyone gets exactly fifteen minutes to finish their task. (The early finishers drum their fingers in boredom, waiting for the moderator to call time.) Finally, the moderator brings up a projection of the prototype, and asks the users to voice their opinions, one-at-a-time, keeping their responses brief, to give everyone time to speak. The process lasts about 1-2 hours, making everyone kind of tired. The participants are paid their incentives, and the moderator drives home, wiping bitter tears from his eyes as he pulls into his driveway.

How could that possibly have been useful? he thinks to himself. What has my life come to?

Continue Reading

So you wanna be a UX researcher?

It’s May! Time for graduations and job hunts, and while B|P isn’t hiring right this minute (we have an awesome team and are busy refining our practices) we will be again, before too long. It seemed like a good time for a post on what I look for when hiring user experience researchers, so here goes, in reverse order of importance:

#4: Education. If you have less than 3 or 4 years of experience, I’m looking for a degree in a related field, and that really can be anything from cognitive science to anthropology to HCI to…well, surprise me, I can be convinced. If it’s from a top school in our field, that’s cool, but it’s no big deal if it’s not. A Master’s is nice for showing a commitment to the field, but doesn’t tell me much about you as a practicing researcher. And whether you do or don’t have a related degree, you’ll definitely grab my attention with interesting people-oriented research projects during your education. (If you have more than 3 or 4 years of experience, I don’t much care where you went to school or what you studied—I’ll be evaluating your professional record exclusively.)

Quiz time!

What do expectant mothers have to do with Aunt Jean’s plans for her family vacation on a cruise?

HOW-TO Take Time-Stamped Collaborative Notes

One thing we’ve always wanted to have is a way to allow multiple people watching one of our research sessions to take automatically time-stamped, collaborative notes–everyone working in the same document, live, with entries labeled by author. Then I realized that it’s been under our noses the whole time: IM chat rooms! With a simple piece of free software, you’ve got a time-stamping, notetaking powerhouse at your disposal. And it even supports smileys. All it takes is a little ingenuity!

Bill Buxton’s Bad Ass CHI 2008 Keynote on Being Human in a Digital Age

Okay, this photo has nothing to do with Bill Buxton’s keynote, except for that it’s a photo planet I made at CHI this year in Florence. The closing plenary talk was the most inspiring talk I heard at CHI. I wasn’t planning on taking notes, but as soon as he said he’d thrown out the talk that he originally planned to give, i got out my laptop and started typing. If I’ve missed any key points, please let me know! Here goes:

B|P Intern Tells All!

The raw, uncut dirt on Bolt|Peters from our intern-for-a-day, UCSD Cog Sci student Jason van Merle. Enjoy! -Tony

——–

So over spring break I decided to skip the margaritas and take my sliver of free time to visit with Bolt | Peters in San Francisco. Last spring, Nate Bolt and Mike Towber had given a presentation to my cognitive engineering class about B|P and the remote usability testing work they do. After spending so much time studying cognitive ethnography, which emphasizes getting your hands dirty and observing users in context, I got curious how one might observe user interaction without being able to actually view the user! They have an impressive client list, and I found their presentation very interesting, so I thought I’d fly up to San Francisco and check them out. They were kind enough to let me intern for the day.

Remote is Better, pt. 1: Getting Clients’ Hands Dirty

In this inaugural entry in our B|P continuing series “Remote is Better”, we discuss how separating the moderator and the user eliminates the need for “two-way mirrors”–now you can get your clients into the driver’s seat with you (metaphorically). We show you how!

Lots of people think of remote research as a trade-off or a compromise–a cheap, quick alternative for when you can’t get users in the lab face-to-face. What often gets overlooked are the many, many qualitative benefits of testing remotely: if done properly, remote research can give you all kinds of data and insight that would be impossible to get otherwise. Of course, doing it properly means you need to know what you’re doing. Wouldn’t it be nice if there were people around with years of remote research experience, who were nice (or dumb) enough to give away all their best practices on their official blog?

Cheddar UX

Call us crazy, but around BP we think that the future of money is a big deal.

Remote Tasting from London

We decided to try beaming in our favorite London-based B|P employee, Captain Ethnio (AKA Mike Towber) at our holiday lunch this year. It was awesome, and even the waitress got a kick out of talking with mike. Turns out restaurant people only look at you funny for a minute when you ask to use their booster seat for your laptop.

Whutup Italy

We’ll be presenting our crazy “simulated native environment for video games” methodology at the CHI 2008 conference in Florence, Italy. We’re excited to finally have a paper accepted at CHI, even though the cool kids don’t go anymore we are still excited. More video game projects this year will be coming, along with the first remote game experience study, everrrrr.

B|P Parties All the Time

Actually, we only party once a year. Usually to celebrate our company anniversary, of which there have now been SIX. So you are formally invited to come celebrate six years of B|P and two years of ethnio this February 21st at our SOMA office. And also come celebrate our five year anniversary since we got really busy last year and spaced out on throwing that party. We know, we know. You can see photos of our space and some previous parties if you’re into that kind of thing, or just RSVP to the official invite below.

Get into My Car

Recently, in addition to our always-cool remote web research, we’ve been getting off the web and into peoples’ cars. In November we had a great time riding along with drivers in different parts of California and observing their use of in-car technology. The goal is to help a car manufacturer design cockpit interfaces around the needs of real people driving. And we broadcast the whole thing to our clients live from the cars using EV-DO! Also, we’ve been getting outta people’s dreams. (Oh c’mon you love that song.)

The New Ethnio. It’s Free.

Our new live recruiting web app, Ethnio recruits real people for all kinds of research – not just remote usability testing. As we get down with research that is more about people’s lives and less about usability, we want you to be able to make the same transition. So use ethnio for free and do better research! Also, plain old remote usability is soooo 2007. You can now recruit participants for in-person focus groups, ethnography, field research, or good old usability testing (if you have to). Sort and filter potential participants in real-time, keep track of who you’ve contacted, and edit any part of the recruiting screener on the fly. Check out the felt stop-motion movie, which breaks it on down:

*EXCLUSIVE* Apple to debut Mac Shuffle at MacWorld

Apple is going to release an ultra-portable tomorrow at MacWorld without a doubt, and I think it will be a Mac Shuffle that lets you do random computing functions every time you press the space bar. Like email all your friends or create a photo album. You can just leave it on random and not even have to worry about what computing function will be next. Genius.

Remote Tasting, or: We Brought Mike to Lunch

We wanted a way to include our B|P homie Mike in our holiday lunch, but he works from London now, and flying him out just for lunch seemed a tad much. So instead, we decided to bring him with us remotely. All we needed was one laptop with a big-ass battery, one EVDO revA card, two skype accounts w/ the new MPEG4 video codec, one booster to place at his chair so the laptop would be at table level, and one friendly waitress at the Slow Club, in San Francisco.

HOW-TO Use UserVue Internationally

At the clubs, people are always asking the same thing: “I love UserVue for remote user research, but I hate that I can’t use it internationally!” For those unhappy party people, here is a guide to dialing internationally with UserVue.

When Live Really Means Live

A couple weeks ago, we were doing research onsite at a client’s office. We had an audience of about 10 and we were live recruiting. It was a high traffic site and we were catching most users within 5 minutes. The interviews were good, but since the focus was on form filling and error message handling, we decided nothing could replace seeing users encounter the pages for the very first time.

B|P Goes to London

Hello, chaps! I’m writing this from the future… ooooooo! Well, not really – but I have moved across the globe to London. Cool your jets, I’m still working for Bolt | Peters as your friendly Captain Ethnio. We figured, if we want to brag about how remote our methodologies are, we might as well walk the walk. So here I am – drop me a line if you’re in the neighborhood. Cheerio!

What We Did On Our Summer Vacation

A month since our last post…you might imagine a bunch of researchers lounging on a beach in an obscure archipelago, unreachable even by IM.

Users Suck

I’m not sure if Todd Wilken’s blog post on Why usability is a path to failure at the adaptive path blog was just venting, or maybe even going for the “99% Something Inflammatory” blog topic methodology, but I took Todd to mean what he said, and I kind of dug it. Here’s why. For years I’ve been noticing that products with a lot of soul or strong vision don’t seem to need usability at all. Big Steve’s stuff of course, and even MySpace before it became dead to me and I deleted my account, seemed to rise above the need for any usability in the traditional sense. But products without a lot of soul, or a vision that’s naturally diluted by large numbers of stakeholders, seem to need usability hella bad. So in honor of products and services that are lucky enough to fall into the first camp, I’ve made this t shirt for Todd that you can order from Zazzle and the rest of us to wear to our next design research meeting for a product. For some unkown reason, the inspiration for this t shirt came from Jeff Veen’s awesome bet combined with that recent Entourage episode.

Peering into the Users’ Technological Ecosystem

Of course by now everyone knows what we mean by “Live Recruits” – they’re usability recruits that we snag when they’re in the midst of visiting the website we’re testing. It’s most often done through mini-surveys on a DHTML overlay (not a popup!). If someone fills out our survey and they’re a match for our target quotas, they’re contacted and interviewed immediately. There’s no scheduling participants and no lag time between the time when a participant is on a site and when they’re interviewed. We talk to them about the tasks they were already doing: no make believe required. It’s the closest way we’ve figured out to observing users interact with a design in their real life circumstances, without artificial barriers of the lab.

Product Detail is King

Just this week, we’ll be presenting the same usability finding that has come out of three separate studies. We’ve been calling the finding “Product Detail is King.”

It may sound obvious, but when users are looking at products online — whether it’s software, healthcare, or crockpots – detailed information about the products is of utmost importance. And a user’s success in finding that fine level of detail can truly make or break whether they have a positive experience on the site.

Upside-Down Architecture

UPA Remote Usability Sessions

Most folks we talked to at the UPA conference this year in Colorado had conducted remote moderated testing using some combination of web conferencing tools and Morae. There were a couple sessions on remote usability tools, and they focused primarily on web apps like WebEx, Breeze, GoToMeeting, etc. One speaker, Mike, did mention that tools like ethnio moving forward will be the way to go since they are made for usability profesionals, which is cool.

UPA, Y’all

I’m in Austin, Texas attending the UPA (Usability Professionals Association) conference. Last night there was a “Birds of a Feather” group for folks interested in remote usability testing. The discussion was moderator by Aaron Marcus and about 30-40 people showed up. It was great to hear how excited and passionate this small group was about these fairly bleeding-edge methodologies, both qualitative and quantitative.