Apple is going to release an ultra-portable tomorrow at MacWorld without a doubt, and I think it will be a Mac Shuffle that lets you do random computing functions every time you press the space bar. Like email all your friends or create a photo album. You can just leave it on random and not even have to worry about what computing function will be next. Genius. 
We wanted a way to include our B|P homie Mike in our holiday lunch, but he works from London now, and flying him out just for lunch seemed a tad much. So instead, we decided to bring him with us remotely. All we needed was one laptop with a big-ass battery, one EVDO revA card, two skype accounts w/ the new MPEG4 video codec, one booster to place at his chair so the laptop would be at table level, and one friendly waitress at the Slow Club, in San Francisco.
Mike and his wife Natalie ate Indian food at the same time we ate our lunch, and we could barely hear what he said during lunch but it was overall awesome. Seriously. Full set of photos on flickr
At the clubs, people are always asking the same thing: “I love UserVue for remote user research, but I hate that I can’t use it internationally!” For those unhappy party people, here is a guide to dialing internationally with UserVue.
WHAT YOU NEED:
- Desk phone
- Computer w/ Skype and UserVue
- Microphone headset for the computer
- Moderator
- Twix brand chocolate bar
LET’S DO THIS:
STEP 1:
Download and register for an account.
STEP 2: Login to Skype and start a UserVue session, like normal.
STEP 3:
Click “Call” in UserVue.
STEP 4:
Put your desk phone number in as Your Phone Number put the Skype call-in number as Participant’s Phone Number, and then click “Dial”.
STEP 5: Have the first Twix bar (there are two to a package). You’ll need the energy for the next few steps.
STEP 6:
Pick up your desk phone when it rings, and then dial “1” when the cold mechanical voice prompts you.
STEP 7:
Answer the call in Skype. Your desk phone will be connected to your Skype line.
STEP 8:
Mute and silence the desk phone. You won’t be speaking through it at all.
STEP 9:
In Skype, click Add Caller, and enter the participant’s number. If the built-in nternational line menu doesn’t work for any reason, then try setting the country to US and then typing in 011, the desired country code, then the participant’s number. Click Call.
STEP 10:
When the user picks up, you win. Direct him/her through the UserVue process like normal.
STEP 11:
Consume the second Twix bar–you’ve earned it. (There should be no more Twix bars. If you still have a Twix bar remaining by the end of this study, then you did it wrong, and the police will already be on their way to your house.)
(OR JAIL)
A WORD OF WARNING
Though it works and you can usually talk to your users just fine, there is a drop in the sound quality of the UserVue recordings. Also, the usual caveats apply when calling internationally with Skype, i.e. it drops out sometimes. You have been warned.
Categories
When Live Really Means Live
A couple weeks ago, we were doing research onsite at a client’s office. We had an audience of about 10 and we were live recruiting. It was a high traffic site and we were catching most users within 5 minutes. The interviews were good, but since the focus was on form filling and error message handling, we decided nothing could replace seeing users encounter the pages for the very first time.
So for the last few, we decided to go really, really live. We wouldn’t call a user more than 30 seconds after they filled out the screener, and instead of scrolling back up the list, we’d sit and wait for a qualified user to come in, each time.
We felt a little funny sitting there not doing anything for minutes at a time…and we found out a lot about our clients’ weekends, kids, and hobbies as we all sat watching the response screen, ready to pounce (actually, that was fun). But the quality of the truly live intercept interviews was unparalleled. Users stumbled, reacted to minor issues, and hardly ever said “I would think this” or “I would do that.” Because they were really doing it, right then.
A great reminder about why we do things this way, and how it’s worth it to wait to catch someone “really” live.
Hello, chaps! I’m writing this from the future… ooooooo! Well, not really – but I have moved across the globe to London. Cool your jets, I’m still working for Bolt | Peters as your friendly Captain Ethnio. We figured, if we want to brag about how remote our methodologies are, we might as well walk the walk. So here I am – drop me a line if you’re in the neighborhood. Cheerio!
A month since our last post…you might imagine a bunch of researchers lounging on a beach in an obscure archipelago, unreachable even by IM.
…but no, in the last month we’ve wrapped 8 research projects, including the most technically complex (and probably coolest) project in the history of the firm and the fastest project in the history of the firm. We had a great time.
Now, we just need a few drinks with umbrellas (to make up for what we missed on that unlisted island we didn’t go to) and we’ll be back in the blog saddle very soon!
Categories
Users Suck
I’m not sure if Todd Wilken’s blog post on Why usability is a path to failure at the adaptive path blog was just venting, or maybe even going for the “99% Something Inflammatory” blog topic methodology, but I took Todd to mean what he said, and I kind of dug it. Here’s why. For years I’ve been noticing that products with a lot of soul or strong vision don’t seem to need usability at all. Big Steve’s stuff of course, and even MySpace before it became dead to me and I deleted my account, seemed to rise above the need for any usability in the traditional sense. But products without a lot of soul, or a vision that’s naturally diluted by large numbers of stakeholders, seem to need usability hella bad. So in honor of products and services that are lucky enough to fall into the first camp, I’ve made this t shirt for Todd that you can order from Zazzle and the rest of us to wear to our next design research meeting for a product. For some unkown reason, the inspiration for this t shirt came from Jeff Veen’s awesome bet combined with that recent Entourage episode.
Of course by now everyone knows what we mean by “Live Recruits†– they’re usability recruits that we snag when they’re in the midst of visiting the website we’re testing. It’s most often done through mini-surveys on a DHTML overlay (not a popup!). If someone fills out our survey and they’re a match for our target quotas, they’re contacted and interviewed immediately. There’s no scheduling participants and no lag time between the time when a participant is on a site and when they’re interviewed. We talk to them about the tasks they were already doing: no make believe required. It’s the closest way we’ve figured out to observing users interact with a design in their real life circumstances, without artificial barriers of the lab.
Categories
Product Detail is King
Just this week, we’ll be presenting the same usability finding that has come out of three separate studies. We’ve been calling the finding “Product Detail is King.”
It may sound obvious, but when users are looking at products online — whether it’s software, healthcare, or crockpots – detailed information about the products is of utmost importance. And a user’s success in finding that fine level of detail can truly make or break whether they have a positive experience on the site.
Categories
Upside-Down Architecture
A couple of recent studies have brought up a big challenge to traditional information architecture. We’ve been working with clients who are trying to create landing pages for customers of one kind or another, and without exception we’re discovering that the customers would never “land” there.
Their initial objective is always 2 or 3 levels down, and these days the way they arrive is straight from Google rather than down the client’s carefully planned hierarchy. Even when customers know exactly which company makes the product they’re interested in, they go to Google and type “company productname” rather than going to company.com and navigating (or even using the homepage search).
So there’s this interesting phenomenon where the user has met an initial task need, but could really benefit from information that’s further “up” in the hierarchy of pages on the site. Traditionally we recommend using wayfinding widgets like breadcrumbs on lower-level pages so the user can self-locate and retrace their path–but that doesn’t seem adequate here. Our clients need to figure out how to design for paths that flow smoothly both “up” and “down” and how to explicitly attract users to navigate upward.
Calls to action need to draw users both directions depending on what they’re doing, and site architects need to assume that users will climb in any and every window before they come knocking on the front door.
What’s the answer? We don’t know yet, but we’re thinking hard about it.
Turning thing upside-down is what the revolution is all about after all.
Categories
UPA Remote Usability Sessions
Most folks we talked to at the UPA conference this year in Colorado had conducted remote moderated testing using some combination of web conferencing tools and Morae. There were a couple sessions on remote usability tools, and they focused primarily on web apps like WebEx, Breeze, GoToMeeting, etc. One speaker, Mike, did mention that tools like ethnio moving forward will be the way to go since they are made for usability profesionals, which is cool.
Categories
UPA, Y’all
I’m in Austin, Texas attending the UPA (Usability Professionals Association) conference. Last night there was a “Birds of a Feather” group for folks interested in remote usability testing. The discussion was moderator by Aaron Marcus and about 30-40 people showed up. It was great to hear how excited and passionate this small group was about these fairly bleeding-edge methodologies, both qualitative and quantitative.
A recurring theme at UPA has been combining qualitative and quantitative methods for more effective analysis. Catriona Campbell from Foviance is giving a talk dedicated to the topic. We’ve seen a lot of interest from clients lately in running these kinds of studies. We’ve conducted a bunch of these combined studies recently and seen exciting results that have helped us understand users better than if we used one method in isolation.
Last week, Nate and I had the honor to speak with a bunch of Cognitive Science students down at our alma mater, UCSD. Almost every year we send Nate and others down to talk to a Cognitive Engineering class about how much our HCI degrees mean to us in our professional lives (shocker: it’s a lot). It’s always a lot of fun and we get to meet a bunch of great people and chat it up with our mentors Dr. Jim Hollan and Dr. Ed Hutchins of the Distributed Cognition and Human Computer Interaction lab. We were joined this year by Kathy Seyama from Qualcomm‘s Usability Group, Ed Langstroth from the Volkswagen Electronics Research Laboratory, and Rod Ebrahimi from Do The Right Thing. Stay classy, San Diego.
Categories
The Booth that Gives Back!
For the sake of humanity, let me bullet point the B|P booth methodology.
- Find a local salvation army thrift store.
- Buy a van-full of the (best/tackiest) furniture you can get your hands on.
- (Optional) buy a slew of records from the 1970s (to play on the turntable you brought).
- Create your virtual living room.
- Donate all of your bounty back to salvation army at the end of the conference (except that amazing red clock – that’s a keeper).
After walking the CHI2007 circuit that called for heavy duty shmoozin & recruitin, it took visitors a second to figure out what to make of the remote usability living room. We co-hosted with our pal Steve August from Revelation.
Since we love talking to users in their most natural, laid back state, we thought it only fitting to provide a retro lounge for our CHI friends, where they could take a break from thinking too hard. People got into it — takin a load off in a recliner, listenin to pat benatar on vinyl, and smokin candy cigarettes.
Which bring us to the second most popular question we heard: where in the world did you get those candy cigs? Nothing but retro candy at groovycandies.com.
See more photos of our CHI 2007 booth here.
“Mediocre?” you may say, “But it works great and we love it…most of the time.” Well, as they say, there’s the rub. It can work, but it can also suck. Bugs abound, features break randomly or are absent altogether. Basically, Interweb gremlins feast on the fatty innards of the Ethnio beast. (Meaning: our system architecture is broken and we really, really want to re-build it.)
Ethnio started as a home-spun solution to help us run remote usability interviews. We patched it together from a collection of PHP scripts and a quick and dirty implementation of the VNC screen sharing protocol. Learning as we went along, we moved from developer to developer and the web application backend grew and grew, making bug fixes and additional features more and more treacherous.
So here we are, three years since Ethnio, the first remote moderated research app ever, became a reality. It’s just Nate and I working on maintaining, supporting, and developing the product – in addition to doing interesting research and the (wonderful) chaos of running a really keen research practice. Since we first started selling Ethnio, about a year ago, the time we’ve been able to spend fixing bugs and developing new features has diminished as we spend more and more time supporting existing customers and chatting with new ones. All the while we’ve been talking about new and exciting features, without the people power or people time to implement them – without the availability to even fix many of the existing bugs and quirks.
We don’t want Ethnio to be relegated to the status of a first-run product that faded, like WordStar or the Diamond Rio. We’re not satisfied with an okay product that mostly works. We are convinced we can make Ethnio an elegant, exceptional solution – and we’re dedicating the next 4 to 6 months to just that. Yes, it does mean that we’re going to stop selling Ethnio and giving trials in the meantime, but we promise it will be so much better for everyone in the long run.
We are still just as passionate about creating a technology that helps people understand people – we just don’t want our technology to get in the way of that ideal.
We’ll post updates on our progress along the way. Watch this space.
We’re doing everything we can do to free up more trial spots for everyone who’s been requesting them. We’d rather your first experience with ethnio was successful then have you see a bunch of bugs, so bear with us as we work out the scalability of our tool. One surefire way to get your name to the top of the list is to email us your experience and desires for remote testing.
Categories
UXI Chicago
I’m hitting up Adaptive Path’s UX Intensive workshop in Chicago this week, and helping them out as the event photographer and box mover. There has been some dope discussions of research methods today by Todd Wilkens.
Categories
This is Not a Dress Rehearsal
In this business we like to say that the person with the freshest eyes has a perspective that is the least tainted. So as the new kid at Bolt | Peters, I thought I’d talk about my take on this whole remote usability thing.
When I first talked to Cyd about how B|P recruits participants in real time, while they’re in the midst of the task we’re actually looking to test, it was a total eureka moment for me. I realized that live recruiting was a brilliant answer to a pain point I hardly realized I had as a usability moderator.
The way it works is, we throw up a DHTML overlay inviting users to participate directly from the website we’re testing. If we’re testing just a single page or sub-section of a site, we recruit from there. Without moving away from what they’re doing, users answer a little questionnaire, which we use to pre-qualify them. We only call users who meet the recruiting requirements. If we’re fast, we can have users on the phone for the study within minutes of filling out the questionnaire. It’s the ultimate in low maintenance recruiting!
Not only is there no pre-session phone interview, no phone tag, no scheduling users, no emailing out confirmations to remind them to show up where and when they said they would. But you actually get real users, who have experience with the site you’re dealing with. They don’t have to fake it.
I can’t tell you how many marginally-qualified people I’ve moderated in the past and what a drag it can be. In the past, whenever I’ve recruited users to bring them into a lab, I would spend the first part of every session getting the user “in the zone.†“Have you ever posted pictures on the Internet? No? Well, could you imagine a situation where you might want to post pictures some day??†It was like being an acting coach, trying to warm up the user and get them into their role. “Imagine you’re an IT manager and you actually do care about purchasing enterprise software.â€
A lot of the users I tested were a little too good at getting into role. Since most of the recruiting was done using Craigslist ads, I’m afraid I had some professional participants on my hands. There seems to be an entire population that’s earning a livelihood from the ETC section of Craigslist. Their unique skill is in working the focus group and usability testing system; the higher the incentive offered, the more they would be able to elaborate on their “qualifications.â€
Of course there is something to be said for the adage that any average Joe should be able to navigate your site. And testing with a Craigslist professional is better than no testing at all. But it’s a very different game when you test with real users. Especially when you catch them very close to the task you’re most interested in learning about. People are relaxed and at ease, and their behaviors are an open book. They don’t need to rehearse; they just need to be themselves.
When we call them up, users often seem pleasantly surprised at actually “winning†something off one of those pop-up boxes (We do offer 75 Amazon bucks for participating). It’s like they’re the subject of a reality TV spot for the duration of the interview. Since they’re in their own space, users sometimes try to multi-task while they’re on the line with us. We see this as a definite plus. We’re peeping into the messiness of their everyday lives and unpredictable online habits. This gives us insights into their actual experience that we just couldn’t get any other way.
One question I did have at the beginning was about whether people would really drop everything, in the middle of their work day to do a 45-minute usability session? What I learned after just a few sessions is that the answer is almost always: “For 75 bucks? You betcha.†That much hasn’t changed. “Lemme just close my office door here, and I’d be glad to talk with you…â€
We’ve stopped issuing new Ethnio accounts for the next six months while we make the product amazing instead of promising but buggy. We’ve had an amazing amount of interest, and three million people have viewed Ethnio live recruiting screeners, but we’re not content with selling a product that’s mediocre. We want more. Mike and Nate want to build the best moderated research web app ever. So we’re taking time to fix bugs and incorporate new functionality. This army of two people has been focusing on support, sales, and maintenance instead of our ideals, and so we’re taking this break to give you a new Ethnio. Read the bloggy blog for updates on our progress, and as always, we’d love to hear from you.
Categories
A Very Funny Thing
Achewood, an internet comic where independently wealthy cats write advice columns and robots lie, comments astutely on the usability profession:
Categories
New Team Members
To support the growth of our research service business and the success of Ethnio, we have some new folks that have joined the team in the past couple of months. Cyd Harrell brings nine years of experience in UX to our team as our new Director of UX Research. Before joining the team, Cyd was a director at Charles Schwab for their representative facing financial service application. That’s RFFSA if you want to get technical.
Also joining the team is Bryan Tsao. Bryan is the 5th graduate of the UCSD HCI program in 5 years to join our team. In addition to helping B|P out with user research and the development of Ethnio, Bryan is getting his masters at from UC Berkeley in the new iSchool (Information Management and Systems). Welcome Cyd and Bryan! Now if you meet people from the iSchool at CAL, don’t say “so you go to iSchool?,” because it sounds like you are asking if they go to high school, and that’s just awkward.
Categories
Remote Dog Treats
We are proud to announce that B|P will be leveraging its experience in remote research to expand into Remote Robotic Dog Treat Dispensers. Working with Lego, or actually, working with the toy robot kit we bought from Lego, Nate and Mike built this little robotic arm that receives IM’s with the word “treat” and then dispenses a small treat to the office dog.
You’ll soon be able to IM the wild pooch a treat and watch it on our office webcam. The venerable Make zine picked it up, and you can check out the video below! We know what you’re thinking, and no, it’s not clear to anyone at the office why we built this, either.
Categories
Just a Little Help, Please
It’s wild how many of our Ethnio customers have been asking for just a little bit of testing assistance for their remote research projects. So many of you are expert practitioners real physical world research, and just need a little assistance in conducting remote research. Whether it’s study design, training, moderating a couple users, helping combine moderated and automated remote methods, using MindCanvas or RelevantView, or just making sure your remote study goes smoothly, we’re down to help. B|P can ensure that the executive team watching your first remote study will denounce corporate social hierarchy and just give you a hug. Unless you don’t want a hug from your executive team, in which case we’ll make sure you get a nice email.
Categories
Ethnio Goes International
B|P is excited to announce that international dialing and call recording is now included free-of-charge with all Ethnio licenses. But what is Ethnio anyway? It’s the world’s first remote usability web application that allows you to watch and record anyone using a computer in the world with the express purpose of making their computer easier to use. Now you can expand your user research to international users! No need to fly to Japan and France to conduct usability studies unless you’re craving sushi or croissants on the corporate dime. Also, some of the latest organizations that have started using Ethnio are Adobe, CareerBuilder, and HP. They’re off and running with both assisted studies, where our practitioners help with set-up, moderating, or live recruiting, and doing all the remote research on their own.
Categories
Genentech Hearts B|P
We completed a rapid remote usability study for Genentech recently for their internal time-reporting tool, and got them up and running with Ethnio. Rather than run off at the mouth about how much we helped them, we’ll just pass along their words:
“The B|P project was a total success. Their research for Genentech actually elevated the importance of usability to the executive level here. It’s now a focus for our entire I.T. organization. The old time-reporting system flooded our Service Desk with support calls the day it launched. The new application launch barely made a ripple. ”
sarah murgel
manager, user experience & interface design, corporate information technology — genentech
