Here's the thing - Bolt | Peters was acquired by Facebook. This site will live on.

Remote is Better, pt. 1: Getting Clients’ Hands Dirty

Posted on Mar 31st, 2008 by Tony Tulathimutte

In this inaugural entry in our B|P continuing series “Remote is Better”, we discuss how separating the moderator and the user eliminates the need for “two-way mirrors”–now you can get your clients into the driver’s seat with you (metaphorically). We show you how!

Lots of people think of remote research as a trade-off or a compromise–a cheap, quick alternative for when you can’t get users in the lab face-to-face. What often gets overlooked are the many, many qualitative benefits of testing remotely: if done properly, remote research can give you all kinds of data and insight that would be impossible to get otherwise. Of course, doing it properly means you need to know what you’re doing. Wouldn’t it be nice if there were people around with years of remote research experience, who were nice (or dumb) enough to give away all their best practices on their official blog?

Use the Right Tools

First things first: your clients need to be able to see and hear everything that you (the moderator) do. This can be as simple as having clients sit alongside the moderator and listen in on the phone conversation, passing notes to the moderator whenever they need to–but that’s kid’s stuff. There are lots of ways to use remote research tools in order to streamline this process. Our favorite, as we’ve mentioned before, is UserVue: it automatically records audio and video, and supports observer chat and video marking for later analysis, but there are tons of screensharing tools with basic built-in observer support: GoToMeeting, NetMeeting, and Adobe Connect. Audio can be shared using any two-line phone with a conference call feature—the moderator can talk to the participant on one line, and clients can listen in on the other. Remember to It’s important that their line remains muted, so that participants don’t hear them.)

Why let clients in on the game?

So, why even bother? Why is it worth releasing a bunch of potential monkeys into your airtight research methodology? Let Tony break it down for you:

Transparency. Letting clients in on the action can get them more invested in the outcome and make the findings easier to understand and present to them. Instead of just seeing a report at the end of the process, participating clients can draw on their own memories of the sessions to put the findings into richer context. You can win over the confidence of those many, many stakeholders who are unfamiliar with user research and are still on the fence about the soundness and validity of the methods (always a big issue for us UX types). When all the stuff that goes into the findings are clear, the value of the research becomes clear, but only, of course, insofar as you’re actually doing good research.

Hitting the right targets. Most research is based on goals and objectives laid out before any research happens. While this is perfect for targeted, specific types of studies (“Does this widget work?”), broader research projects (“How do users respond to our site?”) can benefit from a little flexibility. If a session gets bogged down in topics that aren’t all that important to your clients, then then clients can tell the moderator to move on. Even more useful is the notion of what we at B|P call emerging topics: clients can point out new and unforeseen issues they’re interested in as they come up during the session, allowing for a dynamic scope of research. Of course, it’s up to the moderator’s best judgment to keep the scope manageable—clients may go a little overboard and point out things way outside the domain of the research, and balance must be struck between what is useful to uncover and what the constraints of the research schedule can accommodate.

Expertise and assistance. Since practically any number of people can listen in on a research session, it’s possible to have clients of all kinds of backgrounds and departments participating in the session—designers, engineers, customer service reps, even CEOs can all listen in, supplying any necessary assistance to the moderator. Especially if the project takes specialist-type backgrounds to understand technical jargon, expert clients can help decode or explain obscure user comments. If what’s being tested is a prototype or beta model, the people involved in building the prototype can give the moderator a hand with any malfunctions that arise, and see for themselves if the prototype is working like it’s supposed to.

Note-taking and video annotation. Put em to work! Any notes clients care to take can provide great insight later on what aspects of the session clients are paying the most attention to. If the screen-sharing tool provides live video marking functionality (as UserVue does), clients can even stick notes right in the video file for the moderator to check out later.

Get Em Ready

The best way to make sure that clients don’t pull some crazy-monkey interference on your research is to make sure they know what to expect in advance.

To keep clients questions to the moderator at a minimum, give them a cheat-sheet they can refer to, including clear instructions about how to set up screen-sharing tools, how to chat or mark videos in UserVue, how to properly contact the moderator for requests, and so on.

Clients should also know what to expect about the constraints of the moderator during the session. Since the moderator will both be taking notes and speaking to the user, s/he may not respond right away to clients’ questions. Style is another constraint: the moderator must keep the tasks natural and the language neutral, so s/he might not act on client requests exactly as they would expect. For example, a client may ask, “Ask the user what he thinks of the navigation bar,” but the moderator might want to get at the issue in a more natural and less leading way with a series of questions: “Where would you go to look for laptops?”, “How would you get back to the main page?” etc.

As mentioned above, clients may occasionally ask the moderator to address issues that lie way outside of the scope of the research or would be too much to tackle within the time limit. Encourage your clients to tell you what’s important or interesting to them, but also ask (nicely! Always nicely!) to stick generally to the research goals, and to defer to the moderator’s judgment for balancing emerging topics with the established research goals.

The Care and Feeding of Clients

If you prep them right, you won’t need to do a whole lot of management during the study–your clients can just chug along in the background, contacting you when necessary. But! Don’t assume that clients will follow the preparations to the letter–you’ve got to be prepared to actively manage clients during the study.

If any issue comes up, it’s usually about clients who make more demands than you can handle. It is important to promptly acknowledge their requests to reassure them that you’re listening, but you don’t have to necessarily promise to deliver on every request. To avoid coming off as rushed or irritated, use polite, concise, and unambiguously friendly language: “Understood. I’ll see what I can do! :)” You should also ask for clarification if the request is unclear: “Not sure I quite understand, can you explain? Thanks!” Ultimately, if there are too many requests to handle, the moderator should acknowledge it with the clients when the session ends: “Sorry I couldn’t get to all of your requests—it was important to leave time to achieve all of our study goals. I’ll definitely try to handle as many of your requests as I can, so feel free to keep sending them along!” You want to make clients feel comfortable talking to you, but they also need to understand that you’re just a mortal. (This does not apply if you are Thor.)

There may also be the problem of clients who don’t seem to be paying attention to the session, or who don’t ask any questions at all. If you don’t hear a peep from clients for a long time (~10 minutes), it’s good practice to ask them if there’s anything they’re curious about or want to have you probe further on. If clients respond no, then the moderator can simply continue; if clients are still mum, then the moderator can follow up with a brief assessment of the session to encourage them to respond: “This user seems to have a lot of trouble logging in. I’ll try to find out more about that.”

Finally, since you’re ultimately the one responsible for gathering the feedback, avoid soliciting advice or overrelying on clients for what to tell a participant (e.g. “What should I ask the user now?”). Client requests are more of a tool for refining research objectives, not a playbook for the moderator. (The playbook, of course, consists of the documents based on the research objectives, like the moderator script.)

After Testing

It can be good to hold very short debriefing sessions after each session, in order to get clients’ feedback on the moderating style and findings. When all testing is complete, clients may participate in brainstorming sessions, contributing any insights they may have gathered as outside observers. This is also a good opportunity for moderators to identify from clients’ interpretations any biases they may have about the research. For example, they may take a user’s comment that “the login process was okay” to mean that there are no problems with the login process, whereas a trained moderator may have identified numerous behavioral issues that contradict that perception. Moderators can then be prepared to address this bias during the drafting of the report.

Finally, when the findings are presented, it’s nice to take time to acknowledge the clients’ contributions to the project, specifically calling out instances where they helped. This is valuable for showing clients how they themselves benefited the project, thus increasing their investment and buy-in into the research itself.Greater buy-in will increase the chances of future research projects. The benefits of having those same active, interested, and research-experienced clients helping you out can only increase.

(Photo credit: “looli” @ flickr)

Tags: , , , , , , , , , , , , , , , ,

463 Responses to “Remote is Better, pt. 1: Getting Clients’ Hands Dirty”

  1. hey there and thanks to your information – I have definitely picked up anything new from right here. I did alternatively expertise several technical points the usage of this site, as I skilled to reload the site a lot of instances prior to I could get it to load properly. I have been brooding about if your web hosting is OK? No longer that I am complaining, however sluggish loading cases times will very frequently impact your placement in google and could injury your quality rating if ads and ***********|advertising|advertising|advertising and *********** with Adwords. Well I am adding this RSS to my e-mail and could look out for much extra of your respective fascinating content. Ensure that you replace this again very soon..

  2. Hi, i think that i saw you visited my site so i came to “return thee favor”.I’m attempting to find things to improve my site!I suppose its ok to use a
    few of your ideas!!

  3. Unquestionably believe that which you said. Your favorite reason appeared to be on the net the simplest
    thing to be aware of. I say too you, I definitely get irked while people thin about worries that they just ddo not know about.

    You managed too hit the nail upon the top as well aas
    defiuned out the whole thing without having side-effects ,
    people could take a signal. Will likely be back too get more.

  4. Greetings! Quick question that’s completely off topic. Do you know how to make your site mobile friendly? My blog looks weird when viewing from my apple iphone. I’m trying to find a template or plugin that might be able to resolve this issue. If you have any recommendations, please share. Appreciate it!

  5. Do you mind if I quote a few of your posts as long as I provide credit and sources back to your webpage? My blog site is in the exact same area of interest as yours and my visitors would certainly benefit from a lot of the information you provide here. Please let me know if this ok with you. Thanks a lot!

  6. Hi! I know this is kinda off topic however , I’d figured I’d ask. Would you be interested in trading links or maybe guest authoring a blog post or vice-versa? My website covers a lot of the same topics as yours and I think we could greatly benefit from each other. If you might be interested feel free to send me an e-mail. I look forward to hearing from you! Awesome blog by the way!

  7. Wonderful blog! I found it while surfing around on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Thanks

  8. continuously i used to read smaller contennt which as
    wel clear their motive, and that is also happening with this article which I am readng
    at thnis place.

  9. I cherished up to you’ll obtain performed right here. The caricature is tasteful, your authored subject matter stylish. nonetheless, you command get got an shakiness over that you would like be handing over the following. ill undoubtedly come further beforehand once more since precisely the similar nearly very often within case you protect this hike.

  10. lg air conditioner
    Dec 15, 2016

    Hi! I’ve been following your site for some time now and finally got the courage to go ahead and give you a shout out from Houston Texas! Just wanted to say keep up the good job!

  11. I was curious if you ever thought of changing the layout of your blog? Its very well written; I love what youve got to say. But maybe you could a little more in the way of content so people could connect with it better. Youve got an awful lot of text for only having 1 or two images. Maybe you could space it out better?

  12. Megan Fox photo
    Dec 15, 2016

    I’m studying Journalism at the University of Minnesota, and I am looking for a good website to start blogging on. I’m mostly looking for a good sports blogging website, but a music website would also be of interest. I’ve heard about Bleacher Report, is that one of the main sites?.

  13. Hey, you used to write excellent, but the last few posts have been kinda boring… I miss your tremendous writings. Past few posts are just a little out of track! come on!

  14. This is the right blog for anyone who wants to find out about this topic. You realize so much its almost hard to argue with you (not that I actually would want…HaHa). You definitely put a new spin on a topic thats been written about for years. Great stuff, just great!

  15. sitter websites
    Dec 16, 2016

    Greetings from Carolina! I’m bored to tears at work so I decided to check out your blog on my iphone during lunch break. I love the info you provide here and can’t wait to take a look when I get home. I’m shocked at how quick your blog loaded on my phone .. I’m not even using WIFI, just 3G .. Anyways, wonderful site!

  16. Hey there I am so glad I found your blog, I really found you by mistake, while I was researching on Digg for something else, Nonetheless I am here now and would just like to say thank you for a tremendous post and a all round interesting blog (I also love the theme/design), I don’t have time to read it all at the minute but I have saved it and also added in your RSS feeds, so when I have time I will be back to read a great deal more, Please do keep up the superb job.

  17. buy concert tickets
    Dec 16, 2016

    Good day! I know this is kinda off topic however , I’d figured I’d ask. Would you be interested in trading links or maybe guest authoring a blog post or vice-versa? My website goes over a lot of the same subjects as yours and I feel we could greatly benefit from each other. If you’re interested feel free to shoot me an e-mail. I look forward to hearing from you! Excellent blog by the way!

  18. zap poster printing
    Dec 16, 2016

    Whaat you said mae a great deal of sense. But, think about this, what
    if youu wrote a catchier post title? I ain’t suggesting your information isn’t solid, but
    what if you added a title that makes peoplle desire more?
    I mean Remote iss Better, pt. 1: Gettting Clients’ Hands Dirty | Boltt | Peters is a little plain. You ought to look at Yahoo’s front page
    and watch how they create news headlines to get people interested.
    You might add a video orr a pic or two to grab readers excited
    abot what you’ve got to say. Just myy opinion, it could bring
    youur blog a little bitt more interesting.

  19. power wash house
    Dec 16, 2016

    Hi there! I know this is kinda off topic but I’d figured I’d ask. Would you be interested in trading links or maybe guest writing a blog article or vice-versa? My blog addresses a lot of the same subjects as yours and I think we could greatly benefit from each other. If you happen to be interested feel free to send me an e-mail. I look forward to hearing from you! Excellent blog by the way!

  20. small phone systems
    Dec 16, 2016

    I’m really loving the theme/design of your weblog. Do you ever run into any browser compatibility problems? A number of my blog visitors have complained about my site not operating correctly in Explorer but looks great in Firefox. Do you have any ideas to help fix this issue?

  21. … [Trackback]

    […] There you will find 45576 more Infos: […]

  22. mueller tape ankle
    Dec 16, 2016

    I’m really enjoying the design and layout of your blog. It’s a very easy on the eyes which makes it much more pleasant for me to come here and visit more often. Did you hire out a developer to create your theme? Fantastic work!

  23. mueller inc
    Dec 16, 2016

    First of all I want to say excellent blog! I had a quick question that I’d like to ask if you don’t mind. I was interested to find out how you center yourself and clear your mind prior to writing. I have had difficulty clearing my thoughts in getting my ideas out there. I do enjoy writing however it just seems like the first 10 to 15 minutes tend to be lost simply just trying to figure out how to begin. Any ideas or tips? Many thanks!

  24. Hi, just required you to know I he added your site to my Google bookmarks due to your layout. But seriously, I believe your internet site has 1 in the freshest theme I??ve came across. It extremely helps make reading your blog significantly easier.

  25. Hi there are using WordPress for your site platform? I’m new to the blog world but I’m trying to get started and set up my own. Do you require any html coding expertise to make your own blog? Any help would be really appreciated!

  26. learn this here now
    Dec 17, 2016

    I simply want to say I’m newbie to weblog and truly liked your web-site. Most likely I’m going to bookmark your site . You certainly come with awesome articles and reviews. Thanks for revealing your blog.

  27. Dwight Rambo
    Dec 17, 2016

    Thanks for the information provided! I was obtaining for this details for a long time, but I wasn’t able to uncover a reliable source.

  28. apk downloads
    Dec 17, 2016

    … [Trackback]

    […] There you will find 48817 more Infos: […]

  29. seo hero
    Dec 17, 2016


  30. bonuses
    Dec 17, 2016

    What is the best computer virus remover?. My computer is slow as hell and freezes on me. Most of the time, I have to restart my computer. I think I may have a virus on my computer. What is the best software to remove the viruses of my computer?.

  31. video seo services
    Dec 18, 2016


  32. games
    Dec 18, 2016

    This really answered my problem, thanks!

  33. Andy OS
    Dec 18, 2016

    This is the right blog for anyone who wants to find out about this topic. You realize so much its almost hard to argue with you (not that I actually would want…HaHa). You definitely put a new spin on a topic thats been written about for years. Great stuff, just great!

  34. Wow, marvelous weblog layout! How lengthy have you been blogging for? you make running a blog look easy. The whole look of your website is excellent, as smartly as the content material!

  35. Thank you, I’ve recently been searching for info about this topic for a long time and yours is the best I’ve found out so far. However, what concerning the conclusion? Are you positive concerning the supply?

  36. Does your blog have a contact page? I’m having trouble locating it but, I’d like to shoot you an email. I’ve got some recommendations for your blog you might be interested in hearing. Either way, great website and I look forward to seeing it expand over time.

  37. Good day! This is my first visit to your blog! We are a collection of volunteers and starting a new project in a community in the same niche. Your blog provided us useful information to work on. You have done a outstanding job!

  38. Good post but I was wondering if you could write a litte more on this topic? I’d be very thankful if you could elaborate a little bit further. Kudos!

  39. cbd oil for pain
    Dec 20, 2016

    Hello! Would you mind if I share your blog with my twitter group? There’s a lot of people that I think would really appreciate your content. Please let me know. Thank you

  40. breast lift cost
    Dec 20, 2016

    Hey there! Would you mind if I share your blog with my zynga group? There’s a lot of folks that I think would really enjoy your content. Please let me know. Thanks

  41. try this out
    Dec 20, 2016

    I just want to tell you that I am all new to blogging and site-building and seriously savored this blog. Very likely I’m planning to bookmark your blog post . You certainly have terrific posts. Cheers for sharing your blog.

  42. dna synthese
    Dec 20, 2016

    Hi there! Do you know if they make any plugins to help with Search Engine Optimization? I’m trying to get my blog to rank for some targeted keywords but I’m not seeing very good results. If you know of any please share. Thanks!

  43. Leroy Spiering
    Dec 20, 2016

    It hard to seek out educated individuals on this topic, even so you sound like you realize what you might be speaking about! Thanks

  44. website here
    Dec 20, 2016

    I simply want to mention I’m newbie to weblog and absolutely loved this website. More than likely I’m likely to bookmark your site . You really come with beneficial article content. Many thanks for sharing your website page.

  45. I know this if off topic but I’m looking into starting my own blog and was curious what all is needed to get set up? I’m assuming having a blog like yours would cost a pretty penny? I’m not very web smart so I’m not 100% positive. Any recommendations or advice would be greatly appreciated. Thanks

  46. Admiring the persistence you put into your website and detailed information you provide. It’s good to come across a blog every once in a while that isn’t the same unwanted rehashed information. Great read! I’ve saved your site and I’m including your RSS feeds to my Google account.

  47. Hi there! I realize this is somewhat off-topic but I had to ask. Does building a well-established website like yours require a lot of work? I’m brand new to running a blog however I do write in my diary daily. I’d like to start a blog so I can share my own experience and views online. Please let me know if you have any suggestions or tips for new aspiring blog owners. Appreciate it!

  48. pavlik brace
    Dec 20, 2016

    Have you ever thought about publishing an e-book or guest authoring on other websites? I have a blog centered on the same ideas you discuss and would really like to have you share some stories/information. I know my readers would enjoy your work. If you are even remotely interested, feel free to shoot me an e-mail.

  49. important link
    Dec 20, 2016

    I simply want to say I am all new to blogging and certainly loved this web-site. More than likely I’m likely to bookmark your website . You really come with really good articles and reviews. With thanks for revealing your web site.

  50. magnificent submit, very informative. I wonder why the opposite experts of this sector do not notice this. You must proceed your writing. I’m sure, you’ve a huge readers’ base already!