Here's the thing - Bolt | Peters was acquired by Facebook. This site will live on.

Remote is Better, pt. 1: Getting Clients’ Hands Dirty

Posted on Mar 31st, 2008 by Tony Tulathimutte

In this inaugural entry in our B|P continuing series “Remote is Better”, we discuss how separating the moderator and the user eliminates the need for “two-way mirrors”–now you can get your clients into the driver’s seat with you (metaphorically). We show you how!

Lots of people think of remote research as a trade-off or a compromise–a cheap, quick alternative for when you can’t get users in the lab face-to-face. What often gets overlooked are the many, many qualitative benefits of testing remotely: if done properly, remote research can give you all kinds of data and insight that would be impossible to get otherwise. Of course, doing it properly means you need to know what you’re doing. Wouldn’t it be nice if there were people around with years of remote research experience, who were nice (or dumb) enough to give away all their best practices on their official blog?

Use the Right Tools

First things first: your clients need to be able to see and hear everything that you (the moderator) do. This can be as simple as having clients sit alongside the moderator and listen in on the phone conversation, passing notes to the moderator whenever they need to–but that’s kid’s stuff. There are lots of ways to use remote research tools in order to streamline this process. Our favorite, as we’ve mentioned before, is UserVue: it automatically records audio and video, and supports observer chat and video marking for later analysis, but there are tons of screensharing tools with basic built-in observer support: GoToMeeting, NetMeeting, and Adobe Connect. Audio can be shared using any two-line phone with a conference call feature—the moderator can talk to the participant on one line, and clients can listen in on the other. Remember to It’s important that their line remains muted, so that participants don’t hear them.)

Why let clients in on the game?

So, why even bother? Why is it worth releasing a bunch of potential monkeys into your airtight research methodology? Let Tony break it down for you:

Transparency. Letting clients in on the action can get them more invested in the outcome and make the findings easier to understand and present to them. Instead of just seeing a report at the end of the process, participating clients can draw on their own memories of the sessions to put the findings into richer context. You can win over the confidence of those many, many stakeholders who are unfamiliar with user research and are still on the fence about the soundness and validity of the methods (always a big issue for us UX types). When all the stuff that goes into the findings are clear, the value of the research becomes clear, but only, of course, insofar as you’re actually doing good research.

Hitting the right targets. Most research is based on goals and objectives laid out before any research happens. While this is perfect for targeted, specific types of studies (“Does this widget work?”), broader research projects (“How do users respond to our site?”) can benefit from a little flexibility. If a session gets bogged down in topics that aren’t all that important to your clients, then then clients can tell the moderator to move on. Even more useful is the notion of what we at B|P call emerging topics: clients can point out new and unforeseen issues they’re interested in as they come up during the session, allowing for a dynamic scope of research. Of course, it’s up to the moderator’s best judgment to keep the scope manageable—clients may go a little overboard and point out things way outside the domain of the research, and balance must be struck between what is useful to uncover and what the constraints of the research schedule can accommodate.

Expertise and assistance. Since practically any number of people can listen in on a research session, it’s possible to have clients of all kinds of backgrounds and departments participating in the session—designers, engineers, customer service reps, even CEOs can all listen in, supplying any necessary assistance to the moderator. Especially if the project takes specialist-type backgrounds to understand technical jargon, expert clients can help decode or explain obscure user comments. If what’s being tested is a prototype or beta model, the people involved in building the prototype can give the moderator a hand with any malfunctions that arise, and see for themselves if the prototype is working like it’s supposed to.

Note-taking and video annotation. Put em to work! Any notes clients care to take can provide great insight later on what aspects of the session clients are paying the most attention to. If the screen-sharing tool provides live video marking functionality (as UserVue does), clients can even stick notes right in the video file for the moderator to check out later.

Get Em Ready

The best way to make sure that clients don’t pull some crazy-monkey interference on your research is to make sure they know what to expect in advance.

To keep clients questions to the moderator at a minimum, give them a cheat-sheet they can refer to, including clear instructions about how to set up screen-sharing tools, how to chat or mark videos in UserVue, how to properly contact the moderator for requests, and so on.

Clients should also know what to expect about the constraints of the moderator during the session. Since the moderator will both be taking notes and speaking to the user, s/he may not respond right away to clients’ questions. Style is another constraint: the moderator must keep the tasks natural and the language neutral, so s/he might not act on client requests exactly as they would expect. For example, a client may ask, “Ask the user what he thinks of the navigation bar,” but the moderator might want to get at the issue in a more natural and less leading way with a series of questions: “Where would you go to look for laptops?”, “How would you get back to the main page?” etc.

As mentioned above, clients may occasionally ask the moderator to address issues that lie way outside of the scope of the research or would be too much to tackle within the time limit. Encourage your clients to tell you what’s important or interesting to them, but also ask (nicely! Always nicely!) to stick generally to the research goals, and to defer to the moderator’s judgment for balancing emerging topics with the established research goals.

The Care and Feeding of Clients

If you prep them right, you won’t need to do a whole lot of management during the study–your clients can just chug along in the background, contacting you when necessary. But! Don’t assume that clients will follow the preparations to the letter–you’ve got to be prepared to actively manage clients during the study.

If any issue comes up, it’s usually about clients who make more demands than you can handle. It is important to promptly acknowledge their requests to reassure them that you’re listening, but you don’t have to necessarily promise to deliver on every request. To avoid coming off as rushed or irritated, use polite, concise, and unambiguously friendly language: “Understood. I’ll see what I can do! :)” You should also ask for clarification if the request is unclear: “Not sure I quite understand, can you explain? Thanks!” Ultimately, if there are too many requests to handle, the moderator should acknowledge it with the clients when the session ends: “Sorry I couldn’t get to all of your requests—it was important to leave time to achieve all of our study goals. I’ll definitely try to handle as many of your requests as I can, so feel free to keep sending them along!” You want to make clients feel comfortable talking to you, but they also need to understand that you’re just a mortal. (This does not apply if you are Thor.)

There may also be the problem of clients who don’t seem to be paying attention to the session, or who don’t ask any questions at all. If you don’t hear a peep from clients for a long time (~10 minutes), it’s good practice to ask them if there’s anything they’re curious about or want to have you probe further on. If clients respond no, then the moderator can simply continue; if clients are still mum, then the moderator can follow up with a brief assessment of the session to encourage them to respond: “This user seems to have a lot of trouble logging in. I’ll try to find out more about that.”

Finally, since you’re ultimately the one responsible for gathering the feedback, avoid soliciting advice or overrelying on clients for what to tell a participant (e.g. “What should I ask the user now?”). Client requests are more of a tool for refining research objectives, not a playbook for the moderator. (The playbook, of course, consists of the documents based on the research objectives, like the moderator script.)

After Testing

It can be good to hold very short debriefing sessions after each session, in order to get clients’ feedback on the moderating style and findings. When all testing is complete, clients may participate in brainstorming sessions, contributing any insights they may have gathered as outside observers. This is also a good opportunity for moderators to identify from clients’ interpretations any biases they may have about the research. For example, they may take a user’s comment that “the login process was okay” to mean that there are no problems with the login process, whereas a trained moderator may have identified numerous behavioral issues that contradict that perception. Moderators can then be prepared to address this bias during the drafting of the report.

Finally, when the findings are presented, it’s nice to take time to acknowledge the clients’ contributions to the project, specifically calling out instances where they helped. This is valuable for showing clients how they themselves benefited the project, thus increasing their investment and buy-in into the research itself.Greater buy-in will increase the chances of future research projects. The benefits of having those same active, interested, and research-experienced clients helping you out can only increase.

(Photo credit: “looli” @ flickr)

Tags: , , , , , , , , , , , , , , , ,

347 Responses to “Remote is Better, pt. 1: Getting Clients’ Hands Dirty”

  1. It’s a shame you don’t have a donate button! I’d without a doubt donate to this superb blog! I guess for now i’ll settle for book-marking and adding your RSS feed to my Google account. I look forward to fresh updates and will talk about this blog with my Facebook group. Chat soon!

  2. Wow that was odd. I just wrote an extremely long comment but after I clicked submit my comment didn’t appear. Grrrr… well I’m not writing all that over again. Anyway, just wanted to say superb blog!

  3. google+
    Dec 05, 2016

    How do I add the shadows to a Joomla template?

  4. Superb blog! Do you have any tips for aspiring writers? I’m planning to start my own blog soon but I’m a little lost on everything. Would you recommend starting with a free platform like WordPress or go for a paid option? There are so many options out there that I’m totally confused .. Any tips? Appreciate it!

  5. Good day! This is my first comment here so I just wanted to give a quick shout out and tell you I truly enjoy reading through your posts. Can you recommend any other blogs/websites/forums that deal with the same topics? Thanks for your time!

  6. I do believe all of the concepts you have presented for your post. They’re very convincing and can certainly work. Nonetheless, the posts are too short for starters. May just you please extend them a little from subsequent time? Thank you for the post.

  7. Voyance
    Dec 06, 2016

    Awesome List.I adore the “FormSpring” , “CrazyEgg” and “ActionScript” they are so cool ! AMAZINGGGGGGG :B

  8. You are my breathing in, I possess few blogs and sometimes run out from to brand.

  9. Quadcopter Theory
    Dec 06, 2016

    Spot on with this write-up, I truly believe that this web site needs a lot more attention.
    I’ll probably be back again to read more, thanks for the information!

  10. Hey there! I’m at work browsing your blog from my new apple iphone! Just wanted to say I love reading through your blog and look forward to all your posts! Keep up the superb work!

  11. Plumber In DC
    Dec 06, 2016

    I love your writing style genuinely loving this web site.

  12. health sites
    Dec 06, 2016

    My partner and I absolutely love your blog and find nearly all of your post’s to be exactly I’m looking for. can you offer guest writers to write content available for you? I wouldn’t mind writing a post or elaborating on many of the subjects you write related to here. Again, awesome weblog!

  13. window hardware
    Dec 06, 2016

    It’s seldom that I feel compelled to comment on articles, but you’re material is content-worthy. Thank you for being so conscientious about what you write.

  14. prints vip online
    Dec 06, 2016

    This articcle is in fact a nice one it helps new the web viewers, who are wishing for blogging.

  15. alat bantu sex pria
    Dec 06, 2016

    It’s actually a great and helpful piece of information. I am happy that you simply shared this useful info with us. Please keep us informed like this. Thanks for sharing.

  16. Do you have a spam problem on this site; I also am a blogger, and I was curious about your situation; we have developed some nice procedures and we are looking to exchange solutions with other folks, please shoot me an e-mail if interested.

  17. Mamie Miano
    Dec 07, 2016

    Yay google is my king helped me to locate this excellent web internet site ! .

  18. I really enjoy examining on this internet site, it has got superb posts.

  19. Thanks for the sensible critique. Me and my neighbor were just preparing to do a little research about this. We got a grab a book from our area library but I think I learned more from this post. I’m very glad to see such great information being shared freely out there.

  20. Cheap Printing Vip
    Dec 07, 2016

    Thee other day, while I was at work, my sister stole my apple ipad and tested to see if it can ssurvive
    a 40 foot drop, just so she can be a youtube sensation. My iPad is now brokn and
    she has 83 views. I know thyis is totally off topic but I had to share
    it with someone!

  21. I’m wanting to start a blog for my photography business. What free blog would be best?.

  22. … [Trackback]

    […] Read More here: […]

  23. printing vip
    Dec 07, 2016

    What’s up it’sme, I am also visiging this site regularly, this site is truly good and the viewers arre really sharing pleasant

  24. agen bola
    Dec 08, 2016

    I truly enjoy studying on this internet site, it holds excellent articles. “Beware lest in your anxiety to avoid war you obtain a master.” by Demosthenes.

  25. obat vimax bohong
    Dec 08, 2016

    I have recently started a blog, the information you offer on this website has helped me tremendously. Thank you for all of your time & work.

  26. visit their website
    Dec 08, 2016

    I had two computers both connected to a wireless router. Yesterday I connected a third computer to the network and, although the connection goes through for this new computer, one of the original computers now says “Limited or no connectivity” and cannot connect to the Internet. Is it possible that the new computer is using up too much bandwidth and is therefore preventing the other computer from accessing the Internet?.

  27. 55 printing
    Dec 08, 2016

    hi!,I like your writing very much! perrcentage we communicate more
    approximately your article on AOL? I require ann expert in this space to unraavel my problem.

    May be that’s you! Having a look forward to loo you.

  28. www.55 printing
    Dec 08, 2016

    What’s up, I desire to subscribe for this blog tto
    take hottest updates, therefore where can i do it please help out.

  29. judi online
    Dec 08, 2016

    Valuable info. Lucky me I found your web site by chance, and I am surprised why this accident didn’t took place earlier! I bookmarked it.

  30. www.55printing
    Dec 08, 2016

    For most recent news you have to payy a quick visit world-wide-web and on the web I found
    this web site as a mpst excellent siute for most recent updates.

  31. Jerold Zeinert
    Dec 08, 2016

    Superb blog here! In addition your web internet site rather a whole lot up quick! What host are you using? Can I get your affiliate hyperlink for your host? I wish my web site loaded up as quick as yours lol.

  32. You need to be a part of a contest for one of the highjest quality websites
    online. I most certainly woll highly recommend
    this site!

  33. Beatrice Lipari
    Dec 09, 2016

    As I internet website possessor I believe the content material matter here is rattling great , appreciate it for your efforts. You need to maintain it up forever! Excellent Luck.

  34. I am really glad to rread this blog posts which contains
    plenty of valuable data, thanks for providing such data.

  35. hi!,I love your writing ver much! share we keep up a correspondence more about yopur post on AOL?
    I reqhire aan expert in this area to resolve my problem.
    Maybe that is you! Looking forward tto see you.

  36. fifty Fiveprints
    Dec 09, 2016

    It’s amazing to visijt this site and reading the views
    of all mates about this paragraph, while I am also eaer of
    getting familiarity.

  37. I cling on to listening to the news broadcast speak about receiving free online grant applications so I have been looking around for the most excellent site to get one. Could you advise me please, where could i find some?

  38. Nettie Kress
    Dec 09, 2016

    Vous avez le moyen de choisir d’être pauvre ou riche, d’être heureux ou malheureux, BlackMa reste le meilleur endroit pour trouver des solutions à tous ces problèmes qui nous empêche de profiter de la vie. Visitez leur site :

  39. Have you ever thought about writing an e-book or guest authoring on other sites? I have a blog based on the same subjects you discuss and would really like to have you share some stories/information. I know my subscribers would value your work. If you are even remotely interested, feel free to send me an email.

  40. Lucy
    Dec 09, 2016

    Pretty great post. I simply stumbled upon your blog
    and wanted to mention that I’ve really enjoyed surfing
    around your weblog posts. After all I will be subscribing ffor
    your rss feed and I’m hoping you write again soon!

  41. Billy
    Dec 09, 2016

    Hi friends, its fantastic post regarding teachingand fully explained, keep it up all the time.

  42. Lorena
    Dec 09, 2016

    Hi to all, thee contents existing at this site are trujly awesome for people knowledge, well,
    keep up thhe nice work fellows.

  43. Delores
    Dec 09, 2016

    I love lopoking through a post that can make men and women think.
    Also, thanks for allowing for me to comment!

  44. Georgia
    Dec 09, 2016

    Good post. I learn something totally new and challenging on blogs
    I stumbleupon every day. It’s always exciting to read through
    content from other authors and use a little something
    from their websites.

    Dec 09, 2016

    What’s up to every one, the contents existing att this
    web site are really remarkable for people knowledge,
    well, keep up the nice work fellows.

  46. Valuable info. Fortunate me I discovered your site unintentionally,
    and I am surprised why this accident didn’t happened earlier!

    I bookmarked it.

  47. Jere
    Dec 09, 2016

    Please let mee kmow if you’re looking for a article writer for your blog.
    You have some really great articles and I think I would be a good asset.
    If you eer want to take sokme of the load off, I’d love to write
    some content for your blog in exchange for a link back to mine.

    Please send me an e-mail if interested. Regards!

Leave a Reply