Here's the thing - Bolt | Peters was acquired by Facebook. This site will live on.

This is Not a Dress Rehearsal

Posted on Apr 17th, 2007 by Everyone
583 comments

In this business we like to say that the person with the freshest eyes has a perspective that is the least tainted. So as the new kid at Bolt | Peters, I thought I’d talk about my take on this whole remote usability thing.

When I first talked to Cyd about how B|P recruits participants in real time, while they’re in the midst of the task we’re actually looking to test, it was a total eureka moment for me. I realized that live recruiting was a brilliant answer to a pain point I hardly realized I had as a usability moderator.

The way it works is, we throw up a DHTML overlay inviting users to participate directly from the website we’re testing. If we’re testing just a single page or sub-section of a site, we recruit from there. Without moving away from what they’re doing, users answer a little questionnaire, which we use to pre-qualify them. We only call users who meet the recruiting requirements. If we’re fast, we can have users on the phone for the study within minutes of filling out the questionnaire. It’s the ultimate in low maintenance recruiting!

Not only is there no pre-session phone interview, no phone tag, no scheduling users, no emailing out confirmations to remind them to show up where and when they said they would. But you actually get real users, who have experience with the site you’re dealing with. They don’t have to fake it.

I can’t tell you how many marginally-qualified people I’ve moderated in the past and what a drag it can be. In the past, whenever I’ve recruited users to bring them into a lab, I would spend the first part of every session getting the user “in the zone.” “Have you ever posted pictures on the Internet? No? Well, could you imagine a situation where you might want to post pictures some day??” It was like being an acting coach, trying to warm up the user and get them into their role. “Imagine you’re an IT manager and you actually do care about purchasing enterprise software.”

A lot of the users I tested were a little too good at getting into role. Since most of the recruiting was done using Craigslist ads, I’m afraid I had some professional participants on my hands. There seems to be an entire population that’s earning a livelihood from the ETC section of Craigslist. Their unique skill is in working the focus group and usability testing system; the higher the incentive offered, the more they would be able to elaborate on their “qualifications.”

Of course there is something to be said for the adage that any average Joe should be able to navigate your site. And testing with a Craigslist professional is better than no testing at all. But it’s a very different game when you test with real users. Especially when you catch them very close to the task you’re most interested in learning about. People are relaxed and at ease, and their behaviors are an open book. They don’t need to rehearse; they just need to be themselves.

When we call them up, users often seem pleasantly surprised at actually “winning” something off one of those pop-up boxes (We do offer 75 Amazon bucks for participating). It’s like they’re the subject of a reality TV spot for the duration of the interview. Since they’re in their own space, users sometimes try to multi-task while they’re on the line with us. We see this as a definite plus. We’re peeping into the messiness of their everyday lives and unpredictable online habits. This gives us insights into their actual experience that we just couldn’t get any other way.

One question I did have at the beginning was about whether people would really drop everything, in the middle of their work day to do a 45-minute usability session? What I learned after just a few sessions is that the answer is almost always: “For 75 bucks? You betcha.” That much hasn’t changed. “Lemme just close my office door here, and I’d be glad to talk with you…”

583 Responses to “This is Not a Dress Rehearsal”

  1. It’s a shame you don’t have a donate button! I’d certainly donate to this fantastic blog! I guess for now i’ll settle for book-marking and adding your RSS feed to my Google account. I look forward to new updates and will talk about this site with my Facebook group. Chat soon!


  2. crispr cas9 design
    Dec 20, 2016

    Hey! Would you mind if I share your blog with my myspace group? There’s a lot of folks that I think would really appreciate your content. Please let me know. Thank you


  3. acne scars on face
    Dec 20, 2016

    Please let me know if you’re looking for a article writer for your blog. You have some really great posts and I think I would be a good asset. If you ever want to take some of the load off, I’d really like to write some articles for your blog in exchange for a link back to mine. Please send me an e-mail if interested. Kudos!


  4. additional hints
    Dec 20, 2016

    I just want to tell you that I am new to blogs and absolutely enjoyed this blog site. Most likely I’m going to bookmark your site . You certainly come with tremendous writings. Thanks for sharing with us your web site.


  5. Have you ever considered creating an ebook or guest authoring on other sites? I have a blog based on the same subjects you discuss and would love to have you share some stories/information. I know my visitors would enjoy your work. If you are even remotely interested, feel free to send me an e mail.


  6. my site
    Dec 20, 2016

    I simply want to say I am all new to blogging and honestly savored this blog site. Likely I’m planning to bookmark your blog . You absolutely come with perfect stories. Thanks a lot for sharing your web-site.


  7. great site
    Dec 20, 2016

    I simply want to say I am newbie to weblog and honestly savored this website. Likely I’m likely to bookmark your blog . You certainly come with awesome articles and reviews. Cheers for revealing your web site.


  8. pier building
    Dec 20, 2016

    Fantastic site you have here but I was wondering if you knew of any discussion boards that cover the same topics talked about here? I’d really like to be a part of community where I can get comments from other knowledgeable people that share the same interest. If you have any suggestions, please let me know. Thanks a lot!


  9. Kupon
    Dec 20, 2016

    Thank you a bunch for sharing this with all people you actually know what you’re speaking approximately!
    Bookmarked. Kindly also discuss with my website =). We could have a link exchange arrangement between us


  10. her latest blog
    Dec 20, 2016

    I just want to tell you that I am just all new to blogging and site-building and seriously savored you’re web blog. Likely I’m want to bookmark your website . You absolutely come with tremendous article content. Thanks a bunch for sharing with us your web page.


  11. 01 53
    Dec 20, 2016

    Howdy this is somewhat of off topic but I was wondering
    if blogs use WYSIWYG editors or if you have to manually code with
    HTML. I’m starting a blog soon but have no coding knowledge so I wanted
    to get advice from someone with experience. Any help would be greatly
    appreciated!


  12. Cheri
    Dec 20, 2016

    Huawei’s newest expansion comes at a time when client-oriented companies similar to Xiaomi Inc and Apple Inc, anticipating
    an explosion of Web-related dwelling appliances and client devices, have
    sought to construct ecosystems round their
    common handsets.


  13. Does your website have a contact page? I’m having problems locating it but, I’d like to send you an e-mail. I’ve got some ideas for your blog you might be interested in hearing. Either way, great website and I look forward to seeing it develop over time.


  14. click to read
    Dec 21, 2016

    I just want to say I’m beginner to blogging and definitely enjoyed this web-site. Most likely I’m likely to bookmark your blog post . You amazingly have good writings. Many thanks for sharing with us your website page.


  15. Hi! I know this is kinda off topic but I’d figured I’d ask. Would you be interested in exchanging links or maybe guest writing a blog post or vice-versa? My site goes over a lot of the same topics as yours and I think we could greatly benefit from each other. If you’re interested feel free to send me an e-mail. I look forward to hearing from you! Excellent blog by the way!


  16. click for more info
    Dec 21, 2016

    I simply want to say I am just all new to blogs and seriously enjoyed you’re web page. Almost certainly I’m planning to bookmark your website . You absolutely come with good article content. Appreciate it for sharing with us your web site.


  17. Thanks for one’s marvelous posting! I certainly enjoyed reading it, you can be
    a great author. I will be sure to bookmark your blog
    and may come back at some point. I want to encourage yourself to continue your
    great job, have a nice morning!


  18. Right away I am going to do my breakfast, after having my breakfast coming over again to read more
    news.


  19. Hi would you mind sharing which blog platform you’re using? I’m looking to start my own blog soon but I’m having a hard time selecting between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your design and style seems different then most blogs and I’m looking for something unique. P.S Apologies for being off-topic but I had to ask!


  20. check this site out
    Dec 21, 2016

    I just want to mention I’m new to blogging and site-building and absolutely enjoyed this website. Very likely I’m want to bookmark your blog post . You amazingly have impressive posts. Appreciate it for revealing your blog.


  21. Hello would you mind letting me know which web host you’re using? I’ve loaded your blog in 3 completely different internet browsers and I must say this blog loads a lot faster then most. Can you suggest a good hosting provider at a fair price? Many thanks, I appreciate it!


  22. amva display
    Dec 21, 2016

    Hey there! I know this is kind of off-topic but I had to ask. Does building a well-established website such as yours take a lot of work? I’m brand new to writing a blog but I do write in my journal every day. I’d like to start a blog so I can share my experience and feelings online. Please let me know if you have any ideas or tips for brand new aspiring bloggers. Thankyou!


  23. Hey there! I just wanted to ask if you ever have any trouble with hackers? My last blog (wordpress) was hacked and I ended up losing many months of hard work due to no back up. Do you have any solutions to prevent hackers?


  24. Fashionista
    Dec 21, 2016

    It is truly a great and helpful piece of information.
    I am happy that you shared this helpful information with us.

    Please stay us up to date like this. Thank you for sharing.


  25. vitamix 5200 price
    Dec 21, 2016

    Hello! This is my first visit to your blog! We are a group of volunteers and starting a new initiative in a community in the same niche. Your blog provided us beneficial information to work on. You have done a marvellous job!


  26. Awesome things here. I’m very happy to see your article.
    Thanks so much and I am having a look forward to contact
    you. Will you please drop me a mail?


  27. cheap scrubs online
    Dec 21, 2016

    I know this if off topic but I’m looking into starting my own weblog and was wondering what all is required to get setup? I’m assuming having a blog like yours would cost a pretty penny? I’m not very web smart so I’m not 100% positive. Any suggestions or advice would be greatly appreciated. Many thanks


  28. bisexual
    Dec 21, 2016

    My developer is trying to convince me to move to .net
    from PHP. I have always disliked the idea because
    of the expenses. But he’s tryiong none the less.
    I’ve been using WordPress on a number of websites for about
    a year and am worried about switching to another platform.
    I have heard great things about blogengine.net. Is
    there a way I can import all my wordpress posts into it?
    Any kind of help would be greatly appreciated!


  29. Javier Phalen
    Dec 21, 2016

    My brother recommended I might like this web web site. He was entirely correct. This publish actually made my day. It is possible to not consider just how so considerably time I had spent for this info! Thank you!


  30. Wow, that’s what I was seeking for, what a data! existing here at this website,
    thanks admin of this site.


  31. 347669 649275I admire the valuable facts you offer inside your articles. I will bookmark your weblog and also have my children verify up here often. I?m really positive they?ll learn plenty of new points proper here than anybody else! 787327


  32. It’s impressive that you are getting ideas from this paragraph as well as from our discussion made here.


  33. For wordpress there is the wphone which works great for simple themes, how about joomla? any way of updating a joomla website via mobile?.