Here's the thing - Bolt | Peters was acquired by Facebook. This site will live on.

This is Not a Dress Rehearsal

Posted on Apr 17th, 2007 by Everyone
323 comments

In this business we like to say that the person with the freshest eyes has a perspective that is the least tainted. So as the new kid at Bolt | Peters, I thought I’d talk about my take on this whole remote usability thing.

When I first talked to Cyd about how B|P recruits participants in real time, while they’re in the midst of the task we’re actually looking to test, it was a total eureka moment for me. I realized that live recruiting was a brilliant answer to a pain point I hardly realized I had as a usability moderator.

The way it works is, we throw up a DHTML overlay inviting users to participate directly from the website we’re testing. If we’re testing just a single page or sub-section of a site, we recruit from there. Without moving away from what they’re doing, users answer a little questionnaire, which we use to pre-qualify them. We only call users who meet the recruiting requirements. If we’re fast, we can have users on the phone for the study within minutes of filling out the questionnaire. It’s the ultimate in low maintenance recruiting!

Not only is there no pre-session phone interview, no phone tag, no scheduling users, no emailing out confirmations to remind them to show up where and when they said they would. But you actually get real users, who have experience with the site you’re dealing with. They don’t have to fake it.

I can’t tell you how many marginally-qualified people I’ve moderated in the past and what a drag it can be. In the past, whenever I’ve recruited users to bring them into a lab, I would spend the first part of every session getting the user “in the zone.” “Have you ever posted pictures on the Internet? No? Well, could you imagine a situation where you might want to post pictures some day??” It was like being an acting coach, trying to warm up the user and get them into their role. “Imagine you’re an IT manager and you actually do care about purchasing enterprise software.”

A lot of the users I tested were a little too good at getting into role. Since most of the recruiting was done using Craigslist ads, I’m afraid I had some professional participants on my hands. There seems to be an entire population that’s earning a livelihood from the ETC section of Craigslist. Their unique skill is in working the focus group and usability testing system; the higher the incentive offered, the more they would be able to elaborate on their “qualifications.”

Of course there is something to be said for the adage that any average Joe should be able to navigate your site. And testing with a Craigslist professional is better than no testing at all. But it’s a very different game when you test with real users. Especially when you catch them very close to the task you’re most interested in learning about. People are relaxed and at ease, and their behaviors are an open book. They don’t need to rehearse; they just need to be themselves.

When we call them up, users often seem pleasantly surprised at actually “winning” something off one of those pop-up boxes (We do offer 75 Amazon bucks for participating). It’s like they’re the subject of a reality TV spot for the duration of the interview. Since they’re in their own space, users sometimes try to multi-task while they’re on the line with us. We see this as a definite plus. We’re peeping into the messiness of their everyday lives and unpredictable online habits. This gives us insights into their actual experience that we just couldn’t get any other way.

One question I did have at the beginning was about whether people would really drop everything, in the middle of their work day to do a 45-minute usability session? What I learned after just a few sessions is that the answer is almost always: “For 75 bucks? You betcha.” That much hasn’t changed. “Lemme just close my office door here, and I’d be glad to talk with you…”

323 Responses to “This is Not a Dress Rehearsal”

  1. controlling anxiety
    Dec 02, 2016

    Hi there this is kind of of off topic but I was wondering if blogs use WYSIWYG editors or if you have to manually code with HTML. I’m starting a blog soon but have no coding expertise so I wanted to get advice from someone with experience. Any help would be greatly appreciated!


  2. Hi! I know this is kind of off topic but I was wondering if you knew where I could find a captcha plugin for my comment form? I’m using the same blog platform as yours and I’m having difficulty finding one? Thanks a lot!


  3. gift card balance
    Dec 02, 2016

    Hi there! Someone in my Myspace group shared this site with us so I came to take a look. I’m definitely enjoying the information. I’m book-marking and will be tweeting this to my followers! Outstanding blog and amazing design.


  4. ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

    ?????????????????????????????????????????????????


  5. Hello are using WordPress for your blog platform? I’m new to the blog world but I’m trying to get started and set up my own. Do you require any html coding knowledge to make your own blog? Any help would be greatly appreciated!


  6. gift card deals
    Dec 02, 2016

    Write more, thats all I have to say. Literally, it seems as though you relied on the video to make your point. You clearly know what youre talking about, why throw away your intelligence on just posting videos to your site when you could be giving us something enlightening to read?


  7. Anyone know of or have links to Australian copyright infringement cases involving photos or articles? Thanks.?


  8. work boots sale
    Dec 02, 2016

    Hey there! I understand this is kind of off-topic but I had to ask. Does managing a well-established website like yours take a large amount of work? I am completely new to blogging however I do write in my diary daily. I’d like to start a blog so I can share my personal experience and thoughts online. Please let me know if you have any kind of ideas or tips for brand new aspiring blog owners. Thankyou!


  9. work safety boots
    Dec 02, 2016

    Have you ever thought about including a little bit more than just your articles? I mean, what you say is important and all. Nevertheless just imagine if you added some great photos or video clips to give your posts more, “pop”! Your content is excellent but with pics and clips, this website could certainly be one of the best in its field. Superb blog!


  10. lloan
    Dec 02, 2016

    Hi it’s me, I am also visiting this site regularly, this website is truly good and the
    people are really sharing nice thoughts.


  11. Lloan
    Dec 02, 2016

    Excellent, what a web site it is! This weblog presents valuable data to us,
    keep it up.


  12. Do you have a spam problem on this website; I also am a blogger, and
    I was wondering your situation; many of us have developed some nice methods
    and we are looking to swap solutions with other folks, why not shoot me an email if interested.


  13. Hiya, I’m really glad I’ve found this information. Today bloggers publish only about gossips and web and this is really irritating. A good blog with interesting content, that’s what I need. Thank you for keeping this web site, I’ll be visiting it. Do you do newsletters? Can’t find it.


  14. great points altogether, you just won a new reader. What could you recommend about your put up that you simply made a few days in the past? Any positive?


  15. Read This
    Dec 02, 2016

    I as well as my guys were actually digesting the good ideas from your web site then at once I had a terrible feeling I never expressed respect to you for those secrets. The men ended up for that reason warmed to read them and have really been enjoying them. Appreciation for actually being quite kind as well as for making a decision on certain amazing subject matter most people are really wanting to be informed on. My sincere regret for not expressing gratitude to you earlier.


  16. Web Site
    Dec 02, 2016

    I loved as much as you’ll receive carried out right here. The sketch is tasteful, your authored material stylish. nonetheless, you command get bought an impatience over that you wish be delivering the following. unwell unquestionably come more formerly again since exactly the same nearly very often inside case you shield this increase.


  17. Get More Info
    Dec 02, 2016

    You really make it seem so easy with your presentation but I find this matter to be actually something which I think I would never understand. It seems too complex and very broad for me. I’m looking forward for your next post, I will try to get the hang of it!


  18. If yoou wish for to grow your know-how simply keep visiting this weeb site and be updated with thhe newest news posted here.


  19. agen bola 88
    Dec 02, 2016

    Thankyou for helping out, excellent information.


  20. Danish SEO Bureau
    Dec 03, 2016

    My brother recommended I may like this blog. He used to be entirely right. This put up truly made my day. You can not consider just how so much time I had spent for this info! Thanks!


  21. visit here
    Dec 03, 2016

    Thanks for the sensible critique. Me and my neighbor were just preparing to do some research on this. We got a grab a book from our area library but I think I learned more from this post. I’m very glad to see such great info being shared freely out there.


  22. website
    Dec 03, 2016

    certainly like your web site however you need to test the spelling on quite a few of your posts. Many of them are rife with spelling issues and I in finding it very troublesome to inform the truth then again I¡¦ll certainly come again again.


  23. ash bug
    Dec 03, 2016

    Greetings! Quick question that’s entirely off topic. Do you know how to make your site mobile friendly? My web site looks weird when browsing from my iphone4. I’m trying to find a theme or plugin that might be able to correct this problem. If you have any recommendations, please share. Thank you!



Leave a Reply