Bolt | Peters

Conduct Your Own Remote Usability Study


February 16th, 2005

This is our most popular and exciting way of observing your customers as they use a web application. We've developed an innovative way of conducting moderated one-on-one usability sessions over the internet. Since participants never leave their home or office, their behavior is more true than with in-lab studies. Best of all, this is all done while reducing turnaround time!

Step 1: Setup

What you'll need

Why you'll need it

Fast internet connection
Necessary bandwidth for online meeting.
Computer #1 with Ethnio installed (PC) or WebEx and Camtasia (Mac)
To view participant's screen behavior through online meeting.
An 800 Number that records audio, ar a phone recording interface, such as a JK Audio Tap.
To capture audio
Video-editing software
To create highlight mini-documentaries.
Carefully written moderator script
To run a successful interview, maintaining ethnographic validity.


Step 2: Recruiting users


With online recruiting, users are recruited directly from your site. Recruitment times are reduced and actual site visitors participate with real motivations. For some projects, online recruiting is not possible, and you may have to use a recruiting agency. Use the table below to determine the best recruiting method for your usability testing project.

Method

Pros

Cons

Online pop-up survey   • Participants exhibit natural behavior

  • Quick to implement

  • Large sample size of respondents possible

  • Globally accessible audience

  • Affordable
  •Requires (minimal) IT resources to implement

  • Requires a web site (with traffic)
Recruiting Agency  • Otherwise hard to find people can be included (CEO's, younger visitors, etc)

  • New customers can give first impressions

  • Doesn't require an existing web site, or traffic. This is good prototype development, IA activities, feedback on mockups, etc.
  • More time-consuming

  • More costly

  • Less natural


Step 3: Set up the online screen observation


In order to watch the participants as they use your web site, you'll have to see their browser window. This requires desktop-observation software. Ethnio is an affordable, desktop-observation solution for the PC that is ideal -- it is developed as a remote usability testing tool. If you have a Mac, you can use one of many alternatives available to the public (Webex, PlaceWare, etc).

After participants join the meeting and are sharing their webpages, direct them back to the live site, prototypes, mock-ups, etc.

Step 4: Conduct the session


You're now ready to run the users through the session (remember to make sure the camera is recording and you're getting sound). When the session is over, get the participant's address if you plan to send them a check for participation.

There are many good references for writing quality moderator scripts and conducting tests with strong ethnographic rigor. See Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, by Jeffrey Rubin.

Step 5: Analysis and recommendations


The keys to making usability findings actionable are good analysis and recommendations. Review your notes from each user and look for behavioral patterns and self-reported patterns. Assemble a matrix if possible, and refer back to your goals for conducting the study in the first place. Do they include increasing conversion? Or perhaps increasing overall ease-of-use?

Step 6: Drive web strategy with usability findings


Now that you have analyzed usability findings you are ready to share them with the team. You know first-hand which changes need to occur to meet the strategic goals of your product/site from a usability perspective. If you have edited mini-documentaries from your usability sessions, this is the time to leverage their illustrative power to decision makers. This is also the time to brainstorm with your development team about which technical changes may need to occur to realize usability gains on the presentation layer.

If two or three equally promising screen solutions emerge, consider a follow-up study using multi-variable staging. This type of quantitative testing can measure successful completion rate and other dependent variables based on automated user behavior logging across a large sample of users.

 

By Nate Bolt
  Picture Coming Soon