Browse By

3 Steps to Effective Usability Testing on UserTesting.com

UserTesting.com is a great resource for testing your designs on actual users. As a designer, you sometimes become ‘blind’ to what you’re designing, meaning, you develop an inability to discern subtle differences in your work, which leads to a plateau in the design progress. This happens after multiple rounds of reviews with team stakeholders, which help to refine your designs, but can also be grueling. Getting user input is the best way to test whether your product is well designed and achieves user objectives, or whether you still have additional work to do to get to that perfect product-market fit.

UserTesting.com has emerged as the leader in this pursuit. They have a large number of testers standing ready to assist in evaluating your project, whether it’s a mobile application, web site or mobile-web application. One of the best features of UserTesting.com is the responsiveness of the testers. Publish a test and receive feedback within hours (depending on your demographic profile). After the usability test is complete, you receive a video of the user interacting with your app, along with their voice recording, which captures their thoughts as they move through the test tasks. In addition, you receive the answers to 4 follow up questions, which should capture any lingering thoughts or concerns that the user may have while executing your test.

 

In order to setup a test, you must follow a few steps:

1.  A usability test starts with a testing objective that’s sets the scene for the user and let’s them know their mental state. For instance, if you’re looking to test a signup flow for a financial institution (think Fidelity), you might want the user to imagine that they have recently thought about investing some of their savings in the stock market and after seeing an ad on MarketWatch.com, they land on a signup page to get started.

2. After writing your test objective, you’re required to add tasks that the user will walk-through. No more than 10 should be used and they should include probing questions such as, did you find the site to be trustworthy and reliable? How likely are you to return to the site? If you’re not going to return, why? What didn’t you like?

3. Finally, 4 wrap up questions finish the test. For these, it’s important to leave the last question high level and open ended, for example, Is there anything else that would make your experience more enjoyable? Please let us know!

Finally, as always, be sure to thank the user for their time. UserTesting.com testers are paid, so they don’t need added incentive. However, if you’re recruiting your own candidates for in-house testing, you must provide them with an incentive (e.g. a gift card) or your no-show rate will be high. 

 

What do you do with the UserTesting.com test results?

After the tests have completed, it’s important to watch every video at least 3 times. Wear headphones and have a notepad ready to quickly write down any comments, issues or suggestions that arise. Your goal is not to have neatly organized, perfect notes. Your goal is to write down everything you hear, in a stream of consciousness style. After you have watched every video and recorded your notes, write down the consistencies you see. Did every user have trouble with a certain step or grasping a concept? Did every user like something in particular? Did every user hate something in particular? 

After you have written down your initial findings, you should summarize them into high level findings and recommendations. Below is a sample outline for a usability test report that you can present to your manager and team stakeholders. 

 

Sample Outline for Usability Testing Report

PAGE:

  1. High Level Findings
    • Positive
    • Negative
  2. Recommendations
    • Quick-wins
    • Long-term suggestions
  3. User Scorecard
    • Rate each user high, medium or low on the following:
      • Positive Impression?
      • Ability to complete tasks?
      • Clear understanding of X
      • Clear understanding of Y
      • Likelyhood of doing X
  4. Subject #1 Report
    • Video URL
    • User Information
        • Date
        • Test Number
        • Username
        • Country
        • Gender
        • Age
        • Income
    • Computer System Information
      • Operating System
      • Browser
      • Memory (RAM)
      • Display Resolution
      • Color Depth
      • CPU
    • Notes
      • What was the user’s first impression?
      • What did they like?
      • What did they dislike?
      • Write down any findings from their video
    • Post-Test Follow Up Questions
  5. …Repeat for all users
  6. Task Appendix
    • List each task that the user conducted here.

 

Sample Questions for Usability Testing

  1. What was your opinion of the content on “XYZ” page? 
  2. Is there anything missing? 
  3. Is there too much?
  4. Does XYZ company appear to be trustworthy and reliable?
  5. Where would you look for more information on XYZ topic?
  6. What types of XYZ does XYZ company provide?
  7. Where would you look for an overview of XYZ?
  8. What is your first impression of XYZ?
  9. Would you use XYZ yourself?
  10. What frustrated your the most about XYZ?
  11. Do you have any unanswered questions?
  12. Would you be more inclined to do XYZ if we provided ABC?
  13. Did you find any barriers that would keep you from doing XYZ?
  14. What did you find attractive about XYZ?
  15. Any final thoughts?