Lean Usability Testing: Current Best Practices and Resources

After my first post‘s philosophical bent, here’s something specifically targeted at startups. In preparation for a team meeting to discuss our usability testing strategy, I assembled everything I could find on the most current best practices for lean usability testing. This was originally a text file, but here is the blog post version with proper attribution (let me know if I missed anyone).

Last Tuesday I attended Andres Glusman’s presentation on lean usability testing at the NYC Lean Startup meetup (at Meetup.com’s HQ).

Of all the startup events I’ve been to so far in NYC, none was so dense with interesting and actionable information as Andres’ presentation. So I would highly recommend you check out the slides embedded below.

This presentation reminded me of Cindy Alvarez‘s presentation on usability testing that I saw last summer at the Facebook Fund in Palo Alto. I would also highly recommend that you read through all of her slides as well since both Cindy and Andres have had a lot of experience doing these tests yet came to their conclusions completely independently. So it’s interesting to see how their experiences differ.

Helpful tools for conducting lean usability tests

silverback.com (for macs)

techsmith.com/morae.asp (for PCs)

Lean user testing with The Flip: http://www.webgrrls.com/blog/2009/05/21/best-new-tool-for-user-testing-the-flip/

A real world example of how to conduct a usability test from Steve Krug:

Here is the link to see Steve Krug on the Least You Can Do About Usability Testing (Kind of long. Fast forward to 5 minutes in.)

Outsource Your Usability Testing

Short on money, time, and man-power? I would highly recommend looking into outsourcing your user-testing. As Jacob Nielsen notes, 5 user tests will typically identify 85% of errors but 0 user tests will find 0%. Cheap and outsourced usability can be extremely effective if in-house testing isn’t an option.

Outsourced user-testing options:

I have heard the most good things about usertesting.com

And here is some extremely helpful advice on how to effectively use usertesting.com from Chris Neumann (re-printed with his permission from a post to the Lean Startup Circle google group):

“I’ve been a long time Usertesting.com customer, so I thought I’d share
a few  tips from conducting a lot of tests with a bunch of existing
and MVPs with them:

1. The testers tend to want to tell you how awesome your product is,
so I try to hide who it’s for.  I’ll say the URL to test is google.com
and to search for a solution to the problem.  In Ash’s situation, I
might say something like “you want to share pics of your kids with
other parents and family.  Search for a solution for this.  Spend no
more than 5 minutes, and then go to www.cloudfire.com and see if
that’s a good solution.

2. If you get the testers to search Google, then pay attention to what
they type in.  These words can be valuable in SEO, SEM, headlines, etc

3. You can say who can accept the test, so getting someone who
actually wants your solution makes a BIG difference.  There are lots
of stay at home Moms on there, so finding busy parents was pretty
easy.  I’m happy if I get 1 of 5 testers who want whatever I happen to
be testing.

4. The videos can be incredibly persuasive to others inside and
outside the org.  Share them.  If on step 2 of your registration flow
users are flailing, then forward the videos to the UI designer,
engineer, whoever is responsible for that page.  I’ve been shocked at
the speed at which that stuff gets fixed.  I’ve even sent them to
partners.

5. You can also use the videos to end internal debates about feature
requirements.  Recently I was working on a project where a consumer
had to choose from a list of about 20 things.  One person on the team
suggested we add a search box.  I didn’t even respond, but conducted
the user test I was doing anyway that evening, showed him the videos
and said that I didn’t think we needed search.  That was way easier
than getting into an internal debate.  In other words, these videos
can kill HiPPOs.

6.  If you already have a product, I like to say that I’m testing a
competitor, and then have the tester also test my product as a
comparison.  The testers will tend to bash my product since they think
I’m the competitor, so I’ll get some data that I might not have
otherwise gotten.

7. The support person (Aimee) is good, so if you have a question or
problem, send an email to support and they’ll generally get right back
to you.

This one is for Cindy Alvarez: in-person testing gives better data.
However, it takes more time and is harder to coordinate, so I think
that usertesting.com is way better than no testing.”

Other outsourced usability resources I’ve heard good things about:

http://www.easyusability.com/

http://fivesecondtest.com/

http://www.feedbackarmy.com/

And here’s a great post that goes through all these resources, plus many more, and does Pros and Cons:

http://www.usefulusability.com/24-usability-testing-tools/

Guerilla Style Usability Testing

And finally, there is Guerilla or, as Andres calls it, “Commando Style” Usability testing. This lies in-between outsourced and in-house testing. I haven’t yet outsourced usability testing, and have done little in-house testing. But I have ‘gone Commando’ in several coffee houses in Palo Alto and NYC, and think it’s a very powerful – not to mention free and fun – way to perform customer development.

Here is an excellent and entertaining post on how to do $5 Guerilla Usability Testing in bars.

And here is my follow-up post on my own experiences and hacks for doing Commando Style Usability testing for $0!