Steve Krug on DIY Usability Testing

13 October 2011

Last week I attended the Etre Get Together with Anders Ramsay (on Agile Design) Louis Rosenfeld (on Adaptable Information Architecture) and Steve Krug (on DIY Usability Testing). This was a really useful three-day workshop, jam-packed with practical advice.

Here then are a few words of wisdom from Steve Krug. If you’ve read his book, Rocket Surgery Made Easy, these ideas will be familiar to you.

If you want to do more usability testing, but find it difficult to fit it into tight design and development schedules, then DIY usability makes a lot of sense.

The message is simple: test what you have, as soon as you can, as often as you can.

The question isn’t – is it ready to test? It’s – are you working on it? If the answer is yes, don’t wait until you have a fully functioning prototype, or a set of users that match your persona’s, because, guess what? This time won’t come! According to Steve, his only criteria for selecting test participants is that a) they can use a web browser and b) speak English. Easy. Keep the barriers to entry low, and don’t be fussy about your what you’re testing. If what you have is very low fidelity the tests will be quick, but you can talk through it.

With this in mind, here’s a quick list of do’s and don’ts for doing your own discount usability testing.


  • Set aside a Morning a Month for testing (aka the agile approach of Five Users Every Friday)
  • Test with three users (it’s not a scientific test)
  • Make sure the team watches in a separate viewing room (this is where you get the most value)
  • Provide top-notch snacks for the guys in the viewing room (incentives!)
  • Prepare your test scenarios and tasks and do a dry run test
  • Read from a script when you start the test, and stick to this script for each test (available at )
  • Begin by asking the participant a few warm up questions to get them chatting
  • Ask the participant to think aloud, and keep them on track
  • Remind them not to use search (unless you’re testing search)
  • Start with a homepage tour (what can you do here, who is the site for)
  • Probe if you need to, and stick to minimal responses (think the therapists couch!)
  • Change the script on the fly if you need to
  • Note the top three usability problems that you observe, for each test
  • Ask the team in the viewing room also to record the top three usability problems they observe
  • Do a lunchtime de-briefing session and ask the observers to join if they can
  • As a team prioritise the top three problems you witnessed
  • Write a bullet-point email highlighting the issues (keep the documentation light)
  • When fixing usability problems your motto should be “tweak, don’t re-design”
  • Work your way through the top three usability problems, tackling the most serious first
  • Forget any of the other problems observed, if serious these will re-surface at the next round of testing anyway


  • Use entry or exit questions
  • Ask leading questions
  • Engage in conversation about features
  • In the politest possible way, don’t pay attention to opinions (we want to see what they do, not hear what they say)
  • Write a great big “honkin” report (keep it lean, light and fast)

What to test

You will have a gut feeling about what you should test – basically anything that is keeping you awake at night, things you suspect people may have trouble with, or that aren’t as clear as they could be. Make sure you test the business critical features, and any tasks that are crucial for the end-user.

What about the software?

For DIY testing the following is recommended:

For recording

Camtasia Studio (approx £230 for PC/£76 for Mac)
USB Microphone (e.g. Logitech, approx £20)

For viewing

Decent speakers (e.g. Logitech)
Screen share & audio – e.g. GoTo meeting

For mobile

You can use a VNC client for screen capture, plus webcam on arm to capture finger movements, otherwise the protocol is very similar.


The big benefit comes from doing lightweight testing, iteratively, whilst developing.

By making testing a minimum ceremony, regular thing, it will help bake it into your working practices and give you fewer reasons to put it off. If you work in an agency setting, do hallway testing instead of recruiting participants, if that’s easier for you. The rule in usability testing is always: some testing is better than none.

As Steve Krug say’s, it’s not Rocket Surgery. For more of the finer details see his excellent book – Rocket Surgery Made Easy




Subscribe to Email Updates


We are a digital transformation consultancy. We help our clients succeed.

View Services