Usability testing is often a misunderstood part of the web design process. While most designers, developers and business owners these days have the idea of what usability testing is, a good number don’t see what it can do for them, what value it can add to your site by testing, and how to even get started with testing in the first place.

(Image source: Geert Allegaert)

Obviously doing some usability testing can show up potential issues in your site, help you get feedback on what is or isn’t working, and have a much broader understanding of what users are doing and how they interact with your site.

Furthermore, having a usable site means users will enjoy their interaction that much more, are much more likely to return to your site, and much more likely to recommend it to others. Conversions will increase whether that be sales, subscribers or any other goals you are aiming for. In short, it’s good business for you regardless of what goals you are looking to achieve – unless of course you don’t want to succeed, in which case you may as well stop reading now.

Supposing you are still reading, you should see that usability testing can add value for you; the key then, alongside knowing which testing method you’d like to go with, is knowing what type of method would work best for you and what kind of questions to ask to get useful, actionable answers. Read on to find out:

Types of Usability Tasks/Questions

In a remote usability test (or with any type of usability testing), there are a range of different question types you can ask. This is a selection of three of the more common types so you can get an idea of how to do your own testing in the future. You can combine different types of tasks/questions into one overall test.

Single test

These are great for testing different elements of an interface, navigation structure, layout, for finding products or services, and a range of other similar tests. A single test just means you are testing one image at a time, compared either an A/B or a preference test.

A/B test

An A/B test is where you are shown either image A or image B (NOT both – the image below is the result of the test) so you can see how people might interact with changes in your interface, different wording, different navigation options, different coloured buttons – you get the idea. It’s great for comparisons of any changes, redesigns or any little tweaks you are thinking of making.

Preference test

Preference tests are when two images side by side (as per the image below), and you can decide which one you prefer. This can be really useful for testing how people are going to feel about new features you are thinking of adding or changes you are thinking of making.

The image above is comparing UI elements to see which way people better understand the design. Imagine if Twitter had done a test on the quick bar – it surely wouldn’t have launched if they had. Think about what this could do for your site and community – it should be pretty worthwhile.

Good Usability Testing Questions

What part of the interface stands out the most?

This question is useful for a number of reasons; are you trying to sell a product on your site? You probably want the call to action button, text or product to stand out – if everyone is clicking your logo or other information, you may want to consider making some changes. If people are taking a long time to decide which element stands out (shown by average click time), you may want to make some changes. If you are producing content, you’d probably want your RSS feed to stand out. If people can’t find it, again, you’d want to try to make it more visible. Seems simple, but until you test it you don’t know.

How would you fulfill such and such a task ?

People use sites, browsers, computers and technology in a whole range of different ways. The way you perform a task may be different from the way I do, which could be different again from a third person. Watching how people interact with your sites can lead to stunning revelations, and can have a huge impact on your bottom line as the tale of the $300 million dollar button shows.

For example, testing on our own site showed a number of people went to the pricing page, and then looked to sign up. By adding a link back to sign up from the pricing page, our sign ups went up appreciably. Every way in which your optimise your site can make a difference to your users, and in turn to you and your site. It’s all about maximizing your potential.

These sorts of questions can also help you learn about and understand conventions that people are used to following – examples are clicking your logo to return to your home page, the meanings of different colours, the locations of search boxes and so on. You can even lay out a grid and just ask people to click to try to form ideas on a blank slate about what is or isn’t going to work. Maybe people who visit knitting websites like search boxes on the bottom left hand side? Who knows until you test.

What service do we provide?

I recently did a quick test asking people after a quick look at a screenshot of our homepage, did they know what service we provided. Based on the feedback we received we made a few small tweaks to the content of the home page and developed a much clearer message – again leading to more sign ups.

People need to know who you are and what you do to be interested, and short attention spans being what they are, people aren’t generally going to sift through loads of information when thinking of choosing your service. Is the message on your site clear? Are there any changes you should make? Why not do a quick test and find out.

Which design do you prefer?

AKA preference testing. When you have one or more alternatives and you want to compare, you can always see how your views compare with the wisdom of the crowds. Asking these kinds of questions also enables you to elicit feedback on why people like certain designs, certain features, certain layouts and so on. You can pick the best elements of a number of designs and mix and mash until you find the perfect combination.

Some Useful Examples

It’s easy enough to give some example questions, but it’s much easier to see their use when you can see them in action. Below we’ve got a selection of different tests with different kinds of results, and the useful results that can capture:

Test: Gmail vs. Hotmail vs. Yahoo mail

This project features a combination of testing types (more apparent in the results) testing a number of different things; the grids are really useful for studying people’s preconceptions and learned behaviours. The written questions are a quick visual way to gather feedback; for example the obvious preference for threaded email and text ads, and the rest of the questions are really useful for understanding people’s interactions with the site. Success rates (i.e. clicking in the right place) and average click times can really show up which interfaces work the best. All really useful information.

Gmail vs Hotmail vs Yahoo mail: Test | Results

Preference Test: Colours and Psychology

This test uses preference testing to understand people’s learned behaviours and interpretations of colour. There were many insightful comments on this test regarding the cultural implications of colour, as well as accessibility issues for colour blind users. Are these things you need to be considering for your site or users? These kinds of tests will tell you.

Colours and Psychology: Test | Results

A/B Test: Bing vs. Google

This A/B test is really useful for showing up some of the differences between two different designs or layouts, even for two completely different designs (hey, don’t you want to know how you stack up against your competition?).

This test also showed up quite well the most important parts of the interface for each site – search bars being a prominent feature, but the Bing artwork obviously draws a lot of attention, as does the Google logo – useful information if you’ve got elements you need to stand out.

What you can see from all of these tests is the huge array of information you can gather – information that can have a big impact on the usability and user experience of your site. Even shaving a few seconds of a simple task can make a big difference to your users. Even making your call to action button a little more obvious, or learning to follow conventions can make a big difference to your bottom line.

Bing vs Google: Test | Results

So When to test? Now.

Steve Krug recommends testing once a month. This helps you gather fresh information, and to keep pace with the evolving web. Even if you re-run the same test you may well end up with different results and discover small changes that you need to make. Supposing each change only makes a small difference to your website, they will start to add up; 1% for a tweak here, 5% for a change there – think if after a year, just through small usability changes your revenue increased 30% to 40%, that would be well and truly worthwhile.

The point is that it’s never a bad time to start testing, and even making a few small changes can make a big difference. Even if your testing shows your site is working perfectly, that’s great, valuable information to know, and something your core stakeholders will be very happy to see.

Why not get started today and help make the web a more usable place, one website at a time? As I’ve quoted time and time again, ‘build it and they will come; build it well and they will come back’. With an ever-increasing focus on usability, the web will become a more enjoyable place for all, and I think that’s a very grand goal to aim at.

Further resources

• Usability testing demystified (Alistapart)
• Usability testing toolkit (Noupe)
• 8 guidelines for usability testing (Webcredible)
• A simple usability/UX checklist (intuitionhq)
• Getting started with usability testing in 7 simple steps (intuitionhq)

Do you have any usability testing experience you’d like to share? What kind of questions do you think are important to ask? Or do you have questions on how to get started testing your own sites? Be sure to let us know in the comments.