How to test the usability of prototypes (and why you should)

It’s no secret that one of the best ways to evaluate how a website’s design aligns with user experience is to create a functional prototype. And these days, tools like invision provide an incredibly easy way to get static designs into a more practical and interactive medium, where users can begin to interact with the project as it comes to life.

However, launching a website that delivers a phenomenal user experience requires a commitment to usability testing from the get-go. Ideas become sketches. Sketches become wireframes. Wireframes become full-blown page UI. At each of these stages, change becomes exponentially more challenging, and as a result, it’s beneficial to start these usability tests as early as possible.

Specifically, early usability testing allows you to:

• Discover foundational user experience issues
• Collect user feedback in the form of a narrative or “user story” that can help evolve the prototype through the subsequent stages of design
• Break UI iterations into smaller pieces as a direct result of user feedback

The solution: build and test prototypes early and often

Usability testing at the prototyping stage presents many opportunities to check design, functionality, and user experience well before the project makes it to the development stage. What this means is that you can use your prototypes to reduce requirement uncertainty and receive design feedback from clients early on.

Prototypes should be painfully simple: the purpose is to test a concept. Developing several low-fidelity (or even high-fidelity) prototypes will enable your team to test theories and ideas regarding the layout and structure of the website.

Let’s get started.

How to test the usability of prototypes

How often do you test usability in the early stages of design? If you’re like most designers … not very often.

Usability is something that’s refined throughout the design process, and there are definite benefits to taking an iterative approach to your usability testing. The more you focus on gathering usability data and user feedback at the prototyping stage, the easier it will be to move towards an end product that’s both easy to use and designed to spec.

After a few iterations of usability testing your prototype, you’ll have a tested and validated prototype to move on to more high fidelity design tasks.

Step 1: Test before you create the prototype

If you have the resources and a sample target audience to take the tests, you should try to run some usability tests before building a prototype. Usually, these pre-prototype tests are conceptual and designed with the purpose of gathering data to design and structure your prototype’s information architecture in the most effective way possible.

At this stage, you should have a clear idea of who your target users are and how to segment them based on user behavior, skill set, level of experience, and/or whether or not they have prior knowledge about the industry. It’s a good idea to have a few different segments to get the most value out of your usability tests.

Setting up a low-fidelity, paper-based prototype is fairly simple, and you can run various types of tests using this technique.

For instance, use card sorting to find out how users prefer your website’s information architecture or navigation elements to be structured. Another paper prototype test involves creating a paper mockup and asking users to select the menu item they’d click on first to complete a task. And if you’d like to take things a bit further, you could set up an entire tree test that monitors whether users were able to figure out the correct route to complete the assigned task (e.g., your main success scenario).

If you’re wondering how many users you need, you should recruit at least 15 users per user group for testing the usability of low-fidelity prototypes, according to a study conducted by the Nielsen Norman Group.

Since your goal is to test the usability of what will eventually be a fully-functional website, it’s essential that you tell your test-takers to imagine the different digital interactions they would experience if it were the final product. For example, if your mockups are not yet interactive, be sure to set the stage with a clear narrative and instruct your subjects how to navigate the test effectively.

The key benefit of running these pre-prototypes tests is that you’ll have a solid, conceptual foundation for building a functional (and usable) prototype.

Step 2: Find the right people for the tests

Once you’ve designed and built a functional prototype (either using the data you gathered from the pre-prototype testing stage or coming up with one based on collaborative team efforts), it’s time to recruit users to test out the usability of the prototype.

There are many ways you can recruit users to test out the usability of your prototypes:

• Hallway Testing: This is where you recruit users who are likely unfamiliar with the project you’re working on to discover the more obvious problems with your prototype.
• Existing Users: If this isn’t the client’s first project, then there’s a chance they already have a sizeable user base that might already be familiar with the industry.
• Website Testing Service: For more controlled tests, you could use a website testing service to recruit users based on demographics, experience and industry knowledge.

So, how many people should you recruit for a usability test?

According to the same usability study mentioned above, “Testing with five people lets you find almost as many usability problems as you’d find using many more test participants.” That goes for qualitative tests (with open tasks — we’ll get to this in a minute) aimed at collecting insights. However, if you’re thinking of collecting statistics through quantitative tests (with closed tasks), then the study recommends going with at least 20 users.

Step 3: Plan and create tasks

To conduct usability tests that give you meaningful information about how users interact with the prototype and what problems they face along the way, you’ll need to write tasks based on the usability factors you wish to examine.

Here are some of the different types of usability tests you can write:

Direct Task vs. Scenario Task

A direct task gives clear instruction for what the user needs to do whereas a scenario task lays out a specific use case scenario without providing any clear direction on how to complete it. For instance, if you were working on a flight booking website:

Direct Task: Book a round-trip from New York to Miami for two adults and one child.

Scenario Task: You’re planning a vacation to Miami for your family, and you need to book the flight.

Closed Task vs. Open Task

A closed task is written to test out a specific functionality and has one clear success criteria whereas an open task is designed to see how users interact with your prototype. Closed tasks are quantitative whereas open tasks are qualitative. In the case of a closed task, if the user can complete the task and meet the success criteria, then you know that the prototype works.

If they aren’t, you’ll have information about where they got stuck. And in the case of open tasks, you’ll be able to monitor how a user interacts with your prototype, which is especially useful for tasks you can complete in multiple ways.

For instance, if you were working on a page builder web application:

Closed Task: Create a blue button using the page builder and set its dimensions to 100px by 50px.

Open Task: Recreate the given landing page using the page builder.

Best practices for testing the usability of prototypes

There are some common mistakes designers and usability testers make when testing the usability of prototypes. Prototypes — whether they’re low-fidelity, high-fidelity, or somewhere in between — are still prototypes, which by definition have limitations.

Here are a few best practices to get the most out of testing your prototypes:

#1: Conduct moderated tests

Since prototypes aren’t complete products (in that they either lack functional width or functional depth), experts generally recommend that someone moderates the usability tests. What this means is that a human is present while other humans are taking the test to answer any questions they may have.

Quick Tip: Take note of the questions that come up and add it to a repository of questions that you collectively refer to as a team. Like most exercises in life, you get better with experience. Before you begin a new test, refer back to past questions and points of confusion. Be sure to build your new test to accommodate.

Conducting moderated tests gives you another opportunity to take notes about the questions users have regarding the prototype. Is the microcopy unclear? Can a two-step process be condensed to a single step? Should a single step be broken into multiple steps?

#2: Don’t alter the tasks

When users are unable to complete given tasks or experience difficulty in completing them, usability testers are often inclined to make modifications to the tasks to simplify them. This defeats the purpose of conducting a usability test in the first place.

Instead of modifying your test, prepare a list of questions to help you understand why it was difficult for the user to complete the task. For example, if the user ends up at a page that hasn’t yet been developed (or wasn’t meant to be a part of the test), you could ask them why they thought they should take that path or what they were hoping to find there.

This gives you an opportunity to gather valuable feedback directly from the user instead of indulging in unfounded speculation.

#3: Use real content

Prototypes are built in the early stages of the project lifecycle — usually before the copy is finalized. A common mistake with designing prototypes for usability testing is to use generic text and silly names for placeholders. The problem with this is that Lord Skunky, Master of Odors is pretty distracting and can even bias your test results. The same goes for using generic, lorem ipsum text — it doesn’t communicate any context. You should always design with real content wherever possible.

A prototype is also an excellent place to test your messaging and tone of voice. More often than not, your content is meant to move your users in specific directions. Take the opportunity to monitor the reaction of your users as they read out loud. Did your humor land or distract? Is your vernacular commiserate with your audience or are you speaking with the “curse of knowledge”?

It’s a good idea to use data the end user will likely be entering. So, instead of filling email fields with a random string of letters, write out a sample email address (johndoe@email.com). This will give you clean test data and a better understanding of the end user’s experience with your design.

Conclusion

Testing the usability of prototypes gives you valuable insight into how you can refine your project’s design early on.

The key to usability testing prototypes is to make it an iterative process. Create a prototype, test it, collect feedback, update your prototype based on the input, and run it through another round of usability testing. At each iteration, your website will become easier to use, and you’ll effectively be minimizing the likelihood of having to go back and make significant changes post-development.