Unboxing: guerrilla usability testing

About this post

This post is one of a special series of unboxing posts exploring the different options available for usability testing, covering the relative benefits of each as well as when to choose one approach over the other. From low-cost rapid remote testing through to moderated lab-based testing, there is no reason why some form of iterative improvement through user testing shouldn’t be a key part of every one of your digital project, in order to create best-of-breed user experiences that deliver the highest levels of customer satisfaction and return on investment.

What is guerrilla usability testing?

Guerrilla usability testing is a rapid, low-cost method of quickly capturing user feedback that involves a User Experience (UX) consultant asking questions about specific areas of a site or application. Using portable kit, desktop, mobile and tablet usability testing can be conducted anywhere, from libraries to cafes; enabling insights to be gathered ‘out in the wild’ alongside screen activity, audio recordings, and finger gestures which can then be analysed and summarised.

Given their informal nature, guerrilla usability testing sessions typically cover fewer tasks than some other approaches, and last between 15 and 20 minutes.

How many participants take part in guerrilla testing?

The number of participants can vary from between 6 to 12, depending on where and when those sessions are conducted. Participants are not normally formally recruited in advance based on detailed criteria, but rather approached on the day and asked to take part. The test plan is loosely structured around specific key research objectives or hypotheses.

What deliverables will I get from guerrilla usability testing?

Deliverables are typically qualitative rather than quantitative, so the results are often much more anecdotal than heavily metrics-based. The format of the outputs is very much dependent on the nature of a given project but may include:

• A test plan document outlining the proposed test time frame and key research objectives
• A video combining screen and participant recordings
• A summary report covering key findings and next steps
• A presentation covering key findings and next steps

When should I use guerrilla testing?

Guerrilla testing is perfectly suited to:

• Testing frequently throughout the service lifecycle, due to its low cost method and easy setup
• Identifying (and correcting) generic usability issues early in a project lifecycle; for example, you may have an interactive prototype that you want to validate before development starts or test out an idea for a great new concept
• Conducting iterative design testing which can provide regular ‘validation checkpoints’ in Agile sprints in order to keep testing hypotheses/assumptions and refining the experience
• Getting quick baseline measures of an existing website, app or mobile experience (this may even be a competitor’s), including informal user research and insights about likely user expectations and motivations
• Validating common tasks such as online shopping, completing a form or a simple task, where domain-specific knowledge is not required (as opposed, for example, to navigating a dashboard for a financial services analyst)
• Managing tests when access to representative end-users is limited (in this case, these more specific participants could be saved until later in the process once any general usability issues have been ironed out)
• Testing as part of a broader expert usability review in order to help identify and prioritise issues with an existing solution or service

Example project

After creating a basic HTML prototype of a new checkout process we wanted to test it with members of the public as quickly as possible. We visited a local Starbucks, identified five suitable testers, and asked them to use our prototype to buy a product. It took 15 minutes and their feedback was very insightful. We discovered a number of usability issues and were able to fix them later in the day.

Considerations

As mentioned above, guerrilla testing is great for getting quick insights at a relatively low cost meaning that you can use it multiple times throughout the project lifecycle and making it great for A/B testing. There are trade-offs however which should be considered. For example, sessions are informal so there is no opportunity for observation, they are shorter in length so you tend to discover fewer issues, participants may be distracted/less committed or have a reduced skillset, and of course low battery or no WiFi could impede testing. But if you prepare well in advance most of these issues can be overcome.

At Box UK we have a strong team of UX consultants with hundreds of hours’ testing experience. If you’re interested in finding out more about how we can help you, contact us on +44 (0)20 7439 1900 or email uxteam@boxuk.com.