Cross-device testing: why and how you should do it

What is cross-device testing?

Cross-device testing is a software testing technique that checks solutions in multiple formats and on various devices, to provide confidence in their quality and accessibility no matter how a user chooses to interact.

Talk to one of our digital experts

Tom Houdmont

Head of Business Solutions

Do you have an idea or a project you need support with?

Tom leads Box UK’s Business Solutions team and has over 15 years experience in the web industry.  Tom is passionate about creating impactful solutions that solve real problems and deliver the outcomes our clients need.

Book a meeting with Tom

Or call us on 020 8098 2093

Why do I need cross-device testing?

There is a huge amount of competition in the browser market (and has been since the days of cover disks with either IE, Netscape, or AOL on them) and therefore each browser tries to differentiate themselves. Due to this, extra features are added or refined on top of the HTML standard meaning that each browser potentially deals with the same HTML in a different way.

Each of the browsers uses a rendering engine (sometimes called a layout engine) to tell the browsers how to handle the HTML. The difference in the browsers comes from how this engine handles what the website gives it.

The most popular browsers and the engines they use

• Chrome – Blink
• Internet Explorer – Trident
• Firefox – Gecko
• Safari – Webkit

(For more information on web browser engines, see this Wikipedia page.)

Because the browsers use these different engines to render a web page, occasionally the end result can appear differently depending on which browser you are using. This is especially common with more complex websites that use non-standard HTML tags, or complex CSS.

How do I choose what devices to test on?

As there is an overwhelming combination of screen sizes, devices, Operating Systems (OS) and browsers to choose from, and usually limited time and resource to test, it is important to start by establishing the intended range of configurations that need to be supported.

We can’t test all configurations, so we need to make sure we test as many useful variations as possible to cover the largest pool of users that time and budget permits. This means restricting testing to the most appropriate specifications for each individual project.

Sometimes this comes in the form of a single browser, being used on one OS with a standard screen size. These are straightforward, and can be tested according to the exact specifications to ensure what we test is exactly what the customer will see. This could be the case if we’re developing a mobile app where the user will have a specific device on which the app will be installed; for example, when workers outside of the office all use the same tablet to record information.

Products such as a website that will be viewed by members of the public, however, can and will be viewed in any number of ways, on any number of devices. For this reason it is often beneficial to go for a more comprehensive set of tests. To make sure the testing covers the highest number of potential users, often the most common browsers, operating systems, and screen sizes will be used; for example Windows or Mac as the most common operating systems, and Chrome, Firefox and Internet Explorer (for Windows) or Safari (for Mac) as the most popular browsers. As screens can come in almost any size, often a maximum and minimum are chosen.

There are many tools that can be used to gauge which configurations are most popular. For instance http://gs.statcounter.com/ gives up-to-date statistics of market share for OS, browser usage and even what screen size is being used to view pages. Additionally, if the client can provide their own usage analytics, the specifications can be narrowed down even further.

How do I conduct effective cross-device testing?

For the reasons above, when developing or testing a website it is important to view it in as many of the different browsers that the end user may use as possible. This may be just the two main browsers on a Windows desktop, or a variety of platforms such as an Apple Macbook, an Android phone, or Firefox on a Windows tablet.

Ideally, it is best to view these on machines with the same configuration as the end user. However in practice it’s not always possible, or cost-effective, to have every variation available to test on. When this happens, there are a number of options as to how to test. Some of the methods available are:

Emulation

Software is used to make an approximation of the target device or machine to show you how something ‘should’ look. This is often used for testing on mobile devices, as it means the developer can see their changes as they make them. (As it isn’t possible to develop directly on a device, and it can take additional time if each tweak requires the software to be bundled and downloaded to the device each time.)

Virtual machines

Tools like VirtualBox or BrowserStack will give the tester access to an actual copy of the environment. This can be an operating system being run on a server or locally, but using the actual software to show the results.

The actual hardware

This may not always be the best way to develop software, but it is the best way to test it. There is no better substitute to test how something will look on a device than actually seeing it on the device.

Each of these methods has its own advantages and disadvantages, such as cost (for the actual hardware), limited resources (for virtual machines, every copy of an operating system will take up space and resource that could be used for other things), and inaccuracies and best guesses (emulation might not reflect the real-life constraints of the hardware; for example, a desktop emulating Android will have more resources, no issues with signal or battery, and approximate GPS signals).

When testing between devices, browsers and operating systems, it is crucial that the tester can switch between iterations, whether it’s between browser windows on one machine, or between multiple machines. Tools such as Ghostlab are very useful here, as they allow you to make actions on one iteration and have them repeated on many. For example, from just one Chrome browser the tester can control the same website in Firefox on the same PC, Safari on a Mac, Chrome on an Android phone, Safari on an iPhone, IE on a Windows phone, or any other devices or windows they have in front of them. This means they can go through a site once and view it on multiple devices, in various configurations, all at the same time. With minimal configuration, this can save hours of unnecessary cross-browser testing.

Conclusion

To accurately develop and test software that needs to work in multiple places, a number of different approaches and tools are required, and hopefully this post has introduced you to a few that may prove useful. However, to test efficiently it’s important that you plan carefully at the start of the project, to decide which tools are best for the job and how they will be used.