back to the blog

Usability Testing under Pandemic Conditions: First reports from the field

The pandemic had announced itself, sure. Nevertheless, its arrival felt sudden. Fortunately, we were well prepared and can now work (almost) normally. We explain how user testing works well for us under Corona conditions.

18. March 2020
Dr. Lukas Bänninger, Expert Consultant

Expert Consultant

Dr. Christopher H. Müller, Owner, Expert Consultant

Owner, Expert Consultant

A “normal” usability test in our laboratory: A test user travels to our lab at Steinstrasse in Zurich. A receptionist greets her and leads her from the reception area into the lab itself. There, she is introduced and guided through a simulation or tasks with the product we are testing, such as software, websites or devices. Our test manager often sits directly next to her. Our clients observe the test from the adjoining room via a large screen or directly through a mirrored window.

Quite a lot of dangerous contacts! Most test users travel by public transport. They are greeted politely with a handshake. The test manager and test user sit directly next to each other for an hour. All of the test users handle the same product. And in the observation room, our clients sit comfortably, but perhaps a bit too closely together.

The extraordinary situation caused by the COVID-15 pandemic is clearly forcing us all to take rigorous measures. Fortunately, we have been offering our clients the opportunity to monitor tests remotely via streaming for years and had already performed several remote usability tests. This has always been necessary to some degree, since our clients and interview partners sit all over the world. In today's situation, we benefit from this practical remote testing experience. For our clients, this means that we can carry out usability tests with our normal quality and with even more flexibility. And because our test users can participate from home, we maintain the social distance painlessly.

Instant pivot from in-person to remote testing? Our team made it possible
 

The need for remote testing became clear last week when we were in the last steps of preparing a round of 12 test interviews for Swisscom. Shortly before the first round of testing, we decided to carry out the tests remotely as a precautionary measure. We managed to overhaul our plans, preparing for a full remote operation in less than half a day. Of course, Liz and Zuzanna had their hands full during this, our first test under pandemic conditions! The most time-consuming part was contacting the test users who were signed up (some of them set to participate that very day) and communicating to them that they did not have to travel to our lab, but could (and should) carry out the test comfortably at home. 

Despite this tight lead time, our first tests under Corona conditions worked smoothly. Liz and Zuza were able to contact all of the test users in time and test worked almost perfectly from the beginning. Stefanie Luder, our client from Swisscom, agreed: 

"Even under these conditions, the insights gained from the interviews led to a better understanding of our clients and their needs. Since we were able to observe the tests directly as usual, the initial results are already available. Our team can now continue working on our project from the home office with a clear conscience. We are confident that we will be able stick to the ambitious project plan despite Corona". 

Remote tests can deliver equally valuable results


We are convinced that well-prepared remote tests can work as well as "normal" tests in the usability lab. There are certainly limitations and potential disadvantages, like a potential bias in the panel of test users due to equipment limitations at home. But there are clearly also advantages to remote testing, like the lessened demand effects due to the physical absence of the test manager next to user. By the way, we aren’t just saying it: the power of remote testing has also been demonstrated in scientific studies (disclaimer: our own Klaus Heyden was involved in this study during his studies).

Of course, more challenges await us. Today, we are in the midst of preparing a remote focus group. In the coming weeks, we will conduct hardware tests and deep dives at the user's home – all virtually of course. And we will test the usability of even more websites. We are happy to be well prepared for this unusual situation, and that we can keep up our daily testing routine for our clients.
 

Dr. Lukas Bänninger, Expert Consultant

Expert Consultant

Lukas Bänninger has been shaping the Ergonomen since 2009, including a tenure as CEO from February 2019-May 2020. He has PhD in psychology, is a saxophonist and tousle-head. Lukas is an expert in product management, strategy development, UCD and behavioural economics in product development.

Dr. Christopher H. Müller, Owner, Expert Consultant

Owner, Expert Consultant

Dr. Christopher H. Müller, founder and owner of Ergonomen Usability AG, earned his PhD from the Institute for Hygiene and Applied Physiology at ETH Zurich. With over 22 years of experience, he is an expert in usability and user experience. His strong sense of empathy allows him to quickly understand the needs and perspectives of his clients. With creativity and courage, he supports his clients in their digitalization projects and the optimization of products, services, and processes. He takes a practical approach, developing tailored solutions that can be effectively implemented. Dr. Christopher H. Müller is a columnist for Netzwoche. He also serves as a board member for the Zugang für alle Foundation, and is a member of two Swico advisory boards and co-president of the Regional Conference Nördlich Lägern.

Contact