Discover more from The UXR's Annotations
The Past, Present, & Future of Usability Labs
How remote-first research will shape the usability lab
Is the usability lab dead? What does the future of in-lab UX research hold?
Usability labs have been a cornerstone of UX research for decades, but the prevalence of remote work and unmoderated online methods have left many wondering how labs will fit into research practices moving forward.
This issue will explore four significant changes in the history of usability labs, examine the current state, and consider how the current remote-first paradigm will impact the future.
Welcome to The UXR's Annotations ! Subscribe for monthly content about improving UXR practices.
Past: The history of usability labs
The history of usability labs can be broken into four distinct periods.
First, usability labs emerged between 1947 – 1979. Early pioneers like John E. Karlin of Bell Labs founded user-focused labs as early as 1947. Other well-funded computing companies like NCR, IBM, and XEROX built their first usability labs during this time. Few public documents about these early labs exist, but by 1979 usability labs as we think of them (dedicated spaces with specialized equipment for usability testing) had arrived, in small numbers.
The 1980s marked a proliferation of usability labs. With the mass-market computer revolution and a growing case that ease-of-use sells, many computer and software companies invested in building usability labs. In a 1994 survey, Jakob Nielsen found that most groups established their first usability lab between 1980–1990. During this time, in-lab research was formal, experimental, and focused exclusively on summative methods with sample sizes typically between n= 30–501.
The 1990s brought about a paradigm shift in the primary use case of labs. A series of research studies2 showed that smaller sample size formative usability studies could generate efficient and cost-effective results. UX research practices evolved to include more small-sample-size usability tests and inspection methods (like heuristic evaluation), resulting in a greater diversity of in-lab activities.
Finally, the 2000s & 2010s saw the emergence of remote usability testing as a viable alternative to in-lab tests. In the mid-2000s, a bevy of remote UX research tools entered the market (e.g., RelevantView, Webeffective, Loop11, UserTesting & UserZoom, and UserLytics). These new tools were more efficient than lab testing for recruiting participants and simultaneously collecting several data types; in turn, researchers developed best practices for what projects were well suited for in-lab vs. remote research:
Online usability testing and lab testing go very well together…
In some situations, we first go into the lab to identify the most egregious usability issues, refine the design, and test again.
We usually run an online usability study to validate the user experience of the design…
We leverage the power of the large sample size of the online study to give us a clear picture of what will happen once we launch the product.
–Albert, Tullis, & Tedesco from Beyond the usability lab
Present: In-lab research in a remote-first world
We’re in the midst of a fifth major shift for the usability lab: the remote-first workforce. Global events in 2020 forced most office employees into remote work; in turn, most UX research went remote as well.
The latest State of User Research report showed that 46% of respondents do all their research remotely, and 87% perform mostly remote research. These numbers are down from two years ago (77% fully remote, 96% mostly remote in 2021). An interesting nuance to this trend is that this rate is slower than the return to office. The amount of time people spend in office has been increasing since 2021; 49% of respondents work in-office at least once a week. The forced experiment of exclusively remote research may have influenced lasting changes in our preferences and decision criteria.
The events of 2020 forced a shift to remote work and remote-first UX research methods. Several teams used remote methods for the first time which influenced many to invest in a different tool stack better suited for remote work. In turn, the decision criteria practitioners use to decide whether to conduct a study in the lab or online has shifted towards a remote-first approach.
With these data, some folks ask, “Is the usability lab dead?”
Future: How our labs will change
No, the usability lab is not dead. Turning the SOUR data around, 54% of respondents do at least some research in person.
In-lab testing still has unique advantages. For example, labs are particularly well-suited for testing physical products & hardware. Also, labs allow us to simulate any environment, giving us control over the context of use during our testing sessions. Further, many groups have an existing usability lab and have resumed running studies as their offices reopen. The current remote-first approach will not mark the end of the usability lab, but will change what they look like going forward.
The use of traditional usability labs will persist, but I expect that fewer dedicated lab spaces will be built in the future. Instead, a new paradigm with versatile and adaptable spaces will emerge. As hybrid work shakes out and upper management scrutinizes the use of office space, they will prefer spaces that can serve a variety of purposes over specialized rooms that otherwise sit empty. With that, future usability labs will adopt a more modular design; multifunctional spaces that can be rearranged to cater to a variety of activities for design or UX teams.
Further, I’d expect that the two-room and three-room usability lab setup, where observers and testing sessions are separated by one-way mirrors, will become less common. These setups were designed at a time when we did not have the teleconferencing technology of today. Instead of a dedicated observer room next to the dedicated testing room, we will set up live streaming and other communication channels between the lab and a physical conference room for on-site observers, and a virtual conference room for remote observers.
Finally, I expect that remote research tools will start catering to in-lab & in-person use cases. As people continue to return to the office and start doing more in-person research, the remote research tools that saw significant adoption during the remote shift of 2020 will try to retain as much of the market share as possible. These companies will realize that they can make updates and release new features that will turn their remote research tool into a viable way of capturing and recording data from in-person sessions, similar to older tools like Morae.
The bottom line…
The usability lab has always been a centerpiece of user research practices, and will continue to be a valuable resource to our teams. Usability labs have a rich history, with changes that parallel the evolving practice of UX research.
Remote-first work marks yet another paradigm shift. Usability labs will evolve, not become obsolete. As this evolution plays out I expect we will see fewer traditional, dedicated labs built in favor of more modular spaces that can serve a variety of purposes within a larger UX studio. Further, teleconferencing and remote research technologies will adapt to in-person use cases in ways that will support and conform to hybrid work.
The usability lab isn’t dead, it’s just changing with us, like it always has.
Until next time…
Your reactions and perspectives matter to me, and I'd love to hear from you.
You can hit reply to share any responses, suggestions, or questions.
As always, thanks for reading & being part of this community.
All the best,