How we developed and studied a new questionnaire to make sure it reliably measures usability

Co-authored with Johanna Jagow.

Note: The correct way to write the name of the questionnaire is with small caps, like this: Iɴᴜɪᴛ. However, since this is only possible with special Unicode characters here on Medium, we have opted to set it in all caps instead.

TL;DR: INUIT (short for “Interface Usability Instrument”) is a new questionnaire you can use to assess the usability of your user interface. It has been designed to be more diagnostic than existing usability instruments like, e.g., SUS and for use with machine learning, all the while asking fewer questions than other questionnaires. This article explores how and why INUIT has been developed and why we can be sure that it actually measures usability, and reliably so.

A lot of contemporary usability evaluation relies on easily measurable and readily available metrics like conversion rates, task success rates, and time on task, even though it’s questionable how well these are suited for reliably capturing a concept as complex as usability in its entirety.

The same holds for user experience. When an instrument is used to measure usability, e.g., in controlled user studies or via live intercepts, it’s often the simple single ease question, which is generally not a bad choice, but has its limits.¹

Ultimately, when you intend to precisely and reliably measure the usability of a digital product, there’s no way around a scientifically well-founded instrument or, in everyday terms, a “questionnaire.” The most famous one is probably SUS, the System Usability Scale, but there are also some others around. Two examples are UMUX, the Usability Measure for User Experience, and SUMI, the Software Usability Measurement Inventory.

To join this party, in this article, my co-author Johanna Jagow and I introduce INUIT, the Interface Usability Instrument. INUIT is a new usability questionnaire I developed for my Ph.D. thesis, which investigated automatic usability evaluation based on user interactions. This instrument is based on a review of more than 250 usability guidelines and…