Reliable Usability Evaluation by Small User Groups

Anyone can experience complications and irritation when interacting with products, services, or environments.
We compensate for bad design solutions every day by putting in extra effort and patience, most times without noticing it. However, people with reduced physical, cognitive, or sensorial functionality might be unable to do that. For them, certain steps of use may be really difficult or even impossible to perform. Therefore, these groups of users can be great trackers and identify designs that many people will struggle to use. 


The Design for All Test method revolves around the involvement of users with reduced functionality for measuring the usability of products, packaging, services, or environments. The average of the test participants' collected assessment data serves to evaluate how easy the design is to use.

The Swedish Rheumatism Association uses this method for evaluating packaging and products' ease of use. If approved, the company gets an "Easy to Use" certificate for the product. (More information is under the "Easy to Use" page).

Test procedure

Each test participant individually assesses every step  involved in using the product on a scale from 0 = impossible, 1 = very hard, 2 = rather hard, 3 = neither nor, 4 = rather easy, and 5 = very easy. They are also encouraged to motivate their assessment in the comment field that follows each question. The results are collected and compiled for analysis.
 

Image description: A six point assessment scale stretching from "Impossible" to "Very Easy".  Each option is presented with a text and an accompanying image of a facial expression.


The test participants also assess their functionalities, such as mobility, on a six-point scale where 0 = no ability, 1 = very reduced ability, 2 = mildly reduced ability, 3 = neither nor, 4 = good ability, 5 = excellent ability. The options connected with the numbers are adjusted to fit each question. In some cases, the scale assesses obstacles like pain. Then the scale is 0 = total, 1 = very severe, 2 = rather bad, 3 = moderate, 4 = mild, 5 = no obstacles.  All options have a picture with a facial expression that clarifies its meaning. The answers characterize the test participant’s degree of ability.
 

Reliable results

Since we work with relatively small test groups, we reduce the variation between the groups' composition and results by calibrating the assessments. This is done by comparing each test group's reduced functionality with and a reference group of over a hundred people with similar reduced functionality and then adjusting the results accordingly.

Image description: The image is illustrating the calibration of the results.


When creating this method, we conducted extensive testing with large reference groups to describe how different levels of reduced functionality impact the assessment of the various steps of use. By analyzing the gathered data, we established a quantitative correlation between functionality and product assessment. This correlation is linear, which means that the calibration is proportional to the difference in assessed functionality between the test group and the reference group.

The test group’s product evaluation is calibrated proportionately to the correction factor calculated for that group's assessed functionality. The calibration is calculated by:

Calibrated assessment = Assessment – Reference value * Correction Factor
 

Existing reference Groups
of approximately 100 Persons /Group

SENSORIAL
•Eye sight
•Hearing

PHYSICAL
•Back and neck
•Arms
•Hands
•Legs

COGNITIVE
•Memory
•Structure
•Communication
•Learning

Reference Groups
of approximately 100 Persons /Group

•Eye sight
•Hearing
•Back and neck
•Arms
•Hands
•Legs
•Memory
•Structure
•Communication
•Learning
 

Back to Home Page