Reliable Usability Evaluation by Small User Groups

Many of us experience complications and frustration when interacting with products, services, or environments. We often compensate for poor design by exerting extra effort and patience, usually without even realizing it. However, for individuals with reduced physical, cognitive, or sensory functionality, such design flaws can pose significant barriers, making certain tasks extremely difficult or even impossible

.
Users with impairments can provide valuable insights into design challenges that might affect many people. Their experiences can reveal usability issues that others might overlook. This is why our Design for All Test method prioritizes the involvement of users with reduced functionality to assess the usability of products, packaging, services, and environments.
Our method involves collecting and analyzing user feedback to gauge how easily they interact with the design. The average assessment data from test participants helps us evaluate the design’s overall ease of use.
The Swedish Rheumatism Association employs this method to assess the usability of packaging and products. The company is awarded an "Easy to Use" certificate when a product meets the required standards. For more details, please visit our "Easy to Use" page.

Test procedure

Each test participant evaluates every step involved in using the product on a scale from 0 to 5, where:
0 = Impossible
1 = Very Hard
2 = Rather Hard
3 = Neither Hard nor Easy
4 = Rather Easy
5 = Very Easy
Participants are also encouraged to provide detailed comments to explain their ratings in the comment field provided after each question. The collected data is compiled and analyzed to assess usability.

Image description: A six point assessment scale stretching from "Impossible" to "Very Easy".  Each option is presented with a text and an accompanying image of a facial expression.


Additionally, test participants rate their own functional abilities, such as mobility, using a six-point scale:
0 = No Ability
1 = Very Reduced Ability
2 = Mildly Reduced Ability
3 = Neither Reduced nor Enhanced
4 = Good Ability
5 = Excellent Ability
The scale options are tailored to each specific question. In some cases, where the focus is on assessing obstacles such as pain, the scale is:
0 = Total
1 = Very Severe
2 = Rather Bad
3 = Moderate
4 = Mild
5 = No Obstacles
Each option is accompanied by a picture depicting a facial expression to clarify its meaning. This approach helps accurately capture and characterize the test participant's level of ability and the challenges they face.

Reliable results

To ensure the accuracy and consistency of our findings despite working with relatively small test groups, we calibrate the assessments to minimize variation in group composition and results. This calibration process involves comparing each test group's results with those from a reference group of over a hundred individuals who have similar levels of reduced functionality. By aligning the results with this larger reference group, we adjust and standardize the assessments to provide a more accurate and representative evaluation of the product’s usability.

Image description: The image is illustrating the calibration of the results.


In developing this method, we conducted extensive testing with large reference groups to understand how varying levels of reduced functionality affect the assessment of different product use steps. By analyzing this data, we established a quantitative correlation between users' functionality and their evaluations of the product. This correlation is linear, allowing us to apply proportional calibration based on the differences in assessed functionality between the test group and the reference group.
To ensure accurate results, we calibrate the test group's product evaluations using a correction factor tailored to their assessed functionality. The calibration formula is:
Calibrated Assessment = Assessment - (Reference Value * Correction Factor)
This approach ensures that the test group’s evaluations are adjusted proportionately, aligning with the reference group’s standards and providing a more precise measure of usability.

Reference Groups
of approximately 100 Persons /Group

•Eye sight
•Hearing
•Back and neck
•Arms
•Hands
•Legs
•Memory
•Structure
•Communication
•Learning
 

Back to Home Page