Reliable Usability Evaluation by Small User Groups

Anyone can experience it difficulties and annoyance when interacting with products, services, or environments.
Every day we compensate for design solutions that are hard to use by putting extra effort and patience, most times without even noticing it. However, people with reduced physical, cognitive, or sensorial functionality might not be able to do so. For them, certain steps of use might be really difficult or even impossible to perform. Accordingly, are these groups of users great trackers to identify design solutions that many people will struggle to use. 

The Design for All Test method is based on the involvement of users with reduced functionality when measuring the usability levels of products, packaging, services, or environments. The average of the test participants' collected evaluation data serves as the assessment of how easy the design is to use.

The Swedish Rheumatism Association uses this method for evaluating packagings' and products'  ease of use. If approved, the company can label the item as easy-to-use and or easy-to-open. (See SRA Approval.)

Test procedure

Each test participant individually assesses every step that is involved in the use of the product on a scale from 0 = impossible, 1 = very hard, 2 = rather hard, 3 = neither nor, 4 = rather easy and 5 = very easy. They are also encouraged to motivate their assessment in the comment field that follows each question. The results are collected and compiled for analysis.

Image description: A six point assessment scale stretching from "Impossible" to "Very Easy".  Each option is presented with a text and an accompanying image of a facial expression.

The test participants also assess their functionalities, such as mobility, on a six-point scale where 0 = no ability, 1 = very reduced ability, 2 = mildly reduced ability, 3 = neither nor, 4 = good ability, 5 = excellent ability. The options connected with the numbers are adjusted to fit with each question. In some cases, the scale assesses obstacles like pain. Then the scale is 0 = total, 1 = very severe, 2 = rather bad, 3 = moderate, 4 = mild, 5 = no obstacles.  All options have a picture with a facial expression that clarifies its meaning. The answers characterise the test participant’s degree of ability.

Relyable results

Since we use relatively small test groups, we reduce the effects that come with the group’s different compositions by calibrating the usability assessments. This is done by comparing the test group's reduced functionality with and a reference group of over a hundred people with similar reduced functionality and then adjusting the results accordingly.

Image description: A description of the correlation between the test group and the reference group. The text "Test group" is followed by a red circle with the number 10 inside. Next to it is the text "Reference group" which is followed by a red circle with the number 100 inside. Underneath, a red two-sided arrow with the text "Comparison"  is placed between two identical texts  reading "Subjective judgement of own functions and abilities". Underneath that is another red arrow with the text "Calibration" , pointing towards the text "Judgement of ease of use of products".

When creating this method, extensive testing was conducted with reference groups to describe how the level of reduced functionality impacts the assessment of the various steps of use. By analysing the gathered data, a quantitative correlation between functionality and product assessment was established. This correlation is linear which means that the calibration is proportional to the difference of assessment of functionality between the test group and the reference group.

The test group’s assessment is calibrated proportionately to the correction factor calculated for the function of each test group. The calibration is calculated by:
Calibrated assessment = Assessment – Reference value * Correction Factor

Referense Groups
of ca 100 Persons /Group

•Eye sight
•Back and neck

Back to Home Page