Reliable Usability Evaluation by Small User Groups

Each test person judge individually all steps involved in the use of the "product" on a scale from 0 = Impossible, 1 = Very hard, 2 = Rather hard, 3 = Neither nor, 4 = Rather easy and 5 = Very easy. They are also encouraged to motivate their judgement in the comment field that follows each question. The results are collected and compiled for analysis.

The picture is illustrating the scale that is used to judge the usability


The test persons themselves also assess their functions, such as mobility, on a six-point scale where 0 = no ability, 1 = very reduced ability, 2 = rather reduced ability, 3 = neither nor, 4 = rather good ability, 5 = very good ability. In some cases, the scale is used to assess problems e.g. pain. Then the scale is 0 = total, 1 = very severe, 2 = rather bothering, 3 = moderate, 4 = light, 5 = no problems. The assessment scale 0-5 is used in all cases. Each scale point is linked to a picture to clarify its meaning. The answers connected with the numbers are adjusted to match each question.
The answers are used to characterize the test person’s degree of ability.

To reduce the effects of the test group’s composition, a calibration of the judgements of the steps using the products is carried out. This is done on the basis of the test group's reduced function and a reference group of over a hundred people with similar reduced function.
At the creation of this method, extensive testing was conducted with reference groups to describe how the level of reduced function impacts the assessment of the various using steps. By analysing the gathered data, a quantitative correlation between function and product judgement has been established. This correlation is linear which means that the calibration is proportional to the difference of assessment of function between the test group and the reference group.

The test group’s judgement is calibrated proportionately to the correction factor calculated for the function of each test group. The calibration is calculated by:
Calibrated judgement = Judgement – Reference value * Correction Factor

Referense Groups
of ca 100 Persons /Group

•Eye sight
•Hearing
•Back and neck
•Arms
•Hands
•Legs
•Memory
•Structure
•Communication
•Learning
 

Back to Home Page