UIAccess logo skip navigation, jump to content| Home| About| Resources| Links| Site Map|

Usability screening techniques: evaluating for a wider range of environments, circumstances and abilities

Presented at UPA 2000, the Usability Professionals' Assocation Annual Conference, by:
Chris Law, Ergonomist, Trace R&D Center, University of Wisconsin-Madison
Kitch Barnicle, Research Associate, Trace R&D Center, University of Wisconsin-Madison
Shawn Lawton Henry, Director of R&D, Optavia Corporation

Abstract

The need for mobile products that offer hands-free, eyes-free or ears-free use is increasing, along with the need for products that are usable by people with disabilities, as mandated by recent legislative measures in the United States. This paper presents a set of simple, cheap screening tests that can be used by usability professionals to gauge the usability of a product from the perspective of end-users working under sensory or physical constraints. These tests can be used as a set to uncover the major usability flaws in a product that will be used by people with functional limitations early in the design process, and prior to testing with people with disabilities. More involved screening tests are briefly described.

Introduction

An increasing number of products are being developed for use in "unconventional," often constrained, environments and situations. Eyes-free operation of navigation and cellular communications systems in vehicles and ears-free communication of paging and text-messaging systems are just two examples of operation designed for a constrained environment. Likewise, physical manipulation constraints occur when operating devices while wearing gloves, such as in winter or in cold storage areas.

Many users (as many as 72% of those over 75, for example) have functional limitations that may affect their ability to use products, depending on the product's design. Products that are more usable in constrained situations would be more usable by people with disabilities, and people experiencing functional limitations, as well as by people who have a temporary disability.

In addition to marketing motivations for designing for limitations, recent regulation and policy trends are increasing the impetus for designing accessible products. For example, in the United States:

While there is an increased emphasis on creating products for people who have functional limitations or are experiencing situational limitations, most usability testing methods in common use do not include such user constraints in test design. This paper presents a set of techniques designed to "screen out" usability problems due to environmental or user constraints.

Simple usability screening techniques

Concepts

Each of these tests is designed to be a means of identifying major usability problems that people who have functional limitations might experience in using the product. The tests are not intended to replicate the real-life experiences that people with disabilities have:  someone who is blind cannot take off a blindfold at the end of the test - the psychosocial factors and past-life experiences involved in living with disability are not accounted for with the screening tests; however, the simple aim of blindfolding a user is to discover whether the product can be used without relying on vision, and hence to "screen" for major usability problems.

The tests are designed to be administered by designers using themselves or their colleagues as users. The tests are intended to be easy and inexpensive to implement, so that they may be incorporated into already busy product-development cycles. The tests are especially useful early on in the pre-prototype or prototype stages of a product's development where gross design errors will be exposed.

Administering these tests, or variants thereof, can be done as a part of design walkthroughs, heuristic reviews, formal and informal usability testing.

Test priorities

In creating a test program, it is important to bear in mind whether all aspects of the product are to be used by people with functional limitations, i.e. is it intended that someone who is blind should be able to set up the device by themselves, or that the device should be usable by someone who is blind once it has been set up for the first time?  A set of priorities has to be established since designing all aspects of a product for all possible users may be impractical, but the operational functions of the product should be usable. In general, an initial emphasis should be placed on ensuring access to priority 1 and 2 items below, followed by the remaining items, and the priority 1 and 2 items are therefore usually the focus of the screening tests.

  1. Purpose / Functions that the product is designed for
  2. Routine tasks  (e.g. inserting media)
  3. Maintenance  (e.g. changing batteries)
  4. Setup and installation
  5. Periodic Maintenance or repair

Impaired visual function

Several types of low vision simulation glasses called "VisualEyes™" are available from the Lighthouse International (New York, NY http://www.lighthouse.org). These simulation glasses cover a range of visual impairments from loss of color vision to tunnel vision. An effective screening tool among these is the "Overall Blur" glasses, which render most text on a product below a certain size unreadable (Figure 1).

A test subject wearing low vision simulation glasses attempts to dial a telephone.   A photo of a telephone handset in an individual's hands simulates low vision. The image is significantly blurred. Text labels are unreadable and buttons are indistinguishable.

Figure 1:  Use of low-vision simulation glasses. All text is rendered unreadable on this product because of the small legends on the buttons, but the overall shape, colors, and button locations are identifiable. The manual is discovered to be unreadable.

Lack of visual function

Blindfolding is a means of testing whether the product can be used without having to rely on any visual capabilities of the user. Other means include turning off the lights if the product does not emit light, or putting the product in a black bag if it small enough.

Impaired or no ability to read text

One means to render text unreadable is to use "greeking", that is, covering up the text (Figure 2), or by substituting the font for an unreadable font (changing all text to a symbol font), which is the easiest way with printed manuals and on-screen displays:

This is Arial font...

Arial font sample that says: Usability screening techniques.

This is the same text in Symbol font...

Symbol font sample that cannot be read.

Greeking the text is very different from testing with blurred vision. Whilst in both cases the text is rendered unreadable, with greeking all aspects of the product are clearly visible to the user, but the legends and printed text are of no use. This identifies quite separate problems to those with blur, and enlarging the text (a solution for blur) is not a solution here.

An image of a stereo and remote control with white tape over all of the buttons and printed labels.

Figure 2:  A portable stereo with all labeling covered

Impaired manipulative capabilities

Wheelchair access is covered in the next section (a more involved test), but there are many types of physical manipulation impairment that can be simulated very simply. Use with one hand or one finger is easy to enforce with users. Some people with physical disabilities will use a mouthstick or head stick, usually with some sort of rubber grip on the end, and a length of dowel with a pencil eraser head can be used for simulation, usually gripped with just the tip of the thumb and forefinger. It is especially important to test printed manuals with these methods, as it is surprising how much printed material is inaccessible to mouthstick users.

Another means of testing is to use thick oven mitts to simulate low manipulation capability, both allowing and disallowing use of the thumb for different levels of severity. Users will find that most product packaging is inaccessible when using the oven mitt simulations.

Impaired hearing functions

Some products will be used in noisy environments (e.g. building sites) or in quiet environments (e.g. libraries) where audible output is prohibited. It is possible to simulate the effects of noise by using white noise to mask out the sounds of the device (having the subject wear headphones and tuning a radio to the noise in between stations). Ear plugs available from a pharmacy can also be used, or a combination of both.

Lack of auditory function

For people who cannot hear at all, or who cannot rely on auditory functions for any reason, the sounds from the device can be muted, the speaker disconnected, or a jack placed in the headphone socket with no headphones attached. It is important to be aware that there are sometimes sounds from devices that cannot be muted, such as motors operating, mechanical clicks etc.

Other screening techniques

There are other screening tests that are a little more involved than the simple test set described above:

Cognitive stress simulations

An individual's memory, logic processing, reaction times and such cannot be reduced by simulation (except perhaps through the use of drugs, which is usually not an option allowed by the conventional ethics that usability professionals are bound by). Products intended to be used in cognitively demanding situations, when users are under stress, or by people with cognitive disabilities should be assessed using conventional usability test procedures within the desired context rather than screening tests (i.e., using people with cognitive disabilities, or tested in the target environment of a product, if that environment is cognitively demanding).

Notes on Safety

As with all usability tests, safety for test participants is a concern, but especially when one is manipulating the physical and sensory capabilities of the user:  The following are some examples of where to take care, but each test site and procedure should be examined closely on a case-by case basis:

Conclusion

These screening tests, while they are by no means new when considered individually, were first considered as a complete "set" for the purpose of a design tool in 1999 [4], and are therefore still under the scrutiny and development that goes with all new design tools and methodologies. Initial experiences have been positive in revealing errors in designs of products that are intended to be usable by people who experience functional limitations. The authors are very interested to hear from others who either adopt this test set, or who have ideas or experiences with other types of tests that can be used by designers.

References


Note

Partially funded by the National Institute on Disability and Rehabilitation Research of the Department of Education under Grants #H133E980008 and #H133E990006. The opinions contained in this publication are those of the grantee and do not necessarily reflect those of the Department of Education.


separator
| Home | About | Resources | Links | Site Map |