by Dana Chisnell, Amy Lee, and Janice (Ginny) Redish
Between June and October 2003, AARP conducted exploratory usability testing on its own web site, AARP.org. We conducted three rounds of usability studies: in San Francisco, Tampa, and Baltimore.
These studies gave us many insights into design, interaction, navigation, language, content, and information architecture, as well as to how to recruit and work with older adults in usability studies. These insights should be useful to all web site designers.
This article briefly describes the methodology for these studies. See the related articles for more about our findings.
Goals of the usability studies
We had two main goals:
- gather data about how well the site works for its target audience, so that AARP can improve the site
- understand what the experience of using web sites is for people over 50, so that AARP can begin to build heuristics and guidelines for web site designers who have this demographic in their audiences
Because AARP would like to be the repository for information about Americans over 50 years of age, and because using technology is such an important part of the daily lives of so many people today, it made sense to try to understand what the experience of using web sites is like for people in that age group.
Our research questions
We started with a few big questions:
- Are older web users different from younger web users?
- If so, how? Why?
- How do the differences influence web design?
We also had many detailed questions about how older users interact with specific parts of AARP.org.
Who participated
We planned for 12 participants in each location (for a total of 36 participants) and ended up with 34.
As is typical for usability studies, we used a convenience sample- that is, people who are interested in taking part in studies like this contact a firm that recruits people for studies. The firm then selects people from their database who meet the characteristics that we specify. We asked for a mix of ages (50s to 70s), employment (employed/retired), web experience (more than a year), and AARP membership (members/non-members). The following tables tell you more about our participants.
| Age |
Number of participants |
|---|---|
| 50s | 15 |
| 60s | 9 |
| 70s | 10 |
| Employed or retired |
Number of participants |
|---|---|
| employed at least part time | 17 |
| retired | 17 |
|
Participants by age group |
Average years on web |
|---|---|
| 50s | 6.9 |
| 60s | 6.0 |
| 70s | 4.1 |
We required at least a year of web experience to participate. We did this for two reasons:
- We didn't want to have to teach people how to use the web while we were trying to get usability data.
- We assumed that "typical" users would have enough experience using the web that they would want to do task-oriented activities rather than simply surfing to find out what exists on the web.
Their relationship to AARP
Participants did not know ahead of time that the research was being done by or for AARP. Most participants had not been to AARP.org before their sessions. We did not specify that participants had to be AARP members.
|
AARP members or not |
Number of participants |
|---|---|
| AARP members | 22 |
| not AARP members | 12 |
What we did
The usability sessions were designed to gather data and insights about how easily and successfully participants used features of the AARP web site (www.aarp.org), AARP's web site for its magazine (www.aarpmagazine.org), and parts of another site called iVillage.com. Observing how people who are age 50 to 70+ used these sites helped us understand the types of experiences older people have when using the web in general.
The method
The sessions were conducted one-on-one (a moderator and a participant in the room together), using a "think aloud" protocol. Over the three studies, our methodology evolved. We started out with a set of specific tasks that we asked participants to do in a particular order. We learned that some of our tasks were not realistic, and in the second and third studies we included an exercise to help participants generate their own tasks for part of the session. We filled the remainder of the session with tasks we created based on what we had heard our participants say they were interested in doing. We also used "flexible scripting" to allow participants to do tasks in an order that made sense to them and to eliminate some tasks (such as using message boards) that participants said they would not ordinarily do.
The tasks
The tasks participants generated usually had to do with looking up health or medical topics, getting information about or prices for travel or vacations, answering questions about financial planning, or researching work-related topics.
Some of the tasks we created included:
- Update membership information
- Find travel information and discounts
- Register for the annual AARP member event
- Compare AARP's message boards with iVillage's message boards
- Send an email to a representative or senator
- Make a donation to the AARP Foundation
- Find volunteer opportunities in the area
- Look up information about health insurance
Some participants did only some of these tasks.
The rooms
In each location we rented market research facilities that consisted of two rooms divided by a one-way mirror. On the mirror side of the wall, the moderator conducted the session with the participant; on the other side, observers could view the session through the glass. We recorded the sessions on audio and video, including images of what was happening on the computer screen that observers could also view on a television monitor.