Content area
The purpose of this paper is to describe the processes the UCLA Library Website Redesign Team used to develop a new library website responsive to the needs of the broad population of UCLA Library users. Using a combination of structured analyses of the previous library website, user surveys, a card sort protocol and a think-aloud protocol, the Website Redesign Team procured sufficient information to meet its redesign goals: established clear site organization and navigation, utilized user-centered nomenclature, ensured easy access from the library homepage to relevant information, developed a unified institutional visual identity throughout the site, and enabled a content management system. Standard usability methods such as surveys and the card sort and think-aloud protocols are essential tools for evaluating and redesigning complex multi-layered websites. Since the redesign process is not finite, these tools contribute to keeping a website current and responsive to the needs of its users. This case study provides an example that the Redesign Team hope will empower readers with tools and knowledge that they can use to perform similar tasks in their own environment.
Full text
Abstract
Purpose - The purpose of this paper is to describe the processes the UCLA Library Website Redesign Team used to develop a new library website responsive to the needs of the broad population of UCLA Library users.
Design/methodology/approach - Using a combination of structured analyses of the previous library website, user surveys, a card sort protocol and a think-aloud protocol, the Website Redesign Team procured sufficient information to meet its redesign goals: established clear site organization and navigation, utilized user-centered nomenclature, ensured easy access from the library homepage to relevant information, developed a unified institutional visual identity throughout the site, and enabled a content management system.
Findings - Standard usability methods such as surveys and the card sort and think-aloud protocols are essential tools for evaluating and redesigning complex multi-layered websites. Since the redesign process is not finite, these tools contribute to keeping a website current and responsive to the needs of its users.
Originality/value - This case study provides an example that the Redesign Team hope will empower readers with tools and knowledge that they can use to perform similar tasks in their own environment.
Keywords Worldwide web, Design, Academic libraries
Paper type Case study
Introduction
In 2003, the UCLA Library initiated a redesign of the UCLA Library website. The goals of the redesign were:
* to establish clear site organization and navigation;
* to utilize user-centered nomenclature;
* to ensure easy access from the homepage to information relevant to the entire user population;
* to develop a unified institutional visual identity throughout the site; and
* to enable a content management system.
The guiding principle throughout was to move from many individual websites representing units and departments to a single UCLA Library website with consistent design and information organized in as predictable a manner as possible.
The previous website (see Figure 1), which had been in place for about three years, had the following problems:
* Overall organization of the site reflected the library's organizational chart rather than the way users might look for information. The site modeled administrative reporting lines and the geographic distribution of library buildings, organizing information by department or the unit library responsible for collecting and maintaining it.
* A lack of consistency and standards for the placement and labeling of navigational elements made it difficult for users to find information in a predictable location.
* Inconsistent nomenclature on various linked pages within the site and a heavy use of library jargon made it difficult for users to find the information they sought.
* Differences in graphic design and layout across departmental, resource, and service pages due to decentralized planning and management and a lack of coordination among individual library units resulted in as many different visual identities, navigation schemes, and vocabularies as there were individual personalities creating them. Design inconsistencies ranged from color palates and versions of the library logo to the placement and labeling of links to crucial services such as the online catalog.
* Decentralized planning and management among library departments and units led to an overall unresponsiveness to rapid change and varying depth, type, and accuracy of information. The site made it difficult for users to ascertain which pages to trust as authoritative. This confusion resulted in a significant amount of redundant effort and wasted time.
The problems had been identified by library staff using the site daily in their interaction with UCLA Library users and in their day-to-day job responsibilities, by members of the UCLA community, and by Library administration.
That these problems were real and in some cases acute was reinforced by the survey data and user testing undertaken by the Website Redesign Team appointed by the Library administration. This paper describes the processes the team used to develop a website responsive to the needs of the broad population of users of UCLA Library users. This paper will describe the surveys and user testing undertaken to ensure that the site is organized to anticipate user's questions, that terminology and labels are familiar to most library users, that the design is simple and appealing, and that the site as a whole promotes the important collections and services offered by the library.
Methodology
Structured analysis
The work toward a redesigned UCLA Library website began with an inventory of existing library web pages. A structured analysis identified key information links across unit library websites, information that should appear on upper page levels, and duplication of information among existing pages. The analysis consisted of a systematic sweep of the main library web page and all individual unit and department pages. A spreadsheet of the top and lower level links on each page was created in Microsoft Excel to show the organization of the entire site. The spreadsheet allowed easy comparison of pages from the various libraries and departments. It also showed variations in nomenclature describing information, variations in page architecture, and inconsistencies in the organization of links. The Redesign Team used this information as a basis for surveys and user testing.
Surveys
The team conducted two surveys during the redesign process. The first survey collected information on why and how the existing library website was being used, by whom, what information was found easily and what not, and what information was missing. The survey consisted of 18 questions administered online via a link from the library's homepage. It was open to all library users. UCLA staff, students, and faculty members who completed the survey were entered into a draw for a $250 gift certificate at the UCLA Store. Three hundred people responded to the survey. Unfortunately, due to poor survey design, the responses were not as useful as they could have been. Most of the useful data came from answers to four open-ended survey questions:
(1) What other services or functions would you like to see available on the UCLA Library website?
(2) What do you like most about the current UCLA Library website?
(3) What do you like least about the current UCLA Library website?
(4) Do you have any other comments about the UCLA Library website?
The second survey investigated the terms users might choose for key library services from among the various terms appearing on the Library's own web pages, including traditional library terms (Interlibrary Loan) as well as functional terms (Request Materials from non-UCLA Libraries). The survey listed 12 key library services for which the respondents were asked to choose from one of several suggested terms for that service. They could also supply their own. The survey was conducted on paper, rather than electronically, to facilitate even distribution across libraries. The survey was given to undergraduate students in library instruction sessions and made available to all users in campus libraries. The survey asked the status of the respondent (undergraduate, graduate, faculty, or staff) as well as the name of the library where the survey was completed so that the team could identify any patterns in terminology choice based on these factors.
Three and hundred seventy-one users responded to the survey. For eight of the services, there was a clear preference for one of the terms over the other choices. For the other four, two or more of the terms were very close in respondent preference. In many cases, the preferred term or terms were in fact common library service labels, possibly indicating that the survey was answered by experienced library users. The results did not indicate a difference in terminology preference by user category. The terms from the structured analysis as well as the terms identified in this survey laid the groundwork for the next analytical tool, the card sort protocol.
Card sort protocol
The card sort protocol was the first step in planning the organization of the new website. Jakob Nielsen describes this usability method as "generative" and one used when you "don't yet have a design, and [your] goal is to find out how people think about certain issues" (Nielsen, 2004) - the issue for the Redesign Team being how to organize the vast array of library-supported collections and services on the website. The protocol allowed the team to focus solely on the information architecture of the site (i.e. where to put what) without concern for the design.
A card sort protocol is executed using note cards containing words or phrases. Participants are asked to organize the cards into groups most meaningful to them, putting cards with similar concepts together in the same group. Depending on the goals of the protocol the categories are predefined and named, or the participant determines the number of the categories and their names.
The structured analysis of the existing website provided 76 essential links. The names of the links, informed by the terminology survey, were printed on one side of a white index card. Each card was numbered for identification purposes. The reverse side of the card provided the definition of the link. The definitions were created by team members but were also informed by the previous surveys. For example, the link named "Article Databases" had the definition "online index to the contents of magazines and journals" on the reverse side of the card. Participants were provided these definitions so that their sort was not hindered by a lack of knowledge of library services or of the term provided.
The primary user groups of the Library's website are UCLA faculty, students, and campus staff from all academic departments. Administrative faculty and staff as well as library staff, although heavy users of the site, were considered secondary for purposes of the redesign and therefore not recruited for the card sort. Recognizing the valuable input library staff could provide, however, as well as for political expediency, several library staff participated in the card sort protocol.
Using figures provided by the UCLA Office of Analysis and Information Management (within the Department of Budget & Finance), the team determined that 40 participants would be necessary to accurately reflect all targeted user groups (despite the fact that the literature recommends between 15 and 20 or 30 participants; Nielsen, 2004). With target numbers for all user types in hand, recruiting for the protocol could begin.
Since the majority of participants needed were undergraduate and graduate students, the team approached students across campus in areas where they gather primarily the student union and various eating areas near science, humanities, and social science classrooms. The target participant number for students was 28. Faculty and staff were recruited using departmental listservs. For consistency while recruiting, the team used recruitment scripts. Those who agreed to participate were given an informed consent sheet with maps to the test sites. As an added incentive, all participants who completed the card sort were given a gift bag valued at $15.00 that included a library mug, a copy card, candy, pens, and pencils. Recruiting a sufficient number of participants took two weeks.
The UCLA Institutional Review Board (IRB) requires that anyone using human subjects for surveys or experiments clear their research protocol if the results will be discussed outside of the immediate survey team, delivered in a conference paper, or published. The clearance procedure ensures that test participants will not be harmed and that any personal identifying information gathered will be kept completely confidential. Most institutions have an Office for the Protection of Research Subjects (OPRS) where one can obtain the necessary procedures, forms, etc. Because the Redesign Team's testing was cognitive-based, it was able to file for an exemption and its research did not have to receive a full review from the Review Board. Personnel in the OPRS reviewed the Redesign Team's submission, which included copies of the survey instrument and all related parts of the experiment - recruitment methods and scripts, the informed consent form, the calendar for scheduling participants, an explanation of how the confidentiality of test participants was to be preserved, incentives, etc. - and granted it an exemption.
Participants were given one hour to complete the card sort, i.e. to organize the 76 cards into any number of sets that made sense to them and to name the card sets. Participants seemed to approach this task in two different ways. Some immediately started making sets and moving the cards around accordingly; others looked through the entire stack of cards before making any decisions about how to organize them. The team member facilitating the sort noted the cards for which participants referred to the definition printed on the back before sorting the card. Once all of the cards were sorted and the participant seemed satisfied with the results, they were asked to create names for each set and write them on a yellow index card. (The yellow index card is analogous to a category on a web page, and the set of cards it identifies to the links in that category.) At the end of the session, the facilitator noted the terms on each yellow card and the numbers of all of the cards in each pile.
The card sort results were input into SPSS®, a statistical analysis tool. Since participants created many different words and phrases to identify their card sets, the Redesign Team quickly realized that to make meaningful comparisons it would need to standardize the terms, reconciling various phrases used by participants for similar concepts. A second card sort protocol by team members served this purpose. The team compiled a list of all of the labels used on the yellow cards identifying the sorted terms, gathered similar terms together, and then created labels to bring similar participant-created terms together. (For example, "library info", "general information", etc., were normalized into "general library information".) Standard library terminology as well as terms from the structured analysis were used.
The Redesign Team was now able to see trends in how participants organized their cards:
* In a majority of sorts, certain cards appeared in a similar category. For example, about half of the participants put a card that said "Copying and Printing" under the category they called "Services".
* Some cards appeared in a variety of categories. For example, participants put "Subject Experts" in many different categories, including "About the Library", "Contact Us", "Help", and "Reference and Research Help". This indicated a link to information on subject librarians needed to be made from most, if not all, of these categories on the new website.
The results of the card sort protocol were extremely useful in making decisions on organizing the website. When the team began to refine the large general organizational categories for the new site, the input from the card sort was essential to making initial design decisions.
Think aloud protocol
Based on the information gathered from the structured analysis, the surveys, and the card sort protocol, a professional website designer, Guerard Design Office, developed a prototype of the new UCLA Library website. This prototype served as the basis for another usability test known as a "think aloud protocol". For this test, participants are asked to complete tasks using a prototype website and to "think aloud", i.e. to say everything they are thinking while they complete the tasks. One observer notes everything the subject says and if and how the participant completes the task. Another observer directs the participant and answers any questions asked. This type of user test provides essential real-time feedback on potential problems in the design and organization of a website.
Although Nielsen (2000) and Krug (2000) contend that only five participants are needed for significant results from a think aloud protocol, the team opted to use ten subjects, allowing more than one participant from each of the primary user types. The team used similar recruitment methods to those used for the card sort but did not use any participants from any previous tests.
Participants in the think aloud protocol were asked to find information about the following:
* hours for library units;
* departments served by subject libraries;
* how to obtain an article through interlibrary loan;
* how to place materials on course reserve (faculty) or locate them (students);
* how to access electronic resources from off-campus;
* identify a subject specialist for a research consultation session; and
* how to find a wide range of materials for a research project on a specific subject.
As described above, two facilitators conducted the individual sessions: one read the questions and interacted with the participant while the other recorded the participant's actions, including:
* the path taken to find the answer;
* anything said while navigating the site; and
* any observations of the participant's behavior.
After completing the tasks, the participants were asked about their general impression of the site, suggestions for the designer, or any other comments about the website. Finally, there was a brief survey about the participant's previous use and knowledge of the library and the library's website.
Results from the think aloud protocol were collated to pull out quantitative data based on success, partial success and failure, and qualitative data based on comments and observations. The results showed that participants overall navigated the site successfully and answered the questions quickly. (The average think aloud took 30-40 minutes.) Many participants moved on to the next question or even used the "Ask a Librarian" link when they were unable to complete a task after two or three attempts. Participants generally liked the five navigational categories with the corresponding drop-down menus that forced them to look at all of the choices and explore the site when looking for information. The categories seemed to provide a good sense of the included content and were intuitive so that even if the participant was not sure where to look, they could scan easily the categories, menus and sub-menus for the information they sought. The think aloud protocol did not result in any major changes to the design or content of the prototype website. Examples of the changes made include:
* corrected inconsistent terminology (e.g. off campus access versus Bruin Online Proxy Server);
* duplicated links under more than one category (e.g. Subject Librarian list added to "Ask a Librarian");
* moved certain links up in the menu hierarchy (e.g. Proxy Server);
* added arrows, print links, and other functional and navigational clues. The Current UCLA Library website is shown in Figure 2.
Website design
With work on the site structure and nomenclature underway, the Team focused on hiring a professional graphic designer to provide a strong visual identity and logo that could be applied consistently across all library web pages as well as the online catalog interface screens. In addition, the graphic designer was asked to provide a standardized page template to be used with our content management system.
One of the challenges in developing the standardized page template was to create a device that enforced design consistency but was not confining and monotonous. The initial structural analysis revealed a wide range of page types serving different purposes, e.g. informational pages, instructional pages, pages that served as interfaces to databases, media-intensive pages framing digital collections and exhibits, etc. A successful design would be flexible enough to present many types of information and enforce enough restrictions to assist content creators lacking visual arts training in creating professional pages.
The designer developed a five-column grid system to serve as the underlying framework for every page. The grid system can be reconfigured easily and adapted to the nature of the information being presented on the page. A two-by-three column configuration serves as the template for most pages. Navigational links within the site and informational content consistently appear in designated areas, while another section is reserved for graphics. There are variations on the grid system, such as a page spanning five columns to accommodate the presentation of tabular information.
All of the page templates are managed and enforced by a content management system. RedDot Solution's Web Content Management Software (CMS) was selected for its enterprise-class features, scalability, and reasonable price tag. A major advantage of a CMS is the separation of content from its presentation layer. All content is stored in a database while the HTML templates into which it is poured for presentation are managed by the CMS. Content creators and editors are granted limited permissions to reconfigure or edit elements on a page, thereby reducing variability in location of key navigation and labeling elements. While it is difficult to quantify the effectiveness of the graphical layout of grid-based pages, once problems are identified through formal usability testing and/or empirical observations, modifications to the grid system can be easily and globally implemented utilizing the CMS template system.
Conclusion
The new UCLA Library website, the end product of this work, was released in August 2004. Following the release, the team began soliciting feedback in the following ways:
* directed e-mails to faculty asking for comments;
* comments solicited directly via a link on the new website; and
* library staff feedback sessions.
Over the past year, the website has changed in small ways based on this feedback, but the team is reserving larger changes after users gain more in-depth experience with the site and provide it with their comments. Website design is an ongoing process that requires continuous usability testing as the institution it represents and the information it provides evolve and change. To test revisions of the site, the Redesign Team will create a new prototype and conduct a think aloud protocol to test it. At this time, only the central site has been redesigned; unit libraries and internal department pages have not yet been converted. The team and library staff are still discussing the nature of certain pages, whether they are primarily internal to the library or of interest to the campus or outside users. The UCLA Library is not unique in facing internal changes and external pressures. As the Redesign Team continues to improve the Library website, however, it is confident that it has the tools and methodologies to ensure that it provides library users with the information they seek in the easiest and most direct way, and in terms they understand.
References
Nielsen, J. (2000), "Why you only need to test with 5 users", available at: www.useit.com/ alertbox/20000319.html
Nielsen, J. (2004), "Card sorting: how many users to test", available at: www.useit.com/alertbox/ 20040719.html
Krug, S. (2000), Don't Make Me Think: A Common Sense Approach to Web Usability, New Riders Press, Indianapolis, IN.
Dominique Turnbow
Louise M. Darling Biomedical Library, University of California, Los Angeles, California, USA
Kris Kasianovitz
Charles E. Young Research Library, University of California, Los Angeles, California, USA
Lise Snyder
UCLA College Library, University of California, Los Angeles, California, USA
David Gilbert
UCLA Music Library, University of California, Los Angeles, California, USA
David Yamamoto
UCLA Library Administration, University of California, Los Angeles, California, USA
Copyright Emerald Group Publishing, Limited 2005