Modeling a Library Website Redesign Process: Developing a User-Centered Website Through Usability Testing

Abstract

This article presents a model for creating a strong, user-centered web presence by pairing usability testing and the design process. Four rounds of usability testing were conducted throughout the process of building a new academic library web site. Participants were asked to perform tasks using a talk-aloud protocol. Tasks were based on guiding principles of web usability that served as a framework for the new site. Results from this study show that testing throughout the design process is an effective way to build a website that not only reflects user needs and preferences, but can be easily changed as new resources and technologies emerge. [PUBLICATION ABSTRACT]

Headnote

ABSTRACT

This article presents a model for creating a strong, user-centered web presence by pairing usability testing and the design process. Four rounds of usability testing were conducted throughout the process of building a new academic library web site. Participants were asked to perform tasks using a talk-aloud protocol. Tasks were based on guiding principles of web usability that served as a framework for the new site. Results from this study show that testing throughout the design process is an effective way to build a website that not only reflects user needs and preferences, but can be easily changed as new resources and technologies emerge.

INTRODUCTION

In 2008 the Hunter College Libraries launched a two-year website redesign process driven by iterative usability testing. The goals of the redesign were to:

* update the design to position the library as a technology leader on campus;

* streamline the architecture and navigation;

* simplify the language used to describe resources, tools, and services; and

* develop a mechanism to quickly incorporate new and emerging tools and technologies.

Based on the perceived weaknesses of the old site, the libraries' web committee developed guiding principles that provided a framework for the development of the new site. The guiding principles endorsed solid information architecture, clear navigation systems, strong visual appeal, understandable terminology, and user-centered design.

This paper will review the literature on iterative usability testing, user-centered design, and think-aloud protocol and the implications moving forward. It will also outline the methods used for this study and discuss the results. The model used, building the design based on the guiding principles and using the testing to uphold those principles, led to the development of a strong, user-centered site that can be easily changed or adapted to accommodate new resources and technologies. We believe this model is unique and can be replicated by other academic libraries undertaking a website redesign process.

BACKGROUND

The goals of the research were to (1) determine the effectiveness of the Hunter College Libraries website, (2) discover how iterative usability testing resulting in a complete redesign impacts how the students perceive the usability of a college library website, and (3) reveal student information-seeking habits.

A formal usability test was conducted both on the existing Hunter College Libraries website (appendix A) and the following drafts of the redesign (appendix B) with twenty users over an eighteen-month period. The testing occurred before the website redesign began, while the website was

under construction, and after the site was launched. The participants were selected through convenience sampling and informed that participation was confidential. The intent of the usability test was to uncover the flaws in navigation and terminology of the current website and, as the redesign process progressed, to incorporate the users' feedback into the new website's design to closely match their wants and needs.

The redesign of the website began with a complete inventory of the existing webpages. An analysis was done of the website that identified key information, links, units within the department, and placement of information in the information architecture of the website.

We identified six core goals that we felt were the most important for all users of the library's website:

1. User should be able to locate high-level information within three clicks.

2. Eliminate library jargon from navigational system using concise language.

3. Improve readability of site.

4. Design a visually appealing site.

5. Create a site that was easily changeable and expandable.

6. Market the libraries' services and resources through the site.

LITERATURE REVIEW

In 2010, OCLC compiled a report, "The Digital Information Seeker," that found 84 percent of users begin their information searches with search engines, while only 1 percent began on a library website. Search engines are preferred because of speed, ease of use, convenience, and availability.1 Similar studies such as Emde et al., and Gross and Sheridan, have shown that students are not using library websites to do their research.2 Gross and Sheridan assert in their article on undergraduate search behavior that "although students are provided with library skills sessions, many of them still struggle with the complex interfaces and myriad of choices the library website provides."3 This research shows the importance of creating streamlined websites that will compete for our students' attention. In building a new website at the Hunter College Libraries, we thought the best way to do this was through user-centered design.

Web designers both inside and outside the library have recognized the importance of user-centered design. Nielsen advises that website structure should be driven by the tasks the users came to the site to perform.4 He asserts the amount of graphics on webpages should be minimized because they often affect page download times and that gratuitous graphics (including text rendered as images) should be eliminated altogether.5 He also contends it is important to ensure that page designs are accessible to all users regardless of platform or newness of technology.6

In their article, "How do I find an article? Insights from a web usability study," Cockrell and Jayne cited instances when researchers concluded that library terminology contributed to patrons' difficulties when using library websites, thus highlighting the importance of understandable terminology. Hulseberg and Monson found in their investigation of student-driven taxonomy for library website design that "by developing our websites based on student-driven taxonomy for library website terminology, features, and organization, we can create sites that allow students to get down to the business of conducting research."7

Performing usability testing is one way to confirm user-centered design. In his book Don't Make Me Think!, Krug insists that usability testing can provide designers with invaluable input. That, taken together with experience, professional judgment, and common sense, makes design choices easier.8 Ipri, Yunkin, and Brown, in their article "Usability as a Method for Assessing Discovery," emphasize the important role usability testing has in capturing emotional and aesthetic responses users have to websites, along with expressions of satisfaction with the layout and logic of the site. Even the discovery of basic mistakes, such as incorrect or broken links and ineffective wording, can negatively affect discovery of library resources and services.9

In Battleson, Booth, and Weatherford's literature review for their usability testing of an academic library website case study, they summarize Dumas and Redish's discussion of the five facets of formal usability testing: (1) the goal is to improve the usability of the interface, (2) testers should represent real users, (3) testers perform real tasks, (4) user behavior and commentary are observed and recorded, and (5) data are analyzed to recognize problems and suggest solutions. They conclude that when usability testing is "applied to website interfaces, this test method not only results in a more usable site, but also allows the site design team to function more efficiently, since it replaces opinion with user-centered design."10 This allows the designers to evaluate the results and identify problems with the design being tested.11

Usability experts Nielsen and Tahir contend that the earlier and more frequently usability tests are conducted, the more impact the results will have on the final design of the website because the results can be incorporated throughout the design process. They conclude it is better to conduct frequent, smaller studies with a maximum of five users. They assert, "You will always have discovered so many blunders in the design that it will be better to go back to the drawing board and redesign the interface than to discover the same usability problems several more times with even more users."12

Based on the strength of the literature, we decided to use iterative testing for our usability study. Krug points out that testing is an iterative process because designers need to create, test, and fix based on test results, then test again.13 According to the United States Department of Health and Human Services report "Research-Based Web Design and Usability Guidelines," conducting before and after studies when revising a website will help designers determine if changes actually made a difference in the usability of the site.14 Manzari and Trinidad-Christensen found in their evaluation of user-centered design for a library website, iterative testing is when a product is tested several times during development, allowing users' needs to be incorporated into the design. In their study, their aim was that the final draftof their website would closely match the users' information needs while remaining consistent, easy to learn, and efficient.15 Battleson, Booth, and Weintrop report that there is "a consensus in the literature that usability testing be an iterative process, preferably one built into a Web site's initial design."16 They explain that "site developers should test for usability, redesign, and test again-these steps create a cycle for maintaining, evaluating and continually improving a site."17

George used iterative testing in her redesign of the Carnegie Mellon University Libraries website and concluded that it was "necessary to provide user-centered services via the web site."18 Cobus, Dent, and Ondrusek used six students to usability test the "pilot study." Then eight students participated in the first round of testing; then librarians modified the prototype and tested fourteen students in the second and final round. After the second round of testing they used the results of this test to analyze the user recordings and deliver the findings and proposed "fixes" to the prototype pages to the web editor.19

McMullen's redesign of the Roger Williams University library website was able to "complete the usability-refinement cycle" twice before finalizing the website design.20 But continued refinements were needed, leading to another round of usability tests to identify and correct problem areas.21 Bauer-Graham, Poe, and Weatherford did a comparative study of a library websites' usability via a survey and then redesigned the website after evaluating the survey's results. They waited a semester, distributed another survey to determine the functionality of the current site. The survey had the participants view the previous design and the current design in a side-by-side comparison to determine how useful the changes made to the site were.22

When testing participants, in the article "How do I find an article? Insights from a Web usability study," Cockrell and Jayne suggest using a web interface to perform specified tasks while a tester observes, noting the choices made, where mistakes occur, and using a "think aloud" protocol. They found that modifying the website through an ongoing, iterative process of testing, refining, and retesting its component parts improves functionality.23

In conducting our usability testing we used a think-aloud protocol to capture the participants' actions. Van Den Haak, De Jong, and Schellens define think-aloud protocol as relying on a method that asks users to complete a set of tasks and to constantly verbalize their thoughts while working on the tasks. The usefulness of this method of testing lies in the fact that the data collected reflect the actual use of the thing being tested and not the participants' judgments about its usability. Instead, the test follows the individual's thoughts during the execution of the tasks. 24

Nielsen states that think-aloud protocol "may be the single most valuable usability engineering method. . . . One gets a very direct understanding of what parts of the [interface/user] dialog cause the most problems, because the thinking aloud method shows how users interpret each individual interface item."25 Turnbow 's article "Usability testing for web redesign: a UCLA case study" states that using the "think-aloud protocol" provides crucial real-time feedback on potential problems in the design and organization of a website.26

Cobus, Dent, and Ondrusek used the think-aloud protocol in their usability study. They encouraged participants to talk out loud as they answered the questions, audio taped their comments, and captured their on-screen navigation using Camtasia.27 This information was used to successfully reorganize Hunter College Library's website.

METHOD

An interactive draftof Hunter College Libraries redesigned website was created before the usability study was conducted. In spring 2009, the authors created the protocol for the usability testing. A think-aloud protocol was agreed upon for testing both the old site and the drafts of the new site, including a series of post-test questions that would allow participants to share their demographic information and give subjective feedback on the drafts of the site. Draftquestions were written, and we conducted mock usability tests on each other. After several drafts we revised our questions and performed pilot tests on an MLIS graduate student and two undergraduate student library assistants with little experience with the current website. We ascertained from these pilot tests that we needed to slightly revise the wording of several questions to make them more understandable to all users. We made the revisions and eliminated a question that was redundant.

All recruitment materials and finalized questions were submitted to the Institutional Review Board (IRB) for review and went through the certification process. After receiving approval we secured a private room to conduct the study. Participants were recruited using a variety of methods. Signs were posted throughout the library, an e-mail was sent out to several Hunter College distribution lists, and a tent sign was erected in the lobby of the library. Participants were required to be students or faculty. Participants were offered a $10.00 Barnes & Noble giftcard as incentive. Applicants were accepted on a rolling basis. Twenty students participated in the web usability study (appendix C). No faculty responded to our requests for participation so a decision was made to focus this usability test on students rather than faculty because students comprise our core user base. Another usability test will be conducted in the future that will focus on faculty to determine how their academic tasks differ from undergraduates when using the library website. The redesigned site is malleable, which makes revisions and future changes in the design a predicted outcome of future usability tests.

Tests were scheduled for thirty-minute intervals. We conducted four rounds of testing using five participants per round. The two researchers switched questioner and observer roles after each round of testing. Each participant was asked to think aloud while they completed the tasks and navigated the website. Both researchers took notes during the tests to ensure detailed and accurate data was collected. Each participant was asked to review the IRB forms detailing their involvement in the study, and they were asked to consent at that time. Their consent was implied if they participated in the study after reading the form.

The usability test consisted of fifteen task-oriented questions. The questions were identical when testing the old and new draftsite. The first round tested only the old site, while the following three rounds tested only the new draftsite. We tested both sites because we believed that comparing the two sites would reveal if the new site improved performance. The questions (appendix D) were not changed after they were initially finalized and remained the same throughout the entire four rounds of the usability study.

Participants were reminded at the onset of the test and throughout the process that the design and usability of the site(s) were being tested, not their searching abilities. The tests were scheduled for an hour each, allowing participants to take the tests without time restrictions or without being timed. As a result, the participants were encouraged to take as much time as they needed to answer the questions, but were also allowed to skip questions if they were unable to locate answers. Initially the tests were recorded using Camtasia software. This allowed us to record participants' navigation trails through their mouse movements and clicks. But, after the first round of testing, we decided that observing and taking notes was appropriate documentation, and we stopped using the software.

After the participants completed the tests we asked them user preference questions to get a sense of their user habits and their candid opinions of the new draftof the website. These questions were designed to elicit ideas for useful links to include on the website and also to gauge the visual appeal of the site.

RESULTS

DISCUSSION

Hunter College Libraries' website was due for a redesign because the site was dated in its appearance and did not allow new content to be added quickly and easily. As a result, a decision was made to build a new site using a content management system (CMS) to make the site easily expandable and simple to update. This study tested the simple tasks to determine how to structure the information architecture and to reinforce the guiding principles of the redesigned website.

Task Successes and Failures

The high percentage of success of participants finding books on the redesigned website using the online library catalog and easily find library hours reinforced our guiding principle of understandable terminology and clear navigational systems.

Krug contends that navigation educates the user on the site's contents through its visible hierarchy. The result is a site that guides the user through their options and instills confidence in the website and its designers. 28 We found this to be true in the way our users easily found the hours and catalog links on the prototype of our library website. The users on the old site knew where to look for this information because they were accustomed to how to navigate the old site. Given that the prototype was a complete departure from the navigation and design of the old site, it was crucial that the labels and links were clear and understandable in the prototype or our design would fail. We made "Hours" the first link under the "About" heading and "CUNY+/Books" the first link under the "Find" heading and as a result both our terminology and our structure was a success with participants.

On the old website, users rarely used the libraries' online chat client. Despite our efforts to remind students of its usefulness, the website didn't sufficiently place the link in a reasonably visible location on the home page. In the old site, only 40 percent of participants located the link as it was on the bottom leftof the screen and easy to overlook. Instead, on the new site, the "Ask a Librarian" link was prominently featured on the top of the screen. These results upheld the guiding principles of solid information architecture and understandable terminology. It also supported Nielsen's assertion that "site design must be aimed at simplicity above all else, with as few distractions as possible and with a very clear information architecture and matching navigation tools."29 As a result the launch of the redesigned site, the use of the QuestionPoint chat client has more than doubled.

Finding a journal article on a topic was always problematic for users of the old library website. The participants we tested were familiar with the site, and 80 percent erroneously clicked on "Journal Title List" when the more appropriate link would have been "Databases" if they didn't have an exact journal title in mind. Although we taught this in our information literacy courses, it was challenging getting the information across. In order to address this on the new site, "Databases" was changed to "Databases/Articles" and categorized under the heading "Find." The participants using the new site had greater success with the new terminology; 66 percent correctly chose "Databases/Articles." This question revealed an inconsistency with the guiding principals of understandable terminology and clear navigation systems on the old site. These issues were addressed by adding the word "Articles" after "Databases" on the new site to clarify what resources could be found in a database and also by placing the link under the heading "Find" to further explain the action a student would be taking by clicking on the "Databases/Articles" link.

Finding reference materials was challenging for the users of the old site as none of the participants clicked on the intended link "Subject Guides." In an effort to increase usage of the research guides, the library not only purchased the LibGuides tool, but also changed the wording of the link to "Topic Guides." As we neared the end of our study we observed that only one participant knew to click on the "Topic Guides" link for research assistance. The participants suggested calling it "Research Guides" instead of "Topic Guides" and we changed it. Unfortunately, the usability study was completed and we were unable to further test the effectiveness of the rewording of this link. Anecdotally, the rewording of this link appears to be more understandable to users as the research guides are getting more usage (based on hit counts) than the previous guides. The rewording of these guides adhered to both principles of understandable terminology and user-centered design. These results supported Nielsen's assertion that the most important material should be presented up front, using the inverted pyramid principal. "Users should be able to tell in a glance what the page is about and what it can do for them." 30 Our results also supported the HHS report, which states that terminology "plays a large role in the user's ability to find and understand information. Many terms are familiar to designers and content writers, but not to users."31 We concluded that rewriting the link based on student feedback reduces the use of terminology. Although librarians are "Subject Specialists" and "Subject Liaisons" and are familiar with those labels and that terminology, our students were looking for the word "research" instead of "subject" so they were not connecting with the library's LibGuides.

As previously discussed, students of the old site thought the link "Journal Title List" would give them access to the library's database holdings. When asked to find a specific journal title the correct answer to this question on the old site was "Journal Title List," with only 40 percent of the participants answering correctly. Another change to terminology in the new site, both were placed under the heading "Find," and, after testing of the first prototype, "Journal Title List" was changed to "List of Journals and Magazines." In the following tests 66 percent of the participants were able to answer correctly.

The percentages of success in finding circulation policies between the old site and the prototype site were slight, only a 7 percent difference. This can be attributed to the fact that participants on the old site could click on multiple links to get to the correct page, and they were familiar enough with the site to know that. In the prototype of the site there were several paths as well, some direct, some indirect. Testing the wording of this link supported the understandable terminology principle, more so than the old website's "Library Policies" link, yet to be true to our user-centered design principle, we needed to reword it once more. Therefore, after the test was completed and the website was launched, we reworded the link to "Checkout Policies," which utilizes the same terminology that users are familiar with because they checkout books at our checkout desk.

The remaining tasks consisted of locating information, such as finding books on reserve, magazines by title, library staffcontact information, and finding branch information were all met with higher success rates in the prototype site because in the redesign process the links were reworded to support the understandable terminology and user-centered design principles.

Participant Feedback: Qualitative

The usability testing process informed the redesign of our website in many specific ways. If the layout of the site didn't test well with participants, we planned to create another prototype. In their evaluation of Colorado State Universities Libraries' digital collections and the Western Waters Digital Library websites, Zimmerman and Paschal describe the importance of first impressions of a website as the determining factor of whether users return to a website; if it is positive they will return and continue to explore.32

When given an opportunity to give feedback on what they thought of the design of the website the participants commented:

* "There were no good library links at the bottom before and there wasn't the Ask A Librarian link either which I like a lot."

* "The old site was too difficult to navigate, new site has a lot of information, I like the different color schemes for the different things."

* "It is contemporary and has everything I need in front of me."

* "Cool."

* "Helpful."

* "Straightforward."

* "The organization is easier for when you want to find things."

* "Interactivity and rollovers make it easy to use."

* "Intuitive, straight-forward and I like the simplicity of the colors."

* "More professional, more aesthetically pleasing than the old site."

* "The four menu options (About, Find, Services, Help) break the information down easily."

Additional research conducted by Nathan, Yeow, and Murguesan claims attractiveness (referring to aesthetic appeal of a website) is the most important factor in influencing customer decision-making and affects the usability of the website.33 Not only that, but users feel better when using a more attractive product. Fortunately, the feedback from our participants revealed that the website was visually appealing, and the navigation scheme was clear and easy to understand.

Other Changes Made to the Libraries' Website because of Usability Testing

Participants commented that they expected to find library contact information on the bottom of the homepage, so the bottom of the screen was modified to include this information as well as a "Contact Us" link. Participants did not realize that the "About," "Find," "Services," and "Help" headings were also links, so we modified them so they were underlined when hovered over. There were also adjustments to the gray color bars on the top of the page because participants thought they were too bright, so they were darkened to make the labels easier to read.

Participants also commented that they wanted links to various public libraries in New York City under the "Quick Links" section of the homepage. We designed buttons for Brooklyn Public Library, Queens Public Library, and the New York Public Library and reordered this list to move these links closer to the top of the "Quick Links" section.

CONCLUSION

Conducting a usability study of Hunter College Libraries existing website and the various stages of the redesigned website prototypes was instrumental in developing a user-centered design. Approaching the website redesign in stages, with guidance from iterative user testing and influenced by the participants' comments, gave the web librarian and the web committee an opportunity to incorporate the findings of the usability study into the design of the new website. Rather than basing design decisions on assumptions of users' needs and information seeking behaviors, we were able to incorporate what we'd learned from the library literature and the users' behavior into our evolving designs. This strategy resulted in a redesigned website that, with continued testing, user feedback, and updating, has aligned with the guiding principles we developed at the onset of the redesign project. The one unexpected outcome from this study is that we discovered that despite how well a library website is designed, users will still need to be educated in how to use the site with an emphasis on developing strong information literacy skills.

References

REFERENCES

1. "The Digital Information Seeker: Report of the Findings from Selected OCLC, RIN, and JISC User Behaviour Projects," OCLC Research, ed. Lynn Silipigni-Connaway and Timothy Dickey (2010): 6, www.jisc.ac.uk/publications/reports/2010/digitalinformationseekers.aspx.

2. Judith Emde, Lea Currie, Frances A. Devlin, and Kathryn Graves, "Is 'Good Enough' OK? Undergraduate Search Behavior in Google and in a Library Database," University of Kansas Scholarworks (2008), http://hdl.handle.net/1808/3869; Julia Gross and Lutie Sheridan, "Web Scale Discovery: The User Experience," New Library World 112, no. 5/6 (2011): 236, doi: 10.1108/03074801111136275.

3. Ibid, 238.

4. Jakob Nielsen, Designing Web Usability (Indianapolis: New Riders, 1999), 198.

5. Ibid, 134.

6. Ibid, 97.

7. Barbara J. Cockrell and Elaine A. Jayne, "How Do I Find an Article? Insights from a Web Usability Study," Journal of Academic Librarianship 28, no. 3 (2002): 123, doi: 10.1016/S0099-1333(02)00279-3.

8. Steve Krug, Don't Make Me Think! A Common Sense Approach to Web Usability, 2nd ed. (Berkeley, CA: New Riders, 2006), 135.

9. Tom Ipri, Michael Yunkin, and Jeanne Brown, "Usability as a Method for Assessing Discovery," Information Technology & Libraries 28, no. 4 (2009): 181, doi: 10.6017/ital.v28i4.3229.

10. Brenda Battleson, Austin Booth, and Jane Weintrop, "Usability Testing of an Academic Library Web Site: A Case Study," Journal of Academic Librarianship 27, no. 3 (2001): 189-98, doi: 10.1016/S0099-1333(01)00180-X.

11. Ibid.

12. Jakob Nielsen and Marie Tahir, "Keep Your Users in Mind," Internet World 6, no. 24 (2000): 44.

13. Steve Krug, Don't Make Me Think! A Common Sense Approach to Web Usability, 135.

14. Research-Based Web Design and Usability Guidelines, ed. Ben Schneiderman (Washington: United States Dept. of Health and Human Services, 2006), 190.

15. Laura Manzari and Jeremiah Trinidad-Christensen, "User-Centered Design of a Web Site for Library and Information Science Students: Heuristic Evaluation and Usability Testing," Information Technology & Libraries 25, no. 3 (2006): 163, doi: 10.6017/ital.v25i3.3348.

16. Battleson, Booth, and Weintrop, "Usability Testing of an Academic Library Web Site," 190.

17. Ibid.

18. Carole A. George, "Usability Testing and Design of a Library Website: An Iterative Approach," OCLC Systems & Services 21, no. 3 (2005): 178, doi: 10.1108/10650750510612371.

19. Laura Cobus, Valeda Dent, and Anita Ondrusek, "How Twenty-Eight Users Helped Redesign an Academic Library Web Site," Reference & User Services Quarterly 44, no. 3 (2005): 234-35.

20. Susan McMullen, "Usability Testing in a Library Web Site Redesign Project," Reference Services Review 29, no. 1 (2001): 13, doi: 10.1108/00907320110366732.

21. Ibid.

22. John Bauer-Graham, Jodi Poe, and Kimberly Weatherford, "Functional by Design: A Comparative Study to Determine the Usability and Functionality of One Library's Web Site," Technical Services Quarterly 21, no. 2 (2003): 34, doi: 10.1300/J124v21n02_03.

23. Cockrell and Jayne, "How Do I Find an Article?," 123.

24. Maaike Van Den Haak, Menno De Jong, and Peter Jan Schellens, "Retrospective vs. Concurrent Think-Aloud Protocols: Testing the Usability of an Online Library Catalogue," Behavior & Information Technology 22, no. 5 (2003): 339.

25. Battleson, Booth, and Weintrop, "Usability Testing of an Academic Library Web Site," 192.

26. Dominique Turnbowet al., "Usability Testing for Web Redesign: A UCLA Case Study," OCLC Systems & Services 21, no. 3 (2005): 231, doi: 10.1108/10650750510612416.

27. Cobus, Dent, and Ondrusek, "How Twenty-Eight Users Helped Redesign an Academic Library Web Site," 234.

28. Krug, Don't Make Me Think! 59.

29. Nielsen, Designing Web Usability, 164.

30. Ibid., 111.

31. Schneiderman, Research-Based Web Design and Usability Guidelines, 160.

32. Don Zimmerman and Dawn Bastian Paschal, "An Exploratory Evaluation of Colorado State Universities Libraries' Digital Collections and the Western Waters Digital Library Web Sites," Journal of Academic Librarianship 35, no. 3 (2009): 238, doi: 10.1016/j.acalib.2009.03.011.

33. Robert J. Nathan, Paul H. P. Yeow, and Sam Murugesan, "Key Usability Factors of Service-Oriented Web Sites for Students: an Empirical Study," Online Information Review 32, no. 3 (2008): 308, doi: 10.1108/14684520810889646.

Appendix

(ProQuest: Appendix omitted.)