Usability evaluation of an academic library website: Experience with the Central Science Library, University of Delhi

Abstract

Purpose

- The paper aims to evaluate the usability of the website of Central Science Library (CSL), University of Delhi. Multi-method approach of evaluation is used with the use of standard checklist and questionnaire survey of representative users. Besides, the information architecture of the website under study is analysed in this paper.

Design/methodology/approach

- Based on the literature review and the author's experience, a usability assessment tool for library websites was developed by Pant (2013). It comprises standard checklist and questionnaire for users' survey. In the present study, this framework was used to assess the usability of CSL website. The questionnaire survey of 35 representative users was conducted through random sampling. The results of both methods (standard checklist and questionnaire survey) were analysed for evaluating the website usability.

Findings

- Need for improvement of the website was realised in terms of efficiency, effectiveness and learnability for better usability. Notice board, site search facility, list of services, FAQs and user guides were the most sought after features among others as per the analysis of the questionnaire survey. Besides, the need to enhance the visual appeal of the website was felt. However, information resources provided through the CSL website were found useful for users.

Originality/value

- The paper presents an innovative multi-method approach of website usability assessment while considering six usability attributes: Usefulness, Efficiency, Effectiveness, Learnability, Satisfaction and Accessibility. The approach adopted in this paper is cost effective in comparison to formal usability tests and heuristic evaluation. Therefore, this framework is suitable for libraries having limited budget to ensure the user-centred library website with maximum usability. This paper encourages other libraries to conduct similar website usability evaluation to identify the usability problem areas and users' perception for their respective website.

Introduction

University libraries have the challenge of meeting information needs of diverse users. Their users include undergraduate students, graduate/postgraduate students, research scholars and faculty members. In traditional library systems, face-to-face interactions between library staff and users facilitate an understanding of the needs and skills of the individual user that is not present in the virtual world. With the increasing influence of the Internet in day-to-day life, libraries have recreated their presence on the web in the form of library websites. Users can now visit the library electronically using library websites. In the web environment, the challenge for libraries is to provide access to quality content in electronic form, promoting better visibility for their print resources, as well as offering various value-added electronic services. To achieve this, academic libraries are making available e-journals, e-books, electronic databases (full-text and bibliographic), digitised collections, OPAC, virtual information about the library and enabling online feedback and requests through their websites. Besides the content, website design should also meet user expectations. The overall information architecture of a library website should facilitate easy access to its resources by the users. User satisfaction can be achieved only if a library website conforms to the concept of usability .

Statement of the problem

In addition to offering quality content and services, the academic website should provide a user-centred interface. Website usability is a key aspect of the user-centred websites. The purpose of the present research work is to analyse the usability of the website of the Central Science Library (CSL), University of Delhi. It further aims to identify problem areas on the website in terms of usability and to identify user expectations of the website.

Usability

Usability is a quality attribute of a system which assesses the user interface of the system for its ease of use by the users. ISO standard 9,241-11 Guidance on Usability (1998) defines usability as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use" (Usabilitynet.org, 1998). According to Rubin and Chisnell (2008), a system is usable when the user can do their intended task without any frustration. Furthermore, "to be usable, a product or service should be useful, efficient, effective, satisfying, learnable, and accessible". Library websites are said to be usable if their content and services meet users' expectations; users can complete the task quickly with a minimum errors and users feel satisfied after using the website; the process to accomplish a task is easy to learn; and the website is accessible to users with disabilities or under different technical conditions.

Central Science Library

The CSL, University of Delhi, is a prominent library under the Delhi University Library System. It serves the departments and centres of science streams of the University of Delhi under the Faculty of Science and Mathematical Science. It is well equipped with ICT infrastructure and provides an environment conducive for learning, teaching and research. The library has a large collection of books and bound journals in various science disciplines. More than 300 scientific periodicals are currently subscribed to by the CSL. In addition, the CSL has online access to a number of abstracting and indexing databases, full-text databases and thousands of e-journals through UGC-INFONET Digital Library Consortia and subscriptions by the University of Delhi. Total membership of the CSL for the academic year 2012-2013 was 3,031 members (CSL Annual Report, 2012-2013).

Literature review

This section highlights previous studies related to various usability attributes, usability criteria for evaluating library websites and digital libraries and case studies involving usability evaluation. Some of the notable works on website usability and its attributes and methods to evaluate the system usability are discussed here. Nielsen (1993, 2012) defined usability as a quality attribute which assesses a user interface for its ease for use. According to Nielsen, usability has five components: learnability, efficiency, memorability, errors and satisfaction. He emphasised the importance of usability and how to improve it. He asserted that an iterative design process should be adopted based on testing the system with five users at a time and revising the design to fix the usability flaws identified in each round of user testing. Folmer and Bosch (2004) described the concept of usability and gave an overview of the methods for evaluating usability. They provided the framework to evaluate the usability of the software at an architectural level.

Some authors focused on usability attributes of digital libraries and library websites and methods for their evaluation. Raward (2001) advocated for a user-centred design model for academic library web pages. He proposed a usability checklist, developed from human-computer interface (HCI) principles, for designing academic library websites. Buchanan and Salako (2009) provided a measurement framework for evaluating digital libraries based on key attributes identified for system usability and usefulness. The authors conducted a pilot study using a multi-method approach of questionnaire survey and observation. Hariri and Norouzi (2011) reviewed the literature to identify various attributes and evaluation criteria for digital libraries. They suggested evaluation criteria for user interfaces for digital libraries. Joo and Lee (2011) developed a usability measurement instrument for assessing the usability of academic digital libraries through the survey method. Four usability dimensions: efficiency, effectiveness, satisfaction and learnability were used to develop the instrument with some items in each dimension. Reliability and validity of the instrument was tested through a survey of real users. Joo et al. (2011) developed a survey tool as a usability evaluation model based on three usability dimensions: efficiency, effectiveness and learnability. The evaluation tool consisted of 18 measurement items to judge the three dimensions of usability. The evaluation tool was tested for reliability and validity through a survey of real users. Rocha (2012) proposed a website quality evaluation framework based on three dimensions of websites: content quality, service quality and technical quality. This paper suggested the use of the methodologies based on a Likert scale to evaluate these quality dimensions.

Case studies in the area of usability evaluation of digital libraries and library websites are reviewed in this section. Gullikson et al. (1999) assessed the information architecture of the Dalhousie University website using a task-based analysis of participant performance. A short questionnaire examined user perceptions of the site. Recommendations were made for better information architecture. McGillis and Toms (2001) assessed the usability of an academic library website and studied how faculty and students completed typical tasks given to them using the library website. Thirty-three users were tested and they completed 75 per cent of the given tasks. However, the users experienced difficulties in understanding the site's information architecture. Avouris et al. (2003) used three evaluation techniques: questionnaire, user observation and the heuristic evaluation method to evaluate the usability of a departmental website of the University of Patras. On the basis of the findings, a new website was designed and was further evaluated using the heuristic evaluation method. George (2005) discussed the various methods used for collecting feedback from users and the use of a usability study for redesigning the library website of the Carnegie Mellon University Libraries. A web-based survey was used to determine user needs and the think-aloud protocol was used to determine the strengths and weaknesses of the final design of the website.

King and Jannik (2005) described the issues faced by the Georgia Institute of Technology Library with its website in the past. They also described the usability testing of the library website performed to keep it relevant and current by incorporating feedback from the users to meet their needs. Turnbow et al. (2005) discussed the use of structured analyses. They also described user surveys, card sort protocol and think-aloud protocol used to gather information to redesign the UCLA Library website. Jeng (2005) proposed a usability evaluation model for academic digital libraries. Task-based evaluation of two academic library websites was carried out on the basis of effectiveness, efficiency, satisfaction and learnability, which were found to be interrelated with each other. Rogers and Preston (2009) described the usability evaluation of the website of the Main Library of the St Augustine campus, University of the West Indies (UWI), through survey questionnaires, focus groups, formal usability testing and card sort method. The project identified the major strengths and weaknesses of the site by the users and site visitors and emphasised the incorporation of the findings into a redesign. Kalra and Verma (2011) evaluated the usability and usefulness of library websites of selected research institutions in India. They used multiple evaluation methods, including web impact factor, pre-defined checklist of indicators and online questionnaire survey. Methodologies used at an international level for evaluating usability were discussed. Becker and Yannotta (2013) conducted iterative usability testing using the think-aloud protocol during the building of the new library website of the Hunter College Libraries. Task-oriented questions were given to the participants. They advocated that usability testing throughout the redesigning phase led to a user-centred library website.

The above literature review helps to understand usability and its various attributes. The information also reveals that researchers used different methods of usability evaluation. There is no fixed criterion for usability evaluation as various authors proposed different frameworks for library website evaluation. Use of formal usability testing (think-aloud protocol and focus groups) is discussed in many of the case studies. User surveys, user observation and heuristic evaluation methods were also used to evaluate the usability of library websites and digital libraries. On analysing the literature under review, it is noted that a usability evaluation method using a standard checklist and questionnaire as a multi-method approach can be developed and used for library website evaluation. Various usability attributes were explored and identified in the literature review as the evaluation criteria for library websites.

Objectives and research questions

The objectives of the present study are as follows:

To understand the information architecture of the website of the CSL, University of Delhi;

To test the usability of the CSL website using the standard checklist of website usability measures/criteria developed for evaluation of library websites;

To test the usability of the CSL website through a survey of the library website users using the questionnaire developed for the purpose; and

To provide recommendations for enhancing the usability of the CSL website.

To achieve the desired objectives, it is important to pose some research questions. This study aimed to answer following research questions:

RQ1. Is the overall information architecture of the CSL website logical and user-centred to support website usability?

RQ2. What are the key usability measures/criteria where the CSL website is falling short to fulfil overall website usability?

RQ3. How do CSL website users experience the website in getting their required information?

RQ4. How well did the CSL website meet the expectations of its users?

RQ5. What are the most sought after features required by the CSL website users?

Methodology

Overview

For the present study, the information architecture of the CSL website was studied to provide an overview of information available and its availability (logical arrangement) on the website. Further, the usability of the website was studied using a standard checklist and questionnaire survey. The survey was conducted through random sampling of library users. The standard checklist and questionnaire for assessing the usability of library websites were developed by Pant (2013) on the basis of a literature review, the author's experience and discussing the issue with the library staff. This framework uses the six attributes of usability: usefulness, efficiency, effectiveness, learnability, satisfaction and accessibility, as suggested by Rubin and Chisnell (2008) for categorising the overall usability of a library website.

Information architecture of the CSL website

The information architecture of the CSL website was studied in terms of website home page design; information categories; site navigation and links; and content of the website. The URL for the site under study is http://csl.du.ac.in/Old-index.htm . The home page is broadly divided into three sections: banner, body (main content) and footer (having contact information and site hit counter). The body portion of the website is divided into six blocks (major categories): public domain, electronic information resources, federated/common search engines, public search engines, about the library and clientele (Figure 1). The content (navigation links) of each block (major category) is given below:

Public domain

E-Resources on Public Domain.

Digital Library.

Google Scholar.

Google Books.

PubMed Central.

Scirus.

Some Useful Sites.

E-Resource's User Statistics 2010-11 2008 2007.

DU Makes it to List of World's Best.

Electronic information resources

SciFinder Web Version Registration * New.

SciFinder Help Files.

SciFinder Login.

E-Resources at a Glance.

SEARCH A to Z E-Journals (UGC-INFONET).

Electronics Journals & Databases * Latest updated.

Subject-Wise Electronic Resources.

CSL Serials Holdings.

Periodicals Subscription 2012 2011 2010 2009.

Recent Addition of Books 2011-12 2009-2010.

UGC-InfoNet Digital Library Consortium.

DELNET.

NISCAIR.

Federated/Common search engines

Web OPAC (Online Public Access Catalogue) For Books.

Searchable A-Z List of E-journals.

JCCC@UGC-INFONET (ILL Service) for Journals.

Gist Find for Journals.

Public search engines

Google.

Yahoo.

Bing.

Wikipedia.

About library

Profile.

User Orientation Programme.

Annual Reports 2011-12 2010-11 2009-2010 2008-2009.

Status Reports 2011 2010 2009 2008.

Assessment of Requirement of XI Plan.

Clientele

Researchers.

Departments.

Staff.

Suggest-a-Book.

Contact.

Navigation links are provided from the home page to the second-level pages. The second-level pages consist of 17 HTML pages, 17 PDF pages and one PHP web page. Seventeen outside links are provided to other websites from the home page. Of these outside links, two links (WebOPAC and Searchable A-Z List of E-journals) are in the University of Delhi domain, as these services are provided centrally by the Delhi University library system to all of its branches. Similarly, subsequent level web pages provide links to next lower order web pages or outside links. Some links within the website are outdated which results in dead links. It is evident from the home page that vague headings are given to some of the major categories, such as "Federated/Common Search Engines", "Public Search Engines" and "Clientele", that are not reflective of their intended purpose. Also, the headings of the content given for navigation is poorly arranged under six major categories. For example, "Year-wise E-Resource's User Statistics" is listed under the "Public Domain" category; "Year-wise Recent Addition of Books" is given under the "Electronic Information Resources" category; the "Wikipedia" link is given under "Public Search Engines" category; and "Staff", "Suggest-a-Book" and "Contact" links are given under the "Clientele" category. Even in the second-level pages (one click from home page), further vague categorisation of information resources in an unstructured form is found. Some useful links are given in the website, such as "Thesis and dissertation", "Conference alerts", "Online free books", "E-reference sources" and many more. However, these links are not organised under well-structured categories and there are no proper navigation links provided to direct the users to these resources through the home page. Hence, these were not utilised properly by the library website users.

Standard checklist

A standard checklist consisting of 38 statements was used for evaluating the usability of the CSL website. These statements were categorised into five usability attributes: usefulness, efficiency, effectiveness, learnability and accessibility. Each statement of the checklist was checked to identify whether it is "true", "false" or "not applicable" for the CSL website (Table I).

In the above checklist, three statements were not applicable to the website under study, as the items were not available on the website. It is evident from the checklist that out of 35 qualified statements, the website only fulfilled 15 statements. Twenty statements were not fulfilled by the website. These statements were further analysed on the basis of the five usability attributes. It was found that the website fulfilled six statements out of fourteen statements related to site usefulness; it fulfilled two statements out of five statements related to site efficiency; it did not fulfil any statement out of the three statements related to site effectiveness; it fulfilled two statements out of six statements related to site learnability; and it fulfilled five out of seven statements related to site accessibility.

There are some statements in this checklist which are further explained for better understanding. Statements 12, 13 and 14 are included in the "usefulness" attribute of website usability because, nowadays, it is recommended that web forms should be provided on websites. With the help of web forms, the users can communicate with the library staff using the website interface without needing to go to their e-mail account. This saves time and effort on the part of the user. Web forms can be of several types, for example, for sending feedback or for asking questions and getting help from the library staff related to information resources and library services. In the "efficiency" attribute of usability, Statement 1 "Is the website easy to use for a normal user?" is included and has several metrics. These metrics should be evaluated on a case-by-case basis as for a particular website some of them may be relevant while others may not apply. These metrics are more or less closely related to website accessibility and learnability. They affect the efficiency of the website to a certain degree, particularly for users having fewer computer skills required for easier web browsing. Some of these skills are increasing/decreasing the font size of the web page, moving back to a previous web page, use of the page up/page down key on the keyboard and so forth. For the average user, a library website should provide the following features depending on their relevance to the context: provision for maximising/minimising the font size of the web page, provision for navigation back to a previous web page, navigation back to the home page, provision for navigation up and down within a long web page, the website should not block the back button of web browser, clear information about the place where users are at present and use of different colours for hyperlinks visited and unvisited.

Questionnaire survey

A questionnaire survey of 35 CSL website users was conducted to collect feedback from them. The aim was to identify usability problem areas of the CSL website. The questionnaire consisted of 28 Likert items on a five-point rating scale to make the qualitative data quantifiable. The questions were related to six usability attributes: usefulness, efficiency, effectiveness, learnability, accessibility and satisfaction. Some multiple choice selection and open-ended questions were also given. Basic information on users was also gathered but was not included in the data analysis. The questionnaire was pre-tested with five users.

Data analysis

The qualitative data gathered through the Likert items in the questionnaire survey were analysed using the five-point rating scale, with 5 points given to the most positive response. Mean and standard deviation were calculated for each individual question. The number of respondents for each question is represented by 'N'. Six usability attributes: usefulness, efficiency, effectiveness, learnability, accessibility and satisfaction, were analysed on the basis of participant responses. Results are summarised in Tables II to VII.

Usefulness

Nine questions related to site usefulness were asked of the users. Table II shows the means and standard deviations calculated for each question based on user responses. Most of the users rated the electronic resources available through the website as either good or excellent with a mean score > 4. However, the response for the e-reference sources provided through the site was either average or good with a mean score = 3.54. This may be due to the disorganisation of reference sources on the website or less usage of these resources. E-reference sources were not subcategorised in the questionnaire for rating. Most of the users rated the information given about the library as average with mean score of 3.40. Services provided through the site were also rated average with a mean = 3, although some of the users (23 per cent) also rated it as poor. Help for resources available through the site received a mean score of 2.71. Of all, 46 per cent of users responded that Help was poor while 37 evaluated it as average. Services and help for resources were found to be two weak areas of the CSL website in terms of site usefulness.

Efficiency

Three questions related to site efficiency were asked to the users. Table III shows the means and standard deviations calculated for each question based on user responses. Mean scores for all three questions are between poor and average. Most of the users rated the easiness of the website to perform tasks and number of clicks to get required information as either average or poor with a mean score of 2.91 for both questions. However, some of the users rated these questions as good, which might be due to their individual information needs and skills. Users responded to the question related to the ease of information architecture of the website for finding information as either poor (46 per cent) or average (54 per cent) with a mean score of 2.54.

Effectiveness

Three questions related to site effectiveness were asked. Table IV shows the means and standard deviations calculated for each question. Users treated the CSL website as a good primary source for fulfilling their academic information needs with a mean score of 3.78. Questions related to site effectiveness in terms of error handling received a dispersed set of responses. Of all, 34 respondents replied to the question asking "How often are you successful in finding the academic resources using this site without getting an error?" with a result of 9 per cent responding it was poor, 38 per cent labeled it as average, 29 per cent categorised it as good, while 24 per cent called it excellent with a mean score of 3.68 and a standard deviation of 0.94. For the question asking "How often do you perform the tasks using this site without getting an error?", there were 31 respondents, 16 per cent of users responded it was poor, 29 per cent gave it an average rating, 26 per cent labelled it as good, while 29 per cent responded it was excellent. These results provide a highly dispersed response set with a mean score of 3.68 and a standard deviation of 1.08. Receiving errors while using the website might be due to a variety of reasons which could result in a dispersed user response. The information architecture of the website (what information is located where), particular information that users want, individual skills of users to search and technical issues, such as network availability, presence of dead links, missing links or inappropriate links, can all cause errors while using the website.

Learnability

Four questions related to learnability were asked next. These were about terminology used on the website, confidence level of users in using the website, information architecture and site navigation. Table V shows the means and standard deviations calculated for each question based on responses. It is evident that most of the users rated the questions between average and good. Of 33 respondents, 36 per cent responded that their confidence level is good in using the website, while 48 per cent responded their confidence as average. Of 35 respondents, 34 per cent rated information architecture as poor with a mean score of 2.91, while 37 per cent rated site navigation as poor with a mean score of 2.80. Most of the respondents (79 per cent of the 33 respondents) rated the terminology used in the website as average with a mean = 3.21.

Satisfaction

Table VI presents the results for the four questions related to user satisfaction with the website. Most of the users responded that they are keen to recommend this site to their peers with a mean score = 3.66. It is evident from Table VI that for the rest of three questions in this category, most users rated them as average with a mean value very close to 3.

Accessibility

Five questions related to site accessibility were asked of the users and are presented in Table VII. The CSL website was found to be generally accessible in terms of ability to load pages without getting errors on the site (found to be excellent with a mean of 4.74). Speed with which pages load on the site was categorised as good (mean = 3.89). Consistency of speed from page-to-page on the site received a mean of 3.85, while the ease of reading the pages on the site resulted in a mean of 3.77, putting both in the "good" category. However, the visual appeal of the website was rated between poor and average by the respondents (mean = 2.60).

Users were asked to mention their usage of the website for various purposes, as follows: to find e-journal articles, to find books, to contact library personnel, to use interlibrary loan and others. Most of the users frequently used the website to find e-journal articles and books. However, none of the 28 respondents used the website for interlibrary loan. No respondents mentioned any other purpose for their use of the website. Figure 2 provides a comparative view of users' responses.

An open-ended question was asked about the time taken to find answers for specific queries using the content (i.e. searching for information on the website). Users responded to this question with various time intervals, with the mean being around 7 minutes.

A multiple choice selection question was asked to the users about "How do they find e-journal articles?". Twenty-eight users responded to this question and their responses are shown in Figure 3. Almost all users used Google to search for e-journals articles, while 23 users (out of 28 respondents) used the library website to search for e-journals articles. In the column for other search services, some users mentioned the names of different scientific publishers for direct searching of the articles from their websites. These responses were highly dispersed and were not used for data analysis.

Users were asked to name educational websites related to their academic interests which they had visited within the past month. In response to this question, most of the users named institution websites of the CSIR and UGC and scientific publisher websites, such as Wiley, Springer, Elsevier and so forth.

Important information was gathered through a multiple choice selection question asked to the users about the additional features they would like to have on the CSL website. Message board/notice board, site search facility, list of services, FAQs and user guides were the most sought after features. The responses of the users to this question are given in Table VIII.

Research findings, recommendations and implications

It is evident from the data analysis of the questionnaire survey that most of the users rated some of the questions as average, with a mean score close to 3. Even this type of response is seen for the questions which violate the usability as reflected in the usability checklist, especially in the case of the effectiveness attribute. Kelly et al. (2008, as cited in Buchanan and Salako, 2009) gave the explanation for such positive responses from the users. According to the authors, users have a tendency to rate the questions in a questionnaire more positively due to their tendency to agree with the attitude statements given in the questionnaire and/or their perception that it is required to behave in a particular way and/or sometimes users assume that their abilities are being evaluated through the questionnaire rather than that of the system's ability to perform. To overcome this situation, open-ended and multiple choice selection questions were posed in the questionnaire. It was postulated that the results of the usability checklist comparison and data analysis of the questionnaire survey responses need to be analysed together to have a clearer understanding about the usability of the CSL website.

Keeping in mind the desired objectives and research questions posed for conducting the present study, this section describes the major findings and recommendations. The major findings of the research work, derived from the results of data analysis of the questionnaire survey responses and usability checklist, are listed below:

The overall information architecture of the website under study is in an unstructured form. Vague headings are given to the content categories and navigation labels. This creates confusion and misleads the users. Navigation is not intuitive and is tedious. Information architecture gravely impacts the users' ability to find particular information with the minimum time spent and minimum errors found. Moreover, the information architecture of the website is not logical to learn;

Information resources provided through the CSL website were found to be useful for users. However, not enough help is provided for resources available through the website;

There is a need for substantial usability improvement of the website in terms of efficiency, effectiveness and learnability;

Users preferred the website as a primary resource for getting information for their curriculum/research needs. At the same time, the website does not fully meet their expectations of an ideal website;

Notice board, site search facility, list of services, FAQs and user guides were the most sought after features, as indicated by the responses to the questionnaire survey; and

A need to enhance the visual appeal of the website was expressed.

Recommendations for enhancing the usability of the CSL website are as follows:

The overall information architecture of the CSL website needs to be restructured. This includes: reorganisation of information, restructuring the navigation menu, renaming the navigation labels and renaming the headings of the content. Information architecture should be standardised, keeping in view the users' choice and terminology that is familiar to users. Navigation labels should be explicitly related to the underlying concepts. Content categorisation should be distinct and easily understandable to the users.

The most preferred features suggested by the users should be provided on the website as a priority;

Provision of a good site search facility will enhance the efficiency and the effectiveness of the website;

Whenever the content of the website is updated, that date should be reflected in the website as a date of last updated;

Web forms should be provided for sending feedback, asking queries and getting help;

Help should be provided for resources available through the website;

The visual appeal of the website needs to be enhanced by using appropriate background colours, a proper banner, use of relevant images and providing clear and distinctive navigation buttons; and

Usability evaluations should be conducted regularly to keep the library website up to the expectations of its users.

Based on the research findings and recommendations made, the website under study was redesigned by the researcher to enhance the usability of the CSL website. The details of the website redesign are beyond the scope of this paper. However, the image of the redesigned home page of the CSL website is given below in Figure 4 to show the implications of this research study.

Discussion and conclusion

Usability is the key aspect of user-centred websites. Usability is considered as a subjective quality attribute of the system (Folmer and Bosch, 2004). It depends heavily on the perceptions of the individual user about the system under consideration. There is no fixed criterion for usability evaluation. It is evident, however, from the available literature that use of formal usability testing (think-aloud protocol and focus group) is discussed in many of the case studies. User surveys, user observation and heuristic evaluation methods were also used to evaluate the usability of library websites and digital libraries. In user survey-based research works, different usability attributes were undertaken for developing the measurement framework and conducting the research.

The present research work assesses the usability of the website of CSL, University of Delhi, in an objective manner. It further identifies the problem areas on the website in terms of usability and outlines user expectations with the website. A usability evaluation method using a standard checklist and questionnaire survey of representative number of users as a multi-method approach was used. From the data analysis, it is evident that users preferred the website as a primary resource for getting the information for their curriculum and/or research needs. However, there is much work to do to make the CSL website meet the expectations of its users. The overall information architecture of the CSL website is not logical or intuitive and needs to be restructured. It creates confusion and misleads users. It is somewhat difficult to find particular information within the website with a minimum of time spent (efficiency) and with a minimum of errors found (effectiveness). It was found that substantial changes are required in the website to improve the efficiency, effectiveness and learnability, as well as the visual appeal of the website. Help should be provided for resources available through the website for improved utilisation. Notice board, site search facility, list of services, FAQs and user guides were the most sought after features suggested for inclusion on the CSL website.

However, this research has some limitations. The survey of the representative number of users was limited to 35 respondents. It did provide useful feedback related to user perceptions on various aspects of website usability and the additional features they would like to have on the CSL website. The significance of this research work is twofold. First, it had practical implications for the library website under study, as the site was redesigned on the basis of the research findings and the recommendations made. Moreover, the present study is evidence based; therefore, the methodology used in this study might be helpful for other libraries to conduct website usability evaluations to identify usability problem areas and user perceptions of their respective websites. Second, as a methodological contribution of the present study, the approach adopted in this paper, with the use of a standard checklist and questionnaire survey of the users, is cost effective in comparison to formal usability tests and heuristic evaluations. The framework used in this study makes it suitable for libraries having limited budgets to ensure that they develop a user-centred library website with maximum usability. It is highly recommended that such usability evaluations be conducted regularly to keep the library website up to the expectations of its users. The author believes that this research work will serve as an example for further development of measurement frameworks to assess library website usability based on a survey of the users.

Figure 1. Screenshot of CSL website home page

[Image omitted: See PDF]

Figure 2. Purpose of using the CSL website

[Image omitted: See PDF]

Figure 3. Search behaviour of users to find e-journal articles

[Image omitted: See PDF]

Figure 4. Home page of the redesigned CSL website

[Image omitted: See PDF]

Table I. Checklist for usability evaluation

[Image omitted: See PDF]

Table II. Questionnaire results for questions related to the usefulness attribute

[Image omitted: See PDF]

Table III. Questionnaire results for questions related to the efficiency attribute

[Image omitted: See PDF]

Table IV. Questionnaire results for questions related to the effectiveness attribute

[Image omitted: See PDF]

Table V. Questionnaire results for questions related to the learnability attribute

[Image omitted: See PDF]

Table VI. Questionnaire results for questions related to the satisfaction attribute

[Image omitted: See PDF]

Table VII. Questionnaire results for questions related to the accessibility attribute

[Image omitted: See PDF]

Table VIII. Users' choice for additional features to be provided on the CSL website

[Image omitted: See PDF]

About the author

Ankur Pant has a Bachelor's and Master's Degree in Library and Information Science from University of Delhi. He has completed a PGDLAN from IGNOU through distance learning. During his short professional career, he has been instrumental in applying the latest technologies into librarianship. He has worked for Delhi University Library System, University of Delhi; Delhi University Computer Centre; and currently is working in the Defence Scientific Information and Documentation Centre (DESIDOC), Delhi. His areas of interest are digital libraries and its services, information literacy and website design. Ankur Pant can be contacted at: [email protected]

The author wishes to acknowledge the generous support provided by Dr S.C. Jindal, CSL, to conduct the present research work smoothly. He also acknowledges the support provided by Ms Darshana, staff member, CSL, in conducting the questionnaire survey.

References

References

Avouris, N., Tselios, N., Fidas, C. and Papachristos, E. ( 2003 ), " Website evaluation: a usability-based perspective ", 8th Panhellenic Conference on Informatics, PCI 2001, LNCS 2563, Springer-Verlag, Nicosia, pp. 217 - 231 .

Becker, D.A. and Yannotta, L. ( 2013 ), " Modeling a library website redesign process: developing a user-centered website through usability testing ",Information Technology and Libraries , Vol. 32 No. 1, pp. 6 - 22 .

Buchanan, S. and Salako, A. ( 2009 ), " Evaluating the usability and usefulness of a digital library ",Library Review , Vol. 58 No. 9, pp. 638 - 651 .

Central Science Library Annual Report (2012- 2013 ), " Central Science Library web site ", available at: http://csl.du.ac.in (accessed 6 February 2014).

Folmer, E. and Bosch, J. ( 2004 ), " Architecting for usability: a survey ",The Journal of Systems and Software , Vol. 70 Nos 1/2, pp. 61 - 78 .

George, C.A. ( 2005 ), " Usability testing and design of a library website: an iterative approach ",OCLC Systems & Services , Vol. 21 No. 3, pp. 167 - 180 .

Gullikson, S., Blades, R., Bragdon, M., McKibbon, S., Sparling, M. and Toms, E.G. ( 1999 ), " The impact of information architecture on academic web site usability ",The Electronic Library , Vol. 17 No. 5, pp. 293 - 304 .

Hariri, N. and Norouzi, Y. ( 2011 ), " Determining evaluation criteria for digital libraries' user interface: a review ",The Electronic Library , Vol. 29 No. 5, pp. 698 - 722 .

Jeng, J. ( 2005 ), " Usability assessment of academic digital libraries: effectiveness, efficiency, satisfaction, and learnability ",Libri , Vol. 55 Nos 2/3, pp. 96 - 121 .

Joo, S. and Lee, J.Y. ( 2011 ), " Measuring the usability of academic digital libraries: instrument development and validation ",The Electronic Library , Vol. 29 No. 4, pp. 523 - 537 .

Joo, S., Lin, S. and Lu, K. ( 2011 ), " Usability evaluation model for academic library websites: efficiency, effectiveness and learnability ",Journal of Library and Information Studies , Vol. 9 No. 2, pp. 11 - 26 .

Kalra, J. and Verma, R.K. ( 2011 ), " Evaluation indicators of library websites of selected research institutions in India ",Annals of Library and Information Studies , Vol. 58 No. 2, pp. 139 - 150 .

King, H.J. and Jannik, C.M. ( 2005 ), " Redesigning for usability: information architecture and usability testing for Georgia Tech Library's website ",OCLC Systems & Services , Vol. 21 No. 3, pp. 235 - 243 .

McGillis, L. and Toms, E.G. ( 2001 ), " Usability of the academic library web site: implications for design ",College & Research Libraries , Vol. 62 No. 4, pp. 355 - 367 .

Nielsen, J. ( 1993 ),Usability Engineering , Academic Press, Cambridge, MA .

Nielsen, J. ( 2012 ), " Usability 101: introduction to usability ", Nngroup.com, available at: www.nngroup.com/articles/usability-101-introduction-to-usability/ (accessed 8 June 2014).

Pant, A. ( 2013 ), " Development of usability assessment tool for library website evaluation ", paper presented at International Conference on Digital Libraries, 27-29 November, The Energy and Resources Institute, New Delhi .

Raward, R. ( 2001 ), " Academic library website design principles: development of a checklist ",Australian Academic & Research Libraries , Vol. 32 No. 2, pp. 123 - 136 .

Rocha, A. ( 2012 ), " Framework for a global quality evaluation of a website ",Online Information Review , Vol. 36 No. 3, pp. 374 - 382 .

Rogers, R. and Preston, H. ( 2009 ), " Usability analysis for redesign of a Caribbean academic library web site: a case study ",OCLC Systems & Services: International Digital Library Perspectives , Vol. 25 No. 3, pp. 200 - 211 .

Rubin, J. and Chisnell, D. ( 2008 ),Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests , 2nd ed., Wiley, Indianapolis, IN .

Turnbow, D., Kasianovitz, K., Snyder, L., Gilbert, D. and Yamamoto, D. ( 2005 ), " Usability testing for web redesign: a UCLA case study ",OCLC Systems & Services , Vol. 21 No. 3, pp. 226 - 234 .

Usabilitynet.org ( 1998 ), ISO 9241-11: Guidance on Usability (1998 ), available at: www.usabilitynet.org/tools/r_international.htm #20282 (accessed 10 July 2013).