Reference Chatbots in Canadian Academic Libraries

Abstract

Chatbots are "computer agents that can interact with the user" in a way that feels like human-to-human conversation.1 While the use of chatbots for reference service in academic libraries is a topic of interest for both library professionals and researchers, little is known about how they are used in library reference service, especially in academic libraries in Canada. This article aims to fill this gap by conducting a web-based survey of 106 academic library websites in Canada and analyzing the prevalence and characteristics of chatbot and live chat services offered by these libraries. The authors found that only two libraries were using chatbots for reference service. For live chat services, the authors found that 78 libraries provided this service. The article discusses possible reasons for the low adoption of chatbots in academic libraries, such as accessibility, privacy, cost, and professional identity issues. The article also provides a case study of the authors' institution, the University of Calgary, which integrated a chatbot service in 2021. The article concludes with suggestions for future research on chatbot use in libraries.

Headnote

ABSTRACT

Chatbots are "computer agents that can interact with the user" in a way that feels like human-to-human conversation.1 While the use of chatbots for reference service in academic libraries is a topic of interest for both library professionals and researchers, little is known about how they are used in library reference service, especially in academic libraries in Canada. This article aims to fill this gap by conducting a web-based survey of 106 academic library websites in Canada and analyzing the prevalence and characteristics of chatbot and live chat services offered by these libraries. The authors found that only two libraries were using chatbots for reference service. For live chat services, the authors found that 78 libraries provided this service. The article discusses possible reasons for the low adoption of chatbots in academic libraries, such as accessibility, privacy, cost, and professional identity issues. The article also provides a case study of the authors' institution, the University of Calgary, which integrated a chatbot service in 2021. The article concludes with suggestions for future research on chatbot use in libraries.

INTRODUCTION

With the recent launch of artificial intelligence (AI) chatbots (e.g., ChatGPT and Bing-Chat), and Springshare's introduction of a library-focused chatbot product, it seems probable that more libraries will adopt chatbot technology to provide reference services 24/7.2 This article attempts to document these changes by surveying academic libraries in Canada to determine the current popularity of chatbot use in reference service. At this time, most of the chatbots used in the library market are essentially interactive FAQs, where keywords entered by the user return suggestions to pages that may answer the query. However, with the recent emergence of advanced tools, domain-specific enhancements to chatbots leveraging artificial intelligence could cause an explosion of AI chatbot use in the library world.

LITERATURE REVIEW

This research focuses on the use of chatbots by academic libraries in Canada. There are many different terms for chatbots (e.g., digital assistants, conversational agents, etc.), and these terms are used inconsistently in the media and across the literature.3 A chatbot can be defined as a system that simulates a conversation with a human, in real time, using AI technology.4 Chatbots respond to user interactions in sentences "that track the conversation in a meaningful way to humans."5 Although both involve user interactions in a digital space, chatbots are not to be confused with live chat services, which is when a user can converse with a library staff member in a synchronous online session.

Notably, chatbots use natural language processing (NLP) to communicate with users. Broadly speaking, NLP is the "use of computer technology to assist in or complete tasks involving the processing, categorizing, analyzing, or interpreting the meaning of human language."6 In the context of chatbots, NLP has two main functions, natural language understanding (NLU) to interpret inputs and natural language generation (NLG) to produce a language response.7 "Natural language" in this context refers generally to human language, spoken or written, although there is research and advocacy around the further inclusion of signed languages in NLP research and development, including a recent paper by Yin et al.8 Furthermore, NLP also eliminates the use of Boolean operators, traditionally a requirement of library interfaces.

An additional benefit of NLP technologies is that they are intuitively usable for many people, given that users communicate with these systems in the way that humans are used to communicating, through conversation. Due to this minimal barrier to entry for many users, chatbots are increasingly being implemented as online assistants in many different contexts.9 The increasing popularity of NLP technology also means that when users encounter a chatbot in a digital environment, they are likely familiar with how to interact with it. Chatbots are therefore a "cost-effective and accessible supplement to manual customer service."10

With AI technology on the rise, researchers are exploring the implications of integrating these technologies into academic libraries, and several literature reviews have been conducted on this topic over the last few years.11 Noteworthy benefits of chatbots include the potential to free up staff time and the ability for these systems to assist students remotely, 24/7.12

Despite the growing interest in using AI in academic librarianship, as Tait and Pierson observed, much of the existing literature on library applications of AI is predictive and speculates on what the use of AI might look like for libraries in the future.13 For example, research has explored stakeholder's perception of risks associated with library adoption of chatbots in the future as well as the potential benefits of chatbots for millennial users in particular.14

Although there is no shortage of theoretical research, there is very limited literature assessing the extent to which chatbot technologies are currently being employed by academic libraries for reference services, particularly in Canada. Case studies have explored the introduction of chatbots into specific American academic libraries, as well as public libraries, but there do not seem to be comparable studies published for Canadian post-secondary institutions.15 Although research has examined major Canadian research libraries' incorporation of AI in other areas, such as institutional strategic plans and library programming, research is lacking in the field of library adoption of chatbots for reference or customer service purposes in Canada.16

In this way, research on chatbot use in libraries might be lagging behind research investigating the current use of other AI applications in academic libraries. Topics such as the use of AI to analyze transcripts of communication between librarians and users have been examined.17 With recent advances in NLP, such as the launch of ChatGPT, which could have enormous impacts on education overall, it is a good time to shift focus from theorizing about the effects of chatbots in library spaces and begin assessing their use and impact.18 This is especially important given the significant professional and ethical implications these technologies may have for academic librarianship.19

METHODS

Based on the literature review, we identified two key research questions:

1. To what extent, if any, are academic libraries in Canada employing chatbots for reference services?

2. What can be determined about the use of chatbots for reference service in academic libraries in Canada?

An exploratory data analysis approach was used to address the above research questions. Between January and February 2023, the authors visited 106 academic library websites. Libraries were selected by consulting the Canadian Library Directory and filtering for university libraries.20 Information was collected from each library website around the use of chatbots and live chat services offered by each library and recorded in a spreadsheet. To determine whether a chatbot service was offered, the authors began by visually searching the library homepage for a chat text box or icon. If a chat service was not immediately obvious, the authors looked for a webpage outlining library services or equivalent page. If a chat option was visible, the authors interacted with each chat service widget to see if it indicated the use of AI or a human on the other end. If unclear, we asked via the text box. When a chatbot service was offered, we determined the software vendor by viewing the source code of the website. The vendor or platform used to provide live chat service was also recorded.

In addition to determining what kind of online reference services were offered, we also recorded information regarding student population and geographic location of the institution, as well as further details around the chatbot implementation, including how the chatbot is introduced, its name, how long it had been in use (if available), and whether the chatbot engages in user initiated dialogue (UID), in which the interaction begins with a user question, or system directed dialogue (SDD), in which a system prompts a user for information.21 In examining whether a chatbot was used elsewhere on campus, we determined that it was unrealistic to look at every page on the university's website. Instead, we visited each institution's homepage and admissions page, which are common webpages for each institutional website and which have the traffic and volume of user questions to justify a chatbot.

When we reviewed the results of our initial scan, we were surprised by the low occurrences of chatbots on academic library websites. Collecting and analyzing publicly available data on library websites allowed us to efficiently gather data to create an accurate snapshot of chatbot use for a particular time window. This method of data collection does have some limitations, however, including that we do not have information about whether a chatbot was previously used in an academic library and then discontinued, or insights into the reasoning behind a library's decision to use or not use a chatbot for reference. The flexibility of an exploratory approach allowed us to address the later limitation by including a brief case study outlining the reasoning and process used by the authors' institution, the University of Calgary, to integrate a chatbot service. Information delivered in this section was gathered from institutional documentation and authors' firsthand experience. Limitations of this approach are that a single case study is being used to explore this process and that biases may exist because of the authors close relationship to the work.22 Despite these limitations, the use of a single case study can be appropriate when researchers can provide unique insights.23 Therefore, we will include these insights in a case study section to communicate the process taken to launch a chatbot, in the hope that it can supplement our exploration of this topic and be valuable for other researchers.

FINDINGS

Reference Chatbots

After searching Canadian post-secondary library websites for evidence of chatbot reference, we determined that only two out of 106 libraries (1.89%) were currently using a chatbot reference service. Both institutions, the University of Calgary and Mount Royal University, are located in Calgary Alberta. In both cases, the chatbot was created by Ivy.ai, which according to the company is "the leading provider of conversational, artificially intelligent chatbots for higher education."24 At the University of Calgary, Ivy.ai chatbots were also used by units on campus other than the library, such as the registrar.

During our assessment, we looked for trends in the data that might explain why only two academic libraries are currently using a chatbot. The most obvious similarity is that both institutions are in the same city. The proximity between these two institutions likely means greater potential for the sharing of professional practice ideas at localized professional conferences or through personal connections. This proximity may have impacted the clustering of chatbot use in this one city. The fact that both institutions use Ivy.ai is another similarity which might be explained by nearby libraries having a greater awareness of what the other institution is doing, although this is speculation. This trend might suggest that adoption of chatbots may accelerate once precedent has been established and awareness of successful chatbot integration spreads to other institutions. Further research could involve interviewing academic library leadership to compare what factors they consider when it comes to whether or not to integrate chatbot services.

Given the increasing popularity of chatbots, it is surprising that so few academic libraries in Canada are employing chatbots for reference services. The literature offers a few possible explanations for this. According to Kaushal and Yadav, risks associated with accessibility for users, system restrictions, and privacy have slowed the adoption of chatbot technologies in academic libraries.25 We can speculate as to other contributing factors, including lack of familiarity with these technologies among library staff, cost, insufficient need for reference support due to a manageable volume of questions, or that AI companies may not have sought out academic libraries as potential customers in a significant way.

The relatively slow adoption of AI for reference service can also be compared to the library profession's initial hesitancy to incorporate internet searching in library reference. Nelson and Irwin provided a thorough breakdown of the impacts of the rise of internet search on the occupational identity of librarians.26 The authors determined that there can be hesitancy to adopt new technology when a profession has a well-established identity, a technology emerges designed to perform a task that overlaps with that identity, and the method used by that technology contradicts the general idea of how a task should be done by members of that profession.27 When these three factors combine, the professional's "mastery of the existing approach encourages them to devalue solutions that do not match this approach."28 According to Nelson and Irwin, in the early days of the Internet, librarians were very motivated by "the belief that Internet search was not the best way to help patrons."29 Delays in chatbot adoption may be motivated by similar feelings now. Library staff may understandably feel that current practices meet the needs of patrons better than these technologies currently can.

Live Chat Service

Out of 106 academic libraries studied, 78 (73.58%) offered a live chat service, connecting users with library staff online. We found that some libraries give users the option to text questions, but we did not consider this live chat service because it is unclear whether responses to texts happen live or if library staff respond to those queries later, as they might with an emailed question. Of the 78 libraries offering live agent chat reference, there were three predominant platforms in use. Forty-three (53.85%) of the libraries that offered live chat used some version of LibraryHSlp, either individually or in a consortial environment; 18 (21.79%) used LibChatfrom Springshare; and 12 (15.38%) used AskAway, a branded service for libraries in the British Columbian Electronic Library Network (supported by the LibraryHSlp web client). Less common live chat platforms were tawk.to, Zendesk, Facebook Messenger, SkylerAI Live!, and Ivy.ai (Ivy.ai can be used for both live chat and chatbot services), which were used by one institution each.

These findings suggest that the majority of academic libraries in Canada are offering live chat as a remote reference service. Although we do not have data on how popular this approach was before the onset of the COVID-19 pandemic, we can assume that the transition to off-campus learning made this service particularly important for many institutions. One of the main benefits of both live chat services and chatbots is that students can access them from anywhere they have an internet connection. It is possible that live chat services meet the user's need for remote reference support and, thus, institutions do not feel compelled to introduce a chatbot at this time. Further research could explore the relationship between live chat services and the COVID-19 pandemic in greater detail, including how it has impacted decisions around service delivery.

Integrating or Separating Live Agent Chat and Chatbot Reference Services

The two libraries studied that provided both a chatbot and live chat service exemplify two different approaches to providing these services. In the case of the University of Calgary, two different platforms were used to provide the chatbot and live agent chat services. The chat with library staff is provided through Springshare's LibChat platform, and the chatbot is an Ivy.ai system. In contrast, at Mount Royal University, a single Ivy.ai system is used for both, with the chatbot responding after hours and a live chat agent responding the rest of the time.

Interviewing staff at these institutions to determine their reasoning for integrating or not integrating these two services is beyond the scope of this paper. We can, however, speculate as to possible factors that contribute to these decisions. For example, libraries may want separate systems to very clearly distinguish which service is being delivered, so users understand whether they are communicating with a library staff member or a chatbot. Separate systems also give users the option of communicating with a chatbot rather than a human, which previous research suggests may be preferable for some users with relatively straightforward questions.30 A potential benefit of using an integrated system is that the pathway to getting library assistance is always the same for users, and the conversation can be continuous even after it is transferred to a human.

CASE STUDY

In previous sections, we discussed factors that may contribute to an academic library's decision to integrate or not integrate a chatbot on their library website. To pursue this analysis further, the following case study will provide an example of the authors' institution's reasoning and processes for offering a chatbot as part of their reference services.

The incorporation of chatbot technology was a logical next step in the evolution of library service for the University of Calgary Library. Having offered live chat service since 2010, proactive live chat was introduced in November 2018, using a pop-up that invited users to talk to a staff member via instant message when they lingered on a library webpage for more than 30 seconds. This immediately tripled the number of incoming queries. Upon closer analysis of the live chat transcripts, there was a high demand for chat service outside of regular service hours. Additionally, a significant number of questions were frequently asked, for example, basic reference and directional questions such as "What is a peer reviewed article?," "What are your hours?," or "How do I book a workroom?"

During this period, the library's associate university librarian of technology was investigating various methods and technologies to support the staff answering live chat queries. Discussions with University of Calgary IT revealed that the Registrar's Office was developing a request for proposal for a chatbot and the library was able to take advantage of this same initiative in January 2021. A few months later, a team of library staff was formed to work with the software vendor. The library website was scraped for training data, and the vendor worked closely with the library team to test and calibrate the chatbot. In August 2021, the library went live with Taylor Rex (Т-Rex for short), an eponym for the Taylor Family Digital Library. By taking the initial request and triaging the information, the chatbot provided directional and basic reference support and helped library chat operators manage the number of incoming queries.

In this example, the University of Calgary's decision to introduce a chatbot came out of a need to answer FAQs, meet user needs outside of regular operating hours, and assist staff with a large volume of questions. With a similar approach being pursued elsewhere on campus, timing and institutional support helped move the initiative forward. Since introducing the chatbot, live chat statistics have dropped significantly at the University of Calgary and striking a balance between chatbot vs. live chat entrance points across the library's web properties is an ongoing priority.

CONCLUSION AND NEXT STEPS

Our findings suggest that few academic libraries in Canada were using chatbot technologies to provide reference services at the beginning of 2023. Anecdotally, the authors are aware of at least one academic library declaring itself a "bot-free zone" on social media. While it seems very likely that more libraries will adopt this technology in the near future, there may be a backlash against it, and a renewed entrenchment of personalized human services.

This research provides valuable insights into the current state of Al-assisted chatbots in academic libraries in Canada. This information will be useful for library professionals considering implementing this technology in their institutions and for researchers studying the use of AI in library services.

Suggested research areas for the future include similar analysis in different regions and library types, ethical and quality considerations in the use of chatbots for reference service, and professional considerations and implications for these technologies.

Sidebar

About the Authors

Submitted: May 16, 2023. Accepted for Publication: October 12, 2023. Published 18 December 2023.

Footnote

ENDNOTES

1 Ferliana Dwitama and Andre Rusli, "User Stories Collection via Interactive Chatbot to Support Requirements Gathering," TELKOMNIKA Telecommunication, Computing, Electronics and Control 18, no. 2 (2020): 890, https://doi.org/10.12928/TELKOMNIKA.vl8i2.14866.

2 Talia Richards, "Springshare Announces LibAnswers Chatbot," Springshare, February 15, 2023, https://blog.springshare.com/2023/02/15/springshare-announces-libanswers-chatbot/.

3 Michael McTear, Conversational Al: Dialogue Systems, Conversational Agents, and Chatbots (Switzerland: Springer, 2020), 12-13.

4 "The Power of Chatbots Explained," Expert.ai, March 24, 2022, https://www.expert.ai/blog/chatbot/.

5 DeeAnn Allison, "Chatbots in the Library: Is It Time?" Library Hi Tech 30, no. 1 (2012): 95, https://doi.org/10.1108/07378831211213238.

6 Patrick Rafail and Isaac Freitas, "Natural Language Processing," in Foundations, ed. Paul Atkinson etal. (London: SAGE Publications, 2020), https://doi.org/10.4135/9781526421036879118.

7 Sushree Bibhuprada B. Priyadarshini, Amiya Bhusan Bagjadab, and Brojo Kishore Mishra, "A Brief Overview of Natural Language Processing and Artificial Intelligence," in Natural Language Processing in Artificial Intelligence, ed. Brojo Kishore Mishra (Burlington, ON: Apple Academic Press, 2020), 212, https://doi.org/10.1201/9780367808495.

8 Kayo Yin et al., "Including Signed Languages in Natural Language Processing," arXiv preprint, arXiv, https://doi.org/10.48550/arXiv.2105.05222.

9 McTear, Conversational Al, 21.

10 Knut Kvale et al., "Understanding the User Experience of Customer Service Chatbots: What Can We Learn from Customer Satisfaction Surveys?," in Chatbot Research and Design CONVERSATIONS 2020. Lecture Notes in Computer Science 12604, ed. Asbjprn Fplstad et al. (Switzerland: Springer, 2021), 205, https://doi.org/10.1007/978-3-030-68288-0 14.

11 Andrea Gasparini and Heli Kautonen, "Understanding Artificial Intelligence in Research Libraries: Extensive Literature Review," LIBER Quarterly: The Journal of the Association of European Research Libraries 32, no. 1 (2022): 1-36, https://doi.org/10.53377/lq.10934; Asefeh Asemi, Andrea Ko, and Mohsen Nowkarizi, "Intelligent Libraries: A Review on Expert Systems, Artificial Intelligence, and Robot," Library Hi Tech 39, no. 2 (2021): 412-34, https://doi.org/10.1108/LHT-02-2020-0038.

12 Harry E. Pence, "Future of Artificial Intelligence in Libraries," The Reference Librarian 63, no. 4 (2022): 138, https://doi.org/10.1080/02763877.2022.2140741.

13 Elizabeth Tait and Cameron M. Pierson, "Artificial Intelligence and Robots in Libraries: Opportunities in LIS Curriculum for Preparing the Librarians of Tomorrow," Journal of the Australian Library and Information Association 71, no. 3 (2022): 257, https://doi.org/10.1080/24750158.2022.2081111.

14 Vaishali Kaushal and Rajan Yadav, "The Role of Chatbots in Academic Libraries: An Experience-Based Perspective," Journal of the Australian Library and Information Association 71, no. 3 (2022): 215-32, https://doLorg/10.1080/24750158.2022.2106403; Nishad Nawaz and Mohamed Azahim Saldeen, 'Artificial Intelligence Chatbots for Library Reference Services," Journal of Management Information and Decision Sciences 23 (2020): 442-49, ProQuest One Business.

15 Sharesly Rodríguez and Christina Mune, "Uncoding Library Chatbots: Deploying a New Virtual Reference Tool at the San Jose State University Library" Reference Services Review 50, no. 3/4 (2022): 392-405, https://doi.org/10.1108/RSR-05-2022-0020; Joseph Vincze, 'Virtual Reference Librarians (Chatbots)," Library Hi Tech News 34, no. 4 (2017): 5-8, https://doi.org/10.1108/LHTN-03-2017-0016.

16 Amanda Wheatley and Sandy Hervieux, "Artificial Intelligence in Academic Libraries: An Environmental Scan," Information Services & Use 39, no. 4 (2020): 347-56, https://doi.org/10.3233/ISU-190065.

17 Yongming Wang, "Using Machine Learning and Natural Language Processing to Analyze Library Chat Reference Transcripts," Information Technology and Libraries 41, no. 3 (2022): 1-10, https://doi.org/10.6017/itaLv41i3.14967.

18 Xiaoming Zhai, "ChatGPT User Experience: Implications for Education," SSRN Electronic Journal (2022): 1-18, https://doi.org/10.2139/ssrn.4312418.

19 Andrew Cox, "How Artificial Intelligence Might Change Academic Library Work: Applying the Competencies Literature and the Theory of the Professions," Journal of the Association for Information Science and Technology 74, no. 3 (2022): 12, https: //doi.org/10.1002/asi.24635; Mary Lee Kennedy, "What Do Artificial Intelligence (AI) and Ethics of AI Mean in the Context of Research Libraries?," Research Library Issues, no. 299 (2019): 3-13, https://doi.Org/10.29242/rli.299.l.

20 "Canadian Library Directory," Government of Canada, updated June 2, 2021, https: //sigles-symbols.bac-lac.gc.ca/eng/Search.

21 McTear, Conversational Al, 30.

22 Alexander L. George and Andrew Bennett, Case Studies and Theory Development in the Social Sciences (Cambridge: MIT Press, 2005), 81-83, ProQuest Ebook Central; Bent Flyvberg, "Five Misunderstandings about Case-Study Research," in Qualitative Research Practice, ed. Clive Seale, Giampietro Gobo, David Silverman, and Jaber Gubrium (London: SAGE Publications, 2004), 398-99.

23 Robert K. Yin, Case Study Research: Design and Methods, 4th ed. (Los Angeles, CA: SAGE Publications, 2009), 47-49.

24 Ivy.ai, "Ivy.ai Announces AI Chatbot Support to Streamline Name, Image and Likeness Policies," July 15, 2021, https://www.globenewswire.com/en/news-release/2021/07/15/2263873/0/en/Ivy-ai-Announces-AI-Chatbot-Support-to-Streamline-Name-Image-and-Likeness-Policies.html

25 Kaushal and Yadav, "The Role of Chatbots," 25.

26 Andrew J. Nelson and Jennifer Irwin, "Defining What We Do-All Over Again: Occupational Identity, Technological Change, and the Librarian/Internet-Search Relationship," Academy of Management Journal 57, no. 3 (2014): 892-928, https://doi.org/10.5465/ami.2012.0201.

27 Nelson and Irwin, "Defining What We Do," 919-21.

28 Nelson and Irwin, "Defining What We Do," 919.

29 Nelson and Irwin, "Defining What We Do," 919.

30 Julia Guy, "Artificial Interactions: The Ethics of Virtual Assistants," (master's thesis. University of Alberta, 2022), 31, Education & Research Archive, https://doi.org/10.7939/r3-s2wl-lzl5.