Introducing the Journal of the Medical Library Association's policy on the use of generative artificial intelligence in submissions

Abstract

With the arrival of ChatGPT, the academic community has expressed concerns about how generative artificial intelligence will be used by students and researchers alike. After consulting policies from other journals and discussing among the editorial team, we have created a policy on the use of Al on submissions to JMLA. This editorial provides a brief background on these concerns and introduces our policy.

Headnote

With the arrival of ChatGPT, the academic community has expressed concerns about how generative artificial intelligence will be used by students and researchers alike. After consulting policies from other journals and discussing among the editorial team, we have created a policy on the use of Al on submissions to JMLA. This editorial provides a brief background on these concerns and introduces our policy.

Artificial intelligence (Al) technologies and applications have the potential to enhance how librarians and researchers work with information. On November 30, 2022, OpenAI publicly launched ChatGPT, a free web application known as a "chatbot" using Al to interact with people and generate responses to their queries [1]. While chatbots have been employed prior, ChatGPT created waves due to the speed of its responses, the "human sounding" language it used in its responses, and its ability to adapt to subsequent questions. ChatGPT is a large language model (LLM) Al, a generative artificial intelligence that creates sentences using patterns of language structure in its database to generate paragraphs of text like that of a human writing the text. For librarians interested in learning more about the relevant terms and technology in this space, Lorcan Dempsey provides an excellent primer [2].

With the arrival of ChatGPT, concerns about academic integrity quickly surfaced. Professors discovered students submitting papers generated by ChatGPT [3], and researchers testing the Al's abilities discovered it can answer and pass exams in business and law as well as the United State Medical Licensing Exam (USMLE) [4]. ChatGPT also generates false, but very realistic, citations to academic papers, which our readers may have encountered while working with their patrons. Researchers have even published conversations about plagiarism with ChatGPT [5].

Scholarly publishers have likewise been affected by and monitoring the use of Al and ChatGPT within its ecosystem [6]. Early studies have found that ChatGPT is sophisticated enough to write scholarly articles for publication in academic journals [7]. Meanwhile, a recent article in Nature reported four articles published in peer reviewed journals listing an Al tool as co-author, raising concerns with publishing community over whether Al can be listed as an author and other concerns of transparency and responsibility [8]. However, the scientific community is far from consensus that this technology is entirely negative, with some researchers already advocating for seeking out opportunities for leveraging LLMs throughout the research life-cycle [9].

Journal editors, researchers, publishers, and the biomedical scientific community have begun to discuss the appropriate use of Al within research in order to establish policies on the use of Al for writing manuscripts and creation of images for publication. Among the issues currently being contested are: whether Al can be listed as an author [10]; the differences between using Al as part of the research methodology versus using it to write the manuscript [11]; and the use of Al to generate images and figures [12]. Interestingly, journal and publisher policies often settle on slightly different decisions at the margins of each issue, reflecting the divergence of opinion and lack of consensus in the space. We also anticipate that our colleagues' policies are likely to change as the Al landscape develops.

JMLA was confronted with the use of Al in a manuscript submission early in 2023. A manuscript on the future of Al in medical libraries was submitted for consideration by the journal. At first glance, it seemed like a typical submission. However, the citations were the first clue that something was different. Two JMLA articles were cited; and while these citations had the proper volume and issue numbers, the articles did not exist. Upon a closer read of the manuscript, the broad, generalized statements signaled that the paper was not written by a human. We used the Al-detection tool provided by ChatGPT to confirm that it was "highly likely" that the manuscript and citations were written by AL However, these Aldetection tools are not without fault, with prominent tools such as Turnitin reporting false positive rates as high as 4% [13]. In fact, the Al-detection tool provided by ChatGPT that we used in our process was itself sunset in July 20, 2023, due to concerns over its low rate of accuracy, less than 6 months after it was launched [14]. Reviewing this submission, and the inability to rely on tools such as these going forward, highlighted the exigent need for a policy on the use of Al.

After consulting policies from other journals and discussing among the editorial team, we have created a policy on the use of Al on submissions to JMLA. This policy (found on the JMLA site: https://jmla.mlanet.org/ojs/jmla/Alsubmissionpolicy) will certainly change as this technology and its uses evolve. While JMLA's current policy calls for restrictions on the use of LLMs in the production of original content submitted to the journal, we recognize that these technologies have the potential to transform the discovery, processing, and synthesis of research. Indeed, it is the transformative potential of these tools that makes it imperative that they are utilized openly and transparently.

Equally important is for librarians to conduct research using and on the use of these tools for the profession to learn, adapt, and grow with the technology. As editors we encourage authors to submit articles that depict the integration of LLMs and generative artificial intelligence software into information science research and the professional practice of health science librarians. Projects and initiatives along these lines are already underway. The National Library of Medicine currently uses Al to select indexing terms in PubMed [15], while researchers at The Stanford Center for Research on Foundation Models (CRFM) and MosaicML released an Al trained to interpret biomedical language they originally called PubMedGPT but later renamed to BioMedLM [16]. Meanwhile, prompt engineering, the process of creating queries for an LLMpowered chatbot that will produce meaningful and relevant results, has been identified as a possible application of librarian expertise in search and retrieval [17].

The tasks highlighted in these examples (generating metadata, interpreting primary literature, and retrieving information) are core competencies of health information professionals. Twenty years ago, the ascendence of webbased information tools like Google and Wikipedia created angst among librarians that their skills and expertise would lose relevance for students and researchers. Yet over the course of the following two decades, the profession thoroughly integrated these tools into research and practice. Like Wikipedia and Google before it, we anticipate that LLMs, chatbots, and artificial intelligence software may become tools that librarians develop expertise in using to advance the research needs of their communities [18]. We look forward to the opportunity to review articles investigating the integration of Al and medical librarianship.

Sidebar

See end of article for authors' affiliations.

AUTHORS' AFFILIATIONS

Jill T. Boruff, AHIP, [email protected], https://orcid.оrg/OOOO0002-0338-7322, 0002-0338-7322, Co-Lead Editor, Journal of the Medical Library Association. Associate Librarian, SchuIich Library of Physical Sciences, Life Sciences, and Engineering, McGill University, Montreal, QC, Canada.

Michelle Kraft, AHIP, FMLA, [email protected], Co-Lead Editor, Journal of the Medical Library Association. Medical Library Director, Cleveland Clinic Foundation, Cleveland, OH, United States.

Alexander J. Carroll. AHIP, [email protected], https://orcid.org/0000-0003-0248-3811. Associate Director, Stevenson Science and Engineering Library, Vanderbilt University, Nashville, TN, United States.

Received August 2023; accepted September 2023

References

REFERENCES

1. OpenAI. Introducing ChatGPT [Internet]. 2022 [cited 2023 Aug 16]. Available from: https://openai.com/blog/chatgpt.

2. Dempsey L. Generative Al and large language models: background and contexts [Internet]. Lorcan Dempsey's Weblog. 2023 [cited 2023 Aug 16]. Available from: https: / / www.lorcandempsey.net/ intro-gen-ai/.

3. Huang K. Alarmed by A.I. Chatbots, Universities Start Revamping How They Teaclr. Tire New York Times [Internet]. 2023 Jan 16 [cited 2023 Aug 16]; Available from: https:/ / www.nytimes.com/ 2023/01/16/technology/ chatg pt-artiricial-intelligence-universities.html.

4. Varanasi L. Al models like ChatGPT and GPT-4 are acing everything from the bar exam to AP Biology: Here's a list of difficult exams both Al versions have passed [Internet]. Business Insider. 2023 [cited 2023 Aug 16]. Available from: https://www.businessinsider.com/list-here-are-the-examschatgpt-has-passed-so-far-2023-1.

5. King MR, chatGPT. A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education. Cel Mol Bioeng. 2023 Feb l;16(l):l-2. DOI: 10.1007/sl2195022-00754-8.

6. de Waard A, King H, Wang LL, Groth P. Al and Scholarly Publishing: A View from Three Experts [Internet]. The Scholarly Kitchen. 2023 [cited 2023 Aug 16]. Available from: https: / / scholarlykitchen.sspnet.org/ 2023/01/18/ guestpost-ai-and-scholarly-publishing-a-view-from-threeexperts/.

7. Lucey B, Dowling M. ChatGPT: our study shows Al can produce academic papers good enough for journals - just as some ban it [Internet]. The Conversation. 2023 [cited 2023 Aug 16]. Available from: http: / / theconversation.com/ chatgpt-our-study-shows-aican-produce-academic-papers-good-enough-for-journalsjust-as-some-ban-it-197762.

8. Stokel-Walker C. ChatGPT listed as author on research papers: many scientists disapprove. Nature. 2023 Jan 18;613(7945):620-l. DOI: 10.1038/d41586-023-00107-z.

9. Meyer JG, Urbanowicz RJ, Martin PCN, O'Connor К, Li R, Peng PC, Bright TJ, Tatonetti N, Won KJ, GonzalezHernandez G, Moore JH. ChatGPT and large language models in academia: opportunities and challenges. BioData Mining. 2023 Jul 13;16(l):20. DOI: 10.1186/Sİ3040-023-003399.

10. Nature Editorial Board. Tools suda as ChatGPT threaten transparent science; here are our ground rules for their use. Nature. 2023 Jan 24;613(7945):612-612. DOI: 10.1038/d41586023-00191-1.

11. Hosseini M, Rasmussen LM, Resnik DB. Using Al to write scholarly publications. Accountability in Research. 2023 Jan 25;0(0):l-9. DOI: 10.1080/08989621.2023.2168535.

12. Science Editorial Board. Science Journals: Editorial Policies [Internet], [cited 2023 Aug 16]. Available from: https:/ / www.science.org/ content/page/science-journalseditorial-policies.

13. Coley M. Guidance on Al Detection and Why We're Disabling Turnitin's Al Detector [Internet]. Vanderbilt University. 2023 [cited 2023 Aug 16]. Available from: https://www.vanderbilt.edu/brightspace/2023/08/16/gui dance-on-ai-detection-and-why-were-disabling-turnitins-aidetector/.

14.OpenAI. New Al classifier for indicating AI-written text [Internet]. 2023 [cited 2023 Aug 16]. Available from: https:/ / openai.com/blog/new-ai-classifier-for-indicatingai-written-text.

15. National Library of Medicine. MEDLINE 2022 Initiative: Transition to Automated Indexing. NLM Technical Bulletin. 2021 Dec l;(443):e5. Available from: https://www.nlm.nih.gov/ pubs/techbull/nd21/nd21 med line 2022.html.

16. Venigalla A, Frankie J, Garbin M. BioMedLM: a DomainSpecific Large Language Model for Biomedical Text [Internet]. MosaicML. 2023 [cited 2023 Aug 16]. Available from: https://www.mosaicml.com/blog/introducingpubmed-gpt.

17. Tay A. Prompt engineering: Something for librarians here? [Internet]. Musings about librarianship. 2023 [cited 2023 Aug 16]. Available from: http:/ / musingsaboutlibrarianslTip.blogspot.com/ 2023/06/p rompt-engineering-something-for.html.

18. Wang S, Scells H, Koopman B, Zuccon G. Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search? [Internet]. arXiv; 2023 [cited 2023 Aug 16]. Available from: http://arxiv.org/abs/2302.03495.