Establishing a Regulatory Science System for Supervising the Application of Artificial Intelligence for Traditional Chinese Medicine: A Methodological Framework

Abstract

In this study, we reported a methodological framework for the development of a guideline for establishing a regulatory science system for supervising the application of artificial intelligence for traditional Chinese medicine (TCM). It introduced all of the key steps for developing the guideline as follows: the composition of the guideline expert groups, summary steps, agency, purpose, targeted population, writing, publishing, updating, dissemination, dynamic monitoring, and evaluation. The guideline will provide the basis for national authorities to effectively regulate artificial intelligence technology and enrich the supervisory system for TCM, and it will be of great significance to TCM.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Traditional Chinese medicine (TCM) is a medical and pharmaceutical system with a long history, unique theory, and technical methods, which made great contributions to human health [1]. As a result of dissemination of TCM, domestic and foreign scholars have conducted many scientific research studies, and its influence has gradually expanded [2, 3]. Many landmark research studies have been published in top journals, including Journal of the American Medical Association (JAMA), Annals of Internal Medicine, and JAMA Internal Medicine [4–6]. Thus, the academic influence of TCM has greatly improved. However, its international influence is still encountering challenges, which are mainly reflected in the reluctance of some disciplines to accept the validity of the recognition of TCM [7, 8].

As a result of the development of computer science, the application of artificial intelligence (AI) can effectively transform the combined online and offline medical model to achieve multidisciplinary integration, multipath diagnosis, and treatment of the wisdom of the Internet medical, which can improve the efficiency of diagnosis, treatment, and management [9, 10]. In recent years, interest in AI has increased in the application of data mining, computer-aided diagnosis, intelligent decision therapy, and intelligent rehabilitation of TCM [11]. AI can convert TCM’s classical ancient books and clinical treatment experience into data and establish an extremely large TCM database to provide a basis for scientifically explaining TCM treatments. Doing this can greatly improve the overall service level of TCM, reduce the number of medical resources required, and promote the development of TCM [12, 13].

As a result of the wide application of AI technology in the construction of an evaluation system, disease model construction, medical device innovation, and public health crisis response, the establishment of a regulatory science system for supervising the application of AI has become a potential research topic [14, 15]. Until now, no such study has been conducted in this field, and no such guideline has been developed, and it is urgent to develop a guideline in this field. Thus, we will develop such a guideline. In order to make the guideline transparent [16, 17], we reported the current methodological framework. It will help to establish a regulatory science system and to help in the application of AI technology to TCM. It will also fill research gaps and help TCM serve human health [18].

2. Methods

2.1. Summary Steps of Developing Guideline

The guideline’s development will follow the latest definitions of guidelines provided by the United States Institute of Medicine [19], which are based on the methodology developed by the World Health Organization standard guidance [20] and the six major areas of the Appraisal of Guidelines for Research and Evaluation (AGREE II) [21]. We will also use the Reporting Items for Practice Guidelines in Healthcare (RIGHT) [22] and other international standards to help us develop the guideline [23–26]. The key steps of the guideline are shown in Figure 1.

[figure(s) omitted; refer to PDF]

The development of the guideline consists of four phases (Figure 1). In phase 1, we will form the guideline groups and construct the problems and outcomes. Additionally, we will survey experts from universities and scientific research institutions. In phase 2, we will conduct a systematic review of all available guidelines, documents, and papers on supervising the application of AI to TCM and provide evidence for developing items of the guideline. In phase 3, first, we will draft the initial version of the guideline based on the results from the Delphi method and consensus meeting. Subsequently, we will modify, validate, and finalize the guideline through multiple steps. In phase 4, we will establish the regulatory scientific evidence base, dynamic update, and evaluation protocol.

2.2. Explanations of the Terms

TCM [27]: traditional Chinese medicine (TCM) is an alternative medical practice drawn from traditional medicine in China. It includes various forms of herbal medicine, acupuncture, cupping therapy, Guasha, massage (Tuina), bonesetter (Die-Da), exercise (Qigong), and dietary therapy.

AI [28]: artificial intelligence (AI) is a broad umbrella term used to encompass a wide variety of subfields dedicated to creating algorithms to perform tasks that mimic human intelligence. It combines human abilities of learning, reasoning, perception, and an understanding of natural language through computer programs.

Regulatory science [29]: regulatory science is the science of developing new tools, standards, and approaches derived from various scientific disciplines to assess the safety, efficacy, quality, and performance of all Food and Drug Administration (FDA)-regulated products.

Living guideline [30]: A living guideline is a novel approach that operates through, for example, dynamic monitoring, timely inclusion of new evidence, and live updates of recommendations to effectively improve the timeliness of clinical guidelines by periodically obtaining clinical evidence and updating the results of systematic reviews in a timely manner.

2.3. Ethics and Dissemination

The guideline will not require ethical approval because no data linked to individual patient information will be used in our study. Additionally, the findings will be disseminated through peer-reviewed journals or conferences.

2.4. Agency of the Guideline

The guideline will be developed by members from editorial boards of the Journal of Evidence-Based Medicine and the Chinese Journal of Evidence-Based Medicine, in collaboration with a large number of institutions in China, and we will invite experts from multiple units outside of China.

2.5. Guideline Teams

The guideline teams consist of a guideline steering committee, guideline expert group, guideline secretariat group, and external review group. To ensure the authority and comprehensiveness of the guideline, the guideline teams will include experts from main cities around China. Additionally, we will invite regulatory science experts from countries out of China to participate. Members of the guideline steering committee, guideline expert group, guideline external review group, and secretariat group should complete a declaration of interest and declare any conflict of interests before formally participating in the work related to the development of the guideline. All members’ statements of interest will be reported in the final guideline document.

2.6. Patient and Public Involvement Statement

The current study is a methodological study, and patient and public involvement is not needed.

2.7. Purpose and Targeted Population of the Guideline

The purpose of the guideline is to establish a regulatory science system for supervising the application of AI to TCM. The guideline will focus on the scientific regulatory approach, comprehensive evaluation, and dynamic updating of the application of AI to TCM. This guideline applies to personnel at all levels who conduct regulatory science research on the application of AI to TCM, including policy-makers, medical device managers, clinicians, nurses, computer scientists, and journal editors.

2.8. Clarify the Guideline Questions

Guiding questions will be designed by the guideline steering committee using the relevant literature and study objectives. A three-round Delphi study will be conducted to determine the issues that refer to the guidance and the development method of the regulatory scientific system. Each item will be rated by using a five-point Likert scale (not important, of little importance, neutral, important, and very important). The mean value (X), standard deviation (SD), coefficient of variation (CV), and R value will be calculated for each attribute.

2.9. Search Strategy

The search databases will include PubMed, the Cochrane Library, Embase, Chinese Biomedical Literature Service System, China National Knowledge Infrastructure, WanFang Data, China Science and Technology Journal Database, UptoDate, Guideline Central, National Guideline Clearinghouse, Guidelines International Network, and the National Institute for Health and Care Excellence. The following databases will also be searched: NHS Economic Evaluation Database and Health Technology Assessment. Additionally, information will be obtained from official websites, such as the health authorities’ official websites of the international and local organizations anduniversities, Food and Drug Administration, and the official website of the health insurance department or relevant industry association. Finally, we will search Google, Baidu, and other search engines to obtain other literature. According to the retrieval strategy formulated in advance, articles from the above databases will be imported into the Endnote X9 software.

Subject words and text words will be searched. Search terms will include guideline, consensus, standard, process, artificial intelligence, machine learning, traditional Chinese medicine, TCM, regulatory science, and handbook. There will be no limitation to the types of included studies. The languages will be limited to English and Chinese. The search will be performed in May 2022, and the search will be updated when necessary.

2.10. Survey of Problems

We will conduct a survey of problems from TCM universities and research institutions regarding the application of AI to TCM. It will help us to understand the current scenario and problems regarding the application of AI to TCM.

2.11. Literature Screening and Data Extraction

To achieve the stability and consistency of literature retrieval, preretrieval will be performed to achieve a unified standard. The inclusion criteria will be as follows: any study that is helpful to establish a regulatory science system for the application of artificial intelligence in TCM, including guideline, consensus, standard, process, article, or review. The exclusion criteria will be as follows: the study does not report the clear method for establishing a guideline; it does not involve a regulatory science system. When necessary, the criteria will be revised. Two researchers will independently select and screen the studies according to inclusion criteria and exclusion criteria. A third researcher will be consulted to resolve the disagreement. These two researchers will independently extract data, and discrepancies will be resolved by consensus. If information cannot be retrieved, the corresponding author will be contacted.

2.12. Systematic Review

The quality of the included studies will be assessed according to the types of studies. The risk of bias (ROB) will be assessed using ROB 2.0 for RCTs [31]. The Newcastle–Ottawa Scale [32] will be used to assess the quality of cohort studies or case-control studies. The AGREE II will be used to assess the quality of guidelines [33]. A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR 2) will be used to assess the quality of systematic reviews and meta-analyses [34]. The process will be performed by two independent researchers. Disagreement will be discussed or consulted by a third researcher. A systematic review will be performed to assess the potential studies and to provide evidence to develop the guideline. It will also help to provide recommendations for the guideline.

2.13. Generation of a List of Candidate Items

The candidate items in the guideline will be developed using the Delphi method. We will develop the candidate items before sending them the questionnaire. A two-round Delphi questionnaire and a final consensus meeting will be employed [35, 36]. An item will be recommended if the votes are above 50%. The remaining scenarios will be considered if no consensus is reached, and recommendations with no consensus will be subjected to the next round of voting. A subsequent review of the results will be conducted by the steering committee to identify statements where a consensus is almost reached. With the agreement of two-thirds of the members of the consensus expert group, the steering committee may revise and improve important issues existing in the recommendations, and the guideline secretariat group will faithfully record the entire revision process.

2.14. Writing, Publishing, and Updating the Guideline

Once the recommendations are approved, the guideline steering committee will write the guideline according to the RIGHT (http://right-statement.org) reporting standard. The draft guideline will be submitted to the steering committee for review and approval. Then, the introduction and announcement of the guideline will be published. The guideline will also be presented and discussed at relevant professional society academic meetings and through publication in scientific journals. We will establish the regulatory scientific evidence systematic based on the guideline. The guideline will be updated according to the feedback on the actual operation of the guideline and the latest research results in relevant fields.

3. Discussion

Developing a guideline for establishing a regulatory science system for supervising the application of AI to TCM is urgent because the development of AI is so fast. The future guideline will be developed by a multidisciplinary team of experts. To help the guideline, opinions will be obtained from experts, policy-makers, and users. The guideline developers will establish an update and evaluation process to update the guideline and finally help supervise the application of AI in TCM. The guideline will not only track new evidence in real time, promote the practical application of the best research evidence, and implement scientific regulation but also save a large number of resources and optimize the guideline formulation and evaluation process. The guideline will be a new innovation in TCM.

In conclusion, the current study describes the steps of formulating a guideline for the application of AI to TCM. The guideline will be developed in strictly accordance with the standards of an evidence-based guideline. The results will be used as a context for analyzing general strengths and gaps in the current quality of evidence in the application of AI to TCM.

Disclosure

The funding bodies did not play roles in study design, data collection, analysis, interpretation of results, and the manuscript.

Authors’ Contributions

Yonggang Zhang provided ideas and designed the manuscript. Ying He and Yonggang Zhang wrote the first draft of the manuscript. Qian Wen, Ying Wang, Juan Li, Nian Li, and Ning Li revised the manuscript. All authors approved the submitted version. Ying He and Qian Wen contributed equally to this work.

Acknowledgments

The authors thank Maxine Garcia, PhD, from Liwen Bianji (Edanz) (https://www.liwenbianji.cn/), for editing the English text of a draft of this manuscript. The authors have registered the methodological framework for this guideline on the Global Practice Guidelines Registry Platform (registration number: IPGRP-2021CN297). This study was supported by the National Natural Science Foundation of China (82174227).

References

[1] G. Tian, C. Zhao, X. Zhang, W. Mu, Y. Jiang, X. Wei, M. Zhao, Z. Shi, Y. Jin, J. Si, J. Wang, J. Hu, M. Guan, R. Qiu, C. Zhong, M. Li, Y. Sun, L. You, J. Li, H. Shang, "Evidence-based traditional Chinese medicine research: two decades of development, its impact, and breakthrough," Journal of Evidence-Based Medicine, vol. 14 no. 1, pp. 65-74, DOI: 10.1111/jebm.12420, 2021.

[2] H. Liu, Z. g. Ma, "Analysis on situation of traditional Chinese medicine development and protection strategies in China," Chinese Journal of Integrative Medicine, vol. 26 no. 12, pp. 943-946, DOI: 10.1007/s11655-020-3218-0, 2020.

[3] W. Y. Wang, H. Zhou, Y. F. Wang, B. S. Sang, L. Liu, "Current policies and measures on the development of traditional Chinese medicine in China," Pharmacological Research,DOI: 10.1016/j.phrs.2020.105187, 2020.

[4] L. Zhao, D. Li, H. Zheng, X. Chang, J. Cui, R. Wang, J. Shi, H. Fan, Y. Li, X. Sun, F. Zhang, X. Wu, F. Liang, "Acupuncture as adjunctive therapy for chronic stable Angina: a randomized clinical trial," JAMA Internal Medicine, vol. 179 no. 10, pp. 1388-1397, DOI: 10.1001/jamainternmed.2019.2407, 2019.

[5] J. W. Yang, L. Q. Wang, X. Zou, S. Y. Yan, Y. Wang, J. J. Zhao, J. F. Tu, J. Wang, G. X. Shi, H. Hu, W. Zhou, Y. Du, C. Z. Liu, "Effect of acupuncture for postprandial distress syndrome: a randomized clinical trial," Annals of Internal Medicine, vol. 172 no. 12, pp. 777-785, DOI: 10.7326/m19-2880, 2020.

[6] D. L. Hershman, J. M. Unger, H. Greenlee, J. L. Capodice, D. L. Lew, A. K. Darke, A. T. Kengla, M. K. Melnik, C. W. Jorgensen, W. H. Kreisle, L. M. Minasian, M. J. Fisch, N. L. Henry, K. D. Crew, "Effect of acupuncture vs sham acupuncture or waitlist control on joint pain related to aromatase inhibitors among women with early-stage breast cancer: a randomized clinical trial," Journal of the American Medical Association, vol. 320 no. 2,DOI: 10.1001/jama.2018.8907, 2018.

[7] Y. Yang, K. Tian, G. Bai, X. Zhu, X. Yu, L. Shi, "Health technology assessment in traditional Chinese medicine in China: current status, opportunities, and challenges," Global Health Journal, vol. 3 no. 4, pp. 89-93, DOI: 10.1016/j.glohj.2019.11.002, 2019.

[8] A. X. Lin, G. Chan, Y. Hu, D. Ouyang, C. O. L. Ung, L. Shi, H. Hu, "Internationalization of traditional Chinese medicine: current international market, internationalization challenges and prospective suggestions," Chinese Medicine, vol. 13 no. 1,DOI: 10.1186/s13020-018-0167-z, 2018.

[9] S. Vollmer, B. A. Mateen, G. Bohner, F. J. Kiraly, R. Ghani, P. Jonsson, S. Cumbers, A. Jonas, K. S. L. McAllister, P. Myles, D. Grainger, M. Birse, R. Branson, K. G. M. Moons, G. S. Collins, J. P. A. Ioannidis, C. Holmes, H. Hemingway, "Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness," BMJ, vol. 368,DOI: 10.1136/bmj.l6927, 2020.

[10] X. Liu, S. C. Rivera, D. Moher, M. J. Calvert, A. K. Denniston, "Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension," The Lancet Digital Health, vol. 2 no. 10,DOI: 10.1136/bmj.m3164, 2020.

[11] H. Zhang, W. Ni, J. Li, J. Zhang, "Artificial intelligence-based traditional Chinese medicine assistive diagnostic system: validation study," JMIR Medical Informatics, vol. 8 no. 6,DOI: 10.2196/17608, 2020.

[12] X. Chu, B. Sun, Q. Huang, S. Peng, Y. Zhou, Y. Zhang, "Quantitative knowledge presentation models of traditional Chinese medicine (TCM): a review," Artificial Intelligence in Medicine, vol. 103,DOI: 10.1016/j.artmed.2020.101810, 2020.

[13] G. Arji, R. Safdari, H. Rezaeizadeh, A. Abbassian, M. Mokhtaran, M. Hossein Ayati, "A systematic literature review and classification of knowledge discovery in traditional medicine," Computer Methods and Programs in Biomedicine, vol. 168, pp. 39-57, DOI: 10.1016/j.cmpb.2018.10.017, 2019.

[14] H. D. Marble, R. Huang, S. N. Dudgeon, A. Lowe, M. D. Herrmann, S. Blakely, M. O. Leavitt, M. Isaacs, M. G. Hanna, A. Sharma, J. Veetil, P. Goldberg, J. H. Schmid, L. Lasiter, B. D. Gallas, E. Abels, J. K. Lennerz, "A regulatory science initiative to harmonize and standardize digital pathology and machine learning processes to speed up clinical innovation to patients," Journal of Pathology Informatics, vol. 11 no. 1,DOI: 10.4103/jpi.jpi_27_20, 2020.

[15] D. B. Kramer, A. S. Kesselheim, "Trust and transparency in medical device regulation," BMJ, vol. 365,DOI: 10.1136/bmj.l4166, 2019.

[16] Q. Wang, N. Li, J. Li, Y. He, Y. Li, D. Zhong, X. Liu, J. Fan, R. Jin, D. Kang, Y. Zhang, "A protocol of a guideline to establish the evidence ecosystem of acupuncture," Frontiers of Medicine, vol. 8,DOI: 10.3389/fmed.2021.711197, 2022.

[17] Y. He, J. Li, Y. Li, R. Jin, Q. Wen, N. Li, Y. Zhang, "Strengthening the quality of clinical trials of acupuncture: a guideline protocol," BMJ Open, vol. 12 no. 1,DOI: 10.1136/bmjopen-2021-053312, 2022.

[18] Z. Liang, Y. Lai, M. Li, J. Shi, C. I. Lei, H. Hu, C. O. L. Ung, "Applying regulatory science in traditional Chinese medicines for improving public safety and facilitating innovation in China: a scoping review and regulatory implications," Chinese Medicine, vol. 16 no. 1,DOI: 10.1186/s13020-021-00433-2, 2021.

[19] IOMU Trustworthy, C. P. Guidelines, Clinical Practice Guidelines We Can Trust, 2011.

[20] D. Sinclair, R. Isba, T. Kredo, B. Zani, H. Smith, P. Garner, "World health organization guideline development: an evaluation," PLoS One, vol. 8 no. 5,DOI: 10.1371/journal.pone.0063715, 2013.

[21] Mc Brouwers, K. Kerkvliet, K. Spithoff, "The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines," BMJ,DOI: 10.1136/bmj.i1152, 2016.

[22] Y. Chen, K. Yang, A. Marušic, A. Qaseem, J. J. Meerpohl, S. Flottorp, E. A. Akl, H. J. Schunemann, E. S. Chan, Y. Falck-Ytter, F. Ahmed, S. Barber, C. Chen, M. Zhang, B. Xu, J. Tian, F. Song, H. Shang, K. Tang, Q. Wang, S. L. Norris, "A reporting tool for practice guidelines in health Care: the RIGHT statement," Annals of Internal Medicine, vol. 166 no. 2,DOI: 10.7326/m16-1565, 2017.

[23] P. O. Vandvik, L. Brandt, "Future of evidence ecosystem series: evidence ecosystems and learning health systems: why bother?," Journal of Clinical Epidemiology, vol. 123, pp. 166-170, DOI: 10.1016/j.jclinepi.2020.02.008, 2020.

[24] I. Boutron, P. Créquit, H. Williams, J. Meerpohl, J. C. Craig, P. Ravaud, "Future of evidence ecosystem series: 1. Introduction evidence synthesis ecosystem needs dramatic change," Journal of Clinical Epidemiology, vol. 123, pp. 135-142, DOI: 10.1016/j.jclinepi.2020.01.024, 2020.

[25] E. A. Akl, N. R. Haddaway, G. Rada, T. Lotfi, "Future of evidence ecosystem series: evidence synthesis 2.0: when systematic, scoping, rapid, living, and overviews of reviews come together," Journal of Clinical Epidemiology, vol. 123, pp. 162-165, DOI: 10.1016/j.jclinepi.2020.01.025, 2020.

[26] P. Ravaud, P. Créquit, H. C. Williams, J. Meerpohl, J. C. Craig, I. Boutron, "Future of evidence ecosystem series: 3. from an evidence synthesis ecosystem to an evidence ecosystem," Journal of Clinical Epidemiology, vol. 123, pp. 153-161, DOI: 10.1016/j.jclinepi.2020.01.027, 2020.

[27] C. Liu, M. Gu, "Protecting traditional knowledge of Chinese medicine: concepts and proposals," Frontiers of Medicine, vol. 5 no. 2, pp. 212-218, DOI: 10.1007/s11684-011-0142-x, 2011.

[28] G. S. Collins, K. G. M. Moons, "Reporting of artificial intelligence prediction models," The Lancet, vol. 393 no. 10181, pp. 1577-1579, DOI: 10.1016/s0140-6736(19)30037-6, 2019.

[29] G. A. FitzGerald, "Regulatory science: what it is and why we need it," Clinical Pharmacology & Therapeutics, vol. 89 no. 2, pp. 291-294, DOI: 10.1038/clpt.2010.276, 2010.

[30] E. A. Akl, J. J. Meerpohl, J. Elliott, L. A. Kahale, H. J. Schunemann, "Living systematic reviews: 4. Living guideline recommendations," Journal of Clinical Epidemiology, vol. 91, pp. 47-53, DOI: 10.1016/j.jclinepi.2017.08.009, 2017.

[31] K. Slim, E. Nini, D. Forestier, F. Kwiatkowski, Y. Panis, J. Chipponi, "Methodological index for non-randomized studies (minors): development and validation of a new instrument," ANZ Journal of Surgery, vol. 73 no. 9, pp. 712-716, DOI: 10.1046/j.1445-2197.2003.02748.x, 2003.

[32] A. Stang, "Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses," European Journal of Epidemiology, vol. 25 no. 9, pp. 603-605, DOI: 10.1007/s10654-010-9491-z, 2010.

[33] J. Lang, K. Haines, "Clinimetrics: appraisal of guidelines, research and evaluation II," Journal of Physiotherapy, vol. 65 no. 3,DOI: 10.1016/j.jphys.2019.05.003, 2019.

[34] B. J. Shea, B. C. Reeves, G. Wells, M. Thuku, C. Hamel, J. Moran, D. Moher, P. Tugwell, V. Welch, E. Kristjansson, D. A. Henry, "AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both," BMJ,DOI: 10.1136/bmj.j4008, 2017.

[35] P. Alonso-Coello, H. J. Schünemann, J. Moberg, "GRADE evidence to decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction," BMJ, vol. 353, 2016.

[36] P. Alonso-Coello, A. D. Oxman, J. Moberg, R. Brignardello-Petersen, E. A. Akl, M. Davoli, S. Treweek, R. A. Mustafa, P. O. Vandvik, J. Meerpohl, G. H. Guyatt, H. J. Schunemann, "GRADE evidence to decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: clinical practice guidelines," BMJ, vol. 353,DOI: 10.1136/bmj.i2089, 2016.