Correspondence to Mr Jose Angelo Monreal; [email protected] ; Mr. Juan Alfonso Rojas; [email protected]
Strengths and limitations of this study
Comprehensive search strategy: The study will employ an extensive search strategy across multiple databases to include relevant literature.
Adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols and COnsensus-based Standards for the selection of health Measurement INstruments guidelines: The study will follow established protocols, ensuring methodological quality and review process reliability.
Assessment of multiple measurement properties: The study will evaluate various measurement properties, providing a comprehensive analysis of self-report questionnaires.
Heterogeneity in study populations and settings: Differences in populations, settings and questionnaire versions may lead to inconsistencies and affect generalisability.
Subjectivity in content comparison: Comparing self-report questionnaire content will involve some subjectivity, potentially impacting the overall recommendations.
Review question
What is the validity and reliability of the self-report questionnaires used by university students to evaluate blended teaching and learning methods in health sciences programmes?
Introduction
Amidst the COVID-19 pandemic, blended teaching and learning (BTL) has become a prevailing educational model for allied health science university students worldwide. BTL, characterised as the intentional fusion of traditional face-to-face (F2F) classroom instruction with online learning experiences,1–3 has provided educational institutions with a flexible approach to adapt to the challenges posed by the pandemic.
This approach ensures the safety of allied health science students, faculty and staff by minimising in-person interactions.4 It allows universities to transition seamlessly between fully online and in-person instruction. Nevertheless, BTL introduces unique challenges for allied health science university students who heavily rely on hands-on patient care experiences as a fundamental part of their professional training.5–7
To optimise the effectiveness of BTL in health professional education (HPE), accurate assessment of students’ perceptions of this blended learning approach is crucial. While previous studies have employed self-report questionnaires like the Classroom Environment Questionnaire,8 Dental Clinical Learning Environment Instrument9 10 and Student Course Experience Questionnaire11 12 to gauge students’ perspectives on their learning environments, these instruments were primarily designed for traditional F2F settings. In BTL, where students frequently transition between in-person and online learning modes, their perceptions significantly impact their learning approaches, utilisation of online learning technologies13 and engagement with online course materials alongside in-person activities.14
Despite the growing relevance of BTL in HPE, there remains a noticeable gap in the literature regarding the availability of reliable and valid self-report questionnaires explicitly tailored to assess students’ perceptions of BTL from a relational student-learning perspective. Notably, more systematic reviews need to synthesise evidence for reliable, valid and psychometrically sound instruments for BTL assessment. This systematic review protocol addresses this critical gap by identifying and evaluating existing self-report questionnaires designed to assess students’ perceptions of BTL.
By conducting a comprehensive review of the available literature, this study intends to provide valuable insights for educators, administrators and policymakers seeking to enhance the quality of education in the modern era, especially in the context of health professional education during and beyond the COVID-19 pandemic.
Objective
Our primary objective is to critically appraise, compare, and summarise the psychometric property scores of self-report questionnaires evaluating the quality of BTL delivery among health science university students. Specifically, this review aims to determine these measurement properties of the tools used in assessing BTL:
Reliability, measurement error, internal consistency,
Content validity, criterion validity, construct validity, and
Responsiveness.
Study design
Conducting a systematic review and meta-analysis study design ensures a comprehensive examination of the psychometric properties of the self-report questionnaires used by university students to evaluate blended teaching and learning in health sciences programmes. Adhering to the Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) Protocol for systematic reviews and meta-analyses, researchers will follow a 10-step process for implementing systematic reviews of patient-reported outcome measures.
The University of the Philippines’ Research Grants Administration Office exempted this research protocol from ethics review evaluation under protocol number UPMREB 2022–0259-EX and has been registered with PROSPERO. Study procedures will begin with our PROSPERO registration on 11 December 2022, and our methods will be used until the last search date. Figure 1 illustrates the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) 10-step procedure.
Enlarge this image.Figure 1. The 10-step procedure for conducting systematic reviews and meta-analyses of patient-reported outcome measures by COSMIN. 21 COSMIN, COnsensus-based Standards for the selection of health Measurement INstruments; GRADE, Grading of Recommendations Assessment, Development and Evaluation.
Patient and public involvement statement
None.
Data gathering procedure
These databases will be searched: PubMed, EMBASE, Web of Science, MEDLINE (Ovid), PsycInfo (via ProQuest), CINAHL, EBSCOhost, ERIC, Scopus, ScienceDirect, Google Scholar, JSTOR, Acta Medica Philippina, Philippine Journal on Health Research and Development and HERDIN. Pearling will be used to determine relevant studies from the reference lists of the included articles. Reviewers will improve the search strategy and repeat its implementation if many new studies are found through pearling. Zotero manages the references in each database.
In addition to employing Zotero for reference management, the reviewers manually used MS Excel for the search strategy and critical appraisal processes throughout the study. MS Excel proves advantageous for systematic reviews due to its cost-effectiveness, user-friendly interface, flexibility and widespread acceptance. Its compatibility with other tools further enhances its practicality.
Eligibility criteria
The inclusion criteria are as follows:
A study population consisting of health sciences university students (ie, medicine, physical therapy, occupational therapy, speech-language pathology, psychology, nutrition and nursing) engaged in BTL.
Studies that reported the development of self-report questionnaires for health sciences university students evaluating the quality of BTL delivery.
Studies that determined the measurement properties of self-report questionnaires for health sciences university students on BTL.
Studies that reported the distribution of scores, percentage of missing items, floor and ceiling effects, the availability of scores and change of scores or a minimally significant difference of self-reported questionnaires used in evaluating BTL delivery among health sciences university students.
No time or language restrictions will be used.
The exclusion criteria are as follows: (1) studies reporting students’ perceptions, attitudes, learning experience, self-efficacy, satisfaction and learning outcomes on BTL delivery; and (2) biographies, case reports, editorials, newspaper articles, handouts, consensus development conferences, practice guidelines, short communications, abstracts and meetings.
Search strategy and study selection
The PCC (Population, Concept, and Context) framework, adapted from Joanna Briggs Institute, offers a precise and structured approach for formulating the search strategy.
We will use search terms in developing these three concepts:
Context (BTL): These are examples of search terms: blended learning OR delivery blended teaching OR learning flexible learning.
Population (Health Sciences University Students): All university college students currently taking courses in the medical field engaging in BTL, namely medicine, physical therapy, occupational therapy, speech-language pathology, psychology, nutrition and nursing. Examples of search terms are students, Medical OR medical student OR medicine student OR intern OR interns.
Construct (Psychometric Properties including reports on validity and reliability): These are the example search terms: instrumentation OR methods OR validation studies OR comparative study OR psychometrics.
These sample keywords for exclusion will be included in the search strategy: Delphi-technique OR cross-sectional OR biography OR case reports.
The search strategy aims to locate both published and unpublished studies. An initial limited search of MEDLINE (PubMed) and CINAHL (EBSCOhost) was undertaken to identify articles on the topic. The text words contained in the titles and abstracts of relevant articles and the index terms used to describe the articles were used to develop a whole search strategy for PubMed, EBSCO, ProQuest, Google Scholar, and ScienceDirect (see online supplemental appendix A). The search strategy, including all identified keywords and index terms, was adapted for each included information source. The reference lists of all studies selected for critical appraisal were screened for additional studies. Keywords were combined with Boolean operators and truncations to create the search strategy.
An initial pilot search was undertaken to identify pertinent articles for our review. Here is a breakdown of the findings from different databases:
PubMed yielded 272 hits, out of which 55 papers were potentially relevant.
From EBSCO, 54 articles were retrieved and eight were found to be potentially relevant.
ProQuest had three hits with one article being potentially relevant.
A search on Google Scholar returned two hits and one potentially relevant paper.
Lastly, ScienceDirect produced 569 hits with 55 potentially relevant to our review.
Overall, the results from this pilot search indicate many articles that fit our inclusion criteria. This underscores the feasibility of our systematic review, confirming that a satisfactory volume of articles is available for thorough evaluation.
Drawing from the COSMIN search filter and Biomedische Informatie search blocks, the search terms will efficiently identify relevant measurement property studies and outcome measurement instruments. These strategies are detailed in online supplemental appendix A and streamline the search process.
Using the eligibility criteria, two independent reviewers will examine the titles and abstracts of the studies to identify relevant articles (online supplemental appendix B). Two independent reviewers will reassess the relevance of the studies by reading full-text articles (online supplemental appendix C). Throughout the process, the reviewers will reach a consensus by discussion and a third independent reviewer will arbitrate if needed. The PRISMA flow diagram shows the selection process (online supplemental appendix D).
The COSMIN risk of bias checklist
Two independent reviewers will assess each study’s methodological quality using the COSMIN Risk of Bias checklist (online supplemental appendix E) comprising 10 boxes covering various measurement properties. Depending on the properties evaluated by the study authors, not all boxes need to be completed. Focusing on the questionnaire’s development and content validity as crucial properties, self-report questionnaires with inadequate content validity will be excluded from further assessment. If content validity is considered adequate, reviewers will continue appraising papers for more properties such as structural validity, internal consistency and responsiveness. The 10-step procedure for evaluating measurement properties is shown in figure 1, outlining the quality appraisal process using the COSMIN Risk of Bias checklist.
Using the COSMIN checklist is the optimal approach for assessing the methodological quality of studies concerning measurement properties. This preference arises from its versatility as it serves various essential purposes:
The COSMIN checklist can provide valuable guidance for designing and reporting studies on measurement properties.
It is instrumental in evaluating the potential risk of bias within individual studies within a systematic review of outcome measurement instruments.
The COSMIN checklist is instrumental for reviewers and journal editors when critically evaluating the methodological quality of articles or grant applications pertaining to studies involving measurement properties.
Consequently, the multifaceted utility of the COSMIN checklist renders it the preeminent choice for such assessments.15
In step 1, two independent reviewers will assess the content validity of the self-report questionnaires using COSMIN boxes 1 and 2 (online supplemental appendices E.1 and E.2). When rating COSMIN boxes 1 and 2, the guidelines classify standards as ‘very good’ for adequate quality evidence, ‘adequate’ for unreported but assumed sufficient quality, ‘doubtful’ when quality requirements are unclear, ‘inadequate’ for insufficient quality and ‘not applicable’ when a standard is unnecessary in the study.16 Following online supplemental appendix F, they will evaluate relevance, comprehensiveness and comprehensibility by assigning sufficient (+) ratings for the 10 criteria in Appendix E.3. Next, they will use online supplemental appendix G to guide their assessment of each study’s ratings summarising specific criteria from Terwee et al (2018) .16 Finally, they will determine the overall content validity ratings provided by online supplemental appendix H, adhering to Terwee et al’s guidelines and using the ‘worst score counts’ method within the COSMIN boxes.16
In step 2, reviewers will evaluate the internal structure of the questionnaire, focusing on structural validity, internal consistency and cross-cultural validity. They will use COSMIN boxes 3–10 (online supplemental appendix E.4) and the guidelines from Terwee et al 16 to maintain consistency in evaluating measurement properties across the included studies. The quality of each study will be considered by taking the lowest rating of any standard in the box.
In step 3, the reviewers will assess the remaining measurement properties, including reliability, measurement error, criterion validity, hypothesis testing for construct validity and responsiveness. They will follow the guidelines in online supplemental appendix E.4 and Terwee et al.16 If there is no gold standard for measuring the construct of interest, reviewers will not use the box for criterion validity or the criterion approach for responsiveness. Instead, they will formulate hypotheses about the expected direction and magnitude between the self-report questionnaire and a well-defined comparator questionnaire. The study results will be compared with these hypotheses to determine the construct validity of the questionnaire.
Independent reviewers will assess the evidence quality using the modified Grading of Recommendations Assessment, Development and Evaluation approach, grading the trustworthiness of the overall ratings (online supplemental appendix I).16 Content validity evaluation considers three factors: Risk of bias, inconsistency and indirectness. After bias assessment, blinded reviewers will extract measurement property results from self-report questionnaires used by health sciences students to evaluate BTL (online supplemental appendix J). The study results will be compared with the criteria for good measurement properties (online supplemental appendix K) with evidence summarised for each property per questionnaire.
In step 4, reviewers will describe the interpretability and feasibility of the questionnaire, assign qualitative meaning to scores and assess factors affecting its practical application. For interpretability, they will report the distribution of scores in the study population revealing floor and ceiling effects, minimally important change values and response shift for change in scores (online supplemental appendix L) as per Terwee et al.16 For feasibility, they will consider aspects such as comprehensibility for students and teachers, type and ease of administration, instrument length, completion time, ease of standardisation, score calculation, copyright, cost, availability and regulatory requirements (online supplemental appendix M) as outlined by Terwee et al. 16
Data extraction
To avoid missing relevant data, two independent reviewers will extract the following data from the studies: (1) the characteristics of the included self-report questionnaires of health sciences university students evaluating BTL (ie, constructs, target population, recall period, number of items, response options, scoring, original language, available translations) (online supplemental appendix N) and (2) characteristics of the health sciences university students (ie, number of samples, age, sex, setting, country, language, response rate) (online supplemental appendix O). The characteristics of the study samples have all the information necessary for the generalisability of the results and for determining the similarities or dissimilarities of the study samples.16 17 This review will not quantify the proportion of F2F and online classes in blended learning (BTL). It will include all studies combining F2F and online learning regardless of allocation method. Eligibility criteria won’t explicitly consider allocation so that all relevant studies will be included.
Content comparison
Two independent reviewers will compare the contents of the self-report questionnaire for university health sciences students evaluating BTL in an academic setting across studies. Content comparison can help decide the best available measurement by checking the differences in content between several self-report questionnaires.17 Online supplemental appendix P presents a content comparison.
Summary of finding tables
The two reviewers will pool all results per measurement property of the self-report questionnaire (online supplemental appendix Q). The table shows the pooled results, overall rating (+/−/±/?) and quality of evidence (ie, high, moderate, low and very low). The summary of the findings table will recommend the most appropriate self-report questionnaires used by health sciences university students to evaluate BTL. The pooled or summarised results will again be evaluated against the criteria for good measurement properties to obtain an overall rating (online supplemental appendix R).
Meta-analysis
When pooling the measurement property results from different studies, sufficient similarity in the study population, setting, instrument (language) version and administration form are required. The MedCalc statistical software will integrate quantitative findings from similar studies providing a numerical estimate of the overall effect.18 The weights assigned to studies will be based on the inverse of the SE and sample size with more minor standard errors and larger sample sizes carrying more weight.18
A random-effects model will be used assuming the studies are distinct and without a common effect. The summary effect of the meta-analysis represents the distribution of the mean of the actual outcomes.19 For test–retest reliability, weighted mean intraclass correlation coefficients and 95% CIs will be calculated using a standard generic inverse variance random-effects model. Construct validity will involve pooling correlations of self-report questionnaires measuring similar constructs such as student attitudes.18
In the event of inconsistent results, we will investigate the underlying reasons and perform subgroup analyses by country. The overall ratings per subgroup can be determined and more subgroup analyses will be conducted based on study quality (adequate/very good vs inadequate). Subgroup analysis will be performed in this particular order by studying quality, course and country. Study quality will include heterogeneity assessment, quality of evidence control, identification of source variety, creation of guidelines and recommendations, addressing publication bias and implementation of informed decision-making. Course subgroup analysis will also be conducted as differing degrees present varying technical requirements that may or may not be supplied by an online setting. Subgroup analysis by country will consider the various education systems’ varying quality and policies. Doubtful study results will be excluded from the result pooling. If the reasons for inconsistency remain unclear, the overall rating will be based on most of the study results, with evidence quality downgraded for inconsistency.20
Ethics & dissemination
The University of the Philippines’ Research Grants Administration Office exempted this research protocol from ethics review evaluation (protocol number UPMREB 2022–0259-EX) since this study will not collect individual data. The research protocol was registered with PROSPERO. The results will be disseminated through peer-reviewed journals and conferences to aid researchers and professionals in the field of health education to prudently choose effective self-report questionnaires evaluating blended learning.
Ethics statements
Patient consent for publication
Not applicable.
Contributors VCD: Introduction, methodology, proofreading and editing. MTBD: Methodology. DM: Proofreading and editing. MBBPB: Introduction, methodology. RKSF: Introduction, methodology. AM: Introduction, methodology. KCH: Introduction, abstract, methodology, referencing. AN: Introduction, methodology. JJR: Introduction, abstract, methodology. JAR: Introduction, methodology, proofreading and editing. LBS: Introduction, methodology. Our research team includes authors and consultants with distinct expertise. MTBD, currently pursuing her MHPEd, contributes to the field of Health Professions Education. VCD and DM, both holding PhDs, provide their expertise as research methodologists. VCD also acted as the guarantor. Our consulting team is composed of Ma Elizabeth Grageda, an assistant professor specialising in Instructional Design in Health at the National Teacher Training Center for the Health Profession at the University of the Philippines Manila and Michael Palapal Sy, a senior researcher at the Institute of Occupational Therapy in Zurich University of Applied Sciences.
Funding This work was funded by the National Research Council of the Philippines of the Department of Science and Technology with grant number NRCP GIA FY 2022 (Project No. A-109), solely intended to support the research process and does not imply any influence on the design, conduct, analysis, interpretation or reporting of the study results. We have not received any other funding, honoraria, consulting fees or financial benefits from any organisation or individual that has an interest in the outcome of this research.
Competing interests The authors of this research protocol for the systematic review and meta-analysis declare that we received funding from the National Research Council of the Philippines of the Department of Science and Technology. However, the grant giving body did not influence the design, conduct, analysis, interpretation or reporting of this study. The authors are affiliated with the University of Santo Tomas, the University of the Philippines - Manila and the Pamantasan ng Lungsod ng Maynila. We affirm that our primary objective was to produce an unbiased and rigorous synthesis of the available evidence on the topic under investigation, adhering to the highest ethical and methodological standards throughout the research process. We do not own any patents, stocks, shares or other financial assets that the findings of this study might affect. Our affiliations do not pose any conflicts of interest that could influence the results of this systematic review and meta-analysis. We have disclosed all relevant information concerning potential conflicts of interest, including the funding from the National Research Council of the Philippines of the Department of Science and Technology and our affiliations with the aforementioned universities, to our institutional review board and have obtained their approval for this research protocol.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
1 Liu Q, Peng W, Zhang F, et al. The effectiveness of blended learning in health professions: systematic review and meta-analysis. J Med Internet Res 2016; 18: e2. doi:10.2196/jmir.4807
2 Commission on Higher Education. CMO no.4, s. 2020 guidelines on the implementation of flexible learning. 2020.
3 Gray K, Tobin J. Introducing an online community into a clinical education setting: a pilot study of student and staff engagement and outcomes using blended learning. BMC Med Educ 2010; 10: 6. doi:10.1186/1472-6920-10-6
4 Al Zahrani EM, Al Naam YA, AlRabeeah SM, et al. E- Learning experience of the medical profession’s college students during COVID-19 pandemic in Saudi Arabia. BMC Med Educ 2021; 21: 443. doi:10.1186/s12909-021-02860-z
5 Butz N. Motivation in synchronous hybrid graduate business programs: a self-determination approach to contrasting online and on-campus students. J Online Learn Teach 2014. Available: https://www.academia.edu/77969044/Motivation_in_Synchronous_Hybrid_Graduate_Business_Programs_A_Self_Determination_Approach_to_Contrasting_Online_and_On_Campus_Students
6 Shim TE, Lee SY. College students’ experience of emergency remote teaching due to COVID-19. Child Youth Serv Rev 2020; 119: 105578. doi:10.1016/j.childyouth.2020.105578
7 McCutcheon K, Lohan M, Traynor M. A systematic review protocol on the use of online learning versus blended learning for teaching clinical skills to undergraduate health professional students. H E Pedagogies 2016; 1: 82–8. doi:10.1080/23752696.2015.1134204
8 Lyons C, Brown T, Bourke-Taylor H. The Classroom Environment Questionnaire (CEQ): Development and preliminary structural validity. Aust Occup Ther J 2018; 65: 363–75. doi:10.1111/1440-1630.12474
9 Kossioni AE, Lyrakos G, Ntinalexi I, et al. The development and validation of a questionnaire to measure the clinical learning environment for undergraduate dental students (DECLEI). Eur J Dental Education 2014; 18: 71–9. doi:10.1111/eje.12051
10 Krois NR, Kossioni AE, Barlow PB, et al. Steps towards validation of the Dental Education Clinical Learning Instrument (DECLEI) in American dental schools (DECLEI-USA). J Dent Educ 2020; 84: 895–901. doi:10.1002/jdd.12145
11 Barrie S, Ginns P, Prosser M. Early impact and outcomes of an institutionally aligned, student focused learning perspective on teaching quality assurance1. Assess & Eval Higher Educ 2005; 30: 641–56. doi:10.1080/02602930500260761
12 Ginns P, Prosser M, Barrie S. Students’ perceptions of teaching quality in higher education: the perspective of currently enrolled students. Stud Higher Educ 2007; 32: 603–15. doi:10.1080/03075070701573773
13 Ellis RA, Bliuc AM. An exploration into first‐year university students’ approaches to inquiry and online learning technologies in blended environments. Brit J Educational Tech 2016; 47: 970–80. doi:10.1111/bjet.12385
14 Ellis R, Han F, Ellis RA, et al. 2018 Reasons why some university students avoid the online learning environment in blended courses. J Educ Multimed Hypermedia 2018; 27: 137–52.
15 Mokkink LB, Boers M, van der Vleuten CPM, et al. COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study. BMC Med Res Methodol 2020; 20: 293. doi:10.1186/s12874-020-01179-5
16 Terwee CB, Prinsen CAC, Chiarotto A, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res 2018; 27: 1159–70. doi:10.1007/s11136-018-1829-0
17 de Vet H, Terwee C, Mokkink L, et al. Measurement in medicine: a practical guide. Meas Med Pract Guide 2011; 1–338. doi:10.1017/CBO9780511996214
18 Schoonjans F. MedCalc Manual: Easy-to-use statistical software. Amazon Digital Services LLC - Kdp, 2017. Available: https://books.google.com.ph/books?id=WG5AMQAACAAJ
19 Tufanaru C, Munn Z, Stephenson M, et al. Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness. Int J Evid Based Healthc 2015; 13: 196–207. doi:10.1097/XEB.0000000000000065
20 Higgins JPT, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011; 343: d5928. doi:10.1136/bmj.d5928
21 Prinsen CAC, Mokkink LB, Bouter LM, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 2018; 27: 1147–57. doi:10.1007/s11136-018-1798-3
© 2024 Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. http://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ . Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Ma Teresita B Dalusong 2 ; Manlapaz, Donald 3
; Juan Alfonso Rojas 3
; Kaela Celine Ho 3
; Reyes, Jose Joaquin 3
; Lianna Bartolo Sangatanan 3
; Narcelles, Audrey Marie 3
; Ma Bianca Beatriz P Ballesteros 3
; Santos Flores, Ron Kevin 3
; Monreal, Jose Angelo 3
1 Department of Physical Therapy-College of Rehabilitation Sciences, University of Santo Tomas, Manila, Philippines; Center for Health Research and Movement Science – JBI Affiliated Group, University of Santo Tomas, Metro Manila, Philippines
2 Department of Physical Therapy, Pamantasan ng Lungsod ng Maynila College of Physical Therapy, Manila, Philippines
3 Department of Physical Therapy-College of Rehabilitation Sciences, University of Santo Tomas, Manila, Philippines