1. Introduction
Clinical ultrasonography is used as an assessment tool for biofeedback [1,2] and echo-guided procedures [3,4] and diagnosis [5]. The validity and reliability of ultrasound versus gold standards, such as magnetic resonance imaging (MRI) in morphology assessment [6,7] or electromyography (EMG) in contraction assessment [8,9] have demonstrated excellent correlation, raising the possibility of ultrasonography being used in clinical settings with confidence.
Physiotherapy is a branch of health sciences committed to evidence-based practice [10,11]. Some procedures in clinical practice are invasive (e.g., dry needling, electrolysis) and could be a source of bias [12,13]. These invasive procedures, when associated with diagnostic imaging, have been shown to reduce the risk of bias and improve patient experience [12]. Therefore, ultrasound training in physiotherapy has seen increased demand in several countries [14,15,16,17,18,19] along with the use of these invasive procedures. However, as the demand for ultrasound training increases in physiotherapy, security and recommendations has been published through an international reasoning for ultrasound internationally [14]. The validity of novice versus expertise in ultrasound training has been stablished (intraclass correlation coefficient (ICC) > 0.9) [15,16], providing benefits for professionals and patients. Likewise, high rates of validity were reported for minimal training in physiotherapy students [17] when measuring the abdominal wall with dynamic maneuvers.
One main concern about ultrasound training is the access to training and associated high cost [18,19]. To access to an ultrasound course, physiotherapists have to travel and, therefore, stop working for several days. To reduce economic and family costs, an online method for teaching ultrasound in physiotherapy has been proposed [18,19].
A lack of opportunities to access to ultrasound education has been detected in surveys accomplished in different countries [20,21,22,23] reflecting the need to offer other possibilities in addition to onsite. Jedrzejczack et al. [20] stated that physical therapist needs more education and training in the use of real time ultrasound for clinical practice. The current educational offer has a medical perspective [23]; therefore, it is necessary to develop training tools for physiotherapists in clinical and research settings [21,22]. Online education could solve the problems described in previous studies and allows to attend in a low-cost context and repeat the contents indefinitely, in addition to self-evaluate when desired. Increasing the opportunities to access and acquire knowledge.
Previous studies have obtained good results using digital sonography education in other health sciences specialties [24,25,26] and even in physiotherapy [27]. Nevertheless, our study differs from these by being the first to cover all of the basic anatomical regions used in physical therapy.
Finally, we are experiencing a confinement due to the COVID-19 situation, where all onsite education in universities has been canceled. This has forced the educational community to abruptly adapt to on-line education, which could be a risk of bias that should be explored.
Thus, this study aims to analyze whether an online ultrasound training method produces results comparable to the traditional method used to train physiotherapists.
2. Materials and Methods
In this study, we aimed to compare the onsite education method versus the online method through immediate evaluation in physiotherapy, measuring how many hours are needed in online versus traditional onsite education in order to achieve the same results. The online method must be evaluated in order to make conclusions regarding its validity in physiotherapy according to the required training time.
An exploratory observational prospective study was developed, with an onsite group (or traditional education) and an online group. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines were applied [28]. All participants received the information sheet and consent form was signed.
There were two levels in each group, basic and advanced, which were classified according to the number of hours of education received (18/36 h) coincident with the European Federation of Societies for Ultrasound in Medicine and Biology guidelines in education in muscle skeletal sonography [26], with basic education in muscle skeletal sonography and basic approaches for 18 h and advance education with clinical cases review and more difficult approaches for 36 h. The onsite group received the programmed theoretical contents. The scheduled practice hours with sonographs were conducted by a musculoskeletal sonographer with 10 years’ experience. The online group used video recordings uploaded to a website with the same theoretical contents, scheduled practice hours (without the intervention of the musculoskeletal sonographer) as the onsite group. Practice time with sonographs devices (for both groups) was scheduled before the practical exams. The anatomical regions about which attendants were educated were upper limbs (shoulder, elbow, and wrist) and lower limbs (hip, knee, and ankle) for the two groups.
Participants in the onsite weekend courses were recruited via internet advertisements. The “Kahoot” website was used as a gamification tool for the theoretical exam, which adjusted to accommodate for assessment of the two different levels; therefore, online and onsite groups took the same exam (see
The online group received a call for performing practices with a sonograph, allowing the same timing as the onsite group without teacher intervention. The same theoretical and practical exams were also conducted as for the onsite group.
All measurements were compared with the results of a physiotherapist and musculoskeletal sonographer with 10 years’ experience in sonography.
The study was approved by the Francisco de Vitoria University Ethics Committee (N° 37/2018) and was conducted in accordance with the Helsinki Declaration. A research grant from the Docent and Innovation Department of the same University was used to create the online contents and purchase the required materials.
Statistical Analysis
The sample size calculation was using the rule of thumb [25] by Nunnally and Bernstein using 7 variables, 4 groups, and 5 volunteers by group. For statistical analysis, R Version 5.3.1 software was used (R Foundation for Statistical Computing, Institute for Statistics and Mathematics, Vienna, Austria).
The level of significance was set at p < 0.05. The Shapiro–Wilk test was used to determine the non-normal distribution of outcome variables. Qualitative variables were described as absolute values and frequencies and quantitative variables were described as median and interquartile range (IQR).
Since the study design did not randomly identify the subjects in each of the groups, inverse probability of treatment weighting (IPTW) was applied through the generalized boosted model (GBM) machine learning technique, estimating the average treatment effect (ATE) since the students had the same probabilities of receiving some of the training options [24,29]. The tolerance limit for the balanced standardized mean difference (SMD) between groups was set at 0.2 [24,30] and the effect size was calculated adjusted by the Coefficient of Determination (R2).
The degree of agreement between the scores obtained by the students in each group was evaluated with the ICC. The magnitude of the agreement was defined as either poor (<0.5), moderate (0.5–0.75), good (0.75–0.9), or excellent (>0.9).
3. Results
The sample included 136 attendants divided into four unbalanced groups, with a maximum of 52 students in onsite group 1 and 23 in online group 1 for the basic level. The average age ranged between 29.5 (25, 35.25) years in onsite group 1 and 34 (28.5, 40.5) in the Online 1 group, with a generally higher percentage of women in all groups (Table 1).
The average score ranks for the Kahoot_basic test ranged from 6 (5.5, 7) points in the Onsite 1 group to 7 (6.5, 7.5) points in the Online 1 group compared to scores of 6 (5, 6.5) points in the Onsite 2 group and 7 (6.5, 8.5) points in the Online 2 group of the Kahoot_advanced test (Table 2).
The IPTW model stabilized with 3000 iterations in all cases and, in the online groups, this convergence was more accentuated. There was no overlap between the coefficients of each group compared to the rest, which seems to indicate the presence of a large number of extreme values outside the common support area (Figure 1).
The propensity score weighting of the covariates between the groups and baseline variables clearly diminished the differences between them against the unweighted values (Figure 2).
In the Kahoot_basic scores, the model detected a positive, although not very high, difference between the Online 1 and Onsite 1 groups (0.6708, SE = 0.2680) and a significant (t = 2.502, p = 0.0149) and negative and small difference between the Onsite 1 group and the total effect (−0.3354, SE = 0.1340) and a significant difference (t = −2.502, p = 0.0149) with a low effect size (R2adjusted = 0.103). In the Kahoot_advanced scores, the model detected a positive difference between the Online 2 and Onsite 2 groups (1.1832, SE = 0.5425) and a significant (t = 2.181, p = 0.0353) and small negative difference between the Onsite 2 group and the total effect (−0.3354, SE = 0.1340) as well as a significant difference (t = −2.502, p = 0.0149) with a low effect size (R2adjusted = 0.125).
In the Hollow_structure, the model detected a small negative difference between the Online 1 group and the total effect (−0.9830, SE = 0.4386) and a significant difference (t = −2.241, p = 0.0267) with a low effect size (R2adjusted = 0.08). In the Solid_structure scores the model detected a small negative difference between the Onsite 2 group and the total effect (−1.23148, SE = 0.43687) and a significant difference (t = −2.819, p = 0.00558) with a low effect size (R2adjusted = 0.025). In the Solid_structure_surface_distance the model detected a positive difference between the Online 2 and Onsite 1 groups (1.5026, SE = 0.5822) and a significant difference (t = 2.5809, p = 0.0113) with a low effect size (R2adjusted = 0.016).
The results showed significant differences between some of the training groups and the outcome variables; however, it was confirmed that despite the weighted adjustment of the covariates, very large differences remained in the SMDs, above the tolerable limit of 0.2, both between the covariables and between the groups; in fact, between the groups with the least differences between them (Onsite 1 and Onsite 2) no significant differences were detected in any of the outcome variables.
All indices showed very low ICC values, below 0.3 (except ICC in the variable Between_structures_distance in the Online 2 group, which had a score of 0.9) and were not considered significant.
4. Discussion
Nowadays, teaching is shifting towards a non-classroom model due to technological advances and, unfortunately, the situation imposed due to COVID-19. Thus, studying the degree of confidence of an online teaching model and comparing it with a traditional teaching model is especially relevant. The most important results from this study were those from the propensity analysis, which gave higher probability of obtaining better results in terms of theoretical as well as practical exams for both online groups compared to the traditional teaching groups. These results could be due to the fact that online groups had the possibility to watch the videos more than once compared to the onsite groups who attended the theoretical classes only at the scheduled time. To the best of our knowledge, this is the first time that results of online versus onsite education, both theoretical and practical, have been compared, indicating comparable results.
Other studies [29,31] comparing traditional teaching with online teaching showed good results for both groups but without differences between groups. Fernandez-Lao et al. [27] used a mobile app for ultrasound exploration versus traditional exploration, and found no significant difference between groups in the acquisition of theoretical and practical knowledge. Furthermore, a differentiating element between studies was that while Fernández-Lao et al. only evaluated the shoulder, in our study, we evaluated the main basic anatomical regions used in physical therapy. Mckiernan et al. [32] also found an improvement for both groups receiving face-to-face and DVD instruction, though no differences between groups. This discrepancy between our study and theirs when comparing groups could be explained by the possibility our online group had to practice with the ultrasound devices prior to exams.
In other health sciences specialties, digital media have been used for training, finding increases in sensitivity of 86.6% and specificity of 85.6% for the detection of pneumothorax [33], as well as in transesophageal echocardiography [31], where an improvement of 60% was found; however, they used attendants who had previous or basic ultrasound education.
The results obtained in our study for the scores of the theoretical exam did not indicated very large differences between the Online 1 and Onsite 1 groups (0.6708, SE = 0.2680) but significant differences were found between the Online 2 and Onsite 2 groups (1.1832, SE = 0.5425), probably due to cumulative education time in favor of the online group. These results are in discordance with previous studies [29,31]; however, Vegas et al. [31] and Krishnan et al. [33] concluded that online teaching could be beneficial when associated with traditional training.
For the practical exam, our results showed significant differences between the Online 2 and Onsite 1 groups (1.5026, SE = 0.5822) in the Solid_structure_surface_distance variable. The other measurements did not show significant differences between groups. This could be interpreted as indicating that a minimum level of ultrasound education must be offered to trust the precision of practical results that attendants can achieve.
The results of ICC obtained in the Online 2 group were excellent (ICC = 0.94) and the differences were considered high compared with the other groups. This could be due to bad visibility when performing the in-depth measurement or some bias in the collection of the results since the attendants (online and onsite) sent the scores through a Google form. Which could be avoided by a test-retest prior the sampling. It is possible that some samples obtained used different references in the phantom, despite the indications received. Therefore, it would be necessary to improve the agreement at the measurement points considering the differences in the ICCs (Table 3).
Finally, this study advances the training possibilities and access to knowledge for physiotherapists, supporting the results obtained in previous studies; nevertheless, the access to ultrasound education for physiotherapists has been shown to be difficult [20,21,22,23,34]. Highlighting the need to facilitate the access to physiotherapists. To advance this technique of ultrasound training, it would be necessary to carry out studies on a larger sample using a training model that is not heavily dependent on a device that is sometimes not accessible, i.e., such as through virtualization of the ultrasound scanner. Future research should explore the potential cost savings of this method compared with traditional onsite methods.
5. Limitations
A limitation of this study is the sample size, as a larger sample size would have allowed more confidence in the results. Moreover, attendants should have had prior training for image acquisition from phantoms. Hence, it would be interesting to examine the re-test time for phantom measurement acquisition; this could give better outcomes in the analysis intraclass coefficient results.
6. Conclusions
Our results showed that an online ultrasound training method for physiotherapists is valid and reliable when compared to an onsite ultrasound training method. Our results also showed that the Online 2 group obtained better scores for the theoretical and the practical exams than the Onsite 2 group, while in the Online 1 group, scores for the theoretical exam were similar to those of the Onsite 2 group. The online ultrasound training method showed excellent ICC for the Online 2 group, meaning that future online courses should be based on a minimum of 36 h of training for optimal education.
Author Contributions
Conceptualization, S.F.-C.; methodology, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G., and T.G.-I.; software, S.F.-C., J.N.C.-Z.; validation, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; formal analysis, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; investigation, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; resources, S.F.-C., D.P.-M. and T.G.-I.; data curation, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; writing—original draft preparation, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; writing—review and editing, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; visualization, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; supervision, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; project administration, S.F.-C., J.N.C.-Z., D.P.-M., A.A.-O., A.F.-G. and T.G.-I.; funding acquisition, S.F.-C., D.P.-M. and T.G.-I. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by the Department of Docent Innovation of the Francisco de Vitoria University.
Acknowledgments
We acknowledge the facilities and devices borrowed by the enterprise Helios Electromedicina (Madrid) for the philanthropic collaboration to develop the sampling.
Conflicts of Interest
The authors declare no conflict of interest.
Figures and Tables
Enlarge this image.Figure 2. Standardized mean difference between inverse probability of treatment (IPTW) model and unweighted data.
Baseline characteristics of the sample.
| Onsite 1 | Onsite 2 | Online 1 | Online 2 | ||
|---|---|---|---|---|---|
| n | 52 | 31 | 23 | 30 | |
| Age (median (IQR)) | 29.5 (25, 35.25) | 33 (28, 35.5) | 34 (28.5, 4.5) | 25.00 (22.25, 32.75) | |
| Sex (%) | Female | 17 (32.7) | 7 (22.6) | 8 (34.8) | 12 (40) |
| Male | 35 (67.3) | 24 (77.4) | 15 (65.2) | 18 (60) | |
| Study_years_starting (median (IQR)) | 4 (1, 9.25) | 8 (4, 12) | 4 (4, 4) | 10.00 (5.25, 13.75) | |
| Postgraduate_training (%) | Expert | 19 (36.5) | 7 (22.6) | 1 (4.3) | 1 (3.3) |
| Master | 16 (30.8) | 17 (54.8) | 0 (0) | 0 (0) | |
| PhD | 0 (0) | 0 (0) | 0 (0) | 1 (3.3) | |
| Weekend courses | 17 (32.7) | 7 (22.6) | 16 (69.6) | 21 (70) | |
| NA | 0 (0) | 0 (0) | 6 (26.1) | 7 (23.3) | |
| Previous_US_training (%) | No | 35 (67.3) | 4 (12.9) | 22 (95.7) | 29 (96.7) |
| Yes | 17 (32.7) | 27 (87.1) | 1 (4.3) | 1 (3.3) | |
| Type_of_US_traning (%) | Non regulated training | 8 (15.4) | 4 (12.9) | 1 (4.3) | 0 (0) |
| None | 32 (61.5) | 3 (9.7) | 22 (95.7) | 29 (96.7) | |
| Regulated training | 12 (23.1) | 24 (77.4) | 0 (0) | 0 (0) | |
| Self-training | 0 (0) | 0 (0) | 0 (0) | 1 (3.3) | |
| US_training_hours (%) | 15 to 20 | 7 (13.5) | 14 (45.2) | 1 (4.3) | 2 (6.7) |
| 21 to 40 | 4 (7.7) | 6 (19.4) | 0 (0) | 0 (0) | |
| Less than 15 | 9 (17.3) | 1 (3.2) | 1 (4.3) | 3 (10) | |
| More than 40 | 0 (0) | 7 (22.6) | 1 (4.3) | 0 (0) | |
| None | 32 (61.5) | 3 (9.7) | 20 (87) | 25 (83.3) | |
| Hours_week_using_US (%) | 1 to 4 | 13 (25) | 11 (35.5) | 2 (8.7) | 1 (3.3) |
| 5 to 10 | 3 (5.8) | 9 (29) | 1 (4.3) | 2 (6.7) | |
| 11 to 20 | 0 (0) | 3 (9.7) | 1 (4.3) | 0 (0) | |
| More than 20 | 0 (0) | 1 (3.2) | 0 (0) | 0 (0) | |
| Not use | 36 (69.2) | 7 (22.6) | 19 (82.6) | 27 (90) |
Data expressed as median and interquartile range (IQR) or with absolute and relative values (%).
Table 2Scores obtained in the different tests.
| Onsite 1 | Onsite 2 | Online 1 | Online 2 | |
|---|---|---|---|---|
| Kahoot_basic (median (IQR)) | 6 (5.5, 7) | - | 7 (6.5, 7.5) | - |
| Kahoot_advanced (median (IQR)) | - | 6 (5, 6.5) | - | 7 (6.5, 8.5) |
| Hollow_structure (median (IQR)) | 4.6 (4.1, 5.2) | 4.5 (4, 4.95) | 4.7 (4.4, 4.9) | 4.9 (4.62, 5.18) |
| Solid_structure (median (IQR)) | 3.8 (3.6, 4) | 3.8 (3.6, 4) | 4.45 (4.3, 4.7) | 4.45 (3.9, 4.7) |
| Between_structures_distance (median (IQR)) | 11.6 (11, 12.1) | 11.3 (11.07, 11.75) | 12.1 (11.83, 12.65) | 12.2 (11.8, 12.5) |
Data expressed with median and interquartile range (IQR) or with absolute and relative values (%).
Table 3ICC for each group and measurement.
| Hollow Structure | Solid Structure | Distance between Structures | Solid Structure Surface Distance | Hollow Structure Surface Distance | |
|---|---|---|---|---|---|
| Onsite 1 | 0.048 | 0.061 | 0.019 | 0.015 | 0.016 |
| Onsite 2 | 0.047 | 0.006 | 0.029 | 0.015 | 0.044 |
| Online 1 | 0.143 | 0.142 | 0.122 | 0.015 | 0.000 |
| Online 2 | 0.350 | 0.062 | 0.943 | 0.015 | 0.077 |
ICC: intraclass correlation coefficient.
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Pecos-Martin, Daniel 1
; Achalandabaso-Ochoa, Alexander 3
; Ferragut-Garcias, Alejandro 4 ; Gallego-Izquierdo, Tomás 1 1 Department of Physiotherapy and Nursing, Alcalá University, 28801 Alcalá de Henares, Spain;
2 Department of Physical Therapy, Universidad Francisco de Vitoria, Pozuelo de Alarcón, 28223 Madrid, Spain;
3 Department of Physiotherapy and Nursing, Alcalá University, 28801 Alcalá de Henares, Spain;
4 Departamento de Enfermería y Fisioterapia, Islas Baleares University, Palma de Mallorca, 07122 Islas Baleares, Spain;