Effect of the COVID‐19 induced phase of massive telehealth uptake on end‐user satisfaction

Abstract

Background

COVID‐19 has resulted in a massive increase in telehealth utilisation.

Aims

To determine the user and clinician satisfaction during this period and compare to a pre‐COVID‐19 cohort.

Methods

A prospectively collected voluntary questionnaire following the telehealth appointment at a tertiary‐level hospital with all adult and paediatric‐based specialities was conducted over two time periods: COVID‐19 (16 March 2020 to 15 April 2020) and pre‐COVID‐19 (1 January 2019 to 31 December 2019). There were four groups of participants: patients; parents; adult‐based clinicians; and paediatric‐based clinicians. The outcomes assessed included perceived standard of care, willingness for repeat telehealth consultations, and patient and parental perceptions of safety.

Results

Five thousand and thirty‐three telehealth consultations occurred in the COVID‐19 period with 1757 questionnaires completed, compared to 1917 consultations with 271 questionnaires completed in the pre‐COVID‐19 period. Clinicians were more likely to have previously used telehealth in both time periods than end‐users. In COVID‐19, 1240 actual onsite hospital outpatients' visits were prevented. All groups reported a good overall impression of the telehealth quality; patients/parents scored higher compared to clinicians: 3.6/4 versus 3.3/4, P = 0.02 (pre‐COVID‐19) and 3.3/4 versus 2.8/4, P = 0.001 (COVID‐19). The majority of patients and parents (90%, 1379/1528) felt safer by having a telehealth appointment compared to a face‐to‐face appointment in the COVID‐19 pandemic. All participant groups reported an overall good standard of care, good levels of engagement and were strongly willing to use telehealth again in both of the study time periods. Patients and parents consistently rated higher than clinicians.

Conclusions

During a rapid increase in its utilisation and scope due to the COVID‐19 pandemic, telehealth was generally well accepted by patients, parents and clinicians, which was consistent with pre‐COVID‐19 experiences.

Introduction

The utilisation of telehealth technology to deliver healthcare services at a distance is associated with bridging gaps of cost and social or cultural barriers to provide equity of access to healthcare regardless of geographical location. The COVID‐19 pandemic has resulted in a greater number of patients accessing telehealth consultations at Monash Health in Melbourne, Victoria, Australia. Renumeration for the provision of this service in Australia is predominantly through the Medicare Benefits Schedule (MBS) item numbers.¹ The temporary MBS items numbers implemented with the COVID‐19 pandemic response resulted in geographical location restrictions being removed leading to a change in the patient cohort.¹

Prior to the COVID‐19 pandemic in Victoria, less than 1% of all specialist consultations had been conducted through telehealth.² Previous studies in telehealth have often been in either a regional or rural setting focussing on the access to specialist healthcare, reducing travel or time, economic factors and stakeholder satisfaction.³ However, it has been established that it is an acceptable model of care to both clinicians and patients, and previous research has centred on video outpatient consultations with stakeholder satisfaction, safety, acceptance and effectiveness reported over a broad range of specialities.⁴

The introduction of social distancing measures to minimise COVID‐19 transmission, high‐level statewide restrictions and lockdowns has resulted in healthcare services adjusting their approach for clinical consultation delivery.⁵ Telehealth has therefore been proposed as a solution for this by allowing patients and clinicians to maintain a safe distance and reduce unnecessary travel.

While the use of telehealth as a response to emergencies and disasters in the past has been reported, its use in a pandemic like COVID‐19 is unprecedented.⁶ The rapid expansion of telehealth within our healthcare setting gives a unique opportunity to observe telehealth in both paediatric and adult patients within a wide range of clinical specialties. The purpose of this study is to assess and compare the perceived level of care provided during a rapid expansion of telehealth during COVID‐19 pandemic to a pre‐COVID‐19 setting.

Methods

Institutional setting

This study was conducted in a large metropolitan, multi‐campus, tertiary health service with care provided to both paediatric and adult patient populations. The main mode of telehealth in our healthcare network is through synchronous web‐based video, using Web Real‐Time Communication (WebRTC) technology (Health Direct Videocall, Coviu, Sydney, NSW, Australia). Patients connect with the telehealth consultation through their own personal internet enabled device, and clinicians from any location. Patients are allocated to one of the 125 specialty virtual waiting rooms before the commencing of privately secured consultation with the clinician.

Questionnaire

The prospective electronic survey utilised in this study was one developed by our team and is currently used as a longitudinal prospective evaluation tool since 2015. Prior to COVID‐19 this was primarily paediatric based, but was expanded to include all services in March 2020. It was amended to include relevant COVID‐19 questions following institutional Human Research Ethical Committee review. The survey was voluntary at the end of all consultations to all telehealth users on an opt‐in basis. The utilised survey platform was Qualtrics Version 03.20 (Qualtrics, Provo, UT, USA). The survey questions are in detailed in Appendix 1.

Data capture time period

Telehealth activity during the study's data capture period was prospectively collected over a 4‐week period between 16 March and 15 April 2020. This corresponded to the initial time period for the implementation of the new COVID‐19 MBS item numbers. Additional prospectively collected data from 1 January 2019 to 31 December 2019 comprised a pre‐COVID‐19 comparative group.

Participants

Participants were categorised into four stakeholder groups:

Patient

Parent

Adult‐based clinician

Paediatric‐based clinician

Adult and paediatric clinicians were also categorised as ‘Monash Clinician’ to facilitate comparison with the pre‐COVID‐19 dataset, which contained only paediatric patients. A paediatric patient was categorised as ≤18 years old. For the secondary outcomes, participants were categorised into either pre‐COVID‐19 or COVID‐19 subgroups, in both patients and clinicians.

Outcomes measures

The primary outcome measure was to assess the perception of telemedicine during the initial implementation phase of the COVID‐19 pandemic. This included the standard of care, engagement, perceived safety, willingness to undertake telehealth consultation again, as well as the overall global impression. Previous experience with telehealth and confidence in technology was also noted. The overall quality of telehealth included a perceived global assessment of the consultation and its multiple components. These were measured on a 4‐point or 5‐point Likert scale with a higher number indicating a stronger positive outcome (Appendix 1). The secondary outcome measures included the financial and time‐based effect of telemedicine on patients or clinicians, with comparison to the pre‐COVID‐19 era. Cost saved was measured across four bandings (AU$): 1 ($0), 2 ($10–$60), 3 ($61–$120) and 4 ($121–$250). This was an estimate from the participant based on their previous experience. Time saved: 1 (2‐3 h); 2 (0.5 day); 3 (1 day); 4 (overnight trip); and 5 (>1 night). The time‐based effect on clinicians was assessed as an estimation of the additional time taken for telehealth. This was also measured across five bandings: 1 (less); 2 (equivalent); 3 (1–5 min); 4 (5–10 min); and 5 (>15 min).

Data analysis

The prospectively collected anonymised data were analysed using GraphPad Prism version 7.04 (GraphPad Software, La Jolla, CA, USA). Descriptive analysis of the data included distributions, proportions, range and mean with standard deviation (SD). Not all of the respondents completed all the sections therefore analysis occurred on an individual question basis. Data were analysed with the D'Agostino‐Pearson test for normality. Comparison between groups was performed using Mann–Whitney test or Chi‐squared test with a P‐value of <0.05 considered significant.

Human research ethical approval

Ethical approval was obtained from Monash Health Ethics for the quality assurance project (RES‐18‐0000‐498Q).

Results

Respondents and activity

A total of 5033 telehealth consultations occurred in the COVID‐19 study period with 1757 stakeholders completing the questionnaire and a response rate of 34.9% (1757/5033). These respondents included: 49.8% (875/1757) patients; 36.0% (632/1757) parents; 3.5% (62/1757) adult‐based clinicians; and 10.7% (188/1757) paediatric‐based clinicians. During the pre‐COVID‐19 period there were 1917 telehealth consultations, with 271 survey responses, and a response rate of 14% (271/1917). These included: parents 53/271 (19.6%); clinicians 161/271 (59.4%); and unclassified 57/271 (21%).

Previous experience of telehealth

In COVID‐19, a minority of patients and parents had utilised this form of consultation before (10.5%, 155/1478) compared to clinicians (85.4%, 210/246), P = <0.001. This finding was similar in the pre‐COVID‐19 group: 28% (20/71) patient/parent versus 86% (51/59) clinicians, P = 0.0001. A larger proportion of patients/parents had a previous experience of telehealth in the pre‐COVID‐19 group (28% vs 10.5%, P = <0.001).

Confidence in using technology

All groups were very confident in using technology. There was a difference between parents and paediatric clinicians (4.7/5 vs 4.4/5, P = <0.001), while no significant difference was noticed between patients and adult‐based physicians (4.6/5 vs 4.8/5, P = 0.3). There were no differences in the pre‐COVID cohort for patients/parents (4.6/5 vs 4.4/5, P = 0.09) or for clinicians (4.6/5 vs 4.6/5, P = 0.9).

Overall quality of telehealth (COVID‐19 cohort)

In the COVID‐19 group all participant groups reported a good quality overall. Patients and parents both rated it between ‘Good’ and ‘Great’, whereas clinicians rated it between ‘OK’ and ‘Good’. Patients had a higher overall impression compared to adult clinicians (3.3/4 vs 2.8/4, P = <0.001), as did parents compared to paediatric‐based clinicians (3.4/4 vs 2.7/4, P = <0.001). However, there was no difference between either the patients and parents (3.3/4 vs 3.4/4, P = 0.3) nor the adult and paediatric clinicians (2.8/4 vs 2.7/4, P = 0.6). The combined patient/parent group's perceived overall impression of the telehealth consultation was higher than the adult/paediatric clinicians (3.3/4 vs 2.8/4, P = 0.001) (Fig. 1). However, while this discrepancy was also consistent within the pre‐COVID‐19 groups (3.6/4 vs 3.3/4, P = 0.02), both parents/patients (3.6/4 vs 3.3/4, P = 0.001) and adult/paediatric clinicians (3.3/4 vs 2.7/4, P = 0.0004) reported a higher overall impression of telehealth prior to COVID‐19.

[Image Omitted. See PDF.]

Perceived safety of telehealth (COVID‐19 cohort)

The majority of the patients 89.9% (836/931) and parents 88.4% (593/667) used telehealth due to COVID‐19 restrictions. Of this, 81.8% (758/927) of patients and 73.5% (482/656) of parents would have attended the appointment if it was available face to face, indicating their clinical urgency, P = <0.001. Therefore, telehealth potentially prevented 1240 actual outpatient visits to the hospital. In total, 89% (793/891) of patients and 92% (586/637) of parents felt safer being able to see their clinician through telehealth due to COVID‐19 (P = 0.06).

Standard of care

All participant groups reported an overall good standard of care in telehealth compared to a face‐to‐face consultation (Table 1). Within the COVID‐19 groups, patients reported a higher level of care when compared to adult clinicians, as did parents compared to paediatric clinicians. However, there was no difference between the adult and paediatric clinicians nor the patient and parent groups. The difference in perceived standard of care between the providers (clinicians) and the end‐users were consistent in the pre‐COVID‐19 groups (4.2/5 vs 4.7/5, P < 0.0001). However, the standard of care was higher in pre‐COVID‐19 groups when compared to COVID‐19 groups for both clinicians (4.2/5 vs 3.7/5, P = 0.0001) and end‐users (4.7/5 vs 4.3/5, P = 0.001).

Table 1. Comparison between the subgroups for willingness to use telehealth in the future, perceived engagement and perceived standard of care in COVID‐19

	Willingness to use again			Perceived engagement			Perceived standard of care
Subgroup analysis	N	Mean (SD)	P‐value	N	Mean (SD)	P‐value	N	Mean (SD)	P‐value
Patients	874	4.7 (0.6)		797	4.5 (0.8)		875	4.3 (1.0)
versus			0.1			0.9			<0.001
Adult physician	62	4.6 (0.5)		62	4.3 (1.3)		61	3.9 (1.1)
Parents	623	4.8 (0.6)		623	4.3 (1.0)		630	4.3 (1.0)
versus			0.6			0.8			<0.001
Paediatric physician	187	4.8 (0.4)		188	4.0 (1.4)		187	3.6 (1.3)
Patients	874	4.7 (0.6)		797	4.5 (0.8)		875	4.3 (1.0)
versus			0.009			<0.001			0.4
Parents	623	4.8 (0.6)		623	4.3 (1.0)		630	4.3 (1.0)
Adult physician	62	4.6 (0.5)		62	4.3 (1.3)		61	3.9 (1.1)
versus			0.03			0.2			0.09
Paediatric physician	187	4.8 (0.4)		188	4.0 (1.4)		187	3.6 (1.3)

Telehealth consultation engagement

All participant groups reported a good level of engagement and felt they responded well to having the consultation through video compared to a face‐to‐face consultation (Table 1). Patients/parents had a higher perceived engagement than clinicians during the COVID‐19 period (4.4/5 vs 4.0/5, P = 0.03), but this was reversed in the pre‐COVID‐19 period (4.0/5 vs 4.5/5, P = 0.0002). In addition, patients/parents reported a higher level of engagement prior to COVID‐19 (4.4/5 vs 4.0/5, P < 0.00001). However, this was not the same for clinicians (4.0/5 vs 4.5/5, P = 0.2).

Willingness to use telehealth in the future

All participant groups reported that they were strongly willing to use telehealth again (Table 1). An overall comparison between our two study periods showed that patients/parents were more willing to use telehealth prior to COVID‐19 (4.9/5 vs 4.7/5, P = 0.007). However, this was not observed with clinicians (4.8/5 vs 4.7/5, P = 0.7).

Cost saved by patient/parent

There was a significant difference in the comparison of the cost saved by parents/patients between the pre‐COVID‐19 period and COVID‐19 period: ‘AU$61–$120’ versus ‘no money saved’, P < 0.0001. Parents/patients in the pre‐COVID‐19 groups saved significantly more money than COVID‐19 groups across all categories (Table 2).

Table 2. Comparison of cost saved (1, $0; 2, $10–$60; 3, $61–$120; 4, $121−$250) between pre‐COVID‐19 and COVID‐19 period for patients/parents

Categories of cost saved		Mean (SD)
Travel	Pre‐COVID‐19 (n = 72)	3.1 (0.8)
	versus
	COVID‐19 (n = 1343)	1.9 (0.5)
Accommodation	Pre‐COVID‐19 (n = 67)	2.9 (1.3)
	versus
	COVID 19 (n = 1163)	1.2 (0.5)
Loss of income	Pre‐COVID‐19 (n = 67)	2.5 (1.4)
	versus
	COVID 19 (n = 1208)	1.5 (0.8)
Other	Pre‐COVID‐19 (n = 50)	2.0 (1.2)
	versus
	COVID‐19 (n = 1025)	1.3 (0.6)
Overall	Pre‐COVID‐19 (n = 256)	2.7 (1.2)
	versus
	COVID‐19 (n = 4739)	1.5 (0.7)

Time saved by patient/parent and extra time taken by clinicians

For both the patient and parent subgroups, more time was saved during the pre‐COVID‐19 period (‘a day’ vs ‘2–3 h’, P < 0.0001). However, there was no significant difference between the patient and parent subgroups either in the pre‐COVID‐19 period (‘a day’ vs ‘a day’, P = 0.9) or COVID‐19 period (‘2–3 h’ vs ‘2–3 h’, P = 0.9). Clinicians are using the ‘same amount of time’ as they would in a face‐to‐face consultation (Table 3). However, the ‘clinical consultation’ and ‘additional administration for clinicians’ required significantly more time in the COVID‐19 period (Table 3).

Table 3. Comparison of extra time used by clinicians during different part of a telehealth consult between pre‐COVID‐19 and COVID‐19 periods (1, less time; 2, same amount of time; 3, 1−5 min more; 4, 5−10 min more; 5, more than 15 min more)

Categories of extra time used		Mean (SD)	P‐values
Set up/connection	Pre‐COVID‐19 (n = 50)	2.4 (1.2)	0.5
	versus
	COVID‐19 (n = 423)	2.5 (1.2)
Clinical consultation	Pre‐COVID‐19 (n = 50)	1.6 (0.7)	0.009
	versus
	COVID‐19 (n = 415)	2.0 (1.0)
Additional admin for clinicians	Pre‐COVID‐19 (n = 49)	2.2 (0.9)	0.0006
	versus
	COVID‐19 (n = 409)	2.7 (1.1)
Additional admin for non‐clinicians	Pre‐COVID‐19 (n = 49)	2.4 (1.1)	0.07
	versus
	COVID‐19 (n = 410)	2.7 (1.0)
Overall	Pre‐COVID‐19 (n = 198)	2.2 (1.0)	<0.0001
	versus
	COVID‐19 (n = 1657)	2.5 (1.1)

Discussion

The global COVID‐19 pandemic has radically altered many aspects of modern life including when, how and where individuals access healthcare at our institution. There is a desire for social distancing, protection of patients and their families but also the healthcare workers.⁷ It is likely that these changes will remain in place until the pandemic is controlled.⁵ Our telehealth service utilises videocalls as it has been previously established that effective communication is maintained when compared to face‐to‐face interactions.⁸ Although telephone consultations might be clinically acceptable, there is increased satisfaction with the additional visual component of the consultation as demonstrated in previous studies.⁹, ¹⁰

Some of our outcomes included both the cost and time saved by patients when using telehealth. Our results indicate a significant reduction in both of these during the COVID‐19 period, which might be a reflection of the geographical limitations being lifted with the MBS billing changes in response to COVID‐19.¹ This is indicated by the majority of respondents selecting COVID‐19 as the reason for utilising telehealth. They also felt safer by using telehealth, which may potentially help to decrease anxiety. Interestingly, a significant proportion (78.3%) would have still attended the hospital for a face‐to‐face appointment. Therefore, this telehealth expansion initiative has potentially saved 1240 actual visits to the hospital which significantly contributed to social distancing compliance. Patients/parents saved significantly less money than they would have pre‐COVID‐19 secondary to the removal of geographical restrictions with the resultant decreased travel and accommodation costs. However, they still had some cost savings including the cost of travel to and from the hospital . Although more people were potentially working from home due to the statewide lockdown, the limitations on the number of accompanying persons or siblings would have had a significant effect. This is consistent with previous economic studies, specific to rural/remote settings, which showed telehealth leading to cost saving from a societal perspective.¹¹

There was a good level of satisfaction with the overall quality and standard of care provided in both the patient and parents' groups. All respondents were willing to use telehealth again. A strength of this study is the inclusion of both paediatric and adult services, as previous reported stakeholder satisfaction has often been limited to only one of these groups. In contrast to our findings, a previous paediatric‐based questionnaire reported only 38.4% of parents would prefer a video consultation to avoid a clinic visit.¹² A large adult‐based study revealed only 43–63% of participants would use video consultations with their GP depending on the condition's urgency.¹³ In our study, although only the minority had previously used telehealth, the majority were satisfied with the level of care they received and are willing to use telehealth again. This difference could be secondary to the pre‐existing established telehealth service or the effects of the COVID‐19 pandemic.

Differences between adult and paediatric cohorts were also seen. Less parents would have attended the appointment if it was not through telehealth due to the COVID‐19 pandemic. Patients reported a higher level of perceived engagement with the telehealth appointment, which may be secondary to the effect of the parent acting as the paediatric patients' advocate. Despite this difference, the parents were more willing to use telehealth again when compared to patients in our study. Paediatric clinicians also reported a greater willingness to participate in telehealth again compared to their adult‐based colleagues although both rated it extremely highly. This reflects that paediatric but not adult‐based telehealth was well established at our institution for 4 years prior to COVID.

Another strength of the study is the existing long‐standing establishment of the questionnaire, which allowed comparison between the two periods and therefore the effects of COVID‐19. In the pre‐COVID‐19 period, many end‐users had chronic medical conditions and were rurally based. Therefore, they were more likely to have used telehealth previously and this was reflected in our findings . Likewise, they also had a higher overall global impression of telehealth, perceived it to have a high standard of care, better engagement, and were more willing to participate in telemedicine again. Clinicians, on the other hand, had similar level of prior experience to telemedicine across both periods and therefore, for them, the primary outcomes were mostly unaffected in the setting of the COVID‐19 pandemic. Therefore, as telehealth becomes more established the end‐users may have an improved acceptance and higher satisfaction.

The clinician perceptions of telehealth are a critical factor in determining their acceptance of telehealth as a model of care.¹⁴ We found an acceptance of telehealth by clinicians, in contrast to another study where there has been equivalence.¹⁵ This previous study reported a loss of personal engagement for clinicians whereas our study revealed consistent high levels of engagement in all four groups. Our clinicians still rated the overall standard of care highly and were satisfied with the care that they could provide, although this was less than their respective end‐user groups. This could be the result of the different clinical care model with telehealth, possible preconceived bias or capturing only the initial implementation phase. Video latency resulting in subtle changes in communication styles has also been suggested as an explanation for reduced satisfaction of telehealth.¹⁶ Clinicians and end‐users also perceived a higher standard of care in the pre‐COVID compared to the COVID period. This may reflect that during COVID‐19 telehealth was used as a substitute for face‐to‐face appointments even when there was a clinical preference and need for a face‐to‐face appointment. It will be crucial to explore the important issue of clinician's perceptions and acceptance of telehealth with focus groups and longer‐term follow‐up studies.

A recent systemic review examining barriers to the adoption of telemedicine worldwide showed that limitations due to technology‐related challenges was the most frequent barrier identified by staff and providers.¹⁷ However, the majority of clinicians in our study had used telehealth previously in their practice and there was a high confidence in using technology in all of the individual stakeholders. This is supported by the fact that no extra time was spent on the set up/connection as well as the systemic administrative tasks such as billing, booking and scheduling, when compared to face‐to‐face consultations for the clinicians. This was consistent across both periods, which demonstrates that while there was a significant telehealth consultation utilisation during the COVID‐19 period we were well prepared and equipped to respond rapidly and efficient.¹⁸ It should be noted that the use of telehealth has been previously suggested as a response to emergency situations, as highlighted by the North Atlantic Treaty Alliance Multinational Telemedicine System created in 2000.¹⁹ This adaptability and efficiency are some of the key factors influencing the success and sustainability of telehealth and telemedicine, and perhaps therefore signal the possibility of mass‐expansion of telehealth service, even in a post COVID‐19 setting.³

Although our voluntary questionnaire utilised established constructs there are limitations to its use especially with participation bias, but the large numbers of responses in our study minimise this effect.¹¹, ¹⁵ Furthermore, our study was conducted when the geographical restrictions for MBS funding had been removed and as such our dataset is unique.

Conclusion

Our study has revealed a good overall acceptance, satisfaction and willingness for patients, parents and clinicians to use telehealth in the future. As more and more people are exposed to telemedicine and the rapid cyclical disruption of COVID‐19 eventually plateaus, we should look at telehealth beyond a pandemic setting and support its mass expansion in the digital era.

Appendix 1

Questionnaire questions for patients, parent and clinicians

Questionnaire questions for Patients/Parents	Questionnaire questions for clinicians
Prior use of telehealth Before this telehealth video consultation did you ever take part in a telehealth video consultation (either with Monash Hospital or elsewhere)? Yes or No	Prior use of telehealth Ever run a video consultation before (e.g. private practice, elsewhere)? Yes or No
COVID‐19 questions Are you attending through telehealth to avoid coming into the hospital because of coronavirus? Would you have attended this appointment if it was at Monash hospital? Do you feel safer that you are able to see you doctor/clinician through telehealth? Yes or No	COVID‐19 questions Are you providing this consultation in response to the COVID‐19 pandemic? Yes or No
Satisfaction, engagement, willingness to do again, overall satisfaction and confidence. To what extent do you agree with the following statements about your telehealth video consultation. I received the same standard of care during this video consultation as I would have done in a face‐to‐face consultation I am willing to take part in further video consultations if my doctor suggests it My child, I (the patient) was engaged and responded well to having the consultation through video I am generally confident with using computers and technology (Five‐point Likert scale) Strongly disagree Somewhat disagree Neither agree nor disagree Somewhat agree Strongly agree	Satisfaction, engagement, willingness to do again, overall satisfaction and confidence. To what extent do you agree with the following statements about your telehealth video consultation. I was able to provide the same standard of care during this video consultation as I would have done in a face‐to‐face consultation I am willing to provide further video consultations My patient was engaged and responded well to having the consultation through video I am generally confident with using computers and technology (Five‐point Likert scale) Strongly disagree Somewhat disagree Neither agree nor disagree Somewhat agree Strongly agree
Overall quality of consultation Overall, how would you rate the quality of this video consultation? (Four‐point Likert scale) Terrible – it was impossible OK – it was adequate Good – it worked well Great – I loved it	Overall quality of consultation Overall, how would you rate the quality of this video consultation? (Four‐point Likert scale) Terrible – it was impossible OK – it was adequate Good – it worked well Great – I loved it
Cost saved Compared to a face‐to‐face consultation, did this save you any money in AUD? (not including the cost of internet, camera etc.) For travel? For lost income? For accommodation? For other? No money saved Saved $10 ‐ $60 Saved $61 ‐ $120 Saved $121 ‐ $250	Extra time taken About how much extra time did this consultation take? (compared to face‐to‐face) On set up/connection On the clinical consultation On any additional administration for the clinician (faxing script etc.) On any additional administration for other staff (billing, booking, scheduling etc.) Less time Same amount of time 1–5 min more 5–10 min more More than 15 min more
Time Saved If this consultation had been at Monash Hospital, how much time would you have been away from home? For parent/carer For patient/child 2–3 h Half a day A day An overnight trip More than one night away from home	Location of Consultation Where were you for this telehealth video consultation? (selected choices) My home My office The hospital outpatient clinic rooms Somewhere else (e.g. school, work – please specify)

References

1. Department of Health. MBS changes factsheet. 2020 [cited 2020 May 20]. Available from URL: http://www.mbsonline.gov.au/internet/mbsonline/publishing.nsf/Content/Factsheet-TempBB

2. Wade V, Soar J, Gray L. Uptake of telehealth services funded by Medicare in Australia. Aust Heal Rev Publ Aust Hosp Assoc 2014; 38: 528– 32.

3. Bradford NK, Caffery LJ, Smith AC. Telehealth services in rural and remote Australia: a systematic review of models of care and factors influencing success and sustainability. Rural Remote Health 2016; 16: 3808.

4. Kruse CS, Krowski N, Rodriguez B, Tran L, Vela J, Brooks M. Telehealth and patient satisfaction: a systematic review and narrative analysis. BMJ Open 2017; 7: e016242.

5. Dickens BL, Koo JR, Wilder‐Smith A, Cook AR. Institutional, not home‐based, isolation could contain the COVID‐19 outbreak. Lancet Lond Engl 2020; 395: 1541– 2.

6. Lurie N, Carr BG. The role of Telehealth in the medical response to disasters. JAMA Intern Med 2018; 178: 745– 6.

7. Lancet T. COVID‐19: protecting health‐care workers. Lancet Lond Engl 2020; 395: 922.

8. Dirnberger J, Waisbren S. Efficacy of telehealth visits for postoperative care at the Minneapolis VA. Am J Surg 2020; 220: 721– 4. https://doi.org/10.1016/j.amjsurg.2020.01.015.

9. Rush KL, Howlett L, Munro A, Burton L. Videoconference compared to telephone in healthcare delivery: a systematic review. Int J Med Inform 2018; 118: 44– 53.

10. Gilbert AW, Billany J, Adam R, Martin L, Tobin R, Bagdai S et al. Rapid implementation of virtual clinics due to COVID‐19: report and early evaluation of a quality improvement initiative. BMJ Open Qual 2020; 9: e000985.

11. Langbecker D, Caffery LJ, Gillespie N, Smith AC. Using survey methods in telehealth research: a practical guide. J Telemed Telecare 2017; 23: 770– 9.

12. Brownlee GL, Caffery LJ, McBride CA, Patel B, Smith AC. Telehealth in paediatric surgery: accuracy of clinical decisions made by videoconference: Telehealth in paediatric surgery. J Paediatr Child H 2017; 53: 1220– 5.

13. Castle‐Clarke S. What will new technology mean for the NHS and its patients? London, UK: The Health Foundation, The Institute for Fiscal Studies, The King's Fund and the Nuffield Trust; 2018.

14. Wade VA, Eliott JA, Hiller JE. Clinician acceptance is the key factor for sustainable telehealth services. Qual Health Res 2014; 24: 682– 94.

15. Donelan K, Barreto EA, Sossong S, Michael C, Estrada JJ, Cohen AB et al. Patient and clinician experiences with telehealth for patient follow‐up care. Am J Manage Care 2019; 25: 40– 4.

16. Shaw SE, Seuren LM, Wherton J, Cameron D, A'Court C, Vijayaraghavan S et al. Video consultations between patients and clinicians in diabetes, cancer, and heart failure services: linguistic ethnographic study of video‐mediated interaction. J Med Internet Res 2020; 22: e18378.

17. Scott Kruse C, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M. Evaluating barriers to adopting telemedicine worldwide: a systematic review. J Telemed Telecare 2018; 24: 4– 12.

18. Wosik J, Fudim M, Cameron B, Gellad ZF, Cho A, Phinney D et al. Telehealth transformation: COVID‐19 and the rise of virtual care. J Am Med Inform Assoc 2020; 27: 957– 62.

19. Doarn CR, Latifi R, Poropatich RK, Sokolovich N, Kosiak D, Hostiuc F et al. Development and validation of telemedicine for disaster response: the North Atlantic Treaty Organization multinational system. Telemed J E Health 2018; 24: 657– 68.