1. Introduction
Breast cancer (BC) represents one of the most frequently diagnosed cancers in women worldwide [1]. According to the most recent data from the European Cancer Information System (ECIS), there were approximately 355,460 new cases of BC diagnosed across Europe in 2020, with 34,088 of those cases occurring in Spain [2]. However, thanks to early diagnosis and therapeutic advances, BC survival has increased in recent years [3], with a survival rate of around 85% [4]. Increasing prevention and treatment for BC have lowered mortality, but the diagnosis and treatment continue to have a significant impact in many areas of patients’ lives (physical, emotional, cognitive and social) [5]. The diagnosis of BC, which in most cases necessitates an effort to adjust and adapt to the new situation [5,6], is typically perceived as a traumatic event with a significant impact on the health-related quality of life of the women who suffer from it, making them more vulnerable to the potential consequences of using biased or low-quality health information.
Person-centred care (PCC) is defined as the provision of care that considers a patient’s clinical needs, life circumstances, and personal values and preferences [7]. A central component of PCC is to ensure quality communication between patients and healthcare professionals, with the aim of fostering the process of shared decision making (SDM) [8]. SDM-based interventions, such as patient decision aids (PtDAs), have been shown to improve patients’ knowledge about available treatments and their benefits/risks, decisional conflict and other decisional process variables [9]. There is a need to develop interventions to increase knowledge about PCC and digital health literacy (DHL) [10,11], particularly in chronic pathologies such as BC, where the impact of their diagnosis or treatment may increase the number of queries on the Internet and directly influence the understanding of health information [12]. Health literacy (HL) integrates the skills and motivation to find, understand, evaluate and use health information. As a result, HL facilitates informed decision making and improves the ability to manage and address health disparities, giving patients more autonomy and empowerment to take responsibility for their own health, as well as the health of their families and communities. In turn, low HL impacts health outcomes and health-related costs, leading to inefficient healthcare utilization and delivery [13]. DHL is an extension of HL that employs the same operational definition but in the context of information and communication technology resources. It involves both the provision of information and the degree to which information is understood. When these skills are lacking, technology solutions have the potential to either promote or hinder HL [14]. Due to the complexity of health information, it is recommended that DHL interventions be based on a design of co-creation of resources, websites and health tools through collaborative work with patients, allowing them to improve the medical care they receive [15,16,17].
Massive open online courses (MOOCs) are designed to engage a large number of participants learning remotely, offering the general population, clinical subpopulations or health professionals good quality knowledge on health issues through interactive and flexible technological resources, with little or no prior learning required [18]. To date, most MOOCs have been developed for the education of medical students and health professionals [19,20], but they have also been directed at the general population or clinical subpopulations, showing positive effects in several areas such as healthy nutrition habits [21], self-management of diabetes [22] or learning risk factors for dementia [23].
As has been observed in some projects with other populations, the development of educational interventions with a MOOC based on a co-creation design, which combines several resources in different formats and adapts to different educational, cultural levels and needs of the users, could be a strategy to face the HL, self-care and empowerment challenges for women with BC. One example is the IC-Health European project (
In recent years, the framework of participatory action research has been used for the development of eHealth. It is an approach that involves collaboration to develop a process through the construction of knowledge and social change in a community following a cyclical approach and involving stakeholders as co-investigators in the process [27,28]. As occurs in other participatory processes, the co-design of health interventions contributes to improving the services offered, to the extent that they are adjusted to the needs and priorities of its participants while incorporating their own skills [29,30,31].
In general, digital interventions, such as MOOCs, have the potential to improve the quality of life and outcomes for women with BC by providing access to information from anywhere at any time, thereby increasing accessibility and flexibility, as well as support to complement traditional medical treatments. Therefore, the aim of this study is to co-create a MOOC of PCC and DHL for and with women with BC.
2. Materials and Methods
2.1. Design
The MOOC was co-created using a modified experience-based design approach [32]. The co-creation process was divided into three sequential phases: (a) exploratory phase, (b) development phase and (c) evaluation phase (see Section 2.3).
2.2. Participants and Recruitment
Adult women (≥18 years) in any of their cancer stages and BC survivors (regardless of DHL level and knowledge about PCC), their families/carers and any healthcare professionals involved in the management of BC (oncologists, gynaecologists, nurses, psycho-oncologists, etc.) were invited to participate voluntarily in the MOOC co-creation process. A theoretical sample optimized the maximum variability of sociodemographic and clinical profiles (age, educational level, time since diagnosis and active treatment) of women with BC. The recruitment was carried out via snowball sampling [33] through healthcare professionals and expert patients (BC survivors) between May and June 2020. Participants signed an informed consent declaration.
2.3. Procedure
The co-creation process was carried out in three online sessions of 120 min each (via the Zoom platform due to the COVID-19 pandemic and delivered by members of the research team) between June 2020 and March 2021 and was supported by a Moodle platform.
The first session (exploratory phase) was held in June 2020 and consisted of (i) a brief presentation of the participants; (ii) identifying the different diagnosis, treatment and long-term follow-up paths for BC represented through a patient journey map (PJM)—a scheme that aims to reflect the care pathway followed by a person [5,32]—based on their experiences, emotions, feelings and thoughts; (iii) exploring their empowerment and information needs in each phase of the disease; (iv) and exploring patients’ information needs and experiences on patient empowerment and SDM. Health professionals did not participate in the development of the PJM; they offered advice and their experiences on the most frequent concerns found in clinical practice with these patients, according to the phase of the disease.
In the second session (development phase), held in July 2020, the participants reviewed the PJM and designed the structure and proposed the contents of the MOOC (self-care, myths related to BC, strategies to improve DHL, etc.) based on the empowerment and information needs identified in the first session and their previous experiences managing BC information online. At the end of this session, participants were encouraged to continue the process of co-creation online between July and December 2020 through a Moodle platform where the participants were registered and which they accessed with an individual username and password (assessment phase). The research team developed and shared some content proposals weekly for the different units of the MOOC, and participants were asked to provide feedback and/or new content proposals (see Section 3.3). Initially, the content of the units was presented in infographic format (see Supplemental Materials Figure S1) and was mainly related to PCC, self-care and DHL applied to BC. Once all the suggestions for improvement provided by the participants on the content were compiled, a graphic designer developed videos and edited the infographics to provide them with interactivity and visually improve their appearance. Updated contents were shared again with the participants in March 2021. Through questionnaires on the Moodle platform (see Section 2.4.2), they could give feedback on the definitive contents of the MOOC (see Section 2.4.1).
A third session (evaluation phase) was held in March 2021 to offer final feedback about the content and interface of the MOOC (acceptability pilot) and to evaluate the experience in the co-creation of the MOOC by means of specific questionnaires (see Section 2.4). Four gift cards were raffled off as a token of appreciation during this last meeting.
2.4. Measures
2.4.1. Experience in the Co-Creation Process
A 13-item questionnaire was specifically developed to explore patients’ and healthcare professionals’ experience in the co-creation process. The first 6 items were measured using a 5-point Likert scale (from “strongly disagree” to “strongly agree”), addressing satisfaction with communication, objective adequacy, usefulness of patient involvement in the co-creation process, importance of co-creation to design relevant content for patients, self-perception of increased knowledge and feeling of being part of the team project. The following 4 items were also assessed on a 5-point Likert scale (from “insufficient” to “excellent”) and were related to participants’ opinions on the quality and clarity of the co-creation sessions, the methodology employed, the interactions between participants and the researchers’ implication. The last 3 items were open-ended questions about what participants liked the most and the least about the MOOC co-creation process, which aspects they found most useful and which aspects could be improved in the co-creation process (see Section 3.4).
2.4.2. Acceptability Pilot of the MOOC
The MOOC’s acceptability was evaluated using a specific scale created in the context of the project following the technology acceptance model’s (TAM) methodology [34] and based on previous related studies [35,36]. This scale assessed factors such as ease of navigation, clarity of objectives and language, appropriateness of learning activities and quizzes, and other characteristics of the MOOC. The acceptability questionnaire, answered by both patients and healthcare professionals, included 18 items: the first 15 were rated on a 5-point Likert scale, and the last 3 items were open-ended questions about strengths and weaknesses, improvement suggestions and the main points learned throughout the MOOC (see Section 3.5).
2.5. Analysis
The PJM and MOOC content were progressively developed in conjunction with participants. A draft was created with the information obtained from the online co-creation sessions. The different sections of the PJM summarize the experiences of participants with BC or survivors. The research group reviewed the contributions of the participants and proposed a draft version based on a PCC framework. Subsequently, this version of PJM and MOOC content was reviewed by all participants through an iterative process until consensus was reached.
For the experience in the co-creation process and the acceptability pilot of the MOOC measures, means and standard deviations (SD) were calculated for all items assessed, and we also analysed the response distribution for each item.
3. Results
Twenty-eight participants from Tenerife and Gran Canaria (Canary Islands, Spain) were contacted between May and June 2020, of whom 19 participated in the co-creation process: 17 patients (Table 1) and two healthcare professionals (nurses from gynaecology and breast pathology units; mean age 40 (1.41) years and with more than 10 years of professional experience).
3.1. Patient Journey Map
Points of contact, experience with healthcare received, emotions, feelings and thoughts, diagnostic and therapeutic treatments, and perception about own participation in shared decision making for the three stages of the trajectory of care of BC (early detection and diagnosis, treatment and long-term follow-up) were collected on the co-designed PJM (Figure 1).
3.1.1. Early Detection and Diagnosis Stage
Most of the participants received their diagnosis during routine controls (specialized care) or as a result of the presence of symptoms (primary care), and the main emotions that emerged during this time were shock, anxiety, uncertainty and worry about the future. The main diagnostic techniques that the participants underwent were physical examination (palpation), imaging tests (mammography and ultrasound) and biopsy. The experiences collected about the healthcare received in this stage were related to the perception of professionalism, friendliness, a predisposition to resolve doubts and the transmission of calm and encouragement from the healthcare professionals who attended to them. However, the participants expressed that there were other drawbacks in the medical care received at this time related to the challenges of early detection and the complexity of some administrative processes (e.g., medical appointments). Some participants expressed that they would have liked more advice from medical staff. Other participants expressed that they felt involved in the decision-making process in this phase, and this helped them accept the disease and have trust in the therapeutic approach to be used.
3.1.2. Treatment Stage
Participants identified the involvement of other healthcare professionals (e.g., oncology, gynaecology, surgery and rehabilitation, among others). While uncertainty remained the predominant emotion in this stage, other emotions started to emerge as well, including concern for appearance and shock by the physical changes that were occurring as a result of the therapeutic techniques used in this stage (e.g., chemotherapy, radiotherapy, surgery, etc.). In general, the participants experienced empathetic care and a certain psychological accompaniment by the healthcare professionals who assisted them. The participants felt more involved in the decision-making process in the gynaecology units than in the oncology units. They all concurred that the experience of informed participation in their treatment process was positive.
3.1.3. Long-Term Follow-Up Stage
The main experience was less follow-up by healthcare professionals, giving rise to feelings of helplessness or loneliness and uncertainty about self-care. Other concerns, such as going back to work or looking for a new job more adapted to their health needs, were shared among the participants. The treatments at this stage focused on breast reconstruction surgery and medication. All the participants said they had received limited information on self-care, medical care to follow from this stage and possible new treatments required by healthcare professionals. However, they commented that at this stage they felt empowered to choose the aspects of their health in which they wanted to be involved, leading them to request personalized attention and to ask questions in order to be more involved in the decision-making process.
3.1.4. Recommendations of the Participants for Other Women with BC or Survivors
Additionally, and at their own initiative, the participants in the co-creation sessions provided a series of recommendations or tips for other women diagnosed with BC and suggested their inclusion in the MOOC as another resource. These tips were about family, social, work and empowerment areas and specifically for each of the stages worked (Figure 2).
3.2. Empowerment and Information Needs
Figure 3 shows the empowerment and information needs identified in each phase. The main empowerment needs identified were related to strategies for emotional management and guidelines for self-care throughout the process from diagnosis until long-term follow-up. The main information needs were related to the lack of understanding of the meanings of biomarkers, parameters and acronyms found in reports, as well as medical jargon, treatment options and the likelihood of cancer recurrence. The need to have guidelines for accessing information and support resources available online, including association websites and online experiences of other women with BC, was highlighted.
3.3. MOOC Content Development
Between July and October 2020, a weekly activity was published on the Moodle platform to carry out the process of co-design of the MOOC content. Table 2 shows the themes of these activities. Finally, the MOOC was composed of five units: (i) BC (definition, types and stages, diagnostic process, treatments, myths, etc.), (ii) PCC (definition, implementation strategies, tips for preparing consultations with the healthcare professional, etc.); (iii) DHL (definition, guidelines to improve each skill, etc.), (iv) self-care (management of physical side effects, emotional management, etc.) and (v) experiences and advice from patients in different areas (healthcare, family, social and work area) and moments of the disease (diagnosis, treatment and long-term follow-up).
3.4. Experience in the Co-Creation Process
Data was available for seventeen participants (89.47%) (Table 3). All of them strongly agreed or agreed that the general objectives of the project were adequate (item 2) and that the participation of women who have or have had BC is useful for the development of a MOOC on this content (item 3). More than 88% of the participants strongly agreed or agreed that being part of the MOOC co-creation process made the content more relevant to them (item 4) and rated the quality of the activities carried out in the co-creation process (item 7) and the methodology applied (item 8) as very good or excellent. Regarding open questions, participants appreciated the way their experiences were incorporated into the MOOC and how they felt part of something meaningful, sharing experiences with other women in similar situations (item 11). In order to fully engage in the co-creation process, participants expressed that they would have liked to attend a face-to-face session. Additionally, some participants found it challenging to devote more time to the MOOC due to personal issues (item 12). See Supplementary Materials Table S1 to consult illustrative quotes from participants’ responses to open questions.
3.5. Acceptability Pilot of the MOOC
Data was available for seven participants (36.84%) (Table 4). Combining the “totally agree” and “agree” categories, most of the participants positively evaluated the acceptability of the MOOC in terms of language, content, relevance, proposed activities and suitability of the MOOC objectives. Regarding open questions, most participants emphasized the usefulness of the MOOC’s content (especially related to SDM) and the way it is presented (through infographics and other audio-visual materials) as strengths. Nevertheless, one participant pointed out some navigation difficulties, while another emphasized the lengthy process (item 16). When it was possible, improvements suggested by participants were implemented, such as adding an initial summary of the MOOC’s content (item 17). All the contents were mentioned as important topics learned after completing the MOOC (item 18). See Supplementary Materials Table S2 to consult illustrative quotes from participants’ responses to open questions.
4. Discussion
This study presents the development of a MOOC aimed to improve the DHL of women with BC. We used a co-creation approach involving 17 patients and survivors and two nurses. In order to inform the content of the MOOC, we explored participants’ perceptions of the extent they were involved in the decision-making process, as well as their feelings, emotions and information needs throughout the therapeutic process. Most participants indicated that the MOOC co-creation experience was positive and made them feel involved in the project, and they positively valued the final product. Similar results were obtained by our team with other MOOCs developed for pregnant and lactating women [26] and people with type-2 diabetes [25], including larger samples than the one used in this study. In these two studies, participants’ self-perceived DHL significantly improved after completing the MOOC development compared to baseline. Future work is warranted to evaluate the effectiveness of this MOOC at improving BC patients’ actual DHL (not only self-perceived), objective knowledge of the disease and treatments, and their involvement in treatment decisions.
Women in this study pointed out information needs concerning different stages of the cancer, from diagnosis to long-term follow-up, as shown in previous studies [37,38]. Increasingly, these patients want to be involved in the decisions related to their health, and some studies have focused on involving the patient experience to improve the healthcare they receive [31,39]. As a result of an exchange of information and values between patients and healthcare providers, SDM engages patients as partners in their own care and optimizes the decision-making process [40]. To support SDM and the use of PtDA in the practice, it is important that patients also have a certain level of HL to increase patient empowerment and allow them to adopt a more participatory role in their healthcare [41]. Online interventions that provide information and support to women with BC appear to cushion the uncertainty they experience at different stages of the disease, and MOOCs can be an effective educational resource for meeting these unmet needs and promoting both DHL and SDM processes [24,39].
The PJM considered the evolving requirements for empowerment during the stages of diagnosis, treatment and long-term follow-up. Knowledge of the patients’ experiences, through a PJM, facilitates the identification of key moments in which to provide more precise information [5]. As we have seen in the results of this study, depending on the individual experiences of each woman, the care received during various BC periods could be perceived as more or less satisfactory. Based on our results, women with BC positively valued the experience of participating in the co-creation process of the MOOC, which made the content more relevant to them. This result aligns with previous evidence suggesting that a user-centred design process involves the participation of groups of users throughout the entire development cycle, during which they describe the context in which the generated resources will be used, their needs as users, and take part in user tests [42,43]. These are all contributions for designing and building health information technology through iterations [44].
This intervention represents an opportunity to reach a larger population that, due to health, availability and/or travel circumstances, may find it impossible to attend another type of face-to-face training on this subject. Technology provides great options for enhancing patient care; however, disparities in access and DHL continue to negatively impact vulnerable populations because of potential barriers in the digital sphere for those with low HL [11]. This problem can be especially aggravated as more information is provided online and healthcare professionals must be involved in the development of these skills in their patients with BC, but they also require support and a strategy at the institutional level. Therefore, healthcare organizations must prioritize achieving accessibility for all patients when designing eHealth services [11].
In this regard, the integration of educational materials designed by a representative sample of the target population to which they are addressed makes this proposal an opportunity to contribute to obtaining relevant health results for both affected patients and their healthcare professionals and, ultimately, decision-makers with financial capacity. From a managerial perspective, healthcare organizations should reframe their strategies, procedures and approaches, embracing a patient-centred perspective to become health literate [45]. From a policy perspective, it suggests that individual HL and organizational HL should be handled as two complementary tools to empower people and to engage them in self-care and health policy making [45].
The main strength of this project is having involved the intended audience in the creation of MOOCs, which enhances the significance of the material covered and how it is delivered. This is important because they have valuable insights and perspectives on the subject matter and can provide feedback on the relevance and effectiveness of the content and its delivery. This can lead to the creation of more engaging and effective MOOCs that better meet the needs and expectations of the intended audience. Nonetheless, there are several limitations to the study. Initially, it had been proposed that the co-creation process be based on face-to-face sessions with the participants followed by some online sessions through the Moodle platform. However, due to the COVID-19 pandemic, face-to-face sessions were replaced by online sessions carried out through the Zoom platform. This fact made the co-creation process last a few weeks longer than expected by adapting the work rhythms to the availability and web resources of the participants. However, the online sessions had several advantages: participants did not have to travel, the meetings were easier to organize and fewer financial resources were needed to support the development of the sessions. Another limitation is that, although all professionals related to BC were invited to participate, only two nurses did so. Perhaps the participation of other professionals involved in the process (e.g., gynaecologists and oncologists), as well as family members and/or caregivers, could have been beneficial for the generation of more useful resources. Even though women of all educational levels participated, the majority had higher education, so there was not much variability in this regard and lower educational levels may have been under-represented. In addition, there is a need for independent evaluation of acceptability to confirm the results obtained. Likewise, it is necessary to carry out an evaluation of the effectiveness of the MOOC with an independent sample that allows us to know if there really is an improvement in the levels of DHL and a change in knowledge in all the areas that are included in the different modules of the MOOC (BC, PCC, DHL, etc.).
5. Conclusions
The work carried out in this project is an example of how the development of educational interventions in MOOC format directed and designed by women with BC, with resources in different formats adapted to different educational/cultural levels and needs of the users, seems to be a viable strategy to generate higher-quality and useful resources for this population. The co-creation methodology and this type of resource aim to address the literacy and empowerment challenges of women with BC.
Conceptualization, Y.Á.-P., A.D.-D., A.T.-C. (Ana Toledo-Chávarri) and L.P.-P.; methodology, Y.Á.-P., A.D.-D., A.T.-C. (Ana Toledo-Chávarri), A.R.-S. and L.P.-P.; software, Y.Á.-P. and A.D.-D.; formal analysis, Y.Á.-P., A.D.-D. and A.R.-S.; resources, Y.Á.-P., A.D.-D., V.R.-G. and A.T.-C. (Alezandra Torres-Castaño); writing—original draft preparation, Y.Á.-P., A.D.-D., A.A.-S., V.R.-G. and A.T.-C. (Alezandra Torres-Castaño); writing—review and editing, Y.Á.-P., A.D.-D., A.T.-C. (Ana Toledo-Chávarri), A.R.-S. and L.P.-P.; supervision, Y.Á.-P. and L.P.-P.; project administration, Y.Á.-P. and L.P.-P.; funding acquisition, Y.Á.-P. All authors have read and agreed to the published version of the manuscript.
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of Hospital Universitario Nuestra Señora de la Candelaria (protocol code CHUNSC_2019_61) in 2019.
Informed consent was obtained from all subjects involved in the study.
The data presented in this study are available in
The authors wish to acknowledge and thank all the participants and Eduardo Negrín (graphic designer) in this project for their contribution and dedication.
The authors declare no conflict of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Characteristics of the participants (patients).
| Participants with BC or Survivors |
|
|---|---|
| Age (years) (mean, sd) | 47.88 (6.23) |
| Education level (n, %) | |
| Primary school | 1 (5.88) |
| High school | 2 (11.76) |
| Medium/high technical education | 6 (35.29) |
| University degree | 8 (47.07) |
| Employment status (n, %) | |
| Employed | 6 (35.29) |
| Unemployed | 3 (17.65) |
| Sick leave | 8 (47.06) |
| Civil status (n, %) | |
| Married/living with partner | 11 (64.71) |
| Separated or divorced | 3 (17.65) |
| Single | 3 (17.65) |
| Breast cancer diagnosis date (n, %) | |
| ≥5 years ago | 11 (64.71) |
| <5 years ago | 6 (35.29) |
| Currently undergoing treatment (n, %) | |
| Yes | 9 (52.94) |
| Hormone treatment (n, %) | 5 (55.56) |
| Chemotherapy/radiotherapy (n, %) | 2 (22.22) |
| Plastic surgery (breast reconstruction) (n, %) | 2 (22.28) |
| No | 8 (47.06) |
| Previous knowledge about PCC (n, %) | |
| Yes, but superficial, not deep | 3 (17.65) |
| Yes, with previous education | 1 (5.88) |
| No | 13 (76.47) |
Co-creation activities online in Moodle platform (development phase).
|
|
|
|
|
|
|
|
Experience in the co-creation process.
| Questions | Strongly Agree |
Agree |
Not Sure |
Disagree |
Strongly Disagree |
Mean a (sd) |
|---|---|---|---|---|---|---|
| 1. I am satisfied with the communication maintained with the researchers at the different moments of the project (in the online sessions, by WhatsApp, by the platform, by email, etc.) | 14 (82.35) | 2 (11.77) | 1 (5.88) | 4.76 (0.56) | ||
| 2. The general objectives of the project are adequate | 12 (70.59) | 5 (29.41) | 4.71 (0.47) | |||
| 3. I consider that the participation of women who have or have had breast cancer is useful for the development of a MOOC on this content | 16 (94.12) | 1 (5.88) | 4.94 (0.24) | |||
| 4. Being part of the process of co-creating the MOOC made the content more relevant to me | 11 (64.71) | 4 (23.53) | 1 (5.88) | 1 (5.88) | 4.41 (1.06) | |
| 5. Participating in the different online sessions has increased my knowledge about digital health literacy and this has helped me increase my ability to take control of my health | 11 (64.71) | 4 (23.53) | 2 (11.77) | 4.53 (0.72) | ||
| 6. The process of co-creating the MOOC made me feel part of the project | 8 (47.06) | 6 (35.29) | 2 (11.77) | 1 (5.88) | 4.18 (1.07) | |
|
Excellent
|
Very good
|
Good
|
Low
|
Insufficient
|
Mean a (sd) | |
| 7. In general, how would you rate the quality of the activities carried out in the online sessions, the documents and visual resources that were used and the clarity of the presentation and the dynamics carried out? | 8 (47.06) | 7 (41.18) | 2 (11.77) | 4.35 (0.7) | ||
| 8. How would you rate the online methodology that has been carried out for the development of the MOOC materials? | 8 (47.06) | 7 (41.18) | 1 (5.88) | 1 (5.88) | 4.29 (0.85) | |
| 9. How would you rate the level of interaction and participation of the rest of the participants in the development of the MOOC? | 2 (11.77) | 9 (52.94) | 6 (35.29) | 3.76 (0.66) | ||
| 10. How would you rate the involvement of the research team in the development of the MOOC? | 14 (82.35) | 3 (17.65) | 4.82 (0.39) | |||
| 11. Open question: What did you like most about the process of joint creation of the Online Course? What aspects do you consider most useful? | ||||||
| 12. Open question: What did you like least about the process of joint creation of the Online Course? What aspects do you think could be improved? | ||||||
| 13. Open question: If you have any further comments regarding your participation in this process, please include them | ||||||
a Higher scores indicate more positive rating (range 1–5 for all items).
Acceptability of the MOOC.
| Questions | Totally Agree |
Agree |
Not Sure |
Disagree |
Totally Disagree |
Mean a (sd) |
|---|---|---|---|---|---|---|
| 1. The language on the MOOC was easy to understand | 5 (71.43) | 2 (28.57) | 4.43 (0.98) | |||
| 2. The MOOC is easy to navigate, and the information was clearly organized | 4 (57.14) | 2 (28.57) | 1 (14.29) | 4.43 (0.79) | ||
| 3. The information presented was organized in a coherent and clear way | 4 (57.14) | 2 (28.57) | 1 (14.29) | 4.43 (0.79) | ||
| 5. The objectives of the MOOC were clear | 5 (71.43) | 1 (14.29) | 1 (14.29) | 4.57 (0.79) | ||
| 6. The content of the MOOC was consistent with the proposed objectives and with the people to whom the MOOC is directed | 6 (85.71) | 1 (14.29) | 4.86 (0.38) | |||
| 7. I find the MOOC interesting | 5 (71.43) | 1 (14.29) | 1 (14.29) | 4.57 (0.79) | ||
| 8. This MOOC has met my expectations | 5 (71.43) | 1 (14.29) | 1 (14.29) | 4.57 (0.79) | ||
| 9. The description of the content of each unit and of the activities were clear, avoiding possible errors in their interpretation | 5 (71.43) | 1 (14.29) | 1 (14.29) | 4.57 (0.79) | ||
| 10. There was consistency between the name of a section or link and the content that was displayed or the site to which it was directed | 5 (71.43) | 1 (14.29) | 1 (14.29) | 4.57 (0.79) | ||
| 11. The activities proposed in each unit were useful to deepen the understanding of the content worked | 5 (71.43) | 1 (14.29) | 1 (14.29) | 4.57 (0.79) | ||
| 12. The final evaluation questions of each unit adequately assessed the material presented in the MOOC | 3 (42.86) | 2 (28.57) | 2 (28.57) | 4.14 (0.9) | ||
| 14. This MOOC has been able to improve my ability to self-care in my own health by learning to access relevant and reliable information that I can obtain from the Internet | 4 (57.14) | 2 (28.57) | 1 (14.29) | 4.43 (0.79) | ||
| 15. I would recommend this MOOC to other people | 6 (85.71) | 1 (14.29) | 4.86 (0.38) | |||
|
Very high quality
|
High quality
|
Not Sure
|
Low quality
|
Very low quality
|
Mean a (sd) | |
| 4. The quality of the general design of the course and its contents seemed to me | 4 (57.14) | 2 (28.57) | 1 (14.29) | 4.43 |
||
|
Appropriate
|
Insufficient, requires spending very little time
|
Excessive, requires spending too much time
|
Mean a (sd) | |||
| 13. The amount of time needed to spend to view all the MOOC content is | 6 (85.71) | 1 (14.29) | 2.86 |
|||
| 16. Open question: Please provide a short summary of the strengths and weaknesses of the MOOC | ||||||
| 17. Open question: Please provide brief suggestions on how to improve the MOOC | ||||||
| 18. Open question: What are the main points that you have learned through this MOOC? | ||||||
a Higher scores indicate more positive rating (range 1–5 for items 1–3 and 5–15; range 1–3 for item 4).
Supplementary Materials
The following supporting information can be downloaded at:
References
1. Sociedad Española de Oncología Médica (SEOM). Las Cifras del Cáncer en España. Available online: https://seom.org/images/Cifras_del_cancer_en_Espnaha_2021.pdf (accessed on 21 December 2022).
2. European Cancer Information System (ECIS). Cancer Burden Statistics and Trends across Europe. Available online: https://ecis.jrc.ec.europa.eu/ (accessed on 21 December 2022).
3. Clèries, R.; Rooney, R.M.; Vilardell, M.; Espinàs, J.A.; Dyba, T.; Borras, J.M. Assessing predicted age-specific breast cancer mortality rates in 27 European countries by 2020. Clin. Transl. Oncol.; 2018; 20, pp. 313-321. [DOI: https://dx.doi.org/10.1007/s12094-017-1718-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28726040]
4. Gómez-Acebo, I.; Dierssen-Sotos, T.; Palazuelos-Calderón, C.; Pérez-Gómez, B.; Amiano, P.; Guevara, M.; Molina, A.J.; Domingo, L.; Fernández-Ortiz, M.; Moreno, V. et al. Tumour characteristics and survivorship in a cohort of breast cancer: The MCC-Spain study. Breast Cancer Res. Treat.; 2020; 181, pp. 667-678. [DOI: https://dx.doi.org/10.1007/s10549-020-05600-x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32356254]
5. Ciria-Suarez, L.; Jiménez-Fonseca, P.; Palacín-Lois, M.; Antoñanzas-Basa, M.; Fernández-Montes, A.; Manzano-Fernández, A.; Castelo, B.; Asensio-Martínez, E.; Hernando-Polo, S.; Calderon, C. Breast cancer patient experiences through a journey map: A qualitative study. PLoS ONE; 2021; 16, e0257680. [DOI: https://dx.doi.org/10.1371/journal.pone.0257680]
6. Campbell-Enns, H.J.; Woodgate, R.L. The psychosocial experiences of women with breast cancer across the lifespan: A systematic review. Psychooncology; 2017; 26, pp. 1711-1721. [DOI: https://dx.doi.org/10.1002/pon.4281] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27648726]
7. Doyle, C.; Lennox, L.; Bell, D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open; 2013; 3, e001570. [DOI: https://dx.doi.org/10.1136/bmjopen-2012-001570]
8. Elwyn, G.; Durand, M.A.; Song, J.; Aarts, J.; Barr, P.J.; Berger, Z.; Cochran, N.; Frosch, D.; Galasiński, D.; Gulbrandsen, P. et al. A three-talk model for shared decision making: Multistage consultation process. BMJ; 2017; 359, j4891. [DOI: https://dx.doi.org/10.1136/bmj.j4891]
9. Stacey, D.; Légaré, F.; Lewis, K.; Barry, M.J.; Bennett, C.L.; Eden, K.B.; Holmes-Rovner, M.; Llewellyn-Thomas, H.; Lyddiatt, A.; Thomson, R. et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst. Rev.; 2017; 2017, CD001431. [DOI: https://dx.doi.org/10.1002/14651858.CD001431.pub5]
10. Liu, C.; Wang, D.; Liu, C.; Jiang, J.; Wang, X.; Chen, H.; Ju, X.; Zhang, X. What is the meaning of health literacy? A systematic review and qualitative synthesis. Fam. Med. Community Health; 2020; 8, e000351. [DOI: https://dx.doi.org/10.1136/fmch-2020-000351]
11. Smith, B.; Magnani, J.W. New technologies, new disparities: The intersection of electronic health and digital health literacy. Int. J. Cardiol.; 2019; 292, pp. 280-282. [DOI: https://dx.doi.org/10.1016/j.ijcard.2019.05.066]
12. Basagoiti, I. Alfabetización en Salud de la Información a la Acción; ITACA/TSB: Valencia, Spain, 2012; ISBN 978-84-695-5267-4
13. Sørensen, K.; Van den Broucke, S.; Fullam, J.; Doyle, G.; Pelikan, J.; Slonska, Z.; Brand, H. Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health; 2012; 12, 80. [DOI: https://dx.doi.org/10.1186/1471-2458-12-80]
14. Dunn, P.; Hazzard, E. Technology approaches to digital health literacy. Int. J. Cardiol.; 2019; 293, pp. 294-296. [DOI: https://dx.doi.org/10.1016/j.ijcard.2019.06.039]
15. Halvorsrud, K.; Kucharska, J.; Adlington, K.; Rüdell, K.; Brown Hajdukova, E.; Nazroo, J.; Haarmans, M.; Rhodes, J.; Bhui, K. Identifying evidence of effectiveness in the co-creation of research: A systematic review and meta-analysis of the international healthcare literature. J. Public Health; 2021; 43, pp. 197-208. [DOI: https://dx.doi.org/10.1093/pubmed/fdz126] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31608396]
16. Lee, D. Effects of key value co-creation elements in the healthcare system: Focusing on technology applications. Serv. Bus.; 2019; 13, pp. 389-417. [DOI: https://dx.doi.org/10.1007/s11628-018-00388-9]
17. Kinnane, N.A.; Milne, D.J. The role of the Internet in supporting and informing carers of people with cancer: A literature review. Support. Care Cancer; 2010; 18, pp. 1123-1136. [DOI: https://dx.doi.org/10.1007/s00520-010-0863-4] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20336326]
18. Masters, K. A Brief Guide To Understanding MOOCs. Internet J. Med. Educ.; 2011; 1, 2. [DOI: https://dx.doi.org/10.5580/1f21]
19. Nieder, J.; Nayna Schwerdtle, P.; Sauerborn, R.; Barteit, S. Massive Open Online Courses for Health Worker Education in Low- and Middle-Income Countries: A Scoping Review. Front. Public Health; 2022; 10, 891987. [DOI: https://dx.doi.org/10.3389/fpubh.2022.891987]
20. Longhini, J.; Rossettini, G.; Palese, A. Massive open online courses for nurses’ and healthcare professionals’ continuous education: A scoping review. Int. Nurs. Rev.; 2021; 68, pp. 108-121. [DOI: https://dx.doi.org/10.1111/inr.12649] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33855697]
21. Adamski, M.; Truby, H.; Bennett, C.; Gibson, S. Exploring Impacts of a Nutrition-Focused Massive Open Online Course. Nutrients; 2022; 14, 3680. [DOI: https://dx.doi.org/10.3390/nu14183680]
22. Mackenzie, S.C.; Cumming, K.M.; Mehar, S.; Wilson, L.; Cunningham, S.G.; Bickerton, A.; Wake, D.J. Education at scale: Improvements in type 1 diabetes self-management following a massive open online course. Diabet. Med.; 2022; 17, e0267205. [DOI: https://dx.doi.org/10.1111/dme.14842]
23. Farrow, M.; Fair, H.; Klekociuk, S.Z.; Vickers, J.C. Educating the masses to address a global public health priority: The Preventing Dementia Massive Open Online Course (MOOC). PLoS ONE; 2022; 17, e0267205. [DOI: https://dx.doi.org/10.1371/journal.pone.0267205]
24. Perestelo-Perez, L.; Torres-Castaño, A.; González-González, C.; Alvarez-Perez, Y.; Toledo-Chavarri, A.; Wagner, A.; Perello, M.; Van Der Broucke, S.; Díaz-Meneses, G.; Piccini, B. et al. IC-Health Project: Development of MOOCs to Promote Digital Health Literacy: First Results and Future Challenges. Sustainability; 2020; 12, 6642. [DOI: https://dx.doi.org/10.3390/su12166642]
25. Alvarez-Perez, Y.; Perestelo-Perez, L.; Rivero-Santana, A.; Wagner, A.M.; Torres-Castaño, A.; Toledo-Chávarri, A.; Duarte-Díaz, A.; Alvarado-Martel, D.; Piccini, B.; Van den Broucke, S. et al. Cocreation of Massive Open Online Courses to Improve Digital Health Literacy in Diabetes: Pilot Mixed Methods Study. JMIR Diabetes; 2021; 6, e30603. [DOI: https://dx.doi.org/10.2196/30603] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34898453]
26. Álvarez-Pérez, Y.; Perestelo-Pérez, L.; Rivero-Santanta, A.; Torres-Castaño, A.; Toledo-Chávarri, A.; Duarte-Díaz, A.; Mahtani-Chugani, V.; Marrero-Díaz, M.D.; Montanari, A.; Tangerini, S. et al. Co-Creation of Massive Open Online Courses to Improve Digital Health Literacy in Pregnant and Lactating Women. Int. J. Environ. Res. Public Health; 2022; 19, 913. [DOI: https://dx.doi.org/10.3390/ijerph19020913]
27. Cordeiro, L.; Soares, C.B. Action research in the healthcare field. JBI Database Syst. Rev. Implement. Rep.; 2018; 16, pp. 1003-1047. [DOI: https://dx.doi.org/10.11124/JBISRIR-2016-003200] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29634517]
28. Oberschmidt, K.; Grünloh, C.; Nijboer, F.; van Velsen, L. Best Practices and Lessons Learned for Action Research in eHealth Design and Implementation: Literature Review. J. Med. Internet Res.; 2022; 24, e31795. [DOI: https://dx.doi.org/10.2196/31795]
29. Toledo-Chávarri, A.; Ramos-García, V.; Koatz, D.; Torres-Castaño, A.; Perestelo-Pérez, L.; Ramírez-Puerta, A.B.; Tello-Bernabé, M.-E.; García-García, J.-M.; García-García, J.; Pacheco-Huergo, V. et al. Co-Design Process of a Virtual Community of Practice for the Empowerment of People with Ischemic Heart Disease. Int. J. Integr. Care; 2020; 20, 9. [DOI: https://dx.doi.org/10.5334/ijic.5514]
30. Van Beusekom, M.; Cameron, J.; Bedi, C.; Banks, E.; Harris, R.; Humphris, G. Using Co-design With Breast Cancer Patients and Radiographers to Develop “KEW” Communication Skills Training. Front. Psychol.; 2021; 12, 629122. [DOI: https://dx.doi.org/10.3389/fpsyg.2021.629122]
31. Tsianakas, V.; Robert, G.; Maben, J.; Richardson, A.; Dale, C.; Wiseman, T. Implementing patient-centred cancer care: Using experience-based co-design to improve patient experience in breast and lung cancer services. Support. Care Cancer; 2012; 20, pp. 2639-2647. [DOI: https://dx.doi.org/10.1007/s00520-012-1470-3]
32. Bate, P.; Robert, G. Experience-based design: From redesigning the system around the patient to co-designing services with the patient. Qual. Saf. Health Care; 2006; 15, pp. 307-310. [DOI: https://dx.doi.org/10.1136/qshc.2005.016527]
33. Goodman, L.A. Snowball Sampling. Ann. Math. Stat.; 1961; 32, pp. 148-170. [DOI: https://dx.doi.org/10.1214/aoms/1177705148]
34. Venkatesh, V.; Morris, M.; Davis, G.; Davis, F. User Acceptance of Information Technology: Toward a Unified View. MIS Q.; 2003; 27, 425. [DOI: https://dx.doi.org/10.2307/30036540]
35. García Toribio, G.; Polvo Saldaña, Y.; Hernández Mora, J.J.; Sánchez Hernández, M.J.; Nava Bautista, H.; Collazos Ordóñez, C.A.; Hurtado Alegría, J.A. Medición de la usabilidad del diseño de interfaz de usuario con el método de evaluación heurística: Dos casos de estudio. Rev. Colomb. Comput.; 2019; 20, pp. 23-40. [DOI: https://dx.doi.org/10.29375/25392115.3605]
36. Yılmaz, N.G.; Sungur, H.; van Weert, J.C.M.; van den Muijsenbergh, M.E.T.C.; Schouten, B.C. Enhancing patient participation of older migrant cancer patients: Needs, barriers, and eHealth. Ethn. Health; 2022; 27, pp. 1123-1146. [DOI: https://dx.doi.org/10.1080/13557858.2020.1857338] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33316171]
37. Sheehy, E.M.; Lehane, E.; Quinn, E.; Livingstone, V.; Redmond, H.P.; Corrigan, M.A. Information Needs of Patients With Breast Cancer at Years One, Three, and Five After Diagnosis. Clin. Breast Cancer; 2018; 18, pp. e1269-e1275. [DOI: https://dx.doi.org/10.1016/j.clbc.2018.06.007] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30153977]
38. León-Salas, B.; Álvarez-Pérez, Y.; Ramos-García, V.; del Mar Trujillo-Martín, M.; de Pascual y Medina, A.M.; Esteva, M.; Brito-García, N.; González-Hernández, N.; Bohn-Sarmiento, U.; Biurrun-Martínez, M.C. et al. Information needs and research priorities in long-term survivorship of breast cancer: Patients and health professionals’ perspectives. Eur. J. Cancer Care; 2022; 31, e13730. [DOI: https://dx.doi.org/10.1111/ecc.13730]
39. Kemp, E.; Koczwara, B.; Butow, P.; Turner, J.; Girgis, A.; Schofield, P.; Hulbert-Williams, N.; Levesque, J.; Spence, D.; Vatandoust, S. et al. Online information and support needs of women with advanced breast cancer: A qualitative analysis. Support. Care Cancer; 2018; 26, pp. 3489-3496. [DOI: https://dx.doi.org/10.1007/s00520-018-4206-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29693203]
40. Abrams, E.M.; Shaker, M.; Oppenheimer, J.; Davis, R.S.; Bukstein, D.A.; Greenhawt, M. The Challenges and Opportunities for Shared Decision Making Highlighted by COVID-19. J. Allergy Clin. Immunol. Pract.; 2020; 8, pp. 2474-2480.e1. [DOI: https://dx.doi.org/10.1016/j.jaip.2020.07.003]
41. Schulz, P.J.; Nakamoto, K. Health literacy and patient empowerment in health communication: The importance of separating conjoined twins. Patient Educ. Couns.; 2013; 90, pp. 4-11. [DOI: https://dx.doi.org/10.1016/j.pec.2012.09.006] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23063359]
42.
43. Gulliksen, J.; Göransson, B.; Boivie, I.; Blomkvist, S.; Persson, J.; Cajander, Å. Key principles for user-centred systems design. Behav. Inf. Technol.; 2003; 22, pp. 397-409. [DOI: https://dx.doi.org/10.1080/01449290310001624329]
44. Smaradottir, B.F.; Bellika, J.G.; Fredeng, A.; Fagerlund, A.J. User-Centred Design with a Remote Approach: Experiences from the Chronic Pain Project. Integrated Citizen Centered Digital Health and Social Care; IOS Press: Amsterdam, The Netherlands, 2020; pp. 197-201.
45. Palumbo, R. Leveraging Organizational Health Literacy to Enhance Health Promotion and Risk Prevention: A Narrative and Interpretive Literature Review. Yale J. Biol. Med.; 2021; 94, pp. 115-128.
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Duarte-Díaz, Andrea 1
; Toledo-Chávarri, Ana 1 ; Abt-Sacks, Analía 1
; Ramos-García, Vanesa 1
; Torres-Castaño, Alezandra 1
; Rivero-Santana, Amado 1
; Perestelo-Pérez, Lilisbeth 2
1 Canary Islands Health Research Institute Foundation (FIISC), 38109 Tenerife, Spain; Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), 38109 Tenerife, Spain; The Spanish Network of Agencies for Health Technology Assessment and Services of the National Health System (RedETS), 28029 Madrid, Spain
2 Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), 38109 Tenerife, Spain; The Spanish Network of Agencies for Health Technology Assessment and Services of the National Health System (RedETS), 28029 Madrid, Spain; Evaluation Unit (SESCS), Canary Islands Health Service (SCS), 38109 Tenerife, Spain