Correspondence to Dr Samantha Streuli; [email protected]
Strengths and limitations of this study
We use a community co-design process that is supported by previous research to design a virtual reality education experience for an underserved community.
The qualitative aspects of this study provide in-depth information that can help to address complex vaccine hesitancy issues among Somali refugees.
We employed an iterative, phased approach to the development of educational content which allowed us to continuously assess the project and how it impacted the refugee community.
This project is a pilot study with a small sample size which will need to be expanded to truly understand the effect of our virtual reality health innovation on refugee vaccine perceptions and behaviours.
Introduction
Over 300 000 refugees enter the USA every year. While refugee resettlement has increased over time, events as recent as 2015 mark one of the largest exoduses of forced human migration, sparking a crisis as countries struggle to cope with the influx and the social and economic demands that coincide with human resettlement.1 What is largely missing in the public discussion is an appreciation that refugees face extraordinary challenges throughout their migration process from origination to destination. One challenge in particular—the access to proper healthcare—is critical, both for those who have been resettled in the past and those who have recently arrived.2 The other challenge is to collect and monitor healthcare data that can be accessed and disseminated to resettled communities for public health monitoring.3 In recognition of these challenges, important questions range from how to efficiently meet the healthcare demands of a growing population that is effective and sustainable to providing an engagement that uses culturally specific resources that simultaneously enhances health education and drives an increased level of trust in the local healthcare system.4
Upon resettlement in Western countries, many Somali refugees were faced with the widespread diagnosis of autism, which was unfamiliar to them prior to migration.5 On one hand, among a community cluster of resettled Somali children in Minneapolis, the prevalence of autism has exceeded 3% and has eclipsed the national average of 1.9%.6 On the other hand, this specific refugee community was targeted by anti-vaccination activists and the propaganda that vaccination is a cause of autism. This misinformation led to concern by Somali parents and distrust in the healthcare system, which has resulted in a propensity for vaccine hesitancy and non-acceptance.5–7 A lack of effective messaging towards what does cause autism and the dissemination of anti-vaccination propaganda were successful in decreasing vaccination rates in the Somali refugee population in Minnesota from 92% to 42% over the span of a decade.7 8
Effective healthcare education relies on various principles for building essential skills, including communication, assessing the accuracy of information, decision-making, planning, goal setting and self-management.9 At its core, health education must be simple, retained, and must be assimilated within those factors relevant to a given individual’s biases and acceptance towards the delivered message.9 New digital innovations such as virtual reality (VR) have emerged as tools to provide enriching and immersive learning experiences that promote understanding in acute conditions such as anxiety and post-traumatic stress disorder, and chronic conditions such as tobacco cessation.10–12 VR has also been used to provide patients with a more positive experience in hospital settings by using virtual nurses that patients can relate to.13 The ability for VR to increase information retention and to change health behaviours by engaging users in a non-healthcare environment makes VR a potentially valuable platform for visual health education. Therefore, our primary aim was to develop a customised, culturally and linguistically appropriate VR educational programme specifically focused on paediatric vaccinations, and to leverage community-based participatory research (CBPR) models and community co-designs to build, test and deploy VR at the community level among a group that are known to be vaccine hesitant or vaccine resistant.
Methods: study design
SHIFA (Arabic for ‘healing’) is a community innovation programme to design, deploy, and use new healthcare innovations with an inclusive model of community engagement. Within the current programme, community-based participation was incorporated along three complementary approaches.14 These include:
To define the healthcare access barriers in a resettled refugee community with a focus on paediatric vaccination.
Incorporate a community-based participatory model to develop the virtual reality content with community member co-design.
Develop a VR environment that is customised with cultural, linguistic and religious sensitivities to provide appropriate health education.
Qualitative feedback was derived through focus group discussions, interviews, and surveys developed by the investigators and was completed by community members and experts for analysis during each phase of technology development. Focus groups and interviews were conducted by Najla Ibrahim and Samantha Streuli. Najla Ibrahim is a Somali woman who holds a Master of Public Health (MPH) degree and is an expert in community public health issues. Samantha Streuli is a white woman who holds a PhD in anthropology and who has been working with the Somali community for 4 years. The majority of focus group and interview participants were unknown to Samantha and Najla prior to the research project, though some were acquaintances from previous work within the Somali community.
Patient and public involvement
The Somali community was involved in the research from its inception and was regularly consulted as the research was developed. Focus groups and interviews with the community informed the development of research questions, which prioritised the experiences and interests of the Somali community. We consulted with community members and community leaders when designing and conducting the study and developing survey, focus group discussion and interview questions to determine outcome measures. Somali community members were also involved in the recruitment to the study, as much of the recruitment happened via word of mouth. The results of the study will be presented to participants and other community members, who will be further consulted via focus groups on how to best disseminate results.
Participants
Inclusion/exclusion criteria
We identified groups of Somali individuals for community participation and VR co-design. Participants were required to be members of the Somali community in San Diego, California who were over the age of 18 years and were either (1) Somali refugees; (2) Somali immigrants or (3) US-born Somali Americans. We selected a group of seven expert advisors to assist in the development of VR. These advisors included: (1) researchers or other experts in autism and/or vaccination; (2) physicians serving the San Diego Somali community and in paediatric health; (3) leaders within the Somali community.
Setting
San Diego County is the 3rd largest metropolitan area in California and the 12th largest resettlement area in the USA. The organisation responsible for the development and execution of the programme is Somali Family Service (SFS), a fiscal sponsor for the East African Collaborative of eight-community organisations that aims to outreach, educate, and enrol refugees and immigrants in health insurance programmes. This specific community predominantly resides in City Heights, a subdivision of San Diego County that has a population of 75 000 individuals. Socioeconomic statistics of this region include a median household income of $39 330 (national median $55 322), 40% are immigrants and/or refugees, with 31% having an education level of a bachelor’s degree or greater.15
Recruitment
Somali individuals were recruited via telephone and through word of mouth by Somali community health workers (CHWs) and peers. As a non-interventional and non-comparative programme, we did not determine an a priori sample size and identified consecutive participants interested in participating within the community engagement design. As such, we included any individuals without a predefined minimum or maximum sample size. In total, we included 60 community participants and 7 expert advisors. Our target participants were: (a) parents of children between 0 and 2 years of age, (b) pregnant or (c) planning to become pregnant in the next 2 years; however, we included those Somali community members interested in issues of autism and/or vaccination regardless of parental status. All participants consented to have their names and contact information collected for future correspondence. This information was securely stored in an encrypted file and only used to recontact participants who agree to be recontacted. The research objectives, research participants’ rights and description of how data would be used were explained to all participants prior to participation. All participants provided verbal consent, which was approved by The University of California San Diego Institutional Review Board.
Expert advisors were identified by the SHIFA Project leadership team according to their particular areas of specialisation. Advisors signed consent forms to participate in the iterative development of educational content.
CBPR and community co-design
Our CBPR approach involved community members in each step of the design, iterative testing, and development of culturally and linguistically appropriate health education content. This approach to CBPR in the Somali community builds on public health work previously done within refugee communities.16 We developed a community co-design methodology that uses the principles of design thinking combined with community-based research to enable participants to be directly involved in the design and creation of content and products that are developed to benefit them.16 17 This co-design methodology was inclusive in that community members were asked to participate within content curation and to lead certain aspects of VR development.
Phased approach to VR development
We merged each aspect of CBPR and community co-designs within a phased approach to VR development (Vendor: INVIVO, Toronto, Canada). These phases included:
Phase 1: CBPR approaches to assess community needs and concerns important to paediatric vaccination.
Phase 2: VR modality determination for script development. The modality consisted of the type of VR experience and the script development included the specific educational content.
Phase 3: VR prototype and iterative feedback from storyboards, two-dimensional and three-dimensional animation, and visual and audio experiences that incorporate VR design elements including those factors that allow users to engage at different levels of health literacy.
Phase 4: final VR product and testing.
The objective within VR development was to use those tools and devices that were available to the community at large and to ensure cultural and linguistic sensitivities were incorporated.
Data collection and analysis
All data were collected electronically via tablet or computer at the time of focus group discussions, interviews and surveys. Focus group discussions and interviews were either recorded and transcribed, or extensive notes were taken on a computer in the event that groups or individuals declined to be recorded. Focus group discussions took place at the offices of SFS. In total, we conducted 5 focus groups with 57 participants. Some participants attended more than one focus group. Four of the focus groups were structured (see online supplemental materials for discussion guides), while one was an unstructured discussion of the prototype. Interviews also took place at the offices of SFS and were conducted with three Somali parents. While the majority of participants were female, we also had a small number of male participants.
Analyses were conducted thematically and iteratively using the content of the surveys, focus group discussions and interviews during the phased approach for VR development. This approach used five steps: (1) familiarisation, (2) coding, (3) theme development, (4) defining themes and (5) reporting.18 During the process of familiarisation, all sections of the interviews, focus group discussions and surveys relating to the experience of using VR were extracted. Coding was performed using MAXQDA software. Emergent themes from each phase of development were defined and reported in order to inform the subsequent development phases.
The primary data coder was SS. To ensure a rigorous evaluation of the data, a subset of transcripts was also coded by AM. In order to achieve consensus on codes, AM and SS engaged in recurring discussions on the coding process. The analysis was further tested during discussions with expert advisors. The consensus was reviewed and approved by all investigators. Specific quotations were chosen by SS to represent emergent themes in the data.
Results
This programme began in April 2019 and product testing is ongoing. Figure 1 illustrates the phased approach to development and iterative testing. In total, 67 individuals (7 advisors, 60 Somali community members) provided feedback during each phase of VR development and participated within the community co-design.
Enlarge this image.Figure 1. Phased approach to VR development. Assessment of community needs began in April 2019 in advance of the development of the VR. Iterative testing took place throughout 2019 and 2020, with some gaps in testing due to the restrictions of the COVID-19 pandemic. The final VR product has been completed as of June 2020 and is currently being tested with the Somali community. CBPR, community-based participatory research; VR, virtual reality.
Community-based participation
The first step in our community engagement process was to hold a series of three focus group discussions exploring topics of interest to the Somali community including vaccination, autism, paediatric health and technology (n=18 Somali community members). We also conducted interviews with three Somali parents (see online supplemental files for interview guide). Based on this initial feedback from the community, the project team held a design workshop to create a series of three story ideas for the VR. These ideas were then brought to the Somali community in the form of another design workshop where the community evaluated the suitability of the provided stories and suggested changes. Four Somali community members (two Somali CHWs, one Somali woman and one Somali man) participated in this design workshop to develop the general framework of a story. Information learnt from the community was then used to develop a first iteration of a script. This script was then tested for cultural and linguistic suitability and vaccine-promotion potential with 17 Somali women from ages 26 to 78 years and was followed by a discussion with a prominent male community leader to again assess the cultural and linguistic appropriateness of the content to promote vaccine education. Finally, a two-dimensional prototype of the VR animation was developed with a Somali voice-over (figure 2) and was tested with 24 community members in order to determine the effectiveness of the messaging in promoting vaccination and to assess cultural and linguistic elements of the storyline.
Enlarge this image.Figure 2. Still frame from prototype. The prototype consisted of a motion storyboard with a voice-over which was translated into Somali for presentation to the community.
Phased approach to VR technology development
Phase 1: CBPR approaches to assess community needs and concerns important to paediatric vaccination
The Somali community focus group discussions were broken up into three separate sessions, each of which focused on a particular area of community interest (table 1). The first focus group discussion was centred on health concerns of the community, and participants cited autism as a major concern, as well as language barriers that pose a significant problem when engaging with the medical system. In addition, participants brought up issues of trust which were tied to poor communication.
Table 1Focus group questions and responses from Somali community
| Category and questions for community members | Salient responses |
| Session 1: general health | |
| Examples of questions asked in focus groups | |
| ’What are some of your community’s biggest health concerns in the USA?’ | ‘Autism’. |
| ‘One of the biggest health problems that people have that I forgot to mention is that most people don't understand a lot of English.’ | |
| ‘What do you find not trustworthy within the healthcare system?’ | ‘Lack of good communication…especially in primary language.’ |
| ‘Health insurance!’ | |
| Session 2: paediatric health | |
| Examples of questions asked in focus groups | |
| ‘Do you trust your doctor’s recommendations for your child’s health?’ | ‘Yes, whatever recommendation the doctor gives me, I have to take it.’ |
| ‘I mean I always think it’s obviously for a good reason, but for me, I think I always do my own research before I automatically assume that’s what’s best for me. If it’s something very serious like [the doctor] saying for example ‘'you need a surgery,’ that (I) automatically would be like ‘let me get another opinion from another doctor.’’ | |
| Session 3: vaccination | |
| Examples of questions asked in focus groups | |
| ‘What are the topics or things that you would like to know about in relation to vaccines?’ | ‘If they could show how the vaccine works in the child’s body and what it does – if it can be visualized.’ |
| ‘That before the vaccines are given to our children, for it to be explained to us what the risks are - the sided effects, and the benefits. When we compare the two, then make a decision.’ |
Sample questions and salient responses from exploratory focus groups indicated that parents were very concerned about autism and that they found it difficult to trust the healthcare system due to lack of communication in their primary language. While parents trusted certain doctors, they also highlighted the importance of doing their own research to understand their children’s health. Parents also expressed a desire to understand how vaccines work within the body and what the risks and benefits are of vaccination.
The second focus group discussion explored issues of paediatric health. Participants provided information about pregnancy, childbirth, paediatric care and parenting. The central theme of this focus group discussion was issues of trust within the medical system, with many mothers indicating that while they highly valued their doctors’ opinions, they also preferred to do their own research. Mothers relayed to us their desire to receive health education in their own language from a trustworthy doctor.
The third focus group discussion was centred around issues of vaccination and technology. Participants once again indicated issues surrounding trust in medical systems and their desire to make their own educated health decisions. Additionally, participants stated that they would like to receive more detailed information about how vaccinations work:
If they could show how the vaccine works in the child’s body and what it does – if it can be visualized.
That before the vaccines are given to our children, for it to be explained to us what the risks are - the sided effects, and the benefits. When we compare the two, then make a decision.
Themes that emerged from interviews with three Somali parents also included concerns about autism, medical trust and the desire to learn more about vaccination.
Phase 2: VR modality determination for script development
Modality determination: the project team consulted extensively with Somali CHWs to determine an appropriate modality, and chose 360° video due to the ability of this format to be experienced using VR headsets, smart phones or computers in order to make this VR programme widely accessible to all members of the Somali community.
Script development: open-ended survey responses from Somali community members regarding the three potential storylines indicated that an older Somali male doctor would be most suitable to deliver health information in our story, as this character would evoke feelings of trust and respect. Somali community members favoured a scene where a Somali mother could be shown talking to her doctor—this way they could see themselves as a character in the VR story and could see their questions and concerns being addressed directly. The open-ended survey responses from community advisors indicated a preference for a storyline with a strong focus on family and supported the story concept of a Somali mother asking questions to a trusted physician. This information was used to develop the initial script.
Six members of the expert advisory board reviewed the initial script. Advisors were asked to answer a series of six open-ended questions and provided insightful answers that assisted with script development (table 2). Specifically, when advisors were asked for their overall impressions of the story, they stated:
Overall, I like how the story flows and the way in which the educational components are presented.
I like the simplicity of the conversation with its effective focus on the key messaging of the value of the timely vaccination to help raise healthy kids.
Table 2Questions and responses from project advisors
| Questions asked to advisors | Salient responses |
| 1. What are your overall impressions of the story? What did you like the most about the story? Is there anything that you think should change in the story? | ‘Overall, I like how the story flows and the way in which the educational components are presented. I also think it’s a great idea to have the story centered around a meal, as it seems culturally relevant and helps make the situation relatable to users.’ |
| ‘Based on my past feedback, I am glad that this story has been selected. I like the simplicity of the conversation with its effective focus on the key messaging of the value of the timely vaccination to help raise healthy kids.’ | |
| 2. Have you noticed any inaccuracies in scientific and medical facts in the story? | ‘No.’ |
| ‘No, from my knowledge all of the content presented is accurate.’ | |
| 3. Was the story clear and easy to understand? Did the story flow naturally? | ‘Very clear and flowed in a way we would use in teaching in general. Very logical progression of information.’ |
| ‘Yes, the story is clear and flows very naturally.’ | |
| 4. Was the story culturally appropriate? Was the cultural component balanced throughout? | ‘One of the things I liked about this story is the emphasis of the great Somali family bond that can be pivotal in achieving the goals of this project to leverage the great trust Somali parents put on their relative and educated community members.’ |
| ‘Yes, the story was culturally relevant and appropriate.’ | |
| 5. Was the story convincing? Does it have a potential to change attitudes of vaccine-hesitant parents? | ‘I think it gives the information about immunization, the science behind it, and does not focus on the controversies, which have not been supported by medical data.’ |
| ‘Yes. I would just make sure we really take advantage of VR when we show the visuals inside the body and how vaccine’s function within the immune system.’ | |
| 6. In your opinion, does the story increase knowledge? | ‘Yes, it stays with the facts in a positive way, in a healthy environment with the families.’ |
| ‘It can, depending on the background of the parents and audience and their desire to benefit from such educational program.’ |
Sample questions and salient responses from engagement with project advisors. Advisors enjoyed the clarity and flow of the storyline as well as finding it culturally relevant and appropriate. They also believed that the story had significant potential to increase vaccine knowledge.
VR, virtual reality.
We conducted a community focus group discussion during the script review process to engage the community. Seventeen community members were asked to assess how culturally appropriate the storyline was, what they would like to change, how impactful the story was and how they would personally design the content. During this focus group discussion, participants agreed that the story was clear and easy to follow; however, as additional questions were asked about story flow, the feedback turned to autism. We found that even when we did not mention autism, the false association between autism and the measles, mumps and rubella vaccine (MMR) came up as a topic of discussion. Many community members said that they were concerned about autism and the MMR vaccination. One woman asked:
If MMR doesn’t cause autism, why did I see my child stop talking immediately after getting the MMR?
The participants agreed that the VR must address the autism question, and that they would prefer to learn this information from a doctor character in the VR storyline.
Community members were also asked to take a brief survey after reviewing the script. This survey included questions about attitudes toward vaccination as well as vaccination planning. Notably, there was an 18% increase in participants who endorsed being ‘very comfortable’ with MMR vaccination following script exposure. Additionally, those who said they were ‘not at all comfortable’ with MMR vaccination decreased by 12% following script exposure. There was also a 17% increase in those who stated they would allow their child to receive the MMR following script exposure (table 3).
Table 3Survey following initial script exposure
| How comfortable are you with MMR vaccination? (before) | % | How comfortable are you with MMR vaccination? (after) | % | Would you get MMR for your child? (before) | % | Would you get MMR for your child? (after) | % |
| Very comfortable | 52 | Very comfortable | 70+ | Yes, I would | 65 | Yes, I would | 82+ |
| Somewhat comfortable | 24 | Somewhat comfortable | 18− | I would consider it | 23 | I would consider it | 12− |
| Not at all comfortable | 24 | Not at all comfortable | 12− | I don’t know | 6 | I don’t know | 0− |
| No, I would not | 6 | No, I would not | 6 |
This table shows the results of a survey of 17 Somali mothers following exposure to the initial script for the VR. + indicates % increased following exposure to the script, − indicates % decreased following exposure to the script.
MMR, measles, mumps and rubella; VR, virtual reality.
Phase 3: VR prototype and iterative feedback
Three expert advisors provided feedback on the storyboard and prototype that focused on where characters were positioned (ie, husband next to wife), colour scheme, and highlighted the need to describe the immune system’s function in order to retain scientific accuracy in the communication of vaccination information.
The prototype (figure 2) was tested with the Somali community in the context of an in-person focus group discussion and surveys that took place in person at the SFS offices. Participants in the focus group discussion were asked a series of open-ended questions about their experience with the prototype. The primary focus of this discussion was analysis of the storyboards and stylistic elements of the VR experience (eg, colour preferences, imagery, portrayal of characters). Participants indicated that they highly valued the Somali voice-over and preferred to include the discussion of autism in the final VR storyline, as its exclusion would raise more questions for the community. The participants also felt that the father character in the VR storyline seemed somewhat excluded and should be standing near his wife to signal support.
All participants who reviewed the prototype agreed that the inclusion of culturally appropriate characters and a Somali voice-over maximised the educational experience. Thirteen of 24 (54%) participants stated that the prototype made them either more comfortable or much more comfortable with vaccination than they were before exposure to the prototype. Twenty of 24 (83%) participants stated that they would recommend MMR vaccination to members of their community following exposure to the prototype. Additionally, 21 of 24 (88%) said they planned to vaccinate their children following exposure to the prototype.
VR design factors: the key VR design elements that were incorporated within each phase of VR development include passive, non-intrusive experiences, a dynamic and interactive visualisation, and prompts that promote the user towards self-reflection.
Phase 4: final product and testing
The final product is a 4 min 360° video animation (https://youtu.be/NS8GvtxnIk0) available in Somali and English languages. It can be viewed online using a tablet, a smart phone or with VR goggles. Settings include a Somali home populated by a family, and a doctors’ office. In the animation’s introduction, we meet the expectant mother who states that she is expecting her first child and is trying to make decisions about vaccination. Figure 3 illustrates in screenshots each of the four chapters in the VR experience. We plan to test the final product using an A/B testing model with the Somali community where the A group receives the VR education and the B group receives a basic English-language educational video about vaccination. Both groups will be surveyed before and after exposure to the educational materials to assess changes in attitude toward vaccination and willingness to vaccinate.
Enlarge this image.Figure 3. Top row: chapters 1 and 2 of the VR story. Bottom row: chapters 3 and 4 of the VR story. In chapter 1 of the animation, the expectant mother, her husband, and her sister visit the doctor’s office and learn about measles, mumps and rubella (MMR). In chapter 2, the family learns how the immune system works. In chapter 3, the family learns from the doctor what the MMR vaccine does and how it works. In chapter 4, the doctor explains to the family the risks and benefits of MMR vaccination, including a statement debunking the association between autism and vaccination. Finally, the 360° video concludes by showing the new mother and her family—including the new, healthy baby—and the new mother states that after learning all the facts, she has decided to vaccinate her child. VR, virtual reality.
Discussion
The main results of the SHIFA Programme can be summarised as follows: (1) a community participatory research model can be effectively translated for the co-design of a VR educational programme with community members involved in each phase of technology development; (2) cultural and linguistic sensitivities can be incorporated within a VR paediatric immunisation educational programme and are essential factors for effective community engagement; and (3) effective VR utilisation requires flexibility that can be used among community members with varying levels of health and technology literacy. To the best of our knowledge, our VR development is the first such health innovation for vaccination education designed by a community of refugees known for vaccine hesitancy.
Refugee Learners – Vaccination & Autism
This community has common barriers to effective education such as a lack of information and information that is not culturally and/or linguistically appropriate to drive understanding.19 Regarding immunisations, many parents and caregivers in this community already possess medically inaccurate information. We have previously determined that within this refugee community, the reason not to immunise has resulted from misinformation and the perception that vaccination results in autism. Although MMR vaccination rates have fallen in the Somali community (from 92% to 42% over the span of a decade7 8), rates of autism and paediatric learning impairments have remained high (1:32 Somali children have autism compared with the national average of 1:54).20 21 While these results do not support the link between vaccination and autism, many parents are still convinced of an association between MMR and autism. Given these results, it is important to take into consideration the mental and emotional state and the ideation that arises from associating vaccinations with autism. Community health engagement related to immunisation requires education focused on the importance of vaccinations for newly arrived refugees, and a re-education among those who have previously elected not to immunise.
In this context, we performed a community health assessment and identified the drivers for a low rate of vaccination in the Somali community ranging from cultural and language barriers, distrust in the healthcare system and the misinformation that vaccination results in autism. Recognising these drivers for low immunisation rates in this community, our observations for the mechanisms for how VR affects behaviour changes include: content that is culturally relevant, stimulates an awareness and expectation for what vaccines do and do not do, and provides an immersive experience leading to information retention.22
VR CBPR and co-designs
In addition to the mechanisms for how VR affects behavioural change, several design factors must be maintained when considering who interacts with the VR technology, especially among immigrants who may have varying levels of health and digital literacy. Within our programme, most community members experienced a positive interaction with VR. There are several plausible reasons for our observations. Through community co-designs, we leveraged key design factors including a non-intrusive experience (users learn in their own environment), a passive interaction (content that is visual, audio and depicted vs reading), and a dynamic storyline that builds on previous experiences, uses known environments and promotes self-reflection by allowing the user to introspect and contemplate during the VR experience.
Because we recognised the importance of culturally and linguistically appropriate educational materials to deal with issues of low health literacy and medical distrust, we included community-based approaches in each phase of our development. The results of focus groups and surveys conducted within the community revealed several important considerations for the development of our VR storyline. For instance, community members were much more comfortable receiving information from a trusted doctor character. Community members also felt it necessary to include direct and clear information about the lack of relationship between autism and vaccination in our storyline. This was in contrast to our expectation and that doing so would reinforce this misinformation. Following design workshops with the Somali community, we tested the programme that they helped to co-design within the community in order to address its cultural and linguistic appropriateness as well as its ability to promote vaccination behaviour. We also tested the VR storyline with a team of subject matter experts who evaluated the scientific accuracy and usability of our design. In our community testing, we found that many Somali community members felt the VR storyline engendered trust, was relatable, was educational and was convincing. Several focus group and survey participants stated that they planned to vaccinate and to recommend vaccination to others following exposure to the VR. Our testing with our panel of experts found that our content was user friendly, easy to understand and scientifically accurate.
While we appreciate that community co-designs are an important methodology for how a new technology is designed, a foundation of CBPR is necessary to harness community involvement. To employ CBPR, we engaged community members throughout every step of the process. Before developing the idea for the VR storyline, we engaged the community in a series of three focus groups to better understand their needs, strengths and interest in collaboration. Focus group discussion questions were open ended and allowed for participants to bring their interests and concerns into the conversation. Information learnt from these engagements was used to begin to develop the culturally and linguistically appropriate storyline for the VR. Community members were also continuously engaged throughout the development of the VR storyline through community co-design.
It is our plan to develop and leverage an educational curriculum in future deployments of VR. Due to the constraints of COVID-19, we are currently exploring the possibilities of using telehealth and other digital communication platforms to safely and effectively deploy the VR into the community. Addressing vaccine hesitancy is especially relevant within the context of COVID-19, as vaccination rates for preventable diseases have dropped significantly since the beginning of the COVID-19 crisis.23 There are also concerns about the potential of misinformation related to COVID-19 vaccination that is especially relevant for an underserved community that is largely excluded from vaccine clinical trials and communities that have a history of vaccine hesitancy.
Limitations
Our community feedback and focus groups represent a convenience sample for those that are more apt towards vaccine acceptance, and therefore may not completely capture all concerns among those who are vaccine hesitant. While this represents a potential selection bias, our method to include a wide range of community members as well as internal and external advisors may enhance internal validity by incorporating a heterogeneous group for community input. Community-based participation and community co-designs at each stage of VR development from the initial idea through completion of a VR animation may enhance external validation by including the key components related to cultural and linguistic sensitives within the phased approach for VR development. COVID-19 impacted our abilities to recruit more participants and to test our final VR experience within the community in the way that we had initially planned to. We are attempting to mitigate this limitation by administering online surveys in the Somali community after they view the VR at home; however, this may limit the quality of our data as most individuals do not have VR headsets at home and may not get the full immersive effect without them. Finally, a perceived shift from vaccine hesitancy to vaccine acceptance at this point is subjective and requires real-world validation and prospective follow-up confirming vaccine delivery.
Conclusion
We employed community-based participatory approaches and community co-design to develop an innovative vaccine educational technology with Somali refugees using VR. By combining new technology-enabled approaches with the needs, interests and expertise of Somali community members, we have created a methodology that can address vaccination beliefs and behaviours in a vaccine-hesitant refugee population. Future research will include an assessment of the efficacy of the VR platform on vaccination rates over time, as well as continued community engagement for the development of additional VR content which can increase health literacy within underserved populations.
We first extend our sincerest gratitude to the Somali community for their inspiring engagement in this project. Next, we would like to thank Fadumo Jama, Ahmed Dahir and Sahra Nor for their assistance with participant recruitment, facilitation of community engagements, and translation and interpretation services. Finally, we are grateful to Dr Ahmad Bailony and Karen Henken for their guidance and input on this project.
Data availability statement
Data are available upon reasonable request. Data are deidentified participant data and are available from the corresponding author. Reuse is not permitted.
Ethics statements
Patient consent for publication
Not required.
Ethics approval
This study was approved by The University of California San Diego Institutional Review Board (Protocol #171434).
Twitter @SanjeevBhavnani
Contributors SS, NI and AM contributed to the collection and analysis of data for this work. SS and SPB contributed to the initial drafting of the manuscript. SS, NI, MS, ME, IS, MSa, AS and SPB contributed to the conception and design of the work. All authors (SS, NI, AM, MS, ME, IS, CF, MSa, DM, KE-M, RT, AM, AL, EC, AS and SPB) contributed to the interpretation of the data, revising the work critically for important intellectual content, providing final approval of the version to be published, and all agree to be accountable for all aspects for the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding Alliance Healthcare Foundation Grant #18-34728477.
Disclaimer The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Competing interests CF was a paid consultant on the project from inception until August of 2019. AMarchetti is a former employee of technology partner INVIVO and her engagement with the project began while she was with that organisation in 2018. SPB discloses Scripps Clinic Grants for clinical trials related to machine learning and technology designs in cardiovascular imaging, being a Pfizer Health Information Technology Advisory Board member on strategies related to health information technology, being a Bristol Meyers Squibb Digital Health Advisory Board member on strategies related to digital health technologies, being an Anthem AI Consultant on projects related to cardiovascular care, being a Proteus Health Chair DSMB on digital medication monitoring in hepatitis C, being an American College of Cardiology Innovations Advisory Group principal investigator and committee member on various innovation projects in cardiovascular care, having Analytics4Life Stock options as a scientific advisor on artificial intelligence in cardiovascular diagnostics, and having Blumio Stock options as a scientific advisor on new technology designs in hypertension. ME discloses US patents in IllumeSense, using data analytics and machine learning algorithms for clinical decision support for alternative health therapies. ME is also founder of IllumeSense as majority shareholder and principle. IS discloses that he is a paid employee of Distinct Insights. AS, NI and SS were all paid employees of Somali Family Service at the time of this research. AMohamed was also a paid research assistant for this project. All other authors declare that they have no relevant or material financial interests that relate to the research described in this paper.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
1 Nicolai T, Fuchs O, von Mutius E. Caring for the wave of refugees in Munich. N Engl J Med 2015; 373: 1593–5. doi:10.1056/NEJMp1512139 http://www.ncbi.nlm.nih.gov/pubmed/26397950
2 Kumar R. Refugee Articulations of health: a Culture-Centered exploration of Burmese refugees' resettlement in the United States. Health Commun 2021; 36: 682–92. doi:10.1080/10410236.2020.1712035 http://www.ncbi.nlm.nih.gov/pubmed/31931620
3 Westgard B, Martinson BC, Maciosek M, et al. Prevalence of cardiovascular disease and risk factors among Somali immigrants and refugees. J Immigr Minor Health 2021; 23: 680–8. doi:10.1007/s10903-020-01078-3 http://www.ncbi.nlm.nih.gov/pubmed/32940817
4 Shadmi E, Chen Y, Dourado I, et al. Health equity and COVID-19: global perspectives. Int J Equity Health 2020; 19: 104. doi:10.1186/s12939-020-01218-z http://www.ncbi.nlm.nih.gov/pubmed/32586388
5 Decoteau CL. The "Western disease": Autism and Somali parents' embodied health movements. Soc Sci Med 2017; 177: 169–76. doi:10.1016/j.socscimed.2017.01.064 http://www.ncbi.nlm.nih.gov/pubmed/28171816
6 Henneberry S. University of Minnesota estimates autism spectrum disorder (ASD) in Somali and non-Somali children in Minneapolis. press release, 2013. Available: https://rtc.umn.edu/autism/doc/Somali_Autism_study_press_release.pdf
7 Dyer O. Measles outbreak in Somali American community follows anti-vaccine talks. BMJ 2017; 357: j2378. doi:10.1136/bmj.j2378 http://www.ncbi.nlm.nih.gov/pubmed/28512183
8 Hviid A, Hansen JV, Frisch M, et al. Measles, mumps, rubella vaccination and autism. Ann Intern Med 2019; 170: 513–20. doi:10.7326/M18-2101
9 Nabulsi M, Harris J, Letelier L, et al. Effectiveness of education in evidence-based healthcare: the current state of outcome assessments and a framework for future evaluations. Int J Evid Based Healthc 2007; 5: 468–76. doi:10.1111/j.1479-6988.2007.00084.x http://www.ncbi.nlm.nih.gov/pubmed/21631807
10 Merchant Z, Goetz ET, Cifuentes L, et al. Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Comput Educ 2014; 70: 29–40. doi:10.1016/j.compedu.2013.07.033
11 Psotka J. Immersive training systems: virtual reality and education and training. Instr Sci 1995; 23: 405–31. doi:10.1007/BF00896880
12 Riva G, Baños RM, Botella C, et al. Transforming experience: the potential of augmented reality and virtual reality for enhancing personal and clinical change. Front Psychiatry 2016; 7: 164. doi:10.3389/fpsyt.2016.00164 http://www.ncbi.nlm.nih.gov/pubmed/27746747
13 Zhou S, Bickmore T, Paasche-Orlow M. Agent-User Concordance and Satisfaction with a Virtual Hospital Discharge Nurse. In: Bickmore T, Marsella S, Sidner C, eds. Intelligent Virtual Agents. IVA 2014. Lecture notes in computer science, 2014. doi:10.1007/978-3-319-09767-1_63
14 Bhavnani SP, Narula J, Sengupta PP. Mobile technology and the digitization of healthcare. Eur Heart J 2016; 37: 1428–38. doi:10.1093/eurheartj/ehv770 http://www.ncbi.nlm.nih.gov/pubmed/26873093
15 City heights demographics and statistics. (n.d.), 2020. Available: https://www.niche.com/places-to-live/n/city-heights-san-diego-ca/residents/
16 Fisher KE, Yefimova K, Yafi E. Future’s Butterflies: Co-Designing ICT Wayfaring Technology with Refugee Syrian Youth. Proceedings of the The 15th International Conference on Interaction Design and Children (IDC '16), 2016.
17 Israel BA, Schulz AJ, Parker EA, et al. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health 1998; 19: 173–202. doi:10.1146/annurev.publhealth.19.1.173 http://www.ncbi.nlm.nih.gov/pubmed/9611617
18 Miles MB, Huberman AM. Qualitative data analysis: an expanded sourcebook. Sage, 1994.
19 Kim W, Kim I, Baltimore K, et al. Simple contents and good readability: improving health literacy for Lep populations. Int J Med Inform 2020; 141: 104230. doi:10.1016/j.ijmedinf.2020.104230 http://www.ncbi.nlm.nih.gov/pubmed/32688291
20 Maenner MJ, Shaw KA, Baio J, et al. Prevalence of autism spectrum disorder among children aged 8 years — autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveill Summ 2020; 69: 1–12. doi:10.15585/mmwr.ss6904a1
21 Hewitt A, Hall-Lande J, Hamre K, et al. Autism spectrum disorder (ASD) prevalence in Somali and Non-Somali children. J Autism Dev Disord 2016; 46: 2599–608. doi:10.1007/s10803-016-2793-6 http://www.ncbi.nlm.nih.gov/pubmed/27106569
22 Diemer J, Alpers GW, Peperkorn HM, et al. The impact of perception and presence on emotional reactions: a review of research in virtual reality. Front Psychol 2015; 6: 26. doi:10.3389/fpsyg.2015.00026 http://www.ncbi.nlm.nih.gov/pubmed/25688218
23 Santoli JM, Lindley MC, DeSilva MB, et al. Effects of the COVID-19 Pandemic on Routine Pediatric Vaccine Ordering and Administration — United States, 2020. MMWR Morb Mortal Wkly Rep 2020; 69: 591–3. doi:10.15585/mmwr.mm6919e2 http://www.ncbi.nlm.nih.gov/pubmed/32407298
© 2022 Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. http://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ . Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Ibrahim, Najla 2 ; Mohamed, Alia 3 ; Sharma, Manupriya 4 ; Markie Esmailian 5 ; Ibrahim Sezan 6 ; Farrell, Carrie 7 ; Sawyer, Mark 8 ; Meyer, Dan 9 ; El-Maleh, Khaled 10 ; Thamman, Ritu 11 ; Marchetti, Alex 12 ; Lincoln, Alan 13 ; Courchesne, Eric 14 ; Sahid, Ahmed 15 ; Bhavnani, Sanjeev P 16 1 Department of Anthropology, University of California San Diego, La Jolla, California, USA
2 Department of Health and Wellness, Somali Family Service of San Diego, San Diego, CA, USA
3 Department of Biology, San Diego State University, San Diego, CA, USA
4 Department of Physics and Engineering, Palomar College, San Marcos, California, USA
5 IllumeSense, San Diego, California, USA
6 Distinct Insights, San Diego, California, USA
7 School of Public Affairs, San Diego State University, San Diego, California, USA
8 Department of Pediatrics, University of California San Diego School of Medicine, La Jolla, CA, USA
9 Department of Cardiovascular & Thoracic Surgery, The University of Texas Southwestern Medical Center, Dallas, Texas, USA
10 QUALCOMM, San Diego, California, USA
11 Department of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
12 Farmer School of Business, Miami University, Oxford, Ohio, USA
13 Deparment of Clinical Psychology, Alliant International University-San Diego, San Diego, California, USA
14 Department of Neurosciences, University of California San Diego School of Medicine, La Jolla, CA, USA
15 Somali Family Service of San Diego, San Diego, California, USA
16 Healthcare Innovation and Practice Transformation Laboratory, Scripps Clinic La Jolla-Genesee Executive Plaza, San Diego, California, USA