Effects of the COVID-19 Pandemic on K-12 Education: A Systematic Literature Review

Abstract

The American Trends Panel survey of teachers and principals (Hamilton et al., 2020) implemented in late April-early May complements the work of the Center on Reinventing Public Education (Gross & Opalka, 2020) and their systematic review of school districts' and charter management organizations' web sites. Policy briefs developed from this data have focused on three areas of concern, which are also noted in our coded themes: teacher lack of familiarity with learning tools, district ability to offer socio-emotional support to students, and frequency of teacher-student and teacher-family interactions. [...]the EdWeek survey reporters also revealed that morale for teachers, students, and administrators across the country plummeted during the early months of the pandemic. While it is currently unknown how long students will engage in remote learning, we present data-informed policy recommendations that school district personnel and state education officials might implement to improve stakeholder experiences and minimize a widening achievement gap as school disruptions continue.

Introduction

The COVID-19 pandemic has had far-reaching effects on nearly every aspect of society, including education. Schools in the United States, and in most countries in the world, were closed in March 2020 as nations developed lockdown measures to prevent further spread of the virus. While educators transitioned from traditional face-to-face learning to digital platforms for remote teaching, many challenges arose that required quick solutions and changes to policy and procedures to provide equitable and appropriate remote learning to all students.

Although advances in educational technology in the last few decades proved immensely useful during this pandemic (Dhawan, 2020), teachers, students, parents, and other relevant educators faced many challenges that they were not ready to overcome (Chakraborty et al., 2020). School leaders and teachers rushed to completely redesign their educational delivery models, parents balanced work demands with the responsibility of ensuring their children were accessing instruction from home, and students struggled to stay engaged and connected with their teachers and peers online. Many scholars undertook broad research on educational practices to explore how COVID-19 has impacted teaching and learning nationally, with several organizations conducting and reporting surveys in phases throughout the first four months of remote learning.

The American Trends Panel survey of teachers and principals (Hamilton et al., 2020) implemented in late April-early May complements the work of the Center on Reinventing Public Education (Gross & Opalka, 2020) and their systematic review of school districts' and charter management organizations' web sites. The American Institutes of Research (AIR) also launched a national survey on educators' response to coronavirus (Garet et al., 2020). This survey, which continued throughout the spring of 2021, collected data from 2,500 school districts and organizations. The first survey addressed approaches and challenges to distance learning; supports for students with disabilities and English learners; district policies regarding grading, assessment, and graduation; and well-being and safety. Policy briefs developed from this data have focused on three areas of concern, which are also noted in our coded themes: teacher lack of familiarity with learning tools, district ability to offer socio-emotional support to students, and frequency of teacher-student and teacher-family interactions.

Researchers at the RAND Corporation (Gross & Opalka, 2020) found that nearly 90% of principals reported that students in their schools lacked internet access and 40% reported that access to technology and/or internet was also a barrier for their teachers. A third of principals (35%) reported that district policies related to the use of online tools (such as Zoom or G Suite for Education) presented limitations (Hamilton et al., 2020). In addition, in an EdWeek (2020) survey in April, 74% of teachers said their students' current level of engagement was "much lower" or "somewhat lower" than it had been prior to the pandemic, and nearly a quarter of students were "essentially truant." Finally, the EdWeek survey reporters also revealed that morale for teachers, students, and administrators across the country plummeted during the early months of the pandemic. While researchers working on these national studies used online surveys and document analysis methodologies to investigate pre-defined questions, they were unable to fully describe the phenomenon and represent the multiple perspectives of various stakeholders. Qualitative and mixed-methods research studies enriched our understanding of the experience of teaching, learning, leading, and parenting during the pandemic. Systematic literature reviews, considered important scholarly contributions, help to map, consolidate, synthesize, and refine knowledge from all available, relevant literature and springboard theory development as well as direction of future research.

It is important to undertake a systematic review of all the studies conducted during this time at state, district, school, and classroom levels to inform future practice as we continue with remote and hybrid learning models into the following academic years. We will outline the challenges various stakeholders faced as well as data-driven recommendations for learning environments that will benefit teachers and students from all backgrounds. Through a process of systematic literature review and an elaborated concept mapping strategy, we will present a deeper understanding of how remote instruction was experienced during the mandatory school closures and abrupt transition to online teaching. With a focus on the K-12 context in the United States, research studies in this review included sample populations of elementary and secondary students, teachers, parents, and school leaders. Scholarly sources included journal articles, reports, policy briefs, and dissertations that addressed the focus research question, "What are the effects of the COVID-19 pandemic on K-12 education?" The application of a concept mapping strategy enabled us to present common issues, concerns, and recommendations; prioritize outcomes across studies; and identify gaps in the literature for further investigation.

As mandated remote learning is a recent and ongoing phenomenon, it is essential that we also outline systematically the following aspects of these studies: (1) In what ways do the primary components of these studies, such as research questions, datasets, research methods, samples, relevant theories, results, and limitations overlap, and in what ways do they differ? (2) What are the common results beneficial to understanding challenges and implementing successful strategies to support our teachers, parents, students, and school leaders? This study will make a scholarly contribution as a systematic review helps us locate and understand relevant scholarship, along with sub-themes, populations, and geographic areas that require further exploration. We will also inform teaching and learning by reporting findings on effective technology tools, collaboration and communication methods, and leadership practices. While it is currently unknown how long students will engage in remote learning, we present data-informed policy recommendations that school district personnel and state education officials might implement to improve stakeholder experiences and minimize a widening achievement gap as school disruptions continue.

Literature Review

The global pandemic and schoolwide closures mandated in 2020 have no precedent; while the United States has historically experienced many natural disasters and economic crises, none have required widespread transition to remote learning nor has the technology infrastructure been previously available to support this mandate. There is limited research specific to challenges of remote instruction for K-12 students during extended school closures. The ability to use technology to teach learners at a distance has been especially important in times of emergency, such as natural disasters (Hinson et al., 2007; Rush et al., 2014), but fully functioning emergency online schools are possible only through thoughtful planning and development before a disaster strikes. There are many challenges to online schooling, and this planning must address teacher preparation and training needs, strategies to foster student achievement, and clear communication with students and families. We examined prior literature on these issues to gain better understanding of the challenges inherent to remote instruction.

Teachers Struggle with Online Teaching

During the Covid-19 pandemic, teachers were asked to transition to online teaching with little time to prepare, becoming both instructional designers and pedagogical facilitators using tools which few had fluently mastered. Their lack of experience teaching in a virtual environment was reported widely in the media, yet demand for these learning models has in fact been increasing in the K-12 sector (Archambault & Kennedy, 2014; Rice & Deschaine, 2020). Researchers have found that while most teachers regularly use computers in their personal and professional lives, there is a gap between personal use and classroom use. This gap supports the idea that a lack of confidence and teaching beliefs prevents teachers from regularly using technology in their instruction (Bebell & Kay, 2010; Tyminsky et al., 2013). Despite the need for educators who are prepared to teach online, teacher preparation for online programs is essentially non-existent (Kennedy & Archambault, 2012). Online teaching requires many different skills and competencies than traditional teaching (Pulham & Graham, 2018) yet is not fully integrated in pre-service teacher coursework. Teachers must not only have technological expertise, but they also need to develop strategies for building relationships with students and attending to their social, emotional, and academic needs (Borup & Evmenova, 2019; Rice & Carter, 2015). To address these barriers to effective technology integration, researchers have recommended restructuring preservice credentialing programs, updating existing school and district technology plans, providing targeted professional development for administrators and teachers, and prioritizing time for teachers to experiment and practice with technology (Ertmer et al., 2012; U.S. DOE OET, 2016; Wachira & Keengwe, 2011).

Mixed Results of Student Achievement in Virtual Learning

There is no consensus on whether virtual learning is as effective for K-12 students as face-toface instruction, which may be connected to teachers' lack of pedagogical expertise in remote delivery and students' motivation to engage in this model. Prior comparisons of online and traditional public schools have shown mixed results, and there is a dearth of research on elementary learners (Arnesen et al. 2019). Researchers of full-time K-12 virtual schools (Ahn & McEachin, 2017; Gill et al., 2015; Woodworth et al., 2015) have reported students in the virtual schools performing 0.1 to 0.4 SDs below students in traditional public schools, while in some contexts they found null or slightly positive results (Chingos & Schwerdt, 2014). Research on Florida's wellestablished statewide virtual school has shown varied outcomes, with positive effects for online modalities on course grades but negative effects on longer-term outcomes like graduation readiness (Hart et al., 2019). These findings have lead researchers to conclude that different groups of students might respond to virtual learning differently and the results may depend on the alternative options students have. Heissel (2016) found that although virtual education provides rural schools in North Carolina with cost-saving and higher quality teaching, it can lower academic performance in courses such as Algebra I. Prior high achievement, self-discipline, and technological support at home and school are needed to promote online learning success (Heissel, 2016). When examining the participation of students in remote learning environments, a growing body of research suggests that online schooling can come with an "online penalty" for struggling and vulnerable learners (Dynarski, 2018). The concepts of digital divide and "digital use divide" have been frequently cited as an obstacle for students from lower-income and linguistically and racially diverse backgrounds (Thieman & Cevallos, 2017; Warshauer, 2016).

Methods to Support Parent Involvement

Clear and timely communication between teachers, students, and families is a key component of learning that can lead to essential interactions that build classroom community and enhance student academic achievement. Numerous studies have demonstrated the impact parental involvement has on student motivation and achievement (Gonzalez-DeHass et al., 2005; HooverDempsey et al., 2001; Olmstead, 2013); for some, these interactions involve volunteering in the classroom or attending parent-teacher conferences while, for others, the most meaningful involvement occurs in the home. Parents have been shown to play a vital role in monitoring student homework and grades and sharing the values of effort and academic success (Goodall, 2016). There is evidence to suggest that increasing use of technology to communicate with parents as a group (e.g. mass texts or emails about school policy, Facebook posts announcing school events) and as individuals (e.g. teacher emails regarding student progress) can positively support the engagement of parents of children of all ages (Bouffard, 2008; Watkins, 2013). Ho et al. (2013) reported that electronic methods of communication allow parents and teachers to communicate at times that are convenient to both, giving parents more of a chance to be involved in their children's learning. While the proportion of American adults with high-speed broadband service at home increased rapidly between 2000 and 2010, data from the Pew Research Center (2021) reveal that broadband adoption growth has been much more sporadic in recent years and there are still families without access. The latest survey data reported that roughly 75% of American adults have broadband internet service at home and that reliance on smartphones for online access is especially common among younger adults, lower-income Americans, and individuals with a high school education or less (Pew Research Center, 2021). Racial minorities and those with lower levels of education and income are less likely to have broadband service at home and families with access to the content may not have the requisite language or technological skills to benefit greatly from the information produced by schools (Bouffard, 2008). As parents are the first line of communication from the school, it is essential that district personnel employ multiple modes (i.e., Twitter, Facebook, robocalls, texts, bilingual radio stations) to convey information regarding school closures, instruction, meal service, and health information along with telephone help lines for two-way communication (Goodall, 2016; Watkins, 2013).

Lens of This Literature Review

In reviewing concepts from the research literature, we adopted a social justice lens to examine the extent to which studies addressed issues of equity regarding participation in online learning for our traditionally marginalized student populations. As noted in a report by the Education Trust (2020), "Students of color, students from low-income backgrounds, English learners, students with disabilities, and other vulnerable groups such as homeless students and students in foster care, were less likely to have rigorous, engaging, and positive educational experiences before the pandemic" (p. 1). The present study highlights the need for teachers and school leaders to center social justice and social equity concerns to inform decision-making and effective communication with students and families. Although marginalized groups may be more dramatically impacted by stressors from the pandemic and school closures, students from all socio-economic backgrounds experienced difficulties (Gonzalez, 2021). There has been a gap in the research on the effects of prolonged school closure and remote instruction on students and their parents, and this systematic review examined the first set of studies addressing this topic while it was happening.

School closures are detrimental to all students who need live instructional time and additional support to reach their academic potential, but this is particularly important for the most vulnerable student populations. Research on remote teaching and virtual school instruction has consistently recommended high-quality pedagogical and technical preparation for educators to prevent significant widening of opportunity and achievement gaps for traditionally marginalized student populations (Xu & Jaggars, 2014). Therefore, it is necessary to conduct a systematic review of available scholarly sources to gain better understanding of the challenges and successes experienced by all participants in K-12 education in the United States.

Research Methods

A literature review is a multi-stage process that includes scanning information, making notes of reviews, synthesizing and structuring information, writing a critical review of the literature, and building a repertoire of resources (Ali et al., 2017; Rowley & Slack, 2004). In contrast to traditional literature reviews, a systematic review combines a systematic and transparent data collection process with rigorous analysis (Petticrew & Roberts, 2008; Zhang et al., 2021). Conducting a systematic literature review "is an essential research activity in ensuring a good piece of research" (Alias & Suradi, 2008, p. 1). A systematic literature review is an attempt to make sense of a body of existing literature through the aggregation, interpretation, explanation, or integration of existing research (Rousseau et al., 2008). Systematic reviews should be reliable and replicable, which requires the review process to be documented and made transparent (Xiao & Watson, 2019). The process used includes defining the research questions; conducting a search for relevant papers; screening the papers; keywording using abstracts, keywords, and titles; developing the classification scheme; extracting the data; synthesizing results; and reporting gaps, trends, and policy recommendations.

We applied concept mapping to this systematic review process to organize data extraction, analysis of findings, and synthesis of results. This method facilitated identification of research gaps, exploration of conceptual and geographical patterns, and flagging of subgroups of studies for further analysis (Soaita et al., 2020). For this study, we used a systematic approach for collecting data, followed by a qualitative approach to analyze the data that combined inductive and deductive methods to capture the richness of what was being reported and to visually display common themes. This section describes our review methods, data collection and analysis process, and synthesis of results.

Literature Search and Selection

This study began with locating published research addressing the main research question. A range of quantitative, qualitative, and mixed-methods studies were screened for quality and eligibility using an abstract screening tool and revisited multiple times during the coding process. To refine our search process and obtain an effective sample start set, we applied the following standards to our selection criteria:

a.Time: Studies conducted during the initial period of COVID-19 school closures (March-July 2020)

b. Demographics: Studies conducted in the United States, K-12 sample

c. Sources: EBSCO, ERIC, ProQuest, and Google Scholar

d. Keywords: "K12" AND "research" AND "pandemic" AND "United States"

e. Impact: Academic journal articles (peer-reviewed); high-quality reports and policy briefs from established organizations with well-cited references; and grey literature such as dissertations and working papers.

The search with initial keyword identifiers yielded 5,310 sources. After applying the criteria outlined above, the number of sources narrowed to 226. Further exclusion criteria included: articles or essays that did not report on an original research study; studies published in languages other than English; a higher education context; and studies conducted in countries outside of the United States. Search strings were modified with key words from relevant articles, (for example, "COVID-19 pandemic" AND "remote learning" OR "teaching") and forward snowball searching of sources also produced and/or verified essential studies of interest. The number of peer-reviewed studies selected and analyzed based on the inclusion criteria and the purpose of this study was narrowed to 49, with the majority of the literature published in academic journals (61%). The type and frequency of studies is displayed in Table 1.

Literature Review Process

To analyze the sample of studies, we developed a classification scheme and data extraction table in Excel. The main categories included: "Descriptive Information" (authors, year, journal, study type, key words), "Study Design" (sampling, research questions, theoretical framework, design, data sources, data analysis, results), "Limitations" and "Recommendations." We also began noting key issues across articles, which revealed common themes such as equity, teacher preparation, and communication. This classification tool was piloted with a sample of 10 studies and then adjusted and refined by the team to increase consistency. All findings were included as reported by the authors, without inferences or interpretation.

For this study, we applied graphical representation to organize and represent knowledge, concepts, and their relationships through concept mapping. Concept maps utilize circles or boxes that indicate the connecting links between concepts (Novak & Canas, 2006) and help researchers and students visualize and learn information by using nodes and links that reflect a domain knowledge and application (Alias & Suradi, 2008; Carnot, 2006). We used a concept mapping approach (Zhang, 2011; Zhang et al., 2021) that followed three stages: collecting and cataloguing studies through systematic literature review and creating preliminary visual mapping structure; synthesizing key information from studies and coding by theme; and deeply analyzing the maps to identify key issues across studies as well as limitations and gaps in the research. We synthesized the study results using qualitative methods of coding and discussing the represented frequencies, interconnections, and knowledge gaps in themes. Our goal was to answer our research questions and to provide implications for policy and practice as well as identify areas for future research.

There are many kinds of commercial software tools for concept mapping (e.g., 3D Topicscape, Inspiration, Compendium, Cmap Tools, etc.); in this review a program called Inspiration 10 was used. This software is a tool that helps the user visually organize concepts to deepen understanding and discover relationships (Tech Ed Marketing, 2021). By drawing a thematic mapping structure, entering codes from our reviewed literature, revising after multiple full-text readings of articles, adding article numbers that addressed each concept (coded 01,02,03 on the map), we obtained visual cues to aid in our analysis of the data clusters and to draw conclusions about implications. This process could be performed by hand but the software facilitates revisions, collaboration among research team members, and output diagrams suitable for presentation and publication.

Our map has the primary research focus at the center, the challenges the COVID-19 pandemic has created for K-12 education in the United States, with five sub-groups for classification as shown in Figure 1. We selected these concepts as critical elements to answer the research questions. The mapping was achieved through a screening process of entire papers, often multiple times. Each circle corresponds to the number of papers addressing each concept. The first-level structure (as shown in Figure 1) was expanded as we carefully read our sources and coded them, with additional levels identified via color coding and source references (19/24/44, etc.). Arrows show links and connections between concepts. The complete result of our mapping is provided in Figure 2.

Once a map is created, the data output (map and outline) can be saved in Microsoft Word. Including source reference numbers assisted the researchers in quickly locating information, reviewing, and justifying connections among concepts, and easily viewing frequency counts of particular elements. This led to the synthesis of study results, which will be reported in the next section.

Review Findings

This systematic review synthesized and summarized key findings among multiple studies conducted during the COVID-19 pandemic and transition to remote learning across the United States in the spring of 2020. Preliminary results indicated useful recommendations for local and state education agencies to implement during the 2020-21 academic year and beyond, as we continue to provide instruction in remote and/or blended learning environments. Our key findings address concerns about equity, teacher preparation, and communication, which were among the most referenced challenges in our literature. Using the concept mapping technique, we were able to analyze findings from empirical literature and identify similar and different features among studies as well as key ideas that merit future research consideration. Gaps were also revealed in several areas or sub-themes which warrant further study.

Characteristics of Studies Reviewed

Focus of Studies

We used data collection methods, sample population, and participants as second-level concepts connected to the theme of data. Secondary data, for example, was a third-level category that linked to a fourth level of category with specific names of sources, e.g., Google search trends, student grades, i-Ready login data, etc. while the fifth level further connected these data sources to a district, state, or national context. Each concept circle was coded with the relevant numbers of articles that directly addressed this concept. Participants, such as teachers, had connections explored to and from data collection methods, participants, school level, and subject areas. Table 2 contains an Inspiration output outline from the second and third levels of this map for the category of Data.

Our first finding from this review of study characteristics was that the predominant data source was online surveys, which were included in 22 studies (45%), followed by interviews in 11 studies (22%) and then secondary data in 10 studies (20%). These types of data could be easily collected in a virtual environment, and most interviews were conducted through video conferencing platforms. Lesser-used sources were online polls, which ElSaheli-Elhage (2021) integrated throughout teacher training modules; document review of court cases related to provision of instruction for English learners (Sugarman & Lazarin, 2020); participant collaborative notes/ journals in a parent and child participant study (Schaefer et al., 2020); teacher artifacts (Kaden, 2020), observations (Kaden, 2020; Peterson et al., 2020), and diary entries of parents of elementary children (Wang, 2020). Given the social distancing restrictions during this time period, researchers were able to conveniently and safely collect data from participants using online surveys, interviews, and secondary data analysis as seen in the large-scale national reports (EdWeek, 2020; Gross & Opalka, 2020; Hamilton, 2020).

Our second finding concerned the sample populations of teachers of specific content areas, which were the focus of four studies. When we reviewed the sample population of schools, we found representative samples of teachers from the elementary, middle, and high school levels while only a few studies focused on teachers of specific content areas: math (Horn & SchneebergerMcGugan, 2020); science (Pesnell, 2020), agriscience (Lindner et al., 2020) and music (Hash, 2021). This focus on specific content areas and their instructional demands, online resources, and strengths and weaknesses in a remote instruction mode is an interesting avenue of study that merits further investigation. The science and agriscience teachers found it difficult to teach their subjects without face-to-face interaction and hands-on learning opportunities, and students' inability to manage their time wisely and work independently was also cited as a negative factor (Horn & Schneeberger-McGugan, 2020; Lindner et al., 2020). The 474 school band teachers in Hash's (2021) study reported a lack of parental communication and support as a moderate to extreme challenge to remote learning, while all 10 elementary science teachers in Pesnell's (2020) multiple case study reported a decrease in communication from parents over the duration of the experience. The negative effects on student learning that resulted from low parental support and student struggles with engagement are consistent with findings from previous studies of online learning (Dynarski, 2017; Heissel, 2016). These four content-specific studies took place in different states, yet researchers found many similar challenges reported by teachers.

Analytic Approaches of Studies

There were 24 qualitative, 16 quantitative, and nine mixed-methods research designs as represented in Figure 3. The qualitative studies were further classified as grounded theory, phenomenology, case study, document analysis, and autoethnography. Quantitative research methods included statistical models, t-test, factor analysis, and multiple regression while mixedmethods studies included a combination of the above approaches. Table 3 shows the number and percentage of each type of methodology used in our set of studies.

Our first finding was that the personal, lived experience of students during remote learning was best expressed through qualitative data collection methods (Shaefer et al., 2020; Simpson, 2020; & YouthTruth, 2020.) Schaefer et al. (2020) conducted a child-parent collaborative autoethnography to capture the experience of "new teaching methods in new learning spaces" and offered insight into the negotiations teenagers must make to motivate themselves to do their schoolwork at home. This is one of the few studies enriched by the voices of student participants, which were also reported in two surveys with open-ended responses (Simpson, 2020; YouthTruth, 2020). Simpson (2020) interviewed 155 students and their parents to explore which instructional strategies and technology tools they found most effective in remote learning, and areas that they felt needed improvement. YouthTruth (2020) administered their survey in English and Spanish within 20 school systems across 9 states to 20,438 students in grades 5-12, asking students to compare their experience learning from home to their experience while their school building was open.

Next, we found three studies (Becker et al., 2020; Margolius et al., 2020; Oosterhoff et al., 2020) in which researchers collected data through quantitative surveys of students to provide insights into the experience of teenaged learners. Margolius et al. (2020) surveyed 3,300 young people aged 13-19 on their perceptions of the impact of school closures on their learning and their lives and disaggregated their data by urbanicity, race/ethnicity, and parent place of birth. Oosterhoff et al. (2020) used multiple linear regressions on quantitative survey data collected from students to examine unique associations among 683 adolescents' motivations to engage in social distancing, perceived amount of social distancing, anxiety symptoms, depressive symptoms, burdensomeness, and belongingness. Becker et al. (2020) surveyed 238 adolescents with ADHD and their parents and employed three quantitative measures to examine remote learning practices, family financial burden, learning difficulties, parent confidence in remote learning, and parent difficulties with learning support and home-school communication. As noted by Becker et al. (2020), further studies are needed to establish reliability and validity of these COVID-19 measures which could also be considered a limitation of the other quantitative surveys.

Finally, only one other study (Bacher-Hicks et al., 2020) was coded for students (along with teachers and parents) as it was not possible to pinpoint the users of Google search terms who were its research subjects. Using frequency counts of keywords from Google search trends data (nationally and by designated market areas) to measure search intensity for terms related to online learning, the researchers concluded search intensity for learning resources varied substantially by geography and socioeconomic status. This study reported engagement with online resources to be higher in non-rural, higher income areas with better internet access. These results support widely reported concerns regarding equitable participation in online learning for our traditionally marginalized student populations.

Theoretical Frameworks Used in Studies

The theories and frameworks for the studies ranged from personal (i.e., self-efficacy, selfdetermination, psychological safety) to social (i.e., academic communities of engagement, social capital) to technical (i.e., transactional distance theory, emergency remote teaching). Figure 4 highlights the area of the concept map for theories and frameworks, showing interconnections among various theories.

In this section, our first finding was the frequency of application of theories related to personal characteristics, most notably Bandura's theory of self-efficacy (1977) and related theory of selfefficacy in a crisis (Bandura, 1994, as cited in Money & Pacifici, 2020). Kraft et al. (2020) surveyed 7,800 teachers to explore how their sense of success changed over time and the degree to which working conditions moderated these changes. They concluded that teachers who could depend on strong communication, fair expectations, and a recognition of effort from the top, along with targeted professional development and meaningful collaboration with colleagues, were least likely to experience a dip in their sense of success.

According to Cardullo et al. (2021), teachers must perceive that their instructional technology needs are met in order to feel confident in meeting the needs of their students. Their attitude in using their school's learning management system (LMS) influenced their engagement selfefficacy. Participants in Martinez and Broemel's (2020) study were teachers (17) and administrators (9) in an online principal preparation course who reported high levels of selfefficacy during the pandemic. Money and Pacifici (2020) framed their study of 10 principal candidates using Bandura's theory of self-efficacy "during a crisis" and found participants had mixed feelings about their self-efficacy. A high level of survey respondents reported that they would not be able to raise student achievement in their schools during the pandemic due to equity issues, access gaps, and existing district policies, although they did believe they could successfully create a positive learning environment to facilitate teaching and learning for the students in their K-12 schools.

Self-determination theory was the framework for Oosterhoff et al.'s (2020) study of youth psychological consequences of social distancing required by school closures and reduced social interaction with peers. In the context of social distancing, youth may endorse motives connected with personal autonomy, prosocial action, or control; survey results found that youth motivated by autonomous or prosocial reasons may engage in more social distancing and these motivations may be connected with greater mental health. In their phenomenological study of 54 principals across 19 states, Weiner et al. (2021) sought to understand how, during the early days of the COVID-19 pandemic, principals created, or failed to create, psychological safety in their schools and how these efforts and their outcomes may have varied across contexts. Rather than being grounded in environmental conditions (e.g., urbanicity, demographics, etc.), organizational factors such as differences in accountability, principal autonomy, professional culture, and teacher decisionmaking were all key in the degree of psychological safety exhibited. The theory of psychological safety (Schein & Bennis, 1965) merits further use as a framework for studies about teaching and learning during a pandemic, as it is linked to so many important behavioral outcomes (e.g. engagement, task performance, learning behaviors) and would be relevant to many of the studies in this review with teacher and school leader participants.

Leadership theories applied to research with school leaders included professionalism and metaleadership. Stone-Johnson and Weiner (2020) used professionalism to describe how limiting principals' autonomy to make decisions seen as critical to achieving their goals diminished their feelings of efficacy and satisfaction. Hayes et al. (2020) used the tenets of a meta-leadership framework to anchor their investigation of how the COVID-19 pandemic and quarantine informed the leadership practices of rural principals.

Our second finding was that researchers used a variety of social and community frameworks with inter-related concepts. Since so much information was disseminated to parents online, human information behavior theory was also explored in relation to parents' experiences with their children's education (Wang, 2020). Wang (2020) coded participant responses in their diaries according to information needs, information seeking, information evaluation/weighing, information use, and information sharing in an effort to determine influences on parents' emotional responses to their children's required daily academic activities during two months of school closures. Domina et al. (2021) applied the theory of social capital to examine the correlation between engagement with remote learning for students and parents reporting contact with classmates' parents during the course of the lockdown. As the researchers noted, their findings from this survey of 10,000 elementary parent participants were supported by prior research demonstrating the role of social capital in community recovery from disaster (Aldrich and Meyer 2015, as cited in Domina et al., 2021), suggesting the importance of maintaining social ties among families. We believe social capital is a key theory to explain access to and engagement in remote learning and could be infused into more research studies focusing on students, parents, or teachers to explain engagement in remote learning.

This idea of community influence on learning is further emphasized in the Academic Communities of Engagement (ACE) framework Borup et al. (2020) used in their case study to describe two communities that help support student engagement: the course community associated with school (teacher, peers, administrators, counselors) and students' personal community of relationships (parents, siblings, friends). Ferreira (2020) also applied a community-based theory (communication supporting community resilience, Hyvärinen & Vos, 2015) to investigate his school's efficacy in developing school-home partnerships for resilience before, during, and after crises.

Our third finding was that several researchers described the conceptualization of an online learning modality with a relatively new term for use during pandemic and other crisis situationsemergency remote teaching (ERT) (Hodges et al., 2020; Milman, 2020). Five researchers in our review utilized ERT to examine the temporary shift of instructional delivery to an alternate delivery mode due to crisis circumstances. As described by Peterson et al. (2020), the primary objective of ERT is to provide temporary access to instruction and instructional supports in a manner that is quick to set up and is reliably available during an emergency or crisis. The other theories related to technology included the utilization of an extended technology acceptance model (TAM) by Cardullo et al. (2021) to examine teachers' self-efficacy in remote teaching during the COVID-19 pandemic, and Lindner et al.'s (2020) use of transactional distance theory to assess middle and secondary school agriscience teachers' perceptions of remote instruction and distance education and the inherent strengths and weaknesses of each approach. ERT is likely to be used in future studies examining teaching and learning during the pandemic.

Finally, while Wachen and Johnson (2020) provided document analysis explicitly framed by the equity sections of each state's guidelines for the pandemic, the issue of equity was interwoven in the results reported by the majority of our reviewed studies (39 out of 49, or 79.6%). The prevalence of this perspective in our review of empirical literature highlights the need to continue examining practices in K-12 education with a social justice lens.

Contents of Studies Reviewed

Interviews and surveys with teachers across the country about their daily experiences found that teachers most frequently expressed concerns related to three key themes: struggling to communicate with students virtually; their own lack of preparation for extended remote teaching; and exacerbated inequities for students. Lapses in communication were reported from teachers, parents, and students, along with unclear expectations regarding attendance and grading policies. Teachers relied on their support networks of peers for support and were concerned about access issues for their lower socio-economic status students, students in rural areas, English learners, students with disabilities, and homeless students. Barriers to their participation in remote learning included technology issues, lack of hands-on support, and household factors. Table 4 contains the full output outline for the results section of the concept map. This section will address the seven areas of results we extracted from this set of studies.

Communication Practices

Our first finding regarding communication was the need for heightened teaching presence and increased instructor support and feedback, which both student and parent respondents emphasized as important (Pesnell, 2020; Simpson, 2020). In a survey of nearly 5,000 parents conducted by the Pew Research Center (Horowitz, 2020), 64% of parents reported concern about their children falling behind, with live, online instruction from teachers more prevalent in higher socio-economic areas. In two surveys of youth well-being during remote instruction, researchers found that 29% of adolescents did not feel connected to an adult at school (Margolius et al., 2020), and motivation and engagement varied by grade level, socio-economic level, and race (YouthTruth, 2020). Similarly, teachers reported needing help contacting students and their families when a homework gap was discovered the first few weeks into remote instruction (Clausen et al., 2020). Teachers with less experience were more likely to lose contact with their students (Smith et al., 2020). While there is a large body of research literature addressing the use of technology in schools, most of this has referred to pedagogical implementation rather than the use of technology as a tool for communication. Expectations for student participation in remote learning classrooms should be clear, well communicated, and monitored with accountability measures so that students and parents understand the criteria for success. As we completed our first year in remote instruction, many districts learned to communicate with families through alternative platforms such as radio and television, messaging apps with translating capability, and video-conferencing.

Our second finding involved other challenges, such as the "digital use divide," reported by parents. Garbe et al. (2020) studied parents' experiences with remote learning during COVID-19 and found parental involvement to be negatively influenced by economic resources, lack of internet access, lack of interest in using technology, and low digital self-efficacy. These findings were supported by the pre-pandemic literature on parental involvement (Bouffard, 2008; Goodall, 2016) and articulated the need for flexibility in working with parents with varying levels of digital access and efficacy. Lewin and Luckin (2010) posited that parental needs are complex, therefore interactions should be flexibly designed and should concentrate on needs, rather than the technology used to fulfill those needs. With 55% of studies in this review citing communication issues as a challenge, we find this area merits further scholarship as schools and families are increasingly dependent on digital communication methods.

Teacher Supports Needed

Our first finding in this area is the common challenges reported by teachers with less experience with remote teaching before the pandemic. In response to shelter-in-place orders, most teachers were thrust into emergency remote teaching with little to no preparation. In many cases, teachers reported that expectations for live, online presence were unclear and that they were unable to hold students accountable for task completion (Marshall et al., 2020; Midcalf & Boatwright, 2020; Peterson et al., 2020; Trust & Whalen, 2020). Many struggled to engage and motivate students, with teachers working in low-income communities and communities of color that had been disproportionately affected by the virus reporting the most profound challenges (Hamilton et al., 2020; Kraft et al., 2020). Teachers were also learners, with increased searches for distance learning materials and information on video conferencing and learning platforms (Cavanaugh & DeWeese, 2020; Pryor et al., 2020). Those who did not teach online before the pandemic reported that all of their professional job functions such as lesson planning, assessment, and differentiation were much more challenging in a remote environment (Marshall et al., 2020), and they had a lower sense of success and self-efficacy than reported in pre-pandemic surveys (Cardullo et al., 2021; Kraft et al., 2020).

Although many educators enjoy teaching online, those who are new to this practice may face increased workloads, challenges with technology and communicating with students, and difficulty in measuring student outcomes (Kraft et al., 2020; Trust & Whalen, 2020). As noted in previous literature on virtual teaching, online schools struggle against the notion that teachers who have never taught online can shift their pedagogical practice to an online context with little or no preparation, and be immediately successful (Davis & Roblyer, 2005). When schools closed and education transitioned to remote teaching delivery, many teachers were learning new technology platforms and delivery modes at the same time as their students. Participants in 18 studies (36.7%) reported needing significant support with shifting their practice. This lack of teacher preparation of recent teacher education graduates as well as professional development for experienced, in- service teachers was exacerbated by the abrupt transition to "emergency remote instruction." As such, many teachers were also tasked with their own family needs and juggled teaching remotely while supervising their own children's virtual instruction.

Our second finding was that a minimal number of studies provided insight from teachers who felt successful teaching online. Teachers who felt most confident in their remote teaching skills reported work environments with pre-established scheduled professional learning networks and collaboration with colleagues, supportive school leadership, technology help, and meaningful professional development (Kaden, 2020). In Pryor et al.'s (2020) phenomenological study of elementary educators, 15 of 18 teachers indicated that they benefitted from this experience and they planned to incorporate elements of distance education into future teaching. The elements they found most constructive were independent learning, higher level thinking, organization, use of technology to individualize learning, and improved communication with stakeholders (Pryor et al., 2020). The positive effects of remote teaching and learning on a teacher's professional identity is an area that merits further study.

Inequities

Concerns about students' equitable access to resources were reported in 39 studies in this review (79.6%). Our first finding in this section is that access to infrastructure is an issue that needs to be addressed by policy makers, in collaboration with utility providers and telecommunications companies, because it is still a major barrier in many areas of the United States. In order for all students to benefit from remote learning, they need to have reliable technology resources to get online; students in rural areas and lower socio-economic status (SES) households had less access to participate effectively in remote instruction. Students living in households in higher income brackets, in less rural areas of the country with strong Wi-Fi, benefitted from parental assistance with schoolwork and additional learning resources (Bacher-Hicks et al., 2020). Several researchers analyzed their data with a specific focus on equity concerns (Reich et al., 2020; Wachen & Johnson, 2020) and found a range of results from minimal to extensive support from states and schools. It remains unclear how effective remote and virtual learning will be, given that most K12 students and teachers have little experience with online instruction and that large gaps in technology access exist in many parts of the country (Kuhfeld et al., 2020).

Our second finding addresses the challenges faced by students with disabilities and English language learners, who had limited assistance in remote environments without hands-on instruction and classroom supports required for their full participation and access to content (Hamilton et al., 2020; Sugarman & Lazarin, 2020). Educators acknowledged their own struggles to teach students with disabilities (ElSaheli-Elhage, 2021) while Jameson et al. (2020) discussed strategies to ensure the provision of services as mandated by federal law. Teachers must differentiate learning practices to appeal to a wider range of learning styles (Wang, 2020), and address issues of accessibility for students with disabilities, English learners, and other at-risk youth such as homeless students (Kaden, 2020; Reich et al., 2020).

Projected Learning Loss

Our primary finding in this area was that projections for learning loss reinforce persistent achievement disparities and gaps. Discussion about learning loss is connected to the concept of equity, as statistical models have projected learning loss will probably be greatest among lowincome, Black, and Hispanic students (Dorn et al., 2020), along with students from vulnerable populations (Hebert et al., 2020). Models constructed to project student learning loss during school closures have been based on traditional summer learning loss statistics with students grouped by archetypes of remote instruction quality (Dorn et al., 2020). Dorn et al.'s study had real-time in data i-Ready digital-instruction and assessment software, suggesting that only 60 percent of low-income students were regularly logging into online instruction compared to 90 percent of high-income students. With national suspension of annual assessments in the spring of 2020, schools had to administer other forms of assessment for students in face-to-face or remote instruction throughout the 2020-21 school year and try to remediate while at the same time following grade level standards and curriculum. We hope these researchers apply their models to real student data and report the results in future studies.

Student Mental Health/SEL

Our first finding is the detrimental effect of school closures on student social and emotional health and well-being, which affected their engagement in remote learning in all grades. Educator concern for student social and emotional learning (SEL) was cited in eighteen studies (36.7%). Schools serve students' SEL needs and are important places where students connect with their friends and adults. For some students, school may be the place where they engage in their most meaningful relationships. School closures make it more difficult to maintain those relationships, and this may take a toll on students' social and emotional well-being (Oosterhoff et al., 2020; YouthTruth, 2020). Vastly different levels of support were reported from students' personal communities ranging from negative to extremely supportive (Borup et al., 2020). Student engagement in remote learning can be influenced by family social capital including household material, technological resources, and parents' networks. Students with access to high-speed internet and internet-enabled devices consistently reported higher levels of engagement (Domina et al., 2021; Dorn et al., 2020).

Our second finding is the theme of student trauma reported in four studies (Kuhfeld et al., 2020; Margolius et al., 2020; Oosterhoff et al., 2020; Tremmel et al., 2020). Engagement is also connected to student social and emotional well-being, with several researchers noting the unknown impacts of trauma from the COVID-19 crisis that will have long-reaching effects in the following years. While the initial focus during remote instruction was on building relationships and empathy while reviewing content, sustained student participation will require differentiated learning practices and prioritization of interactive, synchronous instruction. There were more concerns reported anecdotally by teachers and school leaders in these studies than in research on students; therefore, we highly recommend that districts formally assess the socioemotional and mental health needs of their students both during and after school closures.

Principal Leadership

In the area of principal leadership, we found the common result that principals and principal candidates felt prepared to maintain a positive learning environment during the pandemic even though they sometimes disagreed with district policies. Teachers with higher levels of self-efficacy reported their school leaders provided social-emotional support and flexibility; communicated frequently and concisely; and maintained a calming and professional presence (Borup et al., 2020; Kaden, 2020; Kraft et al., 2020). Studies focused on school leaders varied; two used participants in school leader/principal preparation courses (Martinez & Broemmel, 2020; Money & Pacifici, 2020) and two focused on principals and noted a caretaker leadership style (Hayes et al., 2020; Weiner et al., 2021. As Hayes et al. (2020) cautioned in their study of 120 principals in nineteen states, the principals' responses might have been indicative of aspirational leadership (i.e., what they hoped to do as principals) rather than actual practice. For this reason, we also encourage researchers to obtain multiple points of view on leadership practices and to triangulate data with more sources than surveys or interviews.

Student Time Management

We found students' ability to effectively manage their time during remote instruction to be a recurrent theme in studies with parent, teacher, and student participants. Wang's (2020) study of nine parents of elementary-aged children reported almost all participants felt negatively about their children's daily academic activities by the end of the school year, describing a sense of burden and frustration because the activities assigned by their teachers interfered with their own routines and competing priorities. Teachers in Lindner et al.'s (2020) study reported weaknesses of distance education in terms of their secondary students' inability to manage their time wisely and work independently. YouthTruth (2020) surveyed students nationally and time management was referenced as a theme in sections of what students liked about remote learning (work pace) and what they found challenging (distractions). One student wrote, "I get distracted a lot more at my house and procrastinate a lot. At school, I was better at time management because I had class time to work. Now I am distracted by tv, pets, games, etc. " (YouthTruth, 2020, p. 82). Some students, however, preferred the pace of learning during remote instruction stating they enjoyed creating their own schedule to follow and were motivated to finish work early so they could participate in other activities. Schaefer et al. (2020) recommended families and schools work together to create a schedule that has firm boundaries, but with aspects of flexibility that can be responsive to students' needs and interests. Further research is needed to elaborate on these initial findings from parent, teacher, and student participants and to determine if, and how, schools were able to provide resources to families to support time management. Students who flourished in the online environment might also consider full-time enrollment in their state virtual schools rather than return to brick-and-mortar classrooms.

Limitations of Studies Reviewed

Researchers in our set of reviewed studies noted several limitations, and time was a common issue. Eight researchers cited the limitation of time constraints of trying to study a current phenomenon, although this limitation could certainly apply to the majority of the studies. Three researchers (Ferreira, 2020; Kraft et al., 2020; Margolius et al., 2020) mentioned the timing of their survey as a potential threat to validity, since it was administered in the first few weeks of the pandemic and participants might have changed their attitudes and behaviors after becoming accustomed to remote learning.

The second limitation often cited was small and/or non-diverse sample size. Eleven studies noted that their sampling size was small, often using participants from only one district (Clausen et al., 2020) and therefore results could not be generalized. Seven studies had samples lacking in diversity, whether that be socio-economic (Garbe et al., 2020; Schaefer et al., 2020; Wang, 2020); geographic (Martinez & Broemmel, 2020; Simpson, 2020), or race/ethnicity (Marshall et al., 2020; Oosterhoff et al., 2020).

The third limitation is specific to online surveys, which by nature exclude those without internet access and may offer a limited perspective, and four researchers noted that participants self-selected themselves into these studies. Online survey was used in 22 studies (44.9%), yet only seven researchers cited the reliability of their survey as a limitation.

The fourth limitation noted by eight researchers was their limited perspective from only sampling one category of participants about their practices, e.g., school leaders (Garet et al., 2020; Hayes et al., 2020) and not collecting data from other participants to gain multiple perspectives.

Finally, we noted other limitations that could cause potential bias, such as participants being recruited from previous studies with a different sampling purpose (Horn & SchneebergerMcGugan, 2020; Weiner et al., 2021); a lack of disaggregated data by urbanicity, SES, and ethnicity in many studies, and data collected from public sources, such as web search trends or support sites (Bacher-Hicks et al., 2020; Cavanaugh & Deweese, 2020), which could not be linked to specific users. Additionally, student outcome data is needed to support predictions of learning loss and the extent to which achievement gaps have widened although this limitation was only cited in four studies (Becker et al., 2020; Domina et al., 2021; Dorn et al., 2020; Gross et al., 2020).

Limitations of Our Study

One limitation of our study was the range of our literature review, since we were studying a recent and ongoing event. After applying our selection criteria discussed in the research methods section above, we significantly narrowed the available studies to capture the experiences of educators and parents during a specific point in time. Several researchers cited their own time constraints in developing and administering surveys or collecting data, which could affect the reliability of those studies. We could continue to expand this study by extending the time period in our search and providing a deep content analysis applying an equity framework.

Synthesis and Conclusions of Literature Review

A systematic literature review "aims to transparently report not only the constructing parameters but also the resulting temporal, geographical, conceptual and thematic trends in the retrieved literature" (Soaita et al., 2020). As such, we will summarize the results from our review findings, explain their contribution to the body of knowledge of the topic of K-12 education during the pandemic, critique any limitations of study results, and explain relevant future research to address these limitations.

Synthesis

Characteristics of Studies Reviewed

Predominant methods of data collection were surveys, interviews, and secondary data analysis, which could all be collected fairly easily in a virtual environment. Many of the secondary data sources included district and state distance learning plans or Google and Microsoft search trends, which were available publicly on the web. Teachers were the most frequent population sampled, followed by parents, school leaders, and students. Only two studies sampled district administrators and two sampled researchers or experts in the field. We noted that four studies with a teacher sample focused on subject-specific instruction, which revealed pedagogical considerations that may be similar or different depending on the content being delivered. In fact, researchers in all four of these studies reported negative effects on student learning that resulted from low parental support. As we know from previous research, parental involvement has a strong impact on student motivation and achievement (Gonzalez-DeHass et al., 2005; Hoover-Dempsey et al., 2001; Olmstead, 2013). The limitations in using any of these methods in isolation is the inability to triangulate data for more reliable and valid results. Additionally, many researchers cited the time constraints they had to create instruments and collect data and several used sample populations they had recruited for previous studies. Future researchers should try to collect data from more diverse participants, and also invest time in observation of teaching and learning in online classrooms.

Analytic Approaches of Studies

This review included 24 qualitative studies, 16 quantitative studies, and nine mixed-methods studies. We found that the personal, lived experience of students during remote learning was best expressed through qualitative data collection methods with embedded quotes provided in the findings to convey participant voice. Data were collected from parent participants primarily from surveys, although parents were also sampled in Schaefer et al.'s (2020) collaborative autoethnography and Wang's (2020) qualitative study using data from electronic diaries and semistructured interviews. Regardless of the approach used, results showed that parents were almost unanimously supportive of the school closures, yet they faced the responsibility of ensuring their children were logging in and participating in remote instruction. A limitation of many of the quantitative studies, including those with models projecting learning loss, is the lack of student achievement data. Further research would connect predictions in spring 2020 to actual achievement scores obtained during the 2020-21 school year. Additionally, studies using document analysis with deductive coding categories may have limitations related to objectivity or confirmability of trustworthiness.

Theoretical Frameworks Used in Studies

The researchers in our review referenced a range of theoretical frameworks as the rationale for conducting their studies, including personal theories, social theories, and technology use theories. Student motives for social distancing were explored in the context of self-determination theory, Bandura's theory of self-efficacy (1977) was cited in three studies of teachers' confidence in meeting the needs of their students, and self-efficacy in crisis was used to review principal reported dispositions. Principal leadership was also analyzed using professionalism, meta-leadership, and psychological safety frameworks. The theory of psychological safety (Schein & Bennis, 1965) merits further use as a framework for studies about teaching and learning during a pandemic, as it connects to many important behavioral outcomes (e.g. engagement, task performance, learning behaviors). This framework could be used in future studies with larger participant pools and more diverse demographics.

Social theories such as the Academic Communities of Engagement (ACE) framework (Borup et al., 2020), communication supporting community resilience (Ferreira, 2020), and social capital (Domina et al., 2021) emphasized the community influence on student learning. These theories are supported by research in our literature review about the importance of developing school-home partnerships and communication channels (Bouffard, 2008; Goodall, 2016; Watkins, 2013). Cardullo et al. (2021) employed a well-known technology framework, the TAM, in their examination of teacher self-efficacy in remote teaching, while Linder et al. (2020) applied transactional distance theory to assess teacher perceptions of remote instruction and distance learning. The most frequent technology framework used was also a new concept, emergency remote teaching, which characterized the abrupt transition to remote teaching during the pandemic and will most likely take hold as a relevant theory to study school closures globally that result from natural disasters.

Contents of Studies Reviewed

A synthesis of the results reported in the studies in this review indicate a universal concern for inequities in technology access, parental support, and social capital for traditionally marginalized students. Efforts to provide students with meaningful learning opportunities through remote instruction were challenging for teachers with limited online teaching knowledge and skills and varying levels of support. Our analyses also indicate that educational leaders should be aware of the risks of social isolation for students and their families. By communicating regularly and consistently with families and providing flexible options for participation, educators can address social-emotional needs and create communities that enrich student engagement and achievement.

There is a lack of consensus regarding the effectiveness of remote learning and methods of evaluating learning gains in remote settings. Nevertheless, it is clear the COVID-19 pandemic has exposed gaps in how successfully education institutions have been able to deliver quality online instruction. The best curricula may not have much impact if teachers have not been trained in pedagogy appropriate to a virtual setting, and learning loss is projected disproportionately for our most vulnerable student populations including students with disabilities and English learners.

Our second research question addressed the common results beneficial to understanding successful strategies, which underscores the importance of data collected from a smaller number of studies in which participants had positive experiences with remote learning. As we know from previous research (Wachira & Keengwe, 2011), when teachers are professionally engaged they are more likely to be active users of technology in their teaching. The teachers who reported more confidence in remote instruction and improved technical skill to benefit their students were also those who belonged to school cultures of collaborative learning before the pandemic that continued to support them. Other teachers cited self-directed learning as another contributing factor to their feelings of self-efficacy. Most teachers will need more opportunities to explore technology platforms and to experience success in this environment through ongoing technology training and support in their schools when they return to face-to-face instruction. School leaders might also address ways to incorporate successful strategies such as flexible pacing, individualized instruction, and rich resources (Cardullo et al., 2021; Linder et al., 2020) into classroom instruction. The positive effects of remote teaching and learning on a teacher's professional identity, a theme that emerged in Pryor et al.'s study (2020), is an area that merits further study.

Students who enjoyed remote learning shared the instructional strategies of their teachers that were most successful in aiding their learning: live videoconferencing, teacher-recorded instructional videos, and game-based learning (Simpson, 2020). While our study participants covered a geographic and socio-economic range of the United States, numerous findings indicated higher student engagement, more live teacher instruction, and social capital support for students in higher socio-economic, non-rural brackets.

Limitations of Studies Reviewed

The temporal nature of this research was cited frequently as a limitation in our reviewed literature, along with time constraints that limited pilot studies of surveys and inter-rater reliability measures. Other limitations included small sample size, non-diverse samples, and samples recruited for previous studies. Using online surveys alone may offer a limited perspective, and four researchers noted that participants self-selected themselves into these studies. Online survey was the data collection instrument used in 22 studies, yet only seven researchers cited the reliability of their survey as a limitation. Another limitation noted by eight researchers was their narrow perspective from only sampling one category of participants about their practices, e.g., school leaders (Garet et al., 2020; Hayes et al., 2020) and not collecting data from other participants to gain multiple perspectives. To address these aforementioned limitations, future studies could employ multiple methods of data collection, take the time for member checking, peer review, and inter-coder reliability training, and also recruit participants using methods other than convenience sampling. As we were also studying this event during a specific period, we could expand our own study by extending the time period in our search and providing a deep content analysis applying an equity framework.

Significant Contributions

It is clear from nearly every study in this review that we need to increase access to infrastructure for online learning, develop digital literacy skills for all stakeholders, and enhance teacher remote delivery competence. There seems to be a broad variation with regard to the strategies that school districts employed in order to handle the uncertainties caused by the COVID19 pandemic. Although advances in educational technology in the last few decades proved immensely useful during this period of mandatory school closures (Dhawan, 2020), teachers, students, parents, and other relevant educators faced many challenges that they were not ready to overcome (Chakraborty et al., 2020). Hence, is essential that teacher and school leader preparation programs support educators in honing their instructional technology skills and meeting the socialemotional and academic needs of all learners to minimize potential achievement and opportunity gaps. It would be helpful to incorporate remote delivery standards into teacher competency frameworks for teacher pre-service and in-service courses.

To respond to the challenges that many teachers reported in delivering instruction online and best prepare for the coming unpredictable combinations of distance learning, blended learning, and in-school learning (Darling-Hammond et al., 2020) it is vital that universities, state departments of education, and local districts develop equitable policies for technology access and accountability. Local school districts require school leaders who are prepared to foster successful, equitable, socially responsible learning and accountability practices for all students.

The results of this study are expected to help guide further research for scholars in the field who are studying the effects of COVID-19 on K-12 education as well as those who are interested in conducting systematic reviews. We may also guide state and district personnel in best allocating their federal funding from recent education stimulus packages.

Future Study

As we continue to study the challenges of COVID-19 for K-12 education, we recommend scholars in the field shift attention to the ways in which the online learning environment has changed teaching practices. Did districts shift professional development during the school year to address teacher needs in remote teaching practice? What effects did online learning have on student achievement? What kinds of students flourished in the online learning environment? When the situation improves and schools across the country re-open fully, will full-time virtual school enrollment increase? Reporting of quantitative data in much of the literature we reviewed was not complete enough to conduct a meta-analysis, but this method could be applied to future studies. In terms of our concept mapping strategy, we have begun to explore other visual mapping software programs, such as VosViewer, which uses bibliographic mapping and graphical representation to express and analyze relational strengths between topics. Visual mapping provides another method to validate qualitative data analyses.

References

References

Ahn, J. & McEachin, A. (2017). Student enrollment patterns and achievement in Ohio's online charter schools. Educational Researcher, 46(1), 44-57. https://doi.org/10.3102/ 0013189X17692999

Ali, A., Mahfouz, A., and Arisha, A. (2017). Analyzing supply chain resilience: Integrating the constructs in a concept mapping framework via a systematic literature review. Supply Chain Management: An International Journal, 22(1), 16-39. https://doi.org/10.1108/SCM-06-20160197

Alias, M. & Suradi, Z. (2008). Concept mapping: A tool for creating a literature review. In A. J. Cañas, P. Reiska, M. Åhlberg & J. D. Novak, (Eds.), Concept Mapping: Connecting Educators Procedure of the Third International Conference on Concept Mapping. OÜ Valli Press.

Archambault, L., & Kennedy, K. (2014). Teacher preparation for K-12 online and blended learning In R. E. Ferdig & K. Kennedy (Eds.), Handbook of research on K-12 online and blended learning (pp. 225-244). ETC Press.

Arnesen, K. T., Hveem, J., Short, C. R., West, R. E., & Barbour, M. K. (2019). K-12 online learning journal articles: Trends from two decades of scholarship. Distance Education, 40(1), 32-53. https://doi.org/10.1080/01587919.2018.1553566

Bacher-Hicks, A., Goodman, J., & Mulhern, C. (2020). Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time. Journal of Public Economics, 193, 104345-104361.

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191-215.

Bebell, D., & Kay, R. (2010). One to one computing: A summary of the quantitative results from the Berkshire Wireless Learning Initiative. Journal of Technology, Learning, and Assessment, 9(2), 1-5.

Becker, S. P., Breaux, R., Cusick, C. N., Dvorsky, M. R., Marsh, N. P., Sciberras, E., & Langberg, J. M. (2020). Remote learning during COVID-19: Examining school practices, service continuation, and difficulties for adolescents with and without attention-deficit/hyperactivity disorder. Journal of Adolescent Health, 67(6), 769-777. https://doi .org/10.1016/jjadohealth.2020.09.002

Borup, J., & Evmenova, A. S. (2019). The effectiveness of professional development in overcoming obstacles to effective online instruction in a college of education. Online Learning, 23(2), 1-20.

Borup, J., Jensen, M., Archambault, L., Short, C. R., & Graham, C. R. (2020). Supporting students during COVID-19: Developing and leveraging academic communities of engagement in a time of crisis. Journal of Technology and Teacher Education, 28(2), 161-169. https://www.learntechlib.org/primary/p/216288/paper_216288.pdf

Bouffard, S. M. (2008). Tapping into technology: The role of the Internet in family-school communication. Harvard Family Research Project.

Cardullo, V., Wang, C. H., Burton, M., & Dong, J. (2021). K-12 teachers' remote teaching selfefficacy during the pandemic. Journal of Research in Innovative Teaching & Learning. https://www.emerald.com/insight/content/doi/10.n08/JRIT-10-2020-0055/fun/html

Carnot, M. J. (2006). Using concept maps to organize information for large scale literature reviews and technical reports: Two case studies. In A. J. Canas & J. D. Novak (Eds.), Concept maps: Theory, methodology, technology: Proceedings of the Second International Conference on Concept Mapping (pp. 296-299). San Jose, Costa Rica.

Cavanaugh, C., & DeWeese, A. (2020). Understanding the professional learning and support needs of educators during the initial weeks of pandemic school closures through search terms and content use. Journal of Technology and Teacher Education, 28(2), 233-238.

Chakraborty, P., Mittal, P., Gupta, M. S., Yadav, S., & Arora, A. (2020). Opinion of students on online education during the COVID-19 pandemic. Human Behavior and Emerging Technologies, 1-9. https://doi.org/10.1002/hbe2.240

Chingos, M. M., & Schwerdt, G. (2014). Virtual schooling and student learning: Evidence from the Florida Virtual School. Harvard Kennedy School, http://k12accountability.org/resources/ Online Education/FLVS_PEPG_working_paper.pdf

Clausen, J. M., Bunte, B., & Robertson, E. T. (2020). Professional development to improve communication and reduce the homework gap in grades 7-12 during COVID-19 transition to remote learning. Journal of Technology and Teacher Education, 28(2), 443-451.

Darling-Hammond, L., & Hyler, M. E. (2020). Preparing educators for the time of COVID... and beyond. European Journal of Teacher Education, 43(4), 457-465. https://doi.org/10.1080/02619768.2020.1816961

Davis, N. E., & Roblyer, M. D. (2005). Preparing teachers for the "schools that technology built": Evaluation of a program to train teachers for virtual schooling. Journal of Research on Technology in Education, 37(4), 399-409.

Dhawan, S. (2020). Online learning: A panacea in the time of COVID-19 crisis. Journal of Educational Technology Systems, 49(1), 5-22.

Domina, T., Renzulli, L., Murray, B., Garza, A. N., & Perez, L. (2021). Remote or removed: Predicting successful engagement with online learning during COVID-19. Socius, 7, 1-15. https://doi.org/10.1177/2378023120988200

Dorn, E., Hancock, B., Sarakatsannis, J., & Viruleg, E. (2020). COVID-19 and student learning in the United States: The hurt could last a lifetime. McKinsey & Company.

Dynarski, S. (2018, January 19). Online courses are harming the students who need the most help. The New York Times. https://www.nytimes.com/2018/01/19/business/online-coursesareharming-the-students-who-need-the-most-help.html

Education Trust. (2020). 10 questions for equity advocates to ask about distance learning. The Education Trust. https://edtrust.org/resource/10-questions-for-equity-advocates-to-ask-aboutdistance-learning/

EdWeek Research Center. (2020, June 25). Survey tracker: Monitoring how K-12 educators are responding to Coronavirus. Education Week. https://www.edweek.org/ew/articles/2020/04/27/ survey-tracker-k- 12-coronavirusresponse.html

ElSaheli-Elhage, R. (2021). Access to students and parents and levels of preparedness of educators during the COVID-19 emergency transition to e-learning. International Journal on Studies in Education (IJonSE), 3(2), 61-69.

Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423-435.

Ferreira, M. C. (2020). COVID-19: Descriptive case study of a K-8 school district's abrupt transition to remote learning from a traditional in-person mode [Doctoral dissertation, University of New England]. DUNE: Digital UNE. https://dune.une.edu/theses/340/

Garbe, A., Ogurlu, U., Logan, N., & Cook, P. (2020). Parents' experiences with remote education during COVID-19 school closures. American Journal of Qualitative Research, 4(3), 45-65. https://doi.org/10.29333/ajqr/8471

Garet, M., Rickles, J., Bowdon, J., & Heppen, J. (2020, July). National survey on public education's coronavirus pandemic response [First look brief]. American Institutes for Research. https://www.air.org/sites/default/files/National-Survey-on-Public-EducationsCoronavirus-Pandemic-Response-First-Look-July-2020.pdf

Gill, B., Walsh, L., Wulsin, C. S., Matulewicz, H., Grau, E., Lee, A., & Kerwin, T. (2015). Inside online charter schools. A report of the national study of online charter schools. Mathematica Policy Research, Inc.

Gonzalez, K. (2021). The impact of school crises on students and families from a social justice perspective. In J. Glanz, Ed. Crisis and pandemic leadership: Implications for meeting the needs of students, teachers, and parents (pp. 113-124). Rowman & Littlefield.

Gonzalez-DeHass, A. R., Willems, P. P., & Holbein, M. F. D. (2005). Examining the relationship between parental involvement and student motivation. Educational Psychology Review, 17, 99-123. https://doi.org/10.1007/s10648-005-3949-7

Goodall, J. S. (2016). Technology and school-home communication. International Journal of Pedagogies and Learning, 11(2), 118-131. https://doi.org/10.1080/22040552.2016.1227252

Gross, B., & Opalka, A. (2020, June). Too many schools leave learning to chance during the pandemic. Center for Reinventing Public Education. https://www.crpe.org/sites/ default/files/final_national_sample_brief_2020.pdf

Hamilton, L. S., Kaufman, J. H., & Diliberti, M. (2020). Teaching and leading through a pandemic: Key findings from the American Educator Panels Spring 2020 COVID-19 surveys. https://www.rand.org/pubs/research_reports/RRA168-2.html

Hart, C. M. D., Berger, D., Jacob, B., Loeb, S., & Hill, M. (2019). Online learning, offline outcomes: Online course taking and high school student performance. AERA Open, 5(1), 117. https://doi.org/10.1177/2332858419832852

Hash, P. M. (2021). Remote learning in school bands during the COVID-19 shutdown. Journal of Research in Music Education, 68(4), 381-397. https://doi.org/10.1177/0022429420967008

Hayes, S. D., Flowers, J., & Williams, S. M. (2020). "Constant communication": Rural principals' leadership practices during a global pandemic. Frontiers in Education, 5, 1-11. https://doi.org/10.3389/feduc.2020.618067

Hebert, M., Goodrich, J. M., & Namkung, J. M. (2020). Characterizing remote instruction provided by elementary school teachers during school closures due to COVID-19. University of Nebraska-Lincoln. https://doi.org/10.35542/osf.io/vsx4q

Heissel, J. (2016). The relative benefits of live versus online delivery: Evidence from virtual Algebra I in North Carolina. Economics of Education Review, 53, 99-115. https://doi.org/10.1016/j.econedurev.2016.05.001

Hinson, J. M., LaPrairie, K. N., & Carroll, E. (2007). Emergency preparedness and e-learning: Recommendations for readiness. Journal of Interactive Instruction Development, 20(2), 3-7.

Ho, L.-H., Hung, C.-L., & Chen, H.-C. (2013). Using theoretical models to examine the acceptance behavior of mobile phone messaging to enhance parent-teacher interactions. Computers & Education, 61, 105-114. https://doi.org/10.1016/j.compedu.2012.09.009

Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020, March 27). The difference between emergency remote teaching and online learning. EduCAUSE Review. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teachingand-online-learning

Hoover-Dempsey, K. V., Battiato, A. C., Walker, J. M., Reed, R. P., DeJong, J. M., & Jones, K. P. (2001). Parental involvement in homework. Educational Psychologist, 36, 195-209.

Horn, I. & Schneeberger-McGugan, K (2020) Adaptive expertise in mathematics teaching during a crisis: How highly committed secondary U.S. mathematics teachers adjust their instruction in the COVID-19 pandemic. National Science Foundation Grant Nos. DRL-1620920. https://osf.io/2w3sx/?view_only=95d825293837460bb6d8414cd89b042e

Horowitz, J. M. (2020, April 15). Lower-income parents most concerned about their children falling behind amid COVID-19 school closures. Pew Research Center. https://www. pewresearch.org/fact-tank/2020/04/15/lower-income-parents-most-concerned-about-theirchildren-falling-behind-amid-covid-19-school-closures/

Hyvärinen, J., & Vos, M. (2015). Developing a conceptual framework for investigating communication supporting community resilience. Societies, 5(3), 583-597.

Jameson, J. M., Stegenga, S. M., Ryan, J., & Green, A. (2020). Free appropriate public education in the time of COVID-19. Rural Special Education Quarterly, 39(4), 181-192. https://doi.org/10.1177/8756870520959659

Kaden, U. (2020). COVID-19 school closure-related changes to the professional life of a K-12 teacher. Education Sciences, 10(6), 1-13. https://doi.org/10.3390/educsci10060165

Kennedy, K., & Archambault, L. (2012). Offering preservice teachers field experiences in K-12 online learning: A national survey of teacher education programs. Journal of Teacher Education, 63(3), 185-200.

Kraft, Matthew A., Simon, N. S., & Lyon, M. A. (2020). Sustaining a sense of success: The importance of teacher working conditions during the COVID-19 pandemic. (EdWorkingPaper: 20-279). Annenberg Institute at Brown University. https://doi.org/10.26300/35nj-v890

Kuhfeld, M., Soland, J., Tarasawa, B., Johnson, A., Ruzek, E., & Liu, J. (2020). Projecting the potential impacts of COVID-19 school closures on academic achievement. (EdWorkingPaper: 20-226). Annenberg Institute at Brown University. https://doi.org/10.26300/cdrv-yw05

Lewin, C., & Luckin, R. (2010). Technology to support parental engagement in elementary education: Lessons learned from the UK. Computers & Education, 54(3), 749-758. https://doi.org/10.1016/j.compedu.2009.08.010

Lindner, J., Clemons, C., Thoron, A., & Lindner, N. (2020). Remote instruction and distance education: A response to COVID-19. Advancements in Agricultural Development, (2)1, 5364. https://doi.org/10.37433/aad.v1i2.39

Margolius, M., Doyle Lynch, A., Pufall Jones, E., & Hynes, M. (2020). The state of young people during COVID-19: Findings from a nationally representative survey of high school youth. Americas Promise Alliance. https://www.americaspromise.org/sites/default/files/d8/ YouthDuringCOVID_FINAL%20%281%29.pdf

Marshall, D. T., Shannon, D. M., & Love, S. M. (2020). How teachers experienced the COVID19 transition to remote instruction. Phi Delta Kappan, 102(3), 46-50. https://doi .org/10.1177/0031721720970702

Martinez, J. A., & Broemmel, A. D. (2020). Pencils down: Educators respond to the uncertainty amidst COVID-19 school closures. International Studies in Educational Administration (Commonwealth Council for Educational Administration & Management (CCEAM)), 49(1), 109-132.

Midcalf, L., & Boatwright, P. (2020). Teacher and parent perspectives of the online learning environment due to COVID-19. Delta Kappa Gamma Bulletin, 87(1), 24-34.

Milman, N. B. (2020). Pandemic pedagogy. Phi Delta Kappan. https://kappanonline.org/ pandemic-pedagogy-covid- 19-online-milman/

Money, K. W., & Pacifici, L. C. (2020). Principal candidates' sense of efficacy: Can they lead during a pandemic? Journal of School Administration Research and Development, 5(1), 4248.

Novak, J. D., & Cañas, A. J. (2006). The origins of the concept mapping tool and the continuing evolution of the tool. Information Visualization, 5(3), 175-184.

Olmstead, C. (2013). Using technology to increase parent involvement in schools. TechTrends, 57(6), 28-37.

Oosterhoff, B. O., Palmer, C. A., Wilson, J., & Shook, N. (2020). Adolescents' motivations to engage in social distancing during the COVID-19 pandemic: Associations with mental and social health. Journal of Adolescent Health, 67, 179-185. https://doi.org/10. 1016/j .jadohealth.2020.05.004

Pesnell, B. (2020). Elementary teachers' experiences with remote learning and its impact on science instruction: Multiple cases from the early response to the Covid-19 pandemic (Order No. 28257717) [Doctoral dissertation, University of Arkansas]. ProQuest Dissertations & Theses Global. https://scholarworks.uark.edu/cgi/viewcontent.cgi?article=5443&context=etd

Peterson, L., Scharber, C., Thuesen, A., & Baskin, K. (2020). A rapid response to COVID-19: One district's pivot from technology integration to distance learning. Information and Learning Sciences, 121(5/6), 461-469. https://doi.org/10.1108/ILS-04-2020-0131

Petticrew, M., & Roberts, H. (2008). Systematic reviews in the social sciences: A practical guide. John Wiley & Sons.

Pew Research Center. (2021). Internet/Broadband Fact Sheet. Retrieved May 28, 2021 from https://www.pewresearch.org/intemet/fact-sheet/intemet-broadband

Pryor, J., Wilson, R. H., Chapman, M., & Bates, F. (2020). Elementary educators' experiences teaching during the COVID-19 school closures: Understanding resources in impromptu distance education. Online Journal of Distance Learning Administration, 23(4), 1-12.

Pulham, E., & Graham, C. R. (2018). Comparing K-12 online and blended teaching competencies: A literature review. Distance Education, 39(3), 411-432.

Reich, J., Buttimer, C. J., Coleman, D., Colwell, R., Faruqi, F., & Larke, L. R. (2020, July). What's lost, what's left, what's next: Lessons learned from the lived experiences of teachers during the pandemic. MIT Teaching Systems Lab. https://doi.org/10.35542/osf.io/8exp9

Rice, M. F., & Carter, R. A. (2015). With new eyes: Online teachers' sacred stories of students with disabilities. In M. F. Rice (Ed.), Exploring pedagogies for diverse learners online (pp. 209-230). Emerald Group Publishing Limited.

Rice, M. F., & Deschaine, M. E. (2020). Orienting toward teacher education for online environments for all students. The Educational Forum, 84(2). 114-125. https://doi .org/10.1080/00131725.2020.1702747

Rousseau, D. M., Manning, J., & Denyer, D. (2008). Evidence in management and organizational science: Assembling the field's full weight of scientific knowledge through syntheses. Academy of Management Annals, 2(1), 475-515.

Rowley, J., & Slack, F. (2004). Conducting a literature review. Management Research News, 27(4), 31-39. https://doi.org/10.1108/01409170410784185

Rush, S. C., Wheeler, J., & Partridge, A. (2014). Emergency online schools as a means of providing schooling and crisis support after school closings due to catastrophic disasters. International Journal of Emergency Management, 10(3/4), 241-258.

Schaefer, M. B., Schamroth Abrams, S., Kurpis, M., Abrams, M., & Abrams, C. (2020). Making the unusual usual: Students' perspectives and experiences of learning at home during the COVID-19 pandemic. Middle Grades Review, 6(2). https://eric.ed.gov/?id=EJ1257609

Schein, E. H., & Bennis, W. G. (1965). Personal and organizational change through group methods: The laboratory approach. Wiley.

Simpson, J. C. (2020). Distance learning during the early stages of the COVID-19 pandemic: Examining K-12 students' and parents' experiences and perspectives. Interaction Design and Architecture Journal, 46, 29-46.

Smith, B. N., Hakim, D., Bassilakis, A., & Owens, E. (2020). The state of American education during the Covid-19 pandemic: A survey of American public, private, and charter school teachers. Intersect: The Stanford Journal of Science, Technology, and Society, 14(1), 1-15.

Soaita, A. M., Serin, B., & Preece, J. (2020). A methodological quest for systematic literature mapping. International Journal of Housing Policy, 20(3), 320-343. https://doi.org/ 10.1080/19491247.2019.1649040

Stone-Johnson, C., & Weiner, J. M. (2020). Principal professionalism in the time of COVID-19. Journal of Professional Capital and Community, 5(3/4). 367-374. https://10.1108/JPCC-052020-0020

Sugarman J., & Lazarin, M. (2020). Educating English Learners during the COVID-19 pandemic. Migration Policy Institute. https://www.migrationpolicy.org/sites/default/files/publications/ mpi-english-learners-covid- 19-final.pdf

Tech Ed Marketing. (2020). The theory of concept mapping. https://www.inspirationat.com/concept-mapping-theory/

Thieman, G., & Cevallos, T. (2017). Promoting educational opportunity and achievement through 1:1 iPads. The International Journal of Information and Learning Technology, 34(5), 409427.

Trust, T., & Whalen, J. (2020). Should teachers be trained in emergency remote teaching? Lessons learned from the COVID-19 pandemic. Journal of Technology and Teacher Education, 28(2), 189-199. https://www.learntechlib.org/primary/p/215995/

Tyminski, A. M., Haltiwanger, L., Zambak, V. S., Horton, R., & Hedetniemi, T. (2013). Developing inquiry practices in middle grades mathematics teachers: Examining the introduction of technology. Contemporary Issues in Technology and Teacher Education, 13(4), 325-359.

U.S. Department of Education, Office of Educational Technology. (2016). Future ready learning: Reimagining the role of technology in education. http://tech.ed.gov/files/2015/12/NETP16.pdf

Wachen, J., & Johnson, M. (2020). Examining equity in remote learning plans: A content analysis of state responses to COVID-19. The Learning Partnership. https://www.jointhepartnership.net/wp-content/uploads/2020/11/State-Remote-LearningAnalysis_TechReport_Nov2020.pdf

Wachira, P., & Keengwe, J. (2011). Technology integration barriers: Urban school mathematics teachers' perspectives. Journal of Science Education and Technology, 20(1), 17-25. https://doi .org/10.1007/s10956-010-9230-y

Wang, K. Y. (2020). Information behavior of parents during COVID-19 in relation to their young school-age children's education. The Serials Librarian, 79(1-2), 62-77. https://doi.org/10.1080/0361526X.2020.1806179

Warschauer, M. (2016). Addressing the social envelope: Education and the digital divide. In C. Greenhow, J. Sonnevend, & C. Agur (Eds.), Education and social media: Toward a digital future (pp. 29-48). MIT Press.

Watkins, A. (2013). Electronic communication and its influence on parental involvement in high school (UMI No. 3589632) [Doctoral dissertation, Walden University.] ProQuest Dissertations and Theses Global.

Weiner, J., Francois, C., Stone-Johnson, C., & Childs, J. (2021, January). Keep safe, keep learning: Principals' role in creating psychological safety and organizational learning during the COVID-19 pandemic. Frontiers in Education (5), 1-17. https://doi.org/10.3389/ feduc.2020.618483

Woodworth, J. L., Raymond, M. E., Chirbas, K., Gonzalez, M., Negassi, Y., Snow, W., & Van Donge, C. (2015). Online charter school study. Center for Research on Education Outcomes, Stanford University. https://charterschoolcenter.ed.gov/sites/default/files/files/field_ publication_attachment/Online%20Charter%20Study%20FinaLpdf

Xiao, Y., & Watson, M. (2019). Guidance on conduction a systematic literature review. Journal of Planning, Education, and Research, 39(1), 93-112.

Xu, D., & Jaggars, S. S. (2014). Performance gaps between online and face-to-face courses: Differences across types of students and academic subject areas. The Journal of Higher Education, 85(5), 633-659.

YouthTruth. (2020). Students weigh in: Learning and well-being during COVID-19. YouthTruth Student Survey. https://youthtruth.surveyresults.org/report_sections/1087936

Zhang, J. (2011, April). A systematic review of cognitive diagnostic assessment and modeling through concept mapping. [Paper presentation]. Annual meeting of the American Educational Research Association in 2011, New Orleans, United States.

Zhang, J., Jang, E., & Chahine, S. (2021). A systematic review of cognitive diagnostic assessment and modeling through concept mapping. Frontiers of Contemporary Education, 2(2), 10-16. https://doi.org/10.22158/fce.v2n1p41