Protocol for a Systematic Review: The Tools of the Mind Curriculum for Improving Self‐Regulation in Early Childhood: A Systematic Review

BACKGROUND

Self-regulation, defined as volitional control of attention, behavior, and executive functions for the purposes of goal-directed action (Blair & Ursache, 2011), is associated with multiple school-related outcomes (Calkins, Howse, & Philippot, 2004; Diamond & Lee, 2011; McClelland & Tominey, 2011). For example, children with robust self-regulation have been shown (Fisher, Hirsh-Pasek, Newcombe, & Golinkoff, 2013; Ramani, 2012) to more cooperatively participate in classroom activities, sustain focus on tasks (Bierman, Nix, & Greenberg, 2008; Drake, Belsky, & Fearon, 2014), and exhibit reduced behavioral issues (Feng et al., 2008; Ponitz, McClelland, Matthews, & Morrison, 2009).

Conversely, lower levels of self-regulation skills are associated with externalizing behaviors (Flouri, Midouhas, & Joshi, 2014; Olson & Lunkenheimer, 2009), diminished attention (Raver et al., 2011; Tough, 2012), and lower academic achievement (Kim, Nordling, Yoon, & Kochanska, 2014; Nota, Soresi, & Zimmerman, 2004; Soares, Vannest, & Harrison, 2009). In addition to academic outcomes, children with poor self-regulatory competencies are more likely to have worse health and financial outcomes in adulthood (Moffitt, Arseneault, & Caspi, 2011; Schlam, Wilson, Shoda, & Mischel, 2013).

Given the role of self-regulation in promoting both child and adult outcomes, early intervention in preschool contexts holds considerable promise for improving a child's development trajectory. As Heckman noted, early “skill begets skill; learning begets learning” (Heckman & Masterov, 2007, p. 3). Consequently, small self-regulatory differences in early childhood can be magnified to progressively larger differences over time (Alexander, Entwisle, & Kabbani, 2001; O'Shaughnessy, Lane, Gresham, & Beebe-Frankenberger, 2003). Thus, early childhood emerges as an especially critical period in which to intervene.

Previous research about the current state of young children's self-regulation development further underscores the need for early intervention. In the United States, a nationally representative survey indicated that 46% of American kindergarten teachers reported at least half of their students to be routinely struggling with self-control (Rimm-Kaufman, Pianta, & Cox, 2000). In fact, American preschool students are three times more likely to be expelled for unmanageable behavior than primary and secondary students (Gilliam, 2005).

Certain subpopulations of children exhibit diminished self-regulation from a young age. For example, children growing up in poverty are more likely to experience self-regulatory problems (Raver, Blair, & Willoughby, 2013; Raver, 2012), which make low-income children susceptible to disciplinary action inside and outside of school (Alloway, Lawrence, & Rodger, 2013; Miller, Nevado-Montenegro, & Hinshaw, 2012). For example, a Washington DC report (Office of the State Superintendent of Education, 2013) revealed that students aged three and four received 181 suspensions during the 2012-2013 school year, most of which went to students in low-income schools.

Moreover, many children in the United States and worldwide have chronic regulatory deficits such as Attention Deficit Hyperactivity Disorder (ADHD) and conduct disorder (CD). In 2013, 11% of American children between the ages 4 of 17 had been diagnosed with ADHD, which reflects a 41% increase over a single decade (Center for Disease Control, 2013). In the UK, 7% of British boys and 3% of British girls aged 5-10 meet the diagnostic criteria for conduct disorder (NICE, 2013), which presents challenges to the educators responsible for student learning (Webster-Stratton, Reid, & Stoolmiller, 2009). Other research indicates that educators across Europe and Asia (Ben-Ari, 1995; Kwon, 2003) have reported rising levels of chronic self-regulatory deficits among young children. Thus, the current state of students' self-regulation has emerged as a pressing problem confronting education systems in countries around the world.

By contrast, the body of evidence on curricula and interventions that significantly improve mainstream students' self-regulation is sparse. For example, the U.S. Department of Education's Institute of Educational Sciences (IES) funded a randomized controlled trial (RCT) that assessed 14 preschool curricula; the results indicated that none of the curricula significantly improved children's self-regulation skills beyond traditional comparator curricula (Preschool Curriculum Evaluation Research Consortium, 2008). Moreover, none of the 14 programs identified self-regulation development as their primary curricular focus, despite abundant research indicating the benefits of self-regulation for young children.

To the best of our knowledge, only one early childhood curriculum emphasizes self-regulation cultivation as its paramount aim: Tools of the Mind (Tools). Since its development in 1993, Tools has been adopted in parts of the United States, Canada, and South America. Twenty U.S. states now have at least one Tools school; in certain areas such as Washington DC, Tools has been implemented in the majority of local preschools (Tools of the Mind, 2015). In the face of the program's proliferation, it is important to establish evidence of Tools' effectiveness on hypothesized outcomes. That is, does Tools enhance children's self-regulation and academic outcomes as compared with traditional ‘business-as-usual’ or other program curricula? This review aims to be the first to address this question.

The Intervention

Tools derives its inspiration from the work of psychologist Lev Vygotsky. In his book Thought and Language (1962), Vygotsky develops the concept of ‘mental tools,‘ which extend mental faculties in the way that physical tools extend physical faculties. For example, although young children typically struggle with task focus, they can be taught to use private speech (e.g., self-talk meant to guide one's actions as opposed to communicate with others) in order to maintain concentration amid distractions. In this case, private speech serves as a mental tool that enables children to focus beyond their baseline abilities (Vygotsky, 1962).

According to the curricular developers, Tools is informed by “neuropsychological research on the development of self-regulation/executive functions in children” (Bodrova & Leong, 2015, Tools' website home page). Unlike several self-regulation interventions, which often involve individualized plans for specific children (Gulchak, 2008; Soares et al., 2009) or a set of exercises to supplement an existing curriculum (Bierman, Domitrovich, Blair, Nelson, & Gill, 2008; Domitrovich, Cortes, & Greenberg, 2007), Tools is intended to be a comprehensive curriculum delivered to all students in a mainstream classroom.

Tools operates through integrating self-regulation oriented activities within academic instruction (Bodrova, Leong, & Akhutina, 2011, p. 18). That is, each Tools activity contains both a target academic skill (e.g., reading a book with a classmate) and a self-regulatory skill (e.g., waiting one's turn to read the book). Overall, Tools includes 61 activities that simultaneously target students' self-regulation as well as their foundational academic skills.

Two activities, Buddy Reading and Make Believe Play, are emblematic of Tools' approach. Buddy Reading involves two students who cooperatively read a book. One child receives a picture of a mouth, which designates him or her as the reader; the other child receives a picture of an ear, which designates him or her as the listener. The reader then reads the story while the other child actively listens and checks for decoding errors. The children then switch roles after the first reader completes the story (Leong & Bodrova, 2011).

Given proper execution, Buddy Reading simultaneously targets literacy and self-regulation. Since self-regulation is defined as the ability to autonomously control attention and behavior, Buddy Reading should theoretically hone children's self-regulation because children must 1) use working memory to remember and act out their roles, 2) demonstrate attentional flexibility by switching across roles, and 3) exhibit inhibitory control to suppress desires to switch roles at inappropriate times (e.g., the listener should not attempt to become the reader before his or her turn).

The second activity emblematic of the Tools approach is Make Believe Play, which is meant to occur every day in Tools classrooms (Bodrova & Leong, 2013). Tools' focus on play originated with Vygotsk/s assertion that pretend play scenarios directly cultivate self-regulation skills: “At every step the child is faced with a conflict between the rule of the game and what he would do if he could suddenly act spontaneously. In the game he acts counter to what he wants … [achieving] the maximum display of willpower” (1933, p. 14). Pretend play thus requires children to focus on a role (e.g., a grocer), enact that role (e.g., help a ‘customer’ bag groceries), and inhibit the impulse to switch roles (e.g., become the grocery store manager instead of the grocer) even when the child wishes to act spontaneously.

Vygotsky (1933) argued that effective play scenarios require three elements: children must 1) determine an imaginary scenario, 2) negotiate roles for themselves and one another, and 3) act out those roles with fidelity (i.e., not switch or cease a role simply because one has lost interest in it). In order to achieve such structured play scenarios, Tools teachers work with students to create play plans as depicted in figure 1.

As observed in figure 1, the play plan includes both textual and pictorial elements. According to the Tools manual (Leong & Bodrova, 2011), play planning involves multiple steps. First, the teacher convenes a group of students who collectively determine a play scenario. Second, students negotiate roles for each child to assume throughout the play block. For example, in figure 1, the children have decided to enact a scenario involving a princess and prince. Each child then creates a play plan that includes his or her name, a picture of the child acting out that role, and a textual description of the play plan. The plan from figure 1 indicates that the student will pretend to be Sleeping Beauty and marry a prince.

Thus, make-believe play planning simultaneously involves writing practice, drawing practice, and goal-oriented thinking to guide the child's subsequent behavior. If students forget their roles, then the teacher and/or other students should reference the play plan (Leong & Bodrova, 2011); in a sense, the play plan constitutes a contract that commits the child to a specific role. This play-planning process precedes the actual play scenario, which is where Vygotsky (1933) argues children's willpower is directly taxed.

In sum, whether children are engaged in literacy, mathematics, or play scenarios, each Tools activity aims to target self-regulation. Tools is designed to be implemented with fidelity by classroom teachers throughout a full academic year (Leong & Bodrova, 2011). This immersive component of Tools differentiates it from other self-regulation programs and emerges as a key mechanism of its purported efficacy.

How the Intervention Might Work

Tools' theory of change contains three stages: 1) students are regulated by their teacher, 2) students regulate one another, and 3) students self-regulate (Bodrova & Leong, 2007). When students first arrive in a classroom, Vygotsky wrote that they are “slaves to their environment,” whereas education's aim must be to transform them into “masters of their own behavior” (L. S. Vygotsky, 1962, p. 147). Bodrova and Leong, the Tools curricular developers, attempted to capture Vygotsky's philosophy through the teacher-regulated, other-regulated, and self-regulated theory of change model depicted in figure 2.

In a quote that encapsulates this theory of change, Vygotsky wrote that “inner regulation of purposeful activity originates in external regulation” (Vygotsky & Luria, 1994, p. 164). That is, before regulating him or herself, a child's thoughts and actions must first be regulated by someone outside of the child (i.e., an adult or more competent peer). In one of his earlier writings, Vygotsky explains that adults must use their superior cognitive control to lead children toward constant improvement of their cognitive control (Vygotsky, 1994, p. 366).

Along a similar line of reasoning, Bruner coined the term ‘scaffolding’ (Wood, Bruner, & Ross, 1976), which refers to a teacher's provision of the minimum support necessary to propel children toward their learning goals. As a child become more competent, the teacher gradually removes support until the child can work independently. The Tools developers refer to both Vygotsky's and Bruner's theories when explaining Tools' theory of change (Bodrova et al., 2011).

Vygotsky's research also informs the second component of Tools' theory of change: Children's co-regulation of one another. Vygotsky argued that children's higher mental function development originates in social interactions before becoming internalized later on (Vygotsky, 1978). The Tools developers explain that the roots of self-regulation originate in other-regulation, which “implies that children act both as subjects of another person's regulatory behaviors and as actors regulating another person's behaviors” (Leong & Bodrova, 2011, p. 73). In sum, the Tools theory of change specifies children's self-regulation as the ultimate goal and then identifies teacher-regulation and other-regulation as the steps toward that end.

Why it is Important to do the Review

Given self-regulation's role in promoting a multitude of desirable life outcomes, it is critical to identify educational practices that improve self-regulation skills. The Tools developers claim that the program effectively promotes children's self-regulation, and Tools has already been implemented in the U.S., Canada, and parts of South America.

Although Tools' proliferation has been consistent in recent years, the findings from Tools evaluation studies have been inconsistent. For example, in a sampling of five randomized Tools evaluation studies, three found small to moderately positive effects on students' self-regulation (Barnett et al., 2008; Blair & Raver, 2014; Diamond, Barnett, Thomas, & Munro, 2007), one found no effect (Lonigan & Phillips, 2012), and one found no effect during pre-kindergarten but negative effects at kindergarten and first grade follow-ups (Farran & Wilson, 2014). Thus, despite relatively similar research designs with considerable methodological overlap, these five studies arrived at substantially different conclusions regarding Tools' effectiveness.

These mixed findings have thus far precluded any authoritative conclusion regarding the curriculum's effectiveness. The literature lacks a systematic review to assess: 1) Tools' overall effect, 2) the heterogeneity of curricular effectiveness across study-level and child-level characteristics, and 3) the quality of the Tools research base as a whole. By investigating those three issues, the present review aims to provide education policymakers and practitioners with useful information regarding whether to implement Tools.

OBJECTIVES

Our central objective is to identify, appraise, and synthesize the available evidence regarding Tools in order to evaluate Tools' effectiveness as compared with other curricula, including business-as-usual and other programs.

Our ancillary objective is to examine study and student characteristics that explain any observed heterogeneity in effect sizes across trials. We can address this secondary objective only if we identify a sufficient number of studies (i.e., >10) to perform meta-regression. If we identify a sufficient number of studies for inclusion, then the study-level moderators for subgroup analyses will include:

• • Study design (i.e., experimental versus quasi-experimental)
• • Study location (i.e., U.S. versus non-U.S.)

The child-level moderators will include:

• • Age: Older children typically have more developed self-regulation than younger children (Piotrowski, Lapierre, & Linebarger, 2013), so does Tools' effect vary for kindergarten versus pre-kindergarten students?
• • Gender: Girls generally exhibit more advanced self-regulation than boys (Barnett et al., 2008; Piotrowski et al., 2013), so does Tools' effect vary across genders?
• • Special education status: Many children with special educational needs exhibit self-regulatory problems (Harris, Friedlander, & Graham, 2005; Soares et al., 2009), so does Tools differentially affect special needs and mainstream students?
• • FRPL eligibility: Students from low-income backgrounds typically exhibit more self-regulation problems than other students (Raver et al., 2013; Raver, 2012), so does Tools' effect change across FRPL and non-FRPL students?

METHODOLOGY

We will conduct this review in accordance with the Campbell Collaboration's systematic review guidelines, which can be found at www.campbellcollaboration.org.

Criteria for including and excluding studies Types of study designs

Included studies should have experimental, quasi-experimental, or non-experimental designs that have adequate statistical mechanisms to control for potential confounds. At the least, studies must have pre- and post-tests on the outcome measures of interest.

Included studies must qualify as one of the following types:

• • Randomized controlled trial: Random assignment of participants to treatment and control groups by the researcher, using a reliable method of randomization (e.g., random source allocation)
• • Regression discontinuity: Researchers assign a threshold or cutoff point (e.g., a birthday cutoff for eligibility into an early childhood program) above or below which the intervention is delivered. Although formal randomization does not occur, comparison of observations lying close to either side of the threshold enables estimation of the treatment effect.
• • Matched control group studies: Treatment group participants are compared against a matched group of controls who are similar on a set of pre-specifred characteristics but do not receive the intervention.
• • Time-series: Participants are observed before, during, and after the intervention to determine whether it had any effect differentiable from underlying trends over time.
• • Pre- and post-design: The treatment and control groups, although not randomly assigned, are tested at the beginning and end of the intervention. The pre-test establishes whether significant group differences exist at the study's outset; the post-test reveals whether a significant treatment effect manifests.

Although we would ideally restrict included studies to randomized trials, randomization is difficult in education research given ethical concerns and school district policies. Thus, this review will accept the quasi-experimental designs described above in order to include as many studies as possible in the review. Nonetheless, we will present results separately for randomized and non-randomized trials.

Types of participants

Students of any age, gender, ethnicity, special education status, language learning status, and socio-economic status will be included in this review.

Types of interventions

We will include any study that analyzes Tools' effect in comparison to one or more “business-as-usual” curricula. Business-as-usual curricula are those that the school had used before the intervention study began. We will also include studies where Tools was implemented alongside another program or intervention that is new for the school.

Types of outcome measures

As indicated above, Tools aims to simultaneously cultivate children's self-regulatory and academic skills. Thus, the primary outcome measures below involve both self-regulation and academic measures. To be eligible for inclusion in this review, studies must include at least one quantitative outcome pertaining to at least one of the domains below:

• • Primary outcomes:
• º Children's self-regulation as reported by teachers, school administrators, parents, and/or researchers. These subjective reports typically derive from observation periods during which a researcher or teacher rates the child's behavior. For example, parents, teachers, or researchers can fill out the Behavioral Rating Inventory of Executive Function – Preschool (BRIEF-P) rating form (Gioia, Espy, & Isquith, 2005), which has 63-items to assess children's inhibitory control, attentional flexibility, working memory, and overall executive control. Any such subjective reports, whether standardized (e.g., BRIEF-P) or unstandardized (e.g., report forms created by the schools), will be included in the analyses. We will perform sensitivity analyses to check the robustness of findings across standardized and unstandardized rating forms.
• º Children's self-regulation skills as indicated by task-based measures. These scores derive from children's task performance on an executive function exercise. For example, the “Heads-Toes-Knees-Shoulders” task involves touching the correct body part based on the teacher's instructions, which change after each round. This activity engages multiple aspects of self-regulation: l) working memory (remembering the teacher's directions and acting upon them), 2) cognitive flexibility (switching among the rules as they change during each round, and 3) inhibitory control (not touching the body part that you hear, but rather the body part that the teacher has previously specified through a rule).
• º Children's academic outcomes as indicated by various assessments. For example, any literacy and numeracy scores on preschool achievement tests (whether standardized or unstandardized) will be included. As with the self-regulation rating scales, we will perform sensitivity analyses to check the robustness of findings across standardized and unstandardized test forms.

Duration of follow-up

We will include data from any follow-up periods included in the original studies. The follow-up data will be classified into three categories: short-term (i.e., data taken between the end of the Tools intervention year to five months following the intervention), medium-term (i.e., data taken between six and 11 months after the end of the Tools intervention), and long-term (i.e., data taken at 12 months or more after the end of the Tools intervention).

Types of settings

We will include studies from any setting where Tools was implemented. Because Tools is a school-based curriculum, we expect that our search will yield only school-based studies. Nonetheless, no a priori setting-based exclusion criteria will be imposed.

Search strategy

In order to conduct a comprehensive search for Tools studies, we will:

• 1. Systematically query the list of databases below. For each database, we will use some variation of “Tools of the Mind” as a search term. We aim to capture every study that mentions Tools at any point in the title, abstract, or text body; thus, a relatively simple search term that includes the program title seems sensible. For example, in the ERIC database, we will use the following search term: AB(“Tools of the Mind”) OR TI(“Tools of the Mind”). We will also search the curricular developers' names (i.e., Bodrova and Leong) in all databases. The full set of databases includes:
• a. ERIC (ProQuest)
• b. ProQuest Dissertations and Theses (ProQuest)
• c. Applied Social Sciences Index and Abstracts (ProQuest)
• d. Sociological Abstracts (ProQuest)
• e. Social Sciences Citation Index (ProQuest)
• f. PsycINFO (Ovid)
• g. LILACS (http://lilacs.bvsalud.org/en/)
• h. MEDLINE (Ovid)
• i. CENTRAL (Cochrane Library)
• j. Embase (Ovid)
• k. OpenGrey (www.opengrey.eu/)

2. Examine the reference lists of relevant primary studies and reviews to identify additional articles. We will also conduct a forward citation search using the Tools' developers' original curricular text (Bodrova & Leong, 2007) as the starting point.

3. Hand search journals including Child Development, Early Childhood Research Quarterly, Early Childhood Education Journal, and Journal of School Psychology.

4. Review the websites of education institutions and technical agencies including:

• a. Tools of the Mind website: (http://www.toolsofthemind.org)
• b. What Works Clearinghouse at the Institute of Educational Sciences: (http://ies.ed.gov/ncee/wwc/)
• c. National Institute for Early Education Research: (http://nieer.org)
• d. Peabody Research Institute: (http://peabody.vanderbilt.edu/research/pri/)

5. Contact experts in the field to inquire about ongoing studies, gray literature, and suggestions for additional contacts.

Description of methods used in primary research

The included primary studies will likely involve either experimental or quasi-experimental research designs. Some studies will involve random assignment of students to either Tools or business-as-usual classrooms; however, other studies may include samples where random assignment was not possible. In instances where district- or school-level policies precluded random assignment, the primary studies should have adequate statistical mechanisms (e.g., pre- and post-test comparisons) to control for potential confounds.

The studies will likely involve academic and self-regulation assessments at the beginning and end of the school year. The academic achievement data may derive from schools' existing tests or from alternative standardized instruments provided by researchers. The self-regulation assessments will likely be provided and administered by researchers, who sample either a group of students in a school/classroom or the entire classroom/school population.

Finally, some primary studies may compare Tools with ‘business-as-usual’ classrooms, whereas others may compare Tools with another intervention group. In this review, all ‘business-as-usual’ and other intervention groups will be considered as the comparator condition, while Tools serves as the treatment condition. We will also run sensitivity analyses to determine whether different effects are observed for Tools versus ‘business-as-usual’ comparisons and Tools versus other intervention comparisons.

Criteria for determination of independent findings

We will presume that any effect sizes from the same study will be statistically dependent regardless of the observed intra-class correlation. Moreover, we will also assume that any instances of multiple reports from the same study will yield data dependency issues. In order to address data dependency, we will use the robust variance estimation SPSS macro described in Tanner-Smith & Tipton (2014). We will also use the metafor package (Viechtbauer, 2010) in R to perform multilevel meta-analysis as a robustness check.

Details of eligibility screening and coding categories

Two researchers (Baron and Melendez-Torres) will independently conduct eligibility screening on all retrieved studies. Specifically, both researchers will screen titles, abstracts, and (where appropriate) full texts in order to determine whether studies are suitable for inclusion in the review. All disagreements of inclusion versus exclusion will be resolved through discussion and consensus.

As for coding, we have developed a data extraction form for this review (see Appendix A). Baron and Melendez-Torres will independently code the studies selected for inclusion. In instances of missing or unclear information, we will contact study authors for clarification. After completing the coding forms for all studies, we will resolve all coding disagreements through discussion and consensus.

In addition to coding study characteristics, Baron and Melendez-Torres will also independently code each RCT for risk of bias using the Cochrane framework (Higgins & Green, 2011). We will rate risk of bias for randomized trials as low-, high-, or unclear-risk across the following categories:

• • Random sequence generation
• • Allocation concealment
• • Blinding of participants and personnel
• • Blinding of outcome assessment
• • Incomplete outcome data
• • Selective reporting

For non-randomized studies, we will use Cochrane's ACROBAT-NSRI framework (Sterne, Higgins, & Reeves, 2014). We will rate risk of bias for non-randomized trials as low-, moderate-, high-, critical- or unclear-risk across the following categories:

• • Confounding bias
• • Selection bias
• • Measurement of intervention bias
• • Departures from intended intervention bias
• • Missing data bias
• • Measurement outcome bias
• • Selection of reported result bias

In the eventual synthesis, each study's risk of bias will be reported using a risk of bias table. Any studies with high risk of bias across multiple categories will be noted in the synthesis, and sensitivity analyses will be performed to determine how results change with and without the biased studies.

Statistical procedures and conventions

We will use the robust variance estimation SPSS macro described in Tanner-Smith & Tipton, (2014) to address data dependency issues. Specifically, some studies will include multiple measures of self-regulation and/or academic achievement. In an attempt to use all available data, we will analyze all relevant effect sizes from each study on both self-regulation and academic achievement while correcting for dependency in effect sizes from the same study. Moreover, we will use the metafor package (Viechtbauer, 2010) in R to perform multi-level meta-analysis with random effects on effect size as a robustness check on the findings.

As for effect size metrics, we will use the standardized mean difference (Hedges' g) for continuous outcomes and the Odds-Ratio (OR) for binary outcomes. After we have calculated effect sizes across all studies, we will convert all included effect sizes into the most common metric (e.g., Hedges' g). To do so, we will use David Wilson's freely available (http://www.campbellcollaboration.org/resources/effect_size_input.php) effect size effect size calculator, which can handle effect size data on both continuous and nominal scales.

Overall, we plan to conduct meta-analyses on four distinct outcomes:

• 1. Task-based self-regulation measures (e.g., HTKS, peg tapping, etc.)
• 2. Informant-based reports of children's self-regulation from teachers and researchers (e.g., BRIEF-P, Child Social Behavior Questionnaire, etc.)
• 3. Measures of children's language and literacy skills (any standardized or unstandardized test)
• 4. Measures of children's numeracy skills (any standardized or unstandardized test)

Effect sizes across different comparison conditions

We will compute effect sizes for each variant of the comparison condition (e.g., business-as-usual, other intervention, no treatment, etc.). For example, if a study compares Tools with another intervention as well as a business-as-usual curriculum, then each of the two comparison conditions will have its own effect size for this review.

Heterogeneity analysis

We will assess heterogeneity across studies using the Q statistic and the I² statistic. For the former, we will use the chi-square test to determine whether the Q-statistic indicates significant heterogeneity among the effect sizes. For the latter, the I² statistic quantifies the ratio of between-study variance to total-variance, which results in a range from 0 to 100%.

Moderation analyses

If we identify a sufficient number of studies (i.e., 10 or more), then we will conduct subgroup analyses to determine whether the intervention effect significantly varies across study-level or participant-level characteristics. We will execute sub-group analyses by comparing the confidence intervals in each sub-group score, which, if not overlapping, signify significant differences across groups.

Once again, the study-level moderators will include:

• • Study design: Do experimental and quasi-experimental designs exhibit consistently different effect sizes and significance values?
• • Study location: Since the intervention was developed in the U.S., then does Tools' effect change across national contexts?

The child-level moderators will include:

• • Age (pre-kindergarten versus kindergarten)
• • Gender (boy versus girl)
• • Special education status (Special education student versus mainstream student)
• • Socio-economic background (FRPL eligibility versus non-FRPL eligibility)

Publication bias assessment

If a sufficient number of studies (more than 10) are identified, then publication bias will be visually inspected using a funnel plot.

Treatment of qualitative research

We do not plan to include qualitative research.

REVIEW AUTHORS

Lead review author:

The lead author is the person who develops and co-ordinates the review team, discusses and assigns roles for individual members of the review team, liaises with the editorial base and takes responsibility for the on-going updates of the review.

[Table omitted. See PDF]

Co-author(s):

[Table omitted. See PDF]

Co-author(s):

[Table omitted. See PDF]

Co-author(s):

[Table omitted. See PDF]

ROLES AND RESPONSIBILITIES

Please give brief description of content and methodological expertise within the review team. The recommended optimal review team composition includes at least one person on the review team who has content expertise, at least one person who has methodological expertise and at least one person who has statistical expertise. It is also recommended to have one person with information retrieval expertise.

Who is responsible for the below areas? Please list their names:

• • Content: Alex Baron and Maria Evangelou
• • Systematic review methods: G.J. Melendez-Torres
• • Statistical analysis: Lars-Erik Malmberg
• • Information retrieval: Alex Baron and G.J. Melendez-Torres

SOURCES OF SUPPORT

The first author, Alex Baron, has funding through the United Kingdom's Economic and Social Research Council.

DECLARATIONS OF INTEREST

The authors have no conflicts of interest.

PRELIMINARY TIMEFRAME

We aim to submit a draft review by February 15, 2016.

PLANS FOR UPDATING THE REVIEW

Alex Baron will be primarily responsible for the review updates, which will occur every three years following the review's publication in Campbell.

AUTHOR DECLARATION Authors' responsibilities

By completing this form, you accept responsibility for preparing, maintaining and updating the review in accordance with Campbell Collaboration policy. The Campbell Collaboration will provide as much support as possible to assist with the preparation of the review.

A draft review must be submitted to the relevant Coordinating Group within two years of protocol publication. If drafts are not submitted before the agreed deadlines, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to de-register the title or transfer the title to alternative authors. The Coordinating Group also has the right to de-register or transfer the title if it does not meet the standards of the Coordinating Group and/or the Campbell Collaboration.

You accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group.

Publication in the Campbell Library

The support of the Coordinating Group in preparing your review is conditional upon your agreement to publish the protocol, finished review, and subsequent updates in the Campbell Library. The Campbell Collaboration places no restrictions on publication of the findings of a Campbell systematic review in a more abbreviated form as a journal article either before or after the publication of the monograph version in Campbell Systematic Reviews. Some journals, however, have restrictions that preclude publication of findings that have been, or will be, reported elsewhere and authors considering publication in such a journal should be aware of possible conflict with publication of the monograph version in Campbell Systematic Reviews. Publication in a journal after publication or in press status in Campbell Systematic Reviews should acknowledge the Campbell version and include a citation to it. Note that systematic reviews published in Campbell Systematic Reviews and co-registered with the Cochrane Collaboration may have additional requirements or restrictions for co-publication. Review authors accept responsibility for meeting any co-publication requirements.

I understand the commitment required to undertake a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:

Form completed by:    Date:

Alex Baron     April 11, 2016

Appendix A: Coding form Tools meta-analysis coding form

[Table omitted. See PDF]