Sustainability of Teacher Implementation of School-Based Mental Health Programs

Abstract

Evidence-based prevention and intervention programs are increasingly being implemented in schools and it therefore is becoming increasingly important to understand the complexities of program implementation under real-world conditions. Much research has focused on the contextual factors that influence program implementation but less work has attempted to provide an integrated understanding of mechanisms (e.g., teacher-training processes) that affect teachers' program implementation. In this paper, we review literature on factors related to teachers' implementation of school-based prevention and intervention programs, then from this review abstract what we believe are four basic ingredients that characterize potentially sustainable teacher-implemented classroom programs. Finally, we present a sequential model, based on these ingredients, of the naturalistic processes underlying sustainability of teachers' program implementation and describe how this sustainability can be enhanced through provision of teacher training and performance feedback from a classroom consultant. [PUBLICATION ABSTRACT]

DOI: 10.1007/s10802-005-7646-2Journal of Abnormal Child Psychology, Vol. 33, No. 6, December 2005, pp. 665679 ( C 2005)Sustainability of Teacher Implementation

of School-Based Mental Health ProgramsSusan S. Han1,3 and Bahr Weiss2Received October 6, 2003; revision received April 16, 2004; accepted June 21, 2005Evidence-based prevention and intervention programs are increasingly being implemented in schools

and it therefore is becoming increasingly important to understand the complexities of program

implementation under real-world conditions. Much research has focused on the contextual factors that

influence program implementation but less work has attempted to provide an integrated understanding

of mechanisms (e.g., teacher-training processes) that affect teachers program implementation. In this

paper, we review literature on factors related to teachers implementation of school-based prevention

and intervention programs, then from this review abstract what we believe are four basic ingredients

that characterize potentially sustainable teacher-implemented classroom programs. Finally, we present

a sequential model, based on these ingredients, of the naturalistic processes underlying sustainability

of teachers program implementation and describe how this sustainability can be enhanced through

provision of teacher training and performance feedback from a classroom consultant.KEY WORDS: program sustainability; program implementation; school-based mental health; teacher training.There is substantial evidence indicating that, when

properly developed and implemented, school-based mental health programs can produce positive effects on childrens behavioral and emotional functioning (Durlak &

Wells, 1997; Greenberg, Domitrovich, & Bumbarger,

2001; Wilson, Lipsey, & Derzon, 2003). As psychosocial factors play a significant role in influencing academic learning (Wang, Haertel, & Walberg, 1997), many

programs targeting socioemotional functioning also note

benefits for childrens academic functioning as well

(Durlak & Wells, 1997; Ialongo et al., 1999; Tremblay,

Pagani-Kurtz, Masse, Vitaro, & Pihl, 1995). Thus, there

is growing recognition that enhancing childrens social

and emotional competencies also facilitates their ability

to learn and achieve academically (Adelman & Taylor,

2000; Greenberg et al., 2003; Zins, Weissberg, Wang, &

Walberg, 2004). At the same time, increased confidence1Institute for Public Policy Studies, Vanderbilt University.2Department of Psychology and Human Development, Vanderbilt University.3Address all correspondence to Susan Han, Vanderbilt Institute for

Public Policy Studies, 1207 18th Avenue South, Nashville, Tennessee

37212; e-mail: [email protected] the basic efficacy of school-based mental health programs has led to more wide-spread program implementation (Clayton, Ballif-Spanvill, & Hunsaker, 2001; Elliot,

1998). One consequence of this movement toward the

dissemination or scaling up of evidence-based programs

is that more attention is being directed to understanding

the complexities of program implementation under realworld conditions (Domitrovich & Greenberg, 2000;

Elias, Zins, Graczyk, & Weissberg, 2003; Pentz, 2004).The literature on scaling up of innovative practices

in schools (e.g., Adelman & Taylor, 2000, 2003; Connell

& Klem, 2000; Elmore, 1996; McLaughlin & Mitra,

2001) has tended to focus on big picture contextual

factors, such as the development of the organizational

infrastructure necessary to support and sustain change

over time (e.g., capacity building, redeployment of resources, integration of services). However, research on

teacher/classroom factors that increase teachers implementation fidelity and the sustainability of quality program implementation by teachers has been relatively

limited, particularly with regard to the integration of

underlying processes into an inclusive model. Although

they have received less attention, the processes that occur within the classroom that lead teachers to implement,6650091-0627/05/1200-0665/0 C 2005 Springer Science+Business Media, Inc.666 Han and Weissand continue to implement, an innovative program with

fidelity are no less critical (Kealey, Peterson, Gaul, &

Dinh, 2000; Rohrbach, Graham, & Hansen, 1993). The

purpose of the present paper is to discuss teacher-level factors that influence program implementation and sustainability, with respect to classroom-based prevention and

intervention mental health programs that are implemented

by teachers.We focus on teacher-implemented mental health programs because it is teachers who often serve as program providers for school-based programs and because,

given the limited resources available for mental health

services relative to the number of children in need of

such services (Tuma, 1989; U.S. Department of Health

and Human Services, 1999), implementation of effective

programs by existing school personnel such as teachers

or school counselors represents one approach to making programs fiscally sustainable (Atkins et al., 1998).

In addition, teacher-implemented programs can be integrated into the general classroom curricula and focused

on the whole classroom. Thus, as central change agents

with a consistent presence in the classroom environment,

teachers can promote childrens positive development and

generalization of positive skills through their ability to

provide children with frequent opportunities to practice

new skills (e.g., Grossman et al., 1997; Hawkins, Von

Cleve, & Catalano, 1991; Weissberg, Barton, & Shriver,

1997).Most programs that have been evaluated and reported in the literature are research projects, financially

supported by federal research grants or other sources of

funding external to the school system that are not available on an ongoing basis (Wilson et al., 2003). When

the research project is completed and external resources

supporting program implementation are withdrawn, it becomes unclear whether an effective classroom-based program will continue to be implemented by teachers who

have been trained and previously monitored by the research project with regard to their program implementation. Several studies do suggest that providing adequate

training with implementation feedback can increase the

likelihood that teachers will continue to implement a program, but degradation in implementation is a recurring

problem (McCormick, Steckler, & McLeroy, 1995; Noell,

Witt, Gilbertson, Ranier, & Freeland, 1997; Rohrbach et

al., 1993). It therefore is essential that researchers examine mechanisms that can be integrated into the training process in order to increase the fidelity with which

evidence-based mental health programs are, and continue

to be, implemented by teachers (Adelman & Taylor, 2003;

Altman, 1995; Elliott & Mihalic, 2004). The examination

of factors in the training process that increase teachers

knowledge of program principles and that sustain a high

level of implementation fidelity is important also for the

scaling up of programs (Coburn, 2003; McLaughlin &

Mitra, 2001), as maintaining quality implementation beyond the training phase is crucial for achieving the positive student outcomes promised by innovative programs

as they are adopted and institutionalized by schools.In this paper, teacher program sustainability is used

to refer to teachers continued implementation of an intervention or prevention program, with ongoing implementation fidelity to the core program principles, after

supplemental resources used to support initial training

and implementation are withdrawn. It is important to note

that program sustainability does not simply imply that

teachers continue to implement a program, but rather that

they continue to implement the program with fidelity and

adherence to program principles. Poor implementation or

failure to achieve treatment fidelity has often been cited

as a major factor underlying the failure of a treatment

program to produce desired effects (e.g., Botvin, Baker,

Dusenbury, Tortu, & Botvin, 1990; Henggeler, Melton,

Brondino, Sherer, & Hanley, 1997; Ialongo et al., 1999;

Rohrbach et al., 1993). If program fidelity cannot be

achieved and maintained by teachers, the programs effects likely will be diminished even if teachers continue

to implement it.CONTEXTUAL FACTORS OVERLYING

TEACHER PROGRAM IMPLEMENTATIONBefore presenting our conceptual model of the processes underlying teachers fidelity in the implementation

of classroom-based mental health programs, it is important first to review the broader contextual factors that may

support, or inhibit, efforts by teachers at program implementation. Teachers program implementation does not

occur within a vacuum, but rather reflects and is strongly

influenced by school reforms and initiatives that occur

within a shifting landscape of sociopolitical priorities and

policies at the county, state, and federal levels (e.g., Safe

and Drug Free Schools Act, No Child Left Behind Act).

This political and legislative landscape to some extent

shapes the district- and school-level policies, priorities,

and resources that in turn influence what teachers teach

in the classroom (Coburn, 2003; Elmore, 1996). More

proximally, district policies and priorities define the conditions under which schools are governed, through the

assignment of principals, allocation of resources, and the

types of in-service training provided to teachers. In addition, the translation of district policies into curricula

selection, reform initiatives, accountability mechanisms,

and evaluation and promotion strategies for principals andTeacher Program Implementation 667teachers communicate values, priorities, and expectations

to teachers (McLaughlin & Mitra, 2001). This backdrop

shapes the types of intervention programs and initiatives

that schools select to implement and support, and the

alignment between the policy context and the prevention

or intervention program under consideration is an important factor in a programs sustainability (Coburn, 2003);i.e., the amount of administrative and instrumental support

provided to implement and sustain a program will be, in

part, determined by its perceived function and importance

in promoting the districts and schools mission.From this perspective, program sustainability is

likely to occur only in the context of institutionalization

of systemic changes in attitudes, expectations, support

mechanisms, and infrastructure (Adelman & Taylor, 2003;

Elmore, 1996). In order for a program to be sustainable

when external or supplemental funding ends, a program

must be integrated into existing school improvement efforts. That is, when the larger system is mobilized to create reinforcing mechanisms to support teachers efforts at

multiple levels, teachers are better able to sustain change

(Bol et al., 1998; Coburn, 2003); this is applicable whether

the innovation is a program developed from a demonstration/research project or part of a program scale-up effort.

Thus, critical steps for program sustainability are the linking of the programs objectives to the mission and priorities of the district and school, and building consensus and

support among key stakeholders at multiple levels to cultivate institutional readiness and support for the program

or initiative (Adelman & Taylor, 2003; Elias et al., 2003).

By ensuring a high level of policy commitment from key

stakeholders, appropriate resources can be garnered for

capacity building and infrastructure development to facilitate and sustain program implementation. However,

the fact that a reform or program has been mandated by

the district or school does not by itself ensure that the

innovation actually will reach the classroom (Rohrbach

et al., 1993). Even under ideal conditions, with district and

school support for an innovative program, there remains

substantial variability in terms of implementation quality

and quantity of the new program delivered at the classroom level (Domitrovich & Greenberg, 2000; McCormick

et al., 1995; Rohrbach et al., 1993). Thus, the present paper

examines teacher-level processes that underlie the implementation of innovative programs in the classroomand

their long-term sustainability.Another reason why it is important to investigate

classroom-level factors is that it is at this level that an

innovative program is adapted, by teachers, to fit the local

classroom context. Such changes usually are necessary,

but they often occur after initial training and supervision

resources are withdrawn, without consultation from program developers or trainers. This raises the question as to

whether a program will retain its effectiveness in the new

setting (Elliott & Mihalic, 2004), since positive outcomes

for students exposed to these programs are very dependent

on the extent to which the program is implemented with

fidelity (Botvin et al., 1990; Dane & Schneider, 1998;

Rorhbach et al., 1993). Thus, it is important to investigate teacher-level factors related to the likelihood that

teachers will maintain a high level of implementation

fidelity. More generally, this highlights the importance

of not only examining the broader district- and schoollevel infrastructure that supports and sustains innovative

programs, but also the lower-level structural and training

mechanisms that enhance their implementation fidelity.

Focusing on this more localbut no less criticallevel

of the classroom, the present paper examines mechanisms

for enhancing teachers program fidelity in implementation as one aspect for increasing program sustainability. In

the next section, we review research on factors proximal

to teachers program implementation, research that serves

as the foundation for our model.RESEARCH ON FACTORS RELATED

TO TEACHER PROGRAM IMPLEMENTATIONIn reviewing factors related to teachers implementation of school-based prevention and intervention

programs, we focus on characteristics of the school,

teacher, and program that are related to the quantity

and quality (i.e., program fidelity) of teachers program

implementation.School- and Teacher-Specific FactorsStudies in the educational and prevention literatures

indicate that teachers implementation of new programs

is related to certain characteristics of the teacher and the

school system in which the new program is to be implemented (in contrast to characteristics that are specific to

the program to be implemented). These teacher and school

characteristics represent preimplementation factors that

are present prior to initial training or implementation of a

new program by teachers. As noted earlier, other factors

in the broader school system (i.e., at the district level) also

contribute to program implementation and sustainability,

such as the alignment between institutional policy and

the programs objectives, flexibility in procedural options

to enact certain policies, allocation of resources, and the

like. But at a more proximal level, the immediate school

context directly defines the conditions that facilitate or

impede teachers efforts and motivation to implement a

program. Although a fairly broad range of variables has668 Han and Weissbeen researched in this area, the following review focuses

on factors most directly related to teachers program implementation, including (a) support of the program by the

school principal, (b) teachers self-efficacy beliefs, and(c) professional burnout. In addition, (d) teachers beliefs

about the acceptability of the program, (e) the compatibility of the program with their own beliefs about student

behavior, and (f) the anticipated effectiveness of the program appear to influence teachers initial motivation to

implement a new program, and hence are reviewed.Administrative Support by the School PrincipalIn their role as leaders of the school, principals serve

as gatekeepers for new curricula and programs that are

introduced and implemented in their schools (Hallinger

& Heck, 1996). Hence, it is not surprising that their attitudes and behavior can significantly affect teachers implementation of new programs (Fullan, Miles, & Taylor,

1980; Gottfredson, Fink, Skroban, & Gottfredson, 1997;

Gottfredson & Gottfredson, 2002). Knowledgeable and

supportive school leadership can be instrumental in making a program a priority within the school, as reflected in

the time, resources, incentives, and training allocated for

the program as well as the expectation of accountability.

In the dissemination of empirically validated programs in

natural conditions, both principal support and a high degree of teacher implementation quality appear to be necessary to produce intervention effects (Kam, Greenberg,

& Walls, 2003). Further, when principals are made aware

of the importance of their support for implementation and

are explicitly requested to encourage and monitor teachers implementation, teachers program implementation

has been found to correspondingly increase (Rohrbach

et al., 1993). Thus, principal support, whether measured

in instrumental support or through affective stance, is an

important factor in teachers implementation.Teacher Self-Efficacy BeliefsTeachers beliefs about their teaching efficacy are

an important form of intrinsic motivation that appear

to strongly influence at least their initial interest in

implementing a new instructional program (Berman,

McLaughlin, Bass, Pauley, & Zellman, 1977; Guskey,

1988; Stein & Wang, 1988). As they relate to academics,

such efficacy beliefs represent a self-judgment of the

teachers capability to affect student performance and

have a strong influence on behavioral output (Tschannen-

Moran, Hoy, & Hoy, 1998). Expectations of personal efficacy in a specific task determine an individuals initiation

of coping behavior, the amount of effort expended on the

task, and persistence on the task despite setbacks; therefore, individuals who perceive themselves as efficacious

tend to produce sufficient effort to generate successful

outcomes when their pertinent actions are also competently executed, whereas those with low self-efficacy are

likely to cease their efforts prematurely and fail in the task

(Bandura, 1997; Bandura, Adams, & Beyer, 1977).As measured by different scales assessing the construct, teachers sense of efficacy has been found to relate to educational outcomes such as instructional behavior (Allinder, 1994), persistence in a teaching situation

(Gibson & Dembo, 1984), enthusiasm (Allinder, 1994;

Guskey, 1988), and commitment to teaching (Coladarci,

1992), as well as student outcomes such as achievement

(Ashton & Webb, 1986), motivation (Midgley, Feldlaufer,

& Eccles, 1989), and students own sense of efficacy

(Anderson, Greene, & Loewen, 1988). Moreover, teachers

with a strong sense of efficacy appear more open to new

ideas and more willing to experiment with new methods to better meet their students needs (Berman et al.,

1977; Guskey, 1988; Stein & Wang, 1988). For example, in a study of federally funded programs introducing

innovative practices in schools, teachers efficacy beliefs

were positively associated with the percentage of project

goals achieved, the amount of teacher change, and the

continued use of project materials and methods after the

project had ended (Berman et al., 1977). Thus, teachers

with a strong sense of efficacy show a higher likelihood

of implementing innovative educational practices and sustaining their effort to produce positive outcomes for their

students, compared to teachers with lower levels of selfefficacy (Berman et al., 1977; Gibson & Dembo, 1984;

Guskey, 1988; Stein & Wang, 1988). Given the empirical

link between self-efficacy and the amount of effort and

persistence expended on a task (Bandura, 1997), teachers

with a greater sense of self-efficacy seem to actually invest

greater effort in program implementation, which in turn

is more likely to lead to successful experiences with new

strategies (if executed competently).Professional BurnoutConversely, teachers program implementation appears to be negatively correlated with their sense of professional burnout. The construct of educator burnout includes three components that appear relevant to program

sustainability. The first is emotional exhaustion, the tired

and fatigued feeling that develops as emotional energies

are drained (Maslach, Jackson, & Schwab, 1996, p. 28),

which can detract from teachers engagement in theirTeacher Program Implementation 669work with students and undermine their interest in new

and innovative practices. A second component of teacher

burnout is depersonalization, wherein educators no longer

have positive feelings about their students and display indifferent or even negative attitudes toward their students.

The third component is a sense of low personal accomplishment from the job, when educators feel that they no

longer are contributing to students development. Teachers level of professional burnout has been found to correlate with turnover intentions, absenteeism, and somatic

problems (Belcastro & Gold, 1983; Jackson, Schwab,

& Schuler, 1986), as well as attributions about students

misbehavior (Bibou, Stogiannidou, & Kiosseoglou, 1999)

and negative interactions with students (Lamude, Scudder,

& Furno-Lamude, 1992). Moreover, in examining teachers attitudes toward implementing an innovative school

program, Evers, Brouwers, and Tomic (2002) found that

teachers with higher levels of burnout endorsed more

negative attitudes about implementing a new program.

Although there is not yet evidence indicating a direct link

between burnout and program adherence per se, its association with other negative teacher outcomes suggests that

teachers who feel burned out are likely to have low selfefficacy and/or invest low effort in instructional behavior,

which likely would adversely affect the implementation

and sustainability of any program.Program Acceptability and Preimplementation

Attributions about the ProgramTeachers initial implementation efforts also may

be influenced by their perceptions and beliefs about a

new program prior to implementation. More specifically,

teachers judgments of the acceptability of an intervention

program significantly influence their interest and willingness to implement a program and the degree to which they

implement the program with fidelity (Reimers, Wacker, &

Koeppl, 1987). Treatment acceptability refers to clients

or laypersons judgments of whether treatment procedures

are appropriate, fair, and reasonable for the problem or

client (Kazdin, 1981, p. 493). In earlier research on the

acceptability of treatments designed for use in clinical

settings, ratings of acceptability were higher for more

effective treatments and when the child problem being

addressed was more serious, but were lower when adverse

side effects were reported (Kazdin, 1981; Kazdin, French,

& Sherrick, 1981).In educational settings, several contextual and

program-specific variables seem to affect teachers preimplementation judgments of a treatments acceptability.

These factors include (a) the severity of the student target

problem being addressed, (b) the type of treatment, and(c) the amount of time required to implement the intervention procedures (Elliott, 1988; Reimers et al., 1987). In addition, preimplementation attributions about (d) the compatibility of the program with teachers own beliefs about

childrens behavior and classroom management (Kealey

et al., 2000), and (e) the anticipated effectiveness of the

program (Von Brock & Elliott, 1987) also appear to influence teachers ratings of a programs acceptabilityand

ultimately the effort that they invest in program implementation.In general, the severity of students problems, operationalized as either the degree to which a student behaves

inappropriately or the number of children who exhibit behavior problems in a classroom, is associated with greater

acceptability for any given treatment (Elliott, 1988). On

the basis of experienced teachers acceptability ratings

for different behavioral interventions, the severity of a

students target problem also influences how complex

an acceptable treatment can be, with the most complex

treatment (such as token economy) being rated as the

most acceptable intervention for the most severe behavior problem (such as destroying property), and the least

complex intervention (such as praise) being rated as the

most acceptable treatment for the least severe problem

behavior (such as daydreaming; Elliott, Witt, Galvin, &

Peterson, 1984). Thus, teachers are more likely to judge

a relatively complex intervention to be more acceptable

when students problems are more severe.In addition, the type of treatment program also influences teachers ratings of treatment acceptability, with

positive treatment procedures being consistently rated as

more acceptable than negative or reductive treatment procedures, such as time-out, response cost, and ignoring

(Elliott et al., 1984; Martens, Peterson, Witt, & Cirone,

1986). Teachers also tend to perceive positive intervention

methods, such as those involving praise, verbal redirection, and differential reinforcement, as the most effective

and easiest to use (Martens et al., 1986).Another salient factor influencing teachers ratings

of treatment acceptability is the amount of time required

to implement an intervention program (Elliott et al., 1984;

Martens et al., 1986; Witt, Martens, & Elliott, 1984). In

rating descriptions of treatment methods that required

varying amounts of teacher time for implementation,

teachers generally preferred treatment methods that were

time-efficient (Witt et al., 1984). However, in judging

intervention strategies for students with severe problems,

teachers allowances for the complexity of a successful

treatmentand consequently the time involved to change

the problem behaviorshifted upwards correspondingly

with the severity of the problem. Thus, although several670 Han and Weissindependent variables contribute to teachers acceptability ratings, teachers expectations about an intervention

program appear to center around contextual appropriateness (i.e., the extent to which the program matches the

problem).Teachers familiarity with an interventions principles also appears to positively influence their ratings of

the programs acceptability. Although an inverse relation

has been found between years of teaching experience and

treatment acceptability ratings (Witt & Robbins, 1985),

teachers with a higher level of knowledge of behavioral principles were found to rate behavioral treatment

techniques (e.g., reinforcement of incompatible behavior, positive practice) as more acceptable than teachers

with a lower level of knowledge (McKee, 1984, as cited

in Elliott, 1988). Moreover, greater teacher implementation of a school-based substance use prevention program was associated with less teaching experience but

with more experience in the specific teaching methods

espoused by the intervention program (Rohrbach et al.,

1993). In addition, higher levels of program implementation were demonstrated by teachers with more favorable

attitudes toward a conflict resolution program (Thorsen-

Spano, 1996) and whose beliefs were congruent with a sex

education programs objectives (DeGaston, Jensen, Weed,

& Tanas, 1994). Hence, judgments of acceptabilityand

consequently the degree of program implementation

may align more with the compatibility between program

principles and teachers beliefs about behavior change or

their experience in specific techniques, rather than level

of teaching experience or knowledge per se.In order to make a fair judgment of a treatment programs acceptability even before they have implemented

the intervention, teachers must understand what the program entails. In their conceptualization of the impact

of treatment acceptability on program implementation,

Reimers et al. (1987) proposed that improved acceptability and potentially increased use of treatment procedures

may be facilitated through increased familiarity with the

interventions principles. In a similar vein, training procedures that aim to increase teachers understanding of an

interventions principles and that focus on building upon

teachers extant experiences and skills that are relevant to

the intervention program may be useful for capitalizing

on teachers positive expectations of a new program and

thus enhancing their motivation to implement and adhere

to the program.Finally, an intervention programs reported effectiveness appears to influence teachers ratings of acceptability,

with teachers judging interventions that are described as

strong and successful to be more acceptable than those that

are described as weak and relatively unsuccessful (Clark

& Elliott, 1988). Conversely, when teachers judge an intervention to be less acceptable, they also rate it as less

effective (Von Brock & Elliott, 1987). Given this relation

between acceptability and perceived effectiveness, providing program effectiveness information may improve

teachers perceptions of the programs acceptability and

thus increase their intention to implement the program.

Akin to outcome expectancies regarding the likely consequences of performing a task at the expected level of competence (Bandura, 1977), teachers positive attributions

about an intervention programs anticipated effectiveness

can serve as a strong incentive for initial implementation

and adherence to the program.Overall, attention to teachers preimplementation

judgments of an intervention programs acceptability is

important because of their potential to influence implementation intentions. Teachers preimplementation perceptions regarding a program significantly affect their

initial motivation to implement a new program, which

will in turn affect the amount of effort that teachers invest

in program implementation and ultimately their level of

program fidelity and sustainability.Program-Specific FactorsIn contrast to the first set of factors reviewed above

that are related to extant characteristics of the school

system and its teachers, the second set of factors focuses on structural and procedural characteristics that

are specific to the program to be implemented. Two important program components that appear to be related to

teacher program implementation are teacher training and

the provision of performance feedback.Teacher TrainingAn important factor in enhancing the quality as well

as quantity of program implementation, and a major determinant of success in school program implementation,

is the amount and quality of the training that teachers

receive in regard to the program (McCormick et al., 1995;

Perry, Murray, & Griffin, 1990; Rohrbach et al., 1993).

Implementation studies of prevention programs indicate

that when teachers are the program implementers, the

provision of adequate training increases the likelihood

that they will implement the curriculum fully and with

integrity (McCormick et al., 1995; Perry et al., 1990). For

example, subsequent to receiving training on the implementation of a school-based smoking prevention program,

trained teachers were more likely to implement, and to implement more of, the curriculum than untrained teachersTeacher Program Implementation 671(McCormick et al., 1995). In addition, trained teachers

were more likely to continue implementing the program

1 year later than teachers who received program materials

but were not trained by the project (McCormick et al.,

1995).However, even among teachers who receive training, degradation in program implementation over time

is a problem often encountered in otherwise effective

programs. Studies of school-based programs to prevent

smoking and substance use indicate that although over

7090% of teachers who receive training (but no ongoing consultation support) may initially implement at

least some of the program, teacher implementation rates

can drop by 2060% 1 year after training (McCormick

et al., 1995; Rohrbach et al., 1993). Thus, the provision of

teacher training may not be sufficient by itself to maintain

teachers program implementation over the long term,

even when the dissemination of a prevention program is

supported by the school district.Performance FeedbackAn important lesson from research on the training

of classroom academic instruction is that the strongest

effects of training are found when classroom practice is

combined with performance feedback (Rose & Church,

1998), which typically consists of feedback (provided

through oral comments, written notes, or graphs) on ways

to improve teachers classroom use of the skills targeted

in training. In their review of studies using direct observation procedures to train teachers, Rose and Church

(1998) found that the provision of feedback consistently

produced the strongest training effect, suggesting that

performance feedback probably is a necessary component for training programs aimed at changing teachers

classroom behavior. A number of studies indicate that

performance feedback provided by a consultant regarding teachers implementation of behavioral plans can

increase teachers use of the intervention program, improve treatment fidelity, and produce greater improvement

in childrens outcomes (Farmer-Dougan, Viechtbauere,

& French, 1999; Jones, Wickstrom, & Friman, 1997;

Mortenson & Witt, 1998; Noell et al., 1997; Witt, Noell,

LaFleur, & Mortenson, 1997).These studies typically have used multiple baseline

designs, wherein teachers initially are taught each step of

a behavioral plan outside of the classroom and then, to

ensure accurate initial implementation, the consultant observes and prompts the teachers as they administer the intervention procedures in the classroom. Although teachers

often initially demonstrate full treatment fidelity, fidelity

has been found to rapidly decrease when teachers were left

to administer the program without the consultant; however, the subsequent provision of ongoing performance

feedback from a consultant results in substantial increases

in fidelity (Mortenson & Witt, 1998; Noell et al., 1997;

Witt et al., 1997), which suggests that consultation support

needs to be of sufficient duration to achieve depth in teachers skills. Other studies of behavioral consultation with

teachers in regard to instructional techniques also show

that direct training procedures in the classroom involving

modeling, rehearsal, and feedback regarding intervention

plans lead to higher treatment fidelity (Sterling-Turner,

Watson, & Moore, 2002). Although in-service workshops

lead to little change in teachers instructional behavior,

the provision of in-classroom performance feedback has

been found to result in change in teacher behavior that is

maintained even after performance feedback is withdrawn

(Leach & Conto, 1999). Although the use of multiple

baseline designs with small sample sizes limits the generalizability of these findings, these studies suggest that

performance feedback during the implementation phase

may be an important component for achieving sustained

program fidelity.It should be noted that some studies that have investigated the effects of different intensities of teacher consultation have not produced significant effects for intensity or

duration of teacher consultation (e.g., Botvin et al., 1990).

However, most of these studies have used relatively weak

manipulations. For instance, in a substance use prevention

study in which teachers either received a 1-day training

workshop with 15 min of implementation feedback and

reinforcement or watched a 2-hr videotape without any

feedback or reinforcement, there was no significant difference in the level of program implementation between

these two groups of teachers (67% vs. 68%) or in their

students outcomes (Botvin et al., 1990). These investigators speculated that the intensity and type of training

and consultation provided to the two groups might not

have been sufficiently different to produce any differential effects; i.e., these findings suggest that the provision

of 15 min of implementation feedback is functionally no

different from no implementation feedback. However, the

minimum amount of ongoing consultation time necessary

to produce increases in fidelity is not yet clear, and further

research is needed to examine the additive effects of different intensities of ongoing consultation provided during

the training and initial implementation phase.INGREDIENTS OF A SUSTAINABLE

SCHOOL-BASED PROGRAMFrom the above literatures, we have abstracted

out four higher-order factors, what might be called672 Han and Weissessential ingredients that characterize potentially sustainable teacher-implemented classroom mental health

programs: A sustainable program must be (a) acceptable to schools and teachers, (b) effective, (c) feasible to

implement on an ongoing basis with minimal (but sufficient) resources, and (d) flexible and adaptable. Although

these ingredients are important in sustaining teachers implementation at the classroom level, these characteristics

by themselves are not sufficient to make a program sustainable. As others have noted (e.g., Adelman & Taylor,

2003), political, bureaucratic, and systemic factors that

define contextual strengths and potential barriers to successful implementation (Ikeda, Tilly, Stumme, Volmer, &

Allison, 1996; Levenson-Gingiss & Hamilton, 1989) also

determine whether a program can and will be sustained

within a particular setting. Thus, when considering implementation of a program, it is important to lay the groundwork to cultivate institutional understanding, readiness,

and support for the program at the school and district

levels (Adelman & Taylor, 2003; Connell & Klem, 2000;

McLaughlin & Mitra, 2001). But fundamentally, any effort to achieve long-term program sustainability must be

based on an effective program with room for adaptation that meets the needs of the school and its students

and is feasible to implement. These essential ingredients that characterize a sustainable program are discussed

below.Acceptability to TeachersIn order for a new program to be effective in the

classroom, teachers first must be willing to make a sincere effort at implementing it. Thus, teachers must view

the program as acceptable, and the programs structure

and content need to motivate and inspire teachers to want

to implement the program. Before implementation, teachers judgments of program acceptability are influenced by

a myriad of contextual and program-specific variables,

including school and district support for the program, the

severity of their students problems, the type of intervention techniques espoused by the program, the amount of

time required to implement the program, the compatibility of the program with their own beliefs about childrens

behavior and classroom management, and the reported

effectiveness of the program (Elliott, 1988; Kealey et al.,

2000; Reimers, Wacker, & Koeppl, 1987; Von Brock &

Elliott, 1987). In evaluating the potential benefits of a

program and their interest in implementing it, teachers

must perceive the program as not only meeting the needs

of their students but also complementing their teaching

style. Only then will potentially effective programs be

given the opportunity to actually be effective in the classroom by teachers who implement it with fidelity and

commitment.Program EffectivenessA basic requirement of a sustainable school-based

program is that the selected program and its intervention

techniques must have the ability to change childrens emotional and behavioral functioning. This is best achieved

by selecting programs that have been demonstrated to be

effective via empirical evaluation. Then, in order for the

program to achieve its potential outcomes, program implementers (i.e., teachers) must achieve program fidelity

in these techniques (e.g., Botvin et al., 1990; Ialongo et al.,

1999; Rohrbach et al., 1993). To increase the likelihood

that program fidelity will be sustained beyond the training period, program fidelity needs to be achieved to an

acceptable-to-superior level during the training period to

ensure that program implementers learn to implement, and

do implement, the program as it was intended (Henggeler

et al., 1997; McCormick et al., 1995). To achieve this standard, training needs to be of sufficient intensity and incorporate in-classroom feedback to establish both depth and

breadth in teachers knowledge and application of the core

principles of the program (Coburn, 2003; McLaughlin &

Mitra, 2001).Although a programs reported effectiveness should

be an important aspect in its selection for implementation, teachers perceptions of the programs effectiveness

for their students after implementation has begun also are

an important component via their effect on motivation

(Kealey et al., 2000). Teachers must believe that there

is a compelling reason for them to practice differently

in their classrooms, and the best direct evidence in this

regard is their perception that their students are behaving

and learning better as a result of their classroom practices (Elmore, 1996). Thus, as they implement a program,

teachers need to observe change in their students behavior, and they also need to attribute this positive change

in their students to their implementation of the program

(Datnow & Castellano, 2000). This experience of success, correctly attributed to their use of the program, is

critical in reinforcing teachers implementation efforts.

Provision of ongoing feedback during the training and

initial implementation phase may be necessary to facilitate this link between teacher program implementation

and improved student outcome (Leach & Conto, 1999;

Rose & Church, 1998). In sum, there are three elements

in the effectiveness ingredient: (a) effective techniques,(b) implementation fidelity, and (c) teacher attributions

linking their teaching practice (i.e., program implementation) to positive effects for their students.Teacher Program Implementation 673Feasibility of Ongoing Implementation,

With Minimal But Sufficient ResourcesSustainable programs need to be not only acceptable to teachers and demonstrably effective, but also must

be practical and feasible to implement in the school setting. A key component to this is that sufficient resources

be available for ongoing program implementation. The

availability of ongoing resources differs across two situations to which program sustainability applies: (a) research

project pull-out, when a demonstration or research project

ends and pulls out of a school system; and (b) program

scale-up, when a program or initiative is disseminated or

scaled up in a school system. In regard to research project

pull-out, funding resources that were available for initial

training and implementation supervision during the active

phase of the research project often are not available when

the project ends; hence, mechanisms and functions to support ongoing implementation need to be planned from the

start of the project, and developed and integrated into the

extant school infrastructure before external training resources are withdrawn (Adelman & Taylor, 2003). In the

case of program scale-up, resources are typically provided

by the district for initial and ongoing implementation, but

in order for the scale-up program to be cost-effective, the

amount of resources required for ongoing implementation

after each set of teachers has been fully trained needs to

level off as the program is sequentially phased in and

replicated in more schools (Taylor, Nelson, & Adelman,

1999).In both situations, however, initial support for training may be funded from an external source (i.e., a research

project or grant) or supplemental resources (for scaling

up), but subsequent critical program functions that serve to

reinforce ongoing implementation and fidelity need to be

subsumed by the extant school infrastructure. For sustainability under both circumstances, programs should require

minimal additional resources for ongoing implementation

after the initial training phase (Atkins et al., 1998), and

the program needs to be suitable for integration within the

schools operations and infrastructure (Adelman & Taylor,

2003). In the current context of limited fiscal resources,

one approach to making a program sustainable is to use

existing school personnel, such as teachers or school counselors, to implement the program. Therefore, to ensure that

school personnel such as teachers implement the program

with fidelity and continue to maintain program fidelity,

teacher training that is of sufficient intensity and that

incorporates classroom practice with performance feedback may be necessary investments (Huberman & Miles,

1984; McCormick et al., 1995; Rose & Church, 1998),

whether teacher training on implementation is embedded

within a demonstration/research project or a scale-up effort. By focusing on the skill development and motivation

of a community of teachers within a school, the initial

influx of training and support resources can be used to

establish self-reinforcing mechanisms to fuel continued

high-quality implementation when this resource for initial

intensive support is diminished.Flexibility and AdaptabilityClassroom circumstances change across the school

year, as students progress or encounter emotional, behavioral, or academic difficulties, and each new school year

brings a new set of students, with different strengths and

needs. Hence, for a program to be sustainable teachers

must be able to adapt the program so that it is appropriate for these changing circumstances and diverse classrooms. This requires two things: (a) the program must be

developed and structured with sufficient flexibility such

that it can be adapted to changing circumstances, and(b) teachers must understand the program well enough

so that they are able to modify it without sacrificing

the core principles and central intervention techniques.

Thus, during the initial training phase, teachers must not

only achieve a superior level of program fidelity but also

develop an in-depth understanding of the core principles underlying the program and its effectiveness, so that

they can understand the limits within which the program

can be modified (McLaughlin & Mitra, 2001). This will

most likely be achieved by providing direct supervision

in program adaptation during the initial training process

(McCormick et al., 1995; Rose & Church, 1998; Sterling-

Turner et al., 2002). In order to maintain adherence to

the core program principles, relatively intensive teacher

training and close initial supervision regarding implementation are imperative, so that teachers fully understand the

programs principles and objectives and can have experience in adapting the program without sacrificing program

fidelity.PROCESS MODEL OF ENHANCED

SUSTAINABILITYIn this final section of the paper we present a sequential model, based on these ingredients, of the naturalistic

processes underlying sustainability in teachers program

implementation, and describe how this sustainability can

be enhanced through provision of teacher training and

performance feedback from a program consultant. In this

model (see Fig. 1), naturally occurring implementation674 Han and WeissPHASE I PHASE IITeacher Training PROJECT SUPPORT Consultant

Feedback Implementation

Activities Attribution

of Student

Functioning

to the Program ONGOING IMPLEMENTATION PROCESSES Experience of

Success Motivation

to Implement

the Program ONGOING IMPLEMENTATION PROCESSES SKILLS Understanding of

Program Principles,

Implementation Techniques PROGRAM IMPLEMENTATION

Program Delivery Program Fidelity

Generalized Use of Program Principles CHANGES IN

STUDENT BEHAVIOR S U P P O R T E D I M P L E M E N T A T I O N P H A S E SUSTA I N A B I L I T Y PHA S E Fig. 1. Process model of sustained program implementation by teachers.processes create a (potentially) self-sustaining feedback

loop. When teachers implement the program correctly

(Program Implementation), given an efficacious program,

this results in improved or maintained appropriate student behavior (Changes in Student Behavior), which in

turn generates teachers Experience of Success in implementing the program. In part through resultant efficacy

beliefs and their Attribution of Student Functioning to

the Program, teachers experiences of success in using the

program leads to increased Motivation to Implement the

Program and increased Skills in implementing the program as teachers learn through experience what is effective in their classrooms, and what is not. This feedback

loop is completed as teacher motivation and skill in program implementation lead to continued correct Program

Implementation.The likelihood of this feedback cycle being established and sustained may be increased through several

training mechanisms, including provision of (a) intensive

Teacher Training and (b) in-classroom Consultant Feedback focused on implementation issues, lasting for a sufficient period of time for the teacher to internalize the core

program principles (Leach & Conto, 1999; McCormick

et al., 1995; Rose & Church, 1998). Subsequent discontinuation of these training activities then marks the shift

to the sustainability phase, by which point the feedback

cycle must have become self-sustaining (although ongoing support from colleagues, etc. remains important) if the

program is to continue to be implemented.The sustainability of the program depends on continued implementation with ongoing program fidelity, even

beyond the training or supported implementation period.Teacher Program Implementation 675Teachers adherence in implementation is the aim of the

feedback cycle, which is facilitated by the training and

consultation activities. As such, when evaluating sustainability it is important to assess teacher program implementation at three levels: (a) program delivery, the quantity

of program lessons and activities that teachers deliver

to their students; (b) program fidelity, the quality of

program implementation, or the degree to which these

program techniques and activities are implemented as

intended (Moncher & Prinz, 1991; Yeaton & Sechrest,

1981); and (c) teachers generalized use of effective program techniques, which reflects a broader application of

program techniques and strategies reflecting core program

principles to various classroom situations.In our model, the sustainability process proceeds

through three phases: (a) Preimplementation Phase, the

time period prior to implementation when teachers and

administrators at a particular school are introduced to the

program, implementation plans for the specific school are

developed, etc.; (b) Supported Implementation Phase,the

time period during which teachers are trained in the program and receive ongoing in-classroom consultation on

program implementation; and (c) Sustainability Phase,

when supplemental or external support for implementation (i.e., training, consultation) has been withdrawn. In

each phase, there are factors that may facilitate or impede

the ultimate sustainability of the program.Preimplementation PhaseFactors that are specific to the school system and its

teachers and that exist prior to the start of the program

define the contextual strengths and potential barriers to

successful implementation (Ikeda et al., 1996; Levenson-

Gingiss & Hamilton, 1989), and can influence program

implementation and long-term sustainability. Such factors, for instance, can impact on teachers intentions to

implement a program; for example, administrative support

for the program, as reflected in principals attitudes and

behaviors, communicate how important principals believe

a new program is and sets the stage for teachers receptivity of the program (Rohrbach et al., 1993). Other teacher

factors that may influence their initial motivation to implement a program include efficacy beliefs regarding their

capabilities as teachers to produce desired student outcomes (Bandura, 1997; Tschannen-Moran & Hoy, 2001),

which have been found to correlate with teacher persistence and willingness to experiment with new methods

(Anderson et al., 1988; Ashton & Webb, 1986; Gibson &

Dembo, 1984); and professional burnout (Maslach et al.,

1996), which correlates with attributions of students misbehavior and negative student-teacher interactions (Bibou

et al., 1999; Lamude et al., 1992). Teachers initial motivation to implement a program is also determined by their

preimplementation perceptions of the program, such as

the acceptability of the program (Reimers et al., 1987), the

compatibility between the programs principles and their

own beliefs (Kealey et al., 2000; Rohrbach et al., 1993),

and the anticipated benefit of the program for their students (Clark & Elliott, 1988; Von Brock & Elliott, 1987).

Even for a demonstrably effective program, it is important

to consider preimplementation factors that can affect the

potential success of the program within a particular school

and with a given teacher. Information about teachers initial attributions about the program, the correctness of their

understanding of the program, the degree of compatibility

with their teaching styles, etc. can suggest areas where

program orientation and training procedures may need to

address teachers misconceptions or concerns.Supported Implementation PhaseThe supported implementation phase is the time period during which supplemental or external resources for

training and implementation support (e.g., consultation)

are provided. The goal of these activities is to maximize

the programs effectiveness, along with helping teachers

to recognize and correctly attribute positive changes in

students behavior to their use of the program. A key

factor in this process is consultant feedback regarding

teachers program implementation and attributions about

the programs impact, with this feedback structured to(a) increase teachers program knowledge and skills, via

direct training as well as corrective feedback and reinforcement of correct implementation; (b) help teachers

learn how to adapt the program to the changing needs

of their classrooms; and (c) support teachers evolving

attributions about the impact of the program on their

students. Initial training for teachers includes learning

about the program principles and the rationale underlying the principles (i.e., theoretical and empirical bases),

the importance of maintaining fidelity to the core principles, classroom lessons and key objectives, implementation techniques and strategies, and acceptable forms of

flexibility and adaptation within the model. Training may

need to occur over the course of several months or more,

as teachers progress through successive modules of the

program. However, it is crucial that teachers begin implementing the program early in the training process, as these

initial efforts not only make the program more concrete

for teachers but also provide opportunities for consultant

feedback.676 Han and WeissConsultation is focused on the feedback loop processes (see Fig. 1) and has two primary functions,

which are to enhance teachers implementation skills and

promote their motivation to implement the program. The

first function of consultation (to help teachers implement

the program accurately and consistently) is achieved by(a) directly observing teachers implementation efforts

and students responses to the program, (b) providing

feedback on implementation and collaboratively resolving

implementation issues, and (c) modeling program techniques and strategies. Consultant feedback is invaluable in

helping teachers to understand potential reasons why their

use of a technique did not produce the desired outcome,

thus pinpointing reasons for failure in their implementation of a technique while at the same time generating

potential solutions. The depth of this training must be sufficient for teachers to be able to differentiate good versus

poor implementation of techniques so that in the future,

when they are no longer receiving consultation, they will

be able to successfully adapt the program to changing

circumstances while still adhering to core principles.The second function of consultation (to promote

teacher motivation to implement the program) is achieved

by the consultant guiding teachers to (a) focus their attention on incremental improvements in students behavior,(b) objectively evaluate and interpret the immediate or

short-term effect of a particular program lesson or technique on their students, and (c) make accurate attributions

of student functioning to their use of the program. The

consultants and teachers joint attention to even small

increments of success is a vital source of positive reinforcement for the teacher, especially during the early

weeks of implementation. If the programs impact is successful but teachers do not perceive any positive impact

of the program early on, their use of the program is

likely to diminish (Gingiss, 1992); thus, program adherence can be influenced by consultant feedback about the

impact of program implementation on classroom functioning (Noell et al., 1997). However, although consultant

feedback and encouragement initially serve as external

reinforcement for teachers successful implementation,

consultation needs to be designed such that as teachers

become successful and their sense of self-efficacy in regard to implementing the program increases, they become

intrinsically motivated via their sense of responsibility for

their students.Sustainability PhaseDuring the previous phase, supplemental or external resources to provide training and consultation support

were available. It is at the point when this implementation

support is no longer available that it becomes necessary

for the feedback cycle to become self-sustaining. This

requires that (a) teacher motivation be primarily intrinsic

and no longer dependent on external reinforcement from

the consultant (although support from the administration,

colleagues, etc. remains crucial), and that (b) teachers

possess the ability to modify the program and evaluate

the utility of these modifications.Success during this time period when external project

support for implementation has ceased is dependent on

the skills of the teacher and his/her ability to continue to

effectively implement the program without the additional

resources provided during the supported implementation

phase. Hence, teachers level of skills and motivation

demonstrated during this sustainability phase is heavily

dependent on the success experienced during the previous

supported implementation phase. In order for teachers

to be able to continue successful implementation, they

must have achieved a high level of implementation during

the previous phase, as indicated by their degree of program delivery, fidelity, and generalized use of program

strategies. As initially developed and refined during the

supported implementation phase, teachers skills in program implementation will produce continued experiences

of success in achieving desired student outcomes during

the sustainability phase. This observable outcome of improved classroom functioning, which teachers can then

properly attribute to their use of the program, will remain

a central component in maintaining teachers motivation

to implement the program.In addition, the program must be sufficiently flexible

and the teacher sufficiently skilled so that s/he can modify the program to meet changing circumstances, without

sacrificing the core principles underlying program effectiveness. Prior experience in implementing the program

successfully, as well as belief in the ability of the program

to improve student behavior (based on prior experience of

success during the supported implementation phase), may

inoculate teachers from prematurely giving up on the

program when strategies do not show immediate effects

with a new cohort of students. Thus, teachers with the

motivation and skills to implement the program are likely

to experience further success in changing their students

behavior in the classroom, which will in turn lead to continued program usethus continuing an evolving (i.e.,

adaptable to changes in the classroom) and ultimately

self-sustaining cycle. In essence, a teacher who is skilled

in implementing a program with fidelity and motivated

to continue implementing the program beyond the training and supported phase is vital to long-term program

sustainability.Teacher Program Implementation 677CONCLUSIONAs we rely increasingly on schools and teachers to

deliver mental health-related intervention programs to students, it is important that research examine the processes

whereby teachers are trained to implement a program accurately and consistently. It is through efficacious programs that are implemented with fidelity that teachers

and schools can achieve the positive outcomes promised

by evidence-based programs (Botvin et al., 1990; Dane

& Schneider, 1998). In turn, it is through observing the

significant impact of such programs on their students that

teachers and schools are likely to continue to implement

these programs (Datnow & Castellano, 2000; Noell et al.,

1997). This cycle, we believe, is the foundation for the

sustainability of program implementation at the classroom

level by teachers. Although district-level factors in the

form of policies, priorities, and resources undoubtedly

influence the systemic conditions that support or interfere with program sustainability (Coburn, 2003; Elmore,

1996), ultimately it rests upon the teacher to actually deliver the program to the classroom with fidelity. Thus, it

is incumbent upon program developers and evaluation researchers to identify effective methods for training teachers on program implementation, so that teachers achieve

program fidelity and also develop an in-depth understanding of the program.In a recent investigation of a semistructured,

cognitive-behavioral skills training program for

prekindergarten children, we (Han, Catron, Weiss,

& Marciel, 2005) provided intensive consultation on

implementation to teachers as they implemented the

classroom program during a supported implementation

phase for one academic year. For teachers who

oftentimes are unfamiliar or uncomfortable with the

demands of mental health curricula that emphasize

interactive skills training methods (e.g., role-playing,

active modeling), implementation support from a

program consultant was beneficial in terms of directly

demonstrating program techniques in the classroom.

Although the results provided preliminary support

for the efficacy of the program on childrens social

skills and behavior problems and for the utility of a

teacher-consultation model for training teachers on

implementation, we were unable to gather data during

what would have been a sustainability phase for these

teachers to determine whether consultation resulted in

teachers continuing to implement the program with

fidelity after the end of the research project.Further research incorporating assessments of both

teacher and student outcomes over multiple years that

cover an initial supported implementation phase as

well an extended sustainability phase is needed. Moreover, given the relatively high cost of providing consultation, the role of implementation consultation needs

to be investigated further to determine whether the investment of intensive in-classroom consultation provided to teachers during the supported implementation phase results in enhanced program implementation and long-term sustainability, compared to providing teachers with only basic training on implementation. Lastly, research on sustainability needs to examine factors that mediate teachers program implementation, such as teachers implementation skills, motivation, and attributions about the programs impact on

students.ACKNOWLEDGMENTSThis paper was supported in part by a grant from

the National Institute of Mental Health (NIMH R01-

MH58275) and by the Vanderbilt Institute for Public

Policy Studies. The authors thank Annalise Caron, Tom

Catron, Carol Guth, Vicki Harris, and Vicky Ngo for their

suggestions, critique, and encouragement.REFERENCESAdelman, H. S., & Taylor, L. (2003). On sustainability of project innovations as systemic change. Journal of Educational and Psychological

Consultation, 14, 125.Adelman, H. S., & Taylor, L. (2000). Moving prevention from the fringes

into the fabric of school improvement. Journal of Educational and

Psychological Consultation, 11, 736.Allinder, R. M. (1994). The relationship between efficacy and the instructional practices of special education teachers and consultants.

Teacher Education and Special Education, 17, 8695.Altman, D. G. (1995). Sustaining interventions in community systems:On the relationship between researchers and communities. Health

Psychology, 14, 526536.Anderson, R., Greene, M., & Loewen, P. (1988). Relationships among

teachers and students thinking skills, sense of efficacy, and student achievement. Alberta Journal of Educational Research, 34,

148165.Ashton, P. T., & Webb, R. B. (1986). Making a difference: Teachers

sense of efficacy and student achievement. New York: Longman.

Atkins, M. S., McKay, M. M., Arvanitis, P., London, L., Madison, S.,Costigan, C., Haney, P., et al. (1998). An ecological model for

school-based mental health services for urban low-income aggressive children. The Journal of Behavioural Health Services and

Research, 5, 6475.Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191215.Bandura, A. (1997). Self-efficacy: The exercise of control. New York:W. H. Freeman.Bandura, A., Adams, N. E., & Beyer, J. (1977). Cognitive processes

mediating behavioral change. Journal of Personality and Social

Psychology, 35, 125139.Belcastro, P. A., & Gold, R. S. (1983). Teacher stress and burnout:Implications for school health personnel. Journal of School Health,

53, 404407.678 Han and WeissBerman, P., McLaughlin, M., Bass, G., Pauley, E., & Zellman, G. (1977).Federal programs supporting education change. Vol. VII: Factors

affecting implementation and continuation (Report No. R-1589/7-

HEW). Santa Monica, CA: The Rand Corporation (ERIC Document Reproduction Service No. 140 432).Bibou, N. I., Stogiannidou, A., & Kiosseoglou, G. (1999). The relation

between teacher burnout and teachers attributions and practices

regarding school behaviour problems. School Psychology International, 20, 209217.Bol, L., Nunnery, J. A., Lowther, D. L., Dietrich, A. P., Pace, J. B.,Anderson, R. S., et al. (1998). Inside-in and outside-in support for

restructuring: The effects of internal and external support on change

in the New American Schools. Education and Urban Society, 30,

358384.Botvin, G. J., Baker, E., Dusenbury, L., Tortu, S., & Botvin, E. M.(1990). Preventing adolescent drug abuse through a multimodal

cognitive-behavioral approach: Results of a 3-year study. Journal

of Consulting and Clinical Psychology, 58, 437446.Clark, L., & Elliott, S. N. (1988). The influence of treatment strength

information on knowledgeable teachers pretreatment evaluations

of social skills training methods. Professional School Psychology,

3, 241251.Clayton, C. J., Ballif-Spanvill, B., & Hunsaker, M. D. (2001). Preventing

violence and teaching peace: A review of promising and effective

antiviolence, conflict-resolution, and peace programs for elementary school children. Applied and Preventive Psychology, 10, 135.

Coburn, C. (2003). Rethinking scale: Moving beyond numbers to deep

and lasting change. Educational Researcher, 32(6), 312.

Coladarci, T. (1992). Teachers sense of efficacy and commitment to

teaching. Journal of Experimental Education, 60, 323337.

Connell, J. P., & Klem, A. M. (2000). You can get there from here:Using a theory of change approach to plan urban education reform. Journal of Educational and Psychological Consultation, 11,

93120.Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary

and early secondary prevention: Are implementation effects out of

control? Clinical Psychology Review, 18, 2345.Datnow, A., & Castellano, M. (2000). Teachers responses to Success

for All: How beliefs, experiences, and adaptations shape implementation. American Educational Research Journal, 37, 775799.

DeGaston, J. F., Jensen, L., Weed, S. E., & Tanas, R. (1994). Teacher

philosophy and program implementation and the impact on sex

education outcomes. The Journal of Research and Development in

Education, 27, 265270.Domitrovich, C. E., & Greenberg, M. T. (2000). The study of implementation: Current findings from effective programs that prevent

mental disorders in school-aged children. Journal of Educational

and Psychological Consultation, 11, 193221.Durlak, J. A., & Wells, A. M. (1997). Primary prevention mental health

programs for children and adolescents. American Journal of Community Psychology, 26, 775802.Elias, M. J., Zins, J. E., Graczyk, P. A., & Weissberg, R. P. (2003).Implementation, sustainability, and scaling up of social-emotional

and academic innovations in public schools. School Psychology

Review, 32, 303319.Elliot, D. S. (1998). Blueprints for violence prevention. Center for

the Study and Prevention of Violence, University of Colorado at

Boulder.Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5, 4753.

Elliott, S. N. (1988). Acceptability of behavioral treatments: Review of

variables that influence treatment selection. Professional Psychology: Research and Practice, 19, 6880.Elliott, S. N., Witt, J. C., Galvin, G. A., & Peterson, R. (1984). Acceptability of positive and reductive interventions: Factors that influence

teachers decisions. Journal of School Psychology, 22, 353360.

Elmore, R. F. (1996). Getting to scale with good educational practice.Harvard Educational Review, 66(1), 125.Evers, W. J., Brouwers, A., & Tomic, W. (2002). Burnout and selfefficacy: A study on teachers beliefs when implementing an innovative educational system in the Netherlands. British Journal of

Educational Psychology, 72, 227244.Farmer-Dougan, V., Viechtbaure, W., & French, T. (1999). Peerprompted social skills: The role of teacher consultation in student

success. Educational Psychology, 19, 207219.Fullan, M., Miles, M. B., & Taylor, G. (1980). Organizational development in schools: The state of the art. Review of Educational

Research, 50, 121183.Gibson, S., & Dembo, M. (1984). Teacher efficacy: A construct validation. Journal of Educational Psychology, 76, 569582.Gingiss, P. L. (1992). Enhancing program implementation and maintenance through a multiphase approach to peer-based staff development. Journal of School Health, 62, 161166.Gottfredson, D. C., Fink, C. M., Skroban, S., & Gottfredson, G. D.(1997). Making prevention work. In R. P. Weissberg, T. P. Gullotta,R. L. Hampton, B. A. Ryan, & G. R. Adams (Vol. Eds.), Issues

in childrens and families lives: Vol. 9: Healthy children 2010:

Establishing preventive services (pp. 219252). Thousand Oaks,

CA: Sage.Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of schoolbased prevention programs: Results from a national survey.

Journal of Research on Crime and Delinquency, 39(1),

335.Greenberg, M. T., Domitrovich, C., & Bumbarger, B., (2001). The prevention of mental disorders in school-aged children: Current state

of the field. Prevention & Treatment, 4, 162.Greenberg, M. T., Weissberg, R. P., OBrien, M. U., Zins, J. E.,Fredericks, L., Resnik, H., et al. (2003). Enhancing school-based

prevention and youth development through coordinated social,

emotional, and academic learning. American Psychologist, 58,

466474.Grossman, D. C., Neckerman, H. J., Koepsell, T. D., Liu, P., Asher,K. N., et al. (1997). Effectiveness of a violence prevention curriculum among children in elementary school: A randomized controlled trial. Journal of the American Medical Association, 277,

16051611.Guskey, T. R. (1988). Teacher efficacy, self-concept, and attitudes toward the implementation of instructional innovation. Teaching and

Teacher Education, 4, 6369.Hallinger, P., & Heck, R. H. (1996). Reassessing the principals role in

school effectiveness: A review of empirical research, 19801995.

Educational Administration Quarterly, 32, 544.Han, S. S., Catron, T., Weiss, B., & Marciel, K. K. (2005). a teacherconsultation approach to social skills training for pre-kindergarten

children: Treatment model and short-term outcome effects. Journal

of Abnormal Chid Psychology, 33, 681693.Hawkins, J. D., Von Cleve, & Catalano, R. F. (1991). Reducing early

childhood aggression: Results of a primary prevention program.

Journal of the American Academy of Child and Adolescent Psychiatry, 30, 208217.Henggeler, S. W., Melton, G. B., Brondino, M. J., Sherer, D. G., &Hanley, J. H. (1997). Multisystemic therapy with violent and

chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting

and Clinical Psychology, 65, 821833.Huberman, A. M., & Miles, M. B. (1984). Innovation up close: How

school improvement works. New York: Plenum.Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S.,& Lin, Y. (1999). Proximal impact of two first-grade preventive

interventions on the early risk behaviors for later substance abuse,

depression, and antisocial behavior. American Journal of Community Psychology, 27, 599641.Ikeda, M. J., Tilly, W. D., Stumme, J., Volmer, L., & Allison, R.(1996). Agency-wide implementation of problem solving consultation: Foundations, current implementation, and future directions.

School Psychology Quarterly, 11, 228243.Teacher Program Implementation 679Jackson, S. E., Schwab, R. L., & Schuler, R. S. (1986). Toward an

understanding of the burnout phenomenon. Journal of Applied Psychology, 71, 630640.Jones, K. M., Wickstrom, K. F., & Friman, P. C. (1997). The effects of observational feedback on treatment integrity in schoolbased behavioral consultation. School Psychology Quarterly, 12,

316326.Kam, C., Greenberg, M. T., & Walls, C. T. (2003). Examining the role

of implementation quality in school-based prevention using the

PATHS curriculum. Prevention Science, 4, 5563.Kazdin, A. E. (1981). Acceptability of child treatment techniques: The

influence of treatment efficacy and adverse side effects. Behavior

Therapy, 12, 493506.Kazdin, A. E., French, N. H., & Sherrick, R. B. (1981). Acceptability of

alternative treatments for children: Evaluation of inpatient children,

parents, and staff. Journal of Consulting and Clinical Psychology,

49, 900907.Kealey, K. A., Peterson, A. V., Jr., Gaul, M. A., & Dinh, K. T. (2000).Teacher training as a behavior change process: Principles and results from a longitudinal study. Health Education and Behavior,

27, 6481.Lamude, K. G., Scudder, J., & Furno-Lamude, D. (1992). The relationship of student resistance strategies in the classroom to teacher

burnout and teacher type-A behavior. Journal of Social Behavior

and Personality, 7, 597610.Leach, D. J., & Conto, H. (1999). The additional effects of process

and outcome feedback following brief in-service teacher training.

Educational Psychology, 19, 441462.Levenson-Gingiss, P., & Hamilton, R. (1989). Determinants of teachers

plans to continue teaching a sexuality education course. Family and

Community Health, 12, 4053.Martens, B. K., Peterson, R. L., Witt, J. C., & Cirone, S. (1986). Teacher

perceptions of school-based interventions. Exceptional Children,

53(3), 213223.Maslach, C., Jackson, S. E., & Schwab, R. L. (1996). Maslach BurnoutInventoryEducators Survey (MBI-ES). In C. Maslach, S. E.

Jackson, & M. P. Leiter (Eds.), MBI manual (3rd ed.). Palo Alto,

CA: Consulting Psychologists Press.McCormick, L. K., Steckler, A. B., & McLeroy, K. R. (1995). Diffusion

of innovations in schools: A study of adoption and implementation

of school-based tobacco prevention curricula. American Journal of

Health Promotion, 9, 210219.McLaughlin, M. W., & Mitra, D. (2001). Theory-based change and

change-based theory: Going deeper, going broader. Journal of Educational Change, 2, 301332.Midgley, C., Feldlaufer, H., & Eccles, J. (1989). Change in teacher

efficacy and student self- and task-related beliefs in mathematics

during the transition to junior high school. Journal of Educational

Psychology, 81, 247258.Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome

studies. Clinical Psychology Review, 11, 247266.Mortenson, B. P., & Witt, J. C. (1998). The use of weekly performance

feedback to increase teacher implementation of a prereferral academic interventions. School Psychology Review, 27, 613627.

Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. T.(1997). Increasing teacher intervention implementation in general

education settings through consultation and performance feedback.

School Psychology Quarterly, 12, 7788.Pentz, M. A. (2004). Form follows function: Designs for prevention

effectiveness and diffusion research. Prevention Science, 5, 2329.

Perry, C. L., Murray, D. M., & Griffin, G. (1990). Evaluating the

statewide dissemination of smoking prevention curricula: Factors

in teacher compliance. Journal of School Health, 60, 501504.

Reimers, T. M., Wacker, D. P., & Koeppl, G. (1987). Acceptability of

behavioral interventions: A review of the literature. School Psychology Review, 16(2), 212227.Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion

of aschool-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237

260.Rose, D. J., & Church, R. J. (1998). Learning to teach: The acquisition

and maintenance of teaching skills. Journal of Behavioral Education, 8, 535.Stein, M. K., & Wang, M. C. (1988). Teacher development and school

improvement: The process of teacher change. Teaching and Teacher

Education, 4, 171187.Sterling-Turner, H. E., Watson, T. S., & Moore, J. W. (2002). The effects

of direct training and treatment integrity on treatment outcomes in

school consultation. School Psychology Quarterly, 17, 4777.

Taylor, L., Nelson, P., & Adelman, H. S. (1999). Scaling-up reforms

across a school district. Reading and Writing Quarterly, 15,

303325.Thorsen-Spano, L. (1996). A school conflict resolution program: Relationships among teacher attitude, program implementation, and job

satisfaction. The School Counselor, 44, 1927.Tremblay, R. E., Pagani-Kurtz, L., Masse, L. C., Vitaro, F., & Pihl,R. O. (1995). A bimodal preventive intervention for disruptive

kindergarten boys: Its impact through mid-adolescence. Journal

of Consulting and Clinical Psychology, 63, 560568.

Tschannen-Moran, M., Hoy, A. W., & Hoy, W. K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research,

68, 202248.Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17,

783805.Tuma, J. M. (1989). Mental health services for children: The state of the

art. American Psychologist, 44, 188199.U.S. Department of Health and Human Services. (1999). Mental health:A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental

Health Services Administration, Center for Mental Health Services, National Institutes of Health, National Institute of Mental

Health.Von Brock, M. B., & Elliott, S. N. (1987). The influence of treatment

effectiveness information on the acceptability of classroom interventions. Journal of School Psychology, 25, 131144.Wang, M. C., Haertel, G. D., & Walberg, H. J. (1997). Learning influences. In H. J. Walberg & G. D. Haertel (Eds.),

Psychology and educational practice (pp. 199211). Berkeley,

CA: McCatchan.Weissberg, R. P., Barton, H. A., & Shriver, T. P. (1997). The Social-Competence Promotion Program for Young Adolescents. In G. W.

Albee & T. P. Gullotta (Eds.), Primary prevention works (pp. 268

290). Thousand Oaks, CA: Sage.Wilson, S. J., Lipsey, M. W., & Derzon, J. H. (2003). The effects

of school-based intervention programs on aggressive behavior: A

meta-analysis. Journal of Consulting and Clinical Psychology, 71,

136149.Witt, J. C., Martens, B. K., & Elliott, S. N. (1984). Factors affecting

teachers judgments of the acceptability of behavioral interventions: Time involvement, behavior problem severity, type of intervention. Behavior Therapy, 15, 204209.Witt, J. C., Noell, G. H., LaFleur, L. H., & Mortenson, B. P. (1997).Teacher usage of interventions in general education: Measurement

and analysis of the independent variable. Journal of Applied Behavior Analysis, 30, 693696.Witt, J. C., & Robbins, J. R. (1985). Acceptability of reductive interventions for the control of inappropriate child behavior. Journal of

Abnormal Child Psychology, 13, 5967.Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice

and maintenance of successful treatments: Strength, integrity, and

effectiveness. Journal of Consulting and Clinical Psychology, 49,

156167.Zins, J. E., Weissberg, R. P., Wang, M. C., & Walberg, H. J. (Eds.) (2004).Building academic success on social and emotional learning: What

does the research say? New York: Teachers College.