Enhancing Student Engagement Through an Institutional Blended Learning Initiative: A Case Study

Abstract

Tertiary education institutions grapple with how to better engage students in their learning in high-enrolment, introductory courses. This paper presents a case study that examines a large-scale, faculty-level course redesign project in which this challenge was addressed through the use of blended learning models. The main research question was: Are students in blended formats engaged in their learning differently than those in the traditional formats? The first part of this paper describes the institutional policies, processes, and practices that were established to implement the course redesign project. The second part of the paper focuses on the effectiveness of the project, presenting the results of a longitudinal research study that examined changes in student engagement using the Classroom Survey of Student Engagement (CLASSE). The implications of the longitudinal evaluation and institutional strategy, structure, and support components are examined critically, as well as the project's impact on students and on the larger university.

Headnote

ABSTRACT

Tertiary education institutions grapple with how to better engage students in their learning in high-enrolment, introductory courses. This paper presents a case study that examines a large-scale, faculty-level course redesign project in which this challenge was addressed through the use of blended learning models. The main research question was: Are students in blended formats engaged in their learning differently than those in the traditional formats? The first part of this paper describes the institutional policies, processes, and practices that were established to implement the course redesign project. The second part of the paper focuses on the effectiveness of the project, presenting the results of a longitudinal research study that examined changes in student engagement using the Classroom Survey of Student Engagement (CLASSE). The implications of the longitudinal evaluation and institutional strategy, structure, and support components are examined critically, as well as the project's impact on students and on the larger university.

KEYWORDS

active learning, blended learning, case study, higher education, student engagement

INTRODUCTION

Large classes present a major challenge to tertiary education institutions, not only to the instructors who teach them, but also to the students sitting in cavernous auditoriums, often struggling to retain focus on a lecture and to resist the easy temptations of the internet and social media (Ragan, Jennings, Massey, & Doolittle, 2014; Fried, 2008). The potential risk of increasingly disengaged students serves as a prompt for institutions to grapple with the question: How can one better engage students in their learning in high-enrolment classes?

Engagement and active learning

Studies show that engagement is key to learning, with higher levels of student engagement leading to better learning outcomes and superior knowledge retention (Kuh, Kinzie, Schuh, Whitt, & associates, 2005). After 20 years of research, Pascarella and Terenzini (1991) conclude, "Simply put, the greater the student's involvement or engagement in academic work or in the academic experience of college, the greater his or her level of knowledge acquisition and general cognitive development" (p. 616).

Engagement is associated with active learning (Zepke & Leach, 2010). When students participate actively in their learning, outcomes such as their retention of concepts over time are enhanced (Foyle, 1995). In a review of the research into active learning, Prince (2004) concluded that active learning strategies are effective if properly implemented; this conclusion is reinforced by a meta-analysis of 225 studies showing that student performance is improved through active learning in science, engineering and mathematics (Freeman et al., 2014). However, while active learning is relatively easy to achieve in small upper-year classes, where effective teaching strategies are easily explored, it is far more challenging to achieve in large introductory lecture classes.

Blended Learning

Over the past decade, institutions have started to redesign courses to address these challenges (Brown, 2016; Twigg, 2000). Here, and throughout, courseis used in the North American understanding of the term as a component of an academic program, equivalent to the British and Australian terms module or unit. Drawing on research related to students' learning processes and innovative approaches to instruction (Ambrose et al., 2010; Mayer, 2008), redesigned courses blend educational technology with improved in-class instructional approaches to enhance the quality of student learning and to improve student success. Vaughan (2007) identified a number of benefits to redesigning courses using a blended learning approach. Because it involves the integration of face-to-face and online learning activities within a course, this approach promotes increased time flexibility for both students and faculty, while also increasing student engagement in learning, and enhancing student-faculty interaction (Vaughan, 2007). By moving the transmission of information online, classroom time can be focused on active learning, where concepts are applied and knowledge is integrated through group work and problem solving. This model is also referred to as a flipped classroom.

In 2010 Means, Toyama, Murphy, Bakia, and Jones published a report that included a meta-analysis conducted to compare the effectiveness of online, face-to-face (traditional lecture) and blended learning approaches. The meta-analysis included empirical studies from 1996 to 2008. Forty-five studies were included in the meta-analysis and provided 50 independent effect sizes (27 effect sizes compared online versus face-to-face learning while 23 effect sizes compared blended versus face-to-face learning). Out of this subset of 45 studies, learners in 40 of these studies included undergraduates from across a range of disciplines including languages, science, and social science. In the case of the 23 studies that compared blended versus traditional face-to-face approaches, results of the meta-analysis found that, on average, learning outcomes for students taught using a blended approach were significantly (p < .001) better than those of students taught using a traditional face-to-face approach (Means et al., 2010).

While the studies examined by Means et al. (2010) were primarily focused on instructor-led course redesign projects, Vaughan (2010) presented an institutional course redesign initiative, the Inquiry Through Blended Learning (ITBL) program, created to support faculty engaged in blended learning. The goal of the ITBL program was to "shift teaching and learning from an essentially passive lecture approach to an engaged and collaborative one" (Vaughan, 2010, p. 60). Evaluation of the program included using a combination of faculty interviews and student surveys. The student survey consisted of items from three of the five National Survey of Student Engagement (NSSE) survey benchmarks: active and collaborative learning, student interactions with faculty members, and level of academic challenge. To highlight the importance of recognizing course redesign for blended learning as an ongoing process, Vaughan (2010) described how an instructor teaching a small third-year course used feedback from student surveys to make changes to their course, resulting not only in significant improvements in aspects of student engagement (e.g., active and collaborative learning) but also increased success and retention rates (as evidenced through student grade distributions). In a second institutional-level project (Garrison & Vaughan, 2013), this time focused on highenrolment first-year courses, results from the Classroom Survey of Student Engagement (CLASSE) (Ouimet & Smallwood, 2005) were compared to student grades in the blended courses. The items selected from the CLASSE are similar to those found in the NSSE. While no causal relationship could be established, students who perceived high levels of active and collaborative learning also received higher grades in the blended courses (Vaughan, Zimmer, & Villamar, 2011).

Building on this type of work, the case study presented in this paper examines a largescale, Faculty-level course redesign project in which high-enrolment introductory courses were transformed into blended models with the goal of enhancing student engagement. The project, while similar to those in the studies discussed above, had several unique features in terms of its design and way in which it was evaluated. Using the framework presented by Graham, Woodfield, and Harrison (2013), which identifies three components for a successful initiative- strategy, structure, and support-the first part of this paper describes the institutional policies and processes that were established to implement the course redesign project. The second part focuses on the effectiveness of the project, presenting the results of the longitudinal research study that examined changes in student engagement. To conclude, the implications of longitudinal evaluation and institutional strategy, structure, and support components are examined critically, as are the project's impact on students and the larger university.

THE COURSE REDESIGN PROJECT

Strategy

Like many postsecondary institutions, the one presented in this case study is currently experiencing the uncomfortable combination of growing enrolment pressures and declining resources. This demographic challenge is manifested in several ways, one of which is that class sizes have expanded at an unprecedented rate. In the Faculty of Arts and Science at this mid-sized university, the expansion of class sizes has been exacerbated by a policy that encourages 3,000 entering first-year students to explore the widest possible array of subjects offered by the 27 departments and schools that make up this faculty, before selecting one or two subjects of concentration at the end of their first year. However, as a result of the flexible curricular structure, class sizes are very large, ranging from 350 to 1,800 enrolments, an outcome somewhat less beneficial to students.

To address this challenge, in 2011 the Faculty of Arts and Science launched the Course Redesign Project, the primary goal of which was to enhance the quality of the student learning experience and improve student engagement in large, introductory classes by enabling and supporting instructors to redesign their courses as blended models. The project built on independent innovations by two instructors of first-year courses in psychology and human geography (e.g., Leger et al., 2013), which served as catalysts, presenting compelling evidence that instructors were ready to start rethinking the ways in which they delivered their courses. However, the personal experiences of the instructors who redesigned their courses were characterized by a range of challenges-an onerous amount of work that was relatively unrecognized by the university, struggles to have the changing needs of their redesigned courses met by central university service units, particularly in the areas of scheduling and room allocation; indifference and occasional hostility from colleagues who felt skeptical or threatened by changes to the status quo-suggesting that there was a need for strong support at the institutional level. In the absence of such support, course redesigns rely on the enthusiasm of an individual faculty member (or members) and can be very difficult to sustain. In order for "change to occur in a desirable and systematic manner," Garrison and Vaughan (2008) believe it is essential for the institutional leadership to provide "clear policy principles and strategic plans" (p. 164). Furthermore, "redesign support is essential to ensure early success and provide prototypes that will help others understand the benefits and possibilities" (p. 165).

To address these issues, the project's key strategic approach was one of integration, with the Faculty Office not only providing resources, but also communicating with all relevant units

in the institution-administrative, academic, and support services-to highlight the benefits and needs of the project and to facilitate and encourage collaboration. Furthermore, the faculty leadership played a strong advocacy role, clarifying to the university community how the Course Redesign Project was helping the institution meet its mission and goals (Niemiec & Otte, 2009). Funded by the university, coordinated by the associate dean of teaching and learning in the Faculty of Arts and Science, and supported by the instructional designers and technology

experts who work in the faculty's online learning department, the project included courses in a range of subjects from the sciences, social sciences, arts, and humanities.

Structure

Proposal submission

Courses were selected for the project through a voluntary proposal submission process that required a formal commitment from not only an instructor but also the home academic department. Since most departments only had one eligible large, introductory-level course, submissions ranged across the disciplines. Departmental support was essential in order to avoid the instructor's feeling isolated in both the development and implementation phases, and to ensure that the course would continue to be offered in its redesigned form once other instructors took over. In an analysis of strategies that led to the successful implementation of course redesigns through the National Center for Academic Transformation, Twigg (2004) identifies "collective decision-making and departmental buy-in as key factors" (p. 15). Requiring departmental support at the proposal submission phase increased the likelihood that the appropriate discussions had taken place and that the blended format was likely to be sustained in the fUture.

While the proposals did not need to include design details at this early stage, they did need to meet the project criteria and to show evidence that the planned redesign would focus on improving student engagement through the inclusion of active learning components. Once proposals were selected, the instructor, the head of the home academic department, and the associate dean (representing the faculty) signed a memorandum of agreement articulating expectations, available resources, and timelines.

Curriculum

In order for a course revision to be more than a temporary pedagogical experiment by an individual instructor, policies need to be developed at an institutional level. A new policy was instated in the Faculty of Arts and Science requiring all blended courses to go through the full curricular approval process as a "course revision." This process formalized the structure of the course, and meant that the redesigned courses had been discussed and approved by the home department, as well as by the faculty's Curriculum Committee, which included representation by both students and instructors from a wide range of disciplines. Furthermore, as an ex officio member of the Faculty Curriculum Committee, the associate dean leading the Course Redesign Project could support blended course proposals as they moved through the approval process.

Course calendar/catalog system

Although many institutions work within a traditional framework in their definition of blended learning, focusing on the percentage of classroom time being replaced by online instruction (Graham et al., 2013), Wallace and Young (2010) propose that "policies will need to be updated to provide the criteria and process whereby classroom contact hours may be reduced when some teaching components are moved online" (p. 5). Before the Course Redesign Project was launched, the Faculty of Arts and Science proactively changed the existing course catalog system in order to accommodate blended courses. Arts and Science courses had always been categorized in the calendar solely in terms of their contact hours, a system that was clearly inadequate for blended courses. A subcommittee of the Faculty Curriculum Committee was charged with reviewing this practice and seeking a more flexible system that could be used for different learning models. On their recommendation, the faculty adopted a system of "student learning hours," whereby the number of hours on task are approximated for the different learning activities involved in each course, including classroom learning, online learning, private study, and so forth, thereby enabling the accurate description of blended courses.

Blended learning models

Following the description suggested by Garrison and Vaughan (2008) in Blended Learning in Higher Education, the Course Redesign Project was based on an understanding of blended learning as the thoughtful, purposeful, and complementary integration of face-to-face learning with online learning. Although a prescribed course template was not appropriate for the range of subjects and disciplines included in the project, the assumptions for a blended course included a classroom focus on engagement through active and collaborative learning, the use of online learning for the fundamental transmission of information, and a reduction in classroom contact time relative to the traditional version of the course.

While classroom time was expected to be focused on active learning involving group work, no single learning strategy was recommended. Instead, faculty members were encouraged to explore the literature supporting different learning strategies and to select those that best

suited their subject matter and learning objectives. Similar flexibility was applied to the online aspects of blended courses. The goal was for students to have an interactive relationship with online materials, but instructors could freely select the most appropriate resources for their subject matter, for example, publisher's materials, open resources, and self-developed materials such as voice-over-slides, podcasts, and vodcasts. The project did not endorse the exclusive use of streamed lecture capture for the online components, because this risked replicating a passive experience for students.

Support

Successful proposals received funding, and instructional design and technology assistance from the Faculty of Arts and Science. Furthermore, the associate dean of teaching and learning coordinated support from other units across the institution, and ran the research study assessing student engagement.

Development team

The need for instructors to receive pedagogical and technological support and

professional development has been identified in several studies (Garrison & Kanuka, 2004; Picciano, 2006). Consequently, a team-based approach was used in the development phase of each course in the project. Most importantly, the project provided each instructor with the services of an instructional designer who provided course design expertise, advising the instructor on best practices, guiding them to design decisions based on evidence in the pedagogical literature, and acting as a project manager for the redesign process. Other team members included an educational technologist, a web developer (if online materials were being created by the instructor), a subject-specialist librarian, to advise on appropriate resources in the discipline, and, in some cases, the graduate student who had been selected to be the lead facilitator for the small group classroom activities.

Incentives

Providing incentives for instructors to adopt blended learning has been shown to increase the chances of successful implementation (Garrison & Kanuka, 2004), not least because the funding compensates them for the time and effort required to redesign a course. The Course Redesign Project paid a development stipend, which either compensated the instructor directly on an overload basis or was used by the home academic department to relieve the instructor of part of their standard workload and to cover some of their teaching. The latter situation was preferable since instructors routinely underestimated the amount of work required to redesign a course and found it challenging to focus on the project while carrying a standard workload and expending their energies according to the normal rhythms and demands of the academic year.

Support during implementation

A large-scale educational initiative requires the integrated support of many different institutional units. Having the Course Redesign Project led from the Faculty Office proved to be particularly effective in raising its profile across the institution and highlighting the need for cooperation during the implementation phase. In particular, the project required support from the Office of the University Registrar-for matters related to scheduling and appropriate classroom space-and from the Information Technology Services unit, since a robust technological infrastructure is critical to successful blended learning implementation (Graham et. al, 2013). Because of ex officio liaisons with these departments, the associate dean was able to ensure that the needs of the Course Redesign Project were discussed at the appropriate levels, so that solutions could be found to potential barriers for instructors before they were realized.

Garrison and Vaughan (2013) emphasize the importance of supporting instructors not only during the design and development phases of the course redesign, but also during implementation and evaluation. In the Course Redesign Project the instructional designers worked one-on-one with each instructor from the earliest stages of the redesign project through to the first delivery of the blended course. Once the post-course assessment of student engagement had been completed, the team met to debrief and to discuss the research reports, and collectively decided on the steps for further development. Improvements were most often focused on class activities and on modes of assessment. For example, if the goal of having higher engagement through more class discussion was not reflected in the data then the team would discuss how to modify specific class activities in order to ensure the active participation of every student. Instructors were encouraged to use the evaluation data to improve their courses as well as to engage in the Scholarship of Teaching and Learning (SoTL), a strategy advocated for by Hutchings, Huber, & Ciccone. (2011).

Peer support

Sharing design ideas, project artifacts, and challenges with peers involved in redesign activities is a valuable way to help instructors to solve problems and to progress in decisionmaking (Garrison & Vaughan, 2013). The associate dean coordinated regular, informal meetings for all team members involved in each redesign, making the gatherings open to those from the community who were not yet part of the project, but who had expressed interest. The attendees were thus not only from different disciplines, but had different levels of experience with course redesign, enabling rich possibilities for peer mentoring. A flexible agenda allowed for ideas to be shared, challenges to be worked through collaboratively, and demonstrations of artifacts, pedagogical approaches or technologies to be mounted, occasionally by experts from outside the institution.

METHODOLOGY

The purpose of this study was to examine the structure of a large-scale course redesign project designed to enhance student engagement and to review its effectiveness. To achieve this purpose, we adopted a case study methodology (Yin, 2009). According to Yin (2009) case study, as a research method, is appropriate when the goal is to understand a real-life phenomenon in depth and such understanding encompasses important contextual conditions.

Data collection

Longitudinal data were gathered to evaluate the effectiveness of the course transformations on student engagement and achievement. As part of the longitudinal data collection, baseline evaluations were completed prior to redesign, and data collection continued thereafter. Ethics approval received for the project also allowed access to demographic and student performance data for further analysis.

To assess the impact of adopting a blended model on student engagement, toward the end of each semester, all students in a participating course were invited to complete a paperbased or online version of the CLASSE. Ouimet and Smallwood (2005) describe the CLASSE as an engagement survey that measures student engagement at the course level. The 40-item CLASSE consists of five sections: engagement activities, cognitive skills, other educational practices, class atmosphere, and demographics. The engagement activities section is made up of 19 items derived from the NSSE (Ouimet & Smallwood, 2005).

Data analysis

To examine the impact of course redesign on student engagement, the 19 items that made up the engagement activities section of the CLASSE were organized into six scalelets based on results of principal components analysis (see Table 1 and Table 2). According to Pike (2006, p. 552) "scalelets consist of sets of survey questions related to specific aspects of students' educational experiences." The six scalelets were labelled: active learning during class, active learning outside class, collaborative learning, course challenge, student-faculty interactions, and writing skills. Examples of items within each scalelet are shown in Table 3.

Scalelet scores were obtaining by calculating the average score for items that made up a scalelet. Each item on the CLASSE is on a 4-point scale, as a result, scalelet scores could range from 1 (indicating no engagement) to 4 (high levels of engagement). Although all scalelets results were analyzed and reviewed with instructors, two scalelets were of particular interest to the Course Redesign Project: active learning during class and student-faculty interactions, and are therefore the primary focus in this paper. Part of course redesign involved the development of active-learning labs within each course. It was, therefore, important to determine whether these labs were perceived by students to promote active learning. Furthermore, although student learning hours remained unchanged between the traditional and blended formats, instructors were concerned about the impact of reduced face-to-face time on students' perceptions of student-faculty interaction. These two scalelets were, therefore, used to answer the question: Are students in blended formats engaged in their learning differently than those in the traditional formats?

To answer this question one-way multivariate analyses of variance (MANOVAs) were conducted for each course. All six scalelets served as dependent variables, and, given that a new cohort took the course each year, year served as the independent variable. For the post-hoc analyses of variance (ANOVAs), to determine where statistically significant differences existed, traditional year served as the control year.

RESULTS

To date, 16 courses have joined the Course Redesign Project: one each from biology, chemistry, classics, drama, film, gender studies, geography, health, history, law, mathematics, sociology, and statistics; and three from psychology. This set of courses includes four year-long courses and 12 semester-long courses. Of the 16 courses, eight completed at least four evaluation cycles and, during the period of data collection, enrolment numbers ranged from 133 in the drama course to 1,537 in a psychology course (Table 4). In Table 4, year-long courses are referenced by the two calendar years that they spanned; semester-long courses are indicated as either Fall or Winter.

Impact on student engagement

With regard to student engagement, two scalelets were of particular interest to the Course Redesign Project: active learning during class and student-faculty interactions. Table 5, Table 6, and Table 7 provide the results from the eight courses that completed at least four evaluation cycles during the period of data collection with regard to students' perceptions of active learning during class and student-faculty interactions.

Overall, across the eight courses, with the exception of the initial blended versions of GNDS 120 and MATH 121, results showed consistent, statistically significant improvements in perceptions of active learning during class from the traditional format to subsequent blended formats. Furthermore with exception of GNDS 120 and PSYC 100, effect sizes were moderate to large, ranging from 0.11 to 0.44. With regard to perceptions of student-faculty interactions, the pattern was not as consistent and effect sizes were small (ranging from 0.01 to 0.09). In comparing the traditional format to the initial blended format, statistically significant improvements in perceptions of student-faculty interactions were noted in five of the eight courses. This trend in improvement was only sustained in MATH 121.

Examples of course redesign

To get a better sense of the types of changes that occurred during the course redesign project, two examples are presented. In one, the course redesign was considered successful from its earliest iteration (PSYC 100), while the other took several years to stabilize (GNDS 120). Although grade distributions were of interest and reviewed in each course, they are not included in this paper due to the recognition that grade distribution changes were confounded by aspects such as changes to assessments during redesign and changes to departmental standards.

Principles of Psychology (PSYC 100)

PSYC 100 is a 6.0-unit course serving as the gateway course for concentrations in Psychology, as a requirement for students in the Bachelor of Education and Bachelor of Nursing Science degrees, and as an elective course for students in other concentrations.

The traditional course was team-taught by six instructors, and required each student to attend two 1.5-hour lectures per week (in sections of450 students), with the option of attending one tutorial per week, led by teaching assistants.

In the first version of the blended format each student completed work online and through readings, attended one hour-long lecture per week and participated in one hour-long, facilitated activelearning lab per week, in which they worked in groups of six. Lectures were delivered by a team of four instructors, and group learning labs were facilitated by trained undergraduate and graduate teaching assistants. The second offering of the blended format was structured similarly to the first. In the third version of the blended format only two instructors shared the lecturing and they incorporated a classroom response system to promote engagement in lectures. The fourth offering of the blended format was structured similarly to the third.

Women, Gender, Difference (GNDS 120)

GNDS 120 is a 3.0-unit course serving as the gateway course for concentrations in Gender Studies, and as an elective course for students in other concentrations.

The traditional course required each student to attend one two-hour lecture per week in a single large section taught by the instructor of the course, and one one-hour, tutorial led by teaching assistants (sections of 25 students).

In the first version of the blended format, each student completed work online and through readings, attended one hour-long lecture per week, and participated in one hour-long, facilitated active-learning lab per week. Lectures were delivered by the instructor in a single large section, while group learning labs were facilitated by graduate teaching assistants, with sections of 25 students working in groups of five.

In the second offering of the blended format the only change to the course design was the inclusion of a new textbook, but there was a change of instructor from the first blended version.

Significant design changes were made to the third offering of the blended format, including: (1) replacement of the "Online Debate" activity with a "Collaborative Digital Report" (in which group work outside class produced a digital text for upload on the course website); (2) replacement of a final exam with a series of weekly online quizzes to test knowledge; (3) revision and reorganization of course content.

DISCUSSION

Several features of the Course Redesign Project differentiate it from previous studies and serve as the key insights from this case study: the approach to evaluation, policies concerning structure and support, and the scale of the project's impact.

Evaluation

Although the published literature supports the assertion that blended learning enhances student engagement and improves learning outcomes (Brown, 2016; Means et al., 2010; Zepke & Leach, 2010), evaluating of the Course Redesign Project was considered necessary in order to be accountable to the university community, and to justify the renewal of funding. The data generated from the CLASSE offered a means to evaluate how effectively the project goal was being met, and to suggest areas in which course improvements could be made (Graham et. al., 2013). They also provided instructors with an incentive to become engaged in the Scholarship of Teaching and Learning (Hutchings et al., 2011). The longitudinal nature of the evaluation- with the last traditional version establishing a baseline for comparison, and the repeated annual assessments revealing trends-allowed for greater confidence in the results than a single assessment would have done, clarifying whether the effectiveness of the redesigns was being sustained.

The statistically significant improvements in active learning during class are likely attributable to the deliberate inclusion in all redesigned courses of weekly group work involving structured learning activities. Prior to being developed into blended formats the courses comprised large lectures only (e.g., classics) or included conventional tutorials where the teaching assistant reviewed the week's materials and answered student questions (e.g., psychology and gender studies). As noted by Zepke and Leach (2010), it is important to "create learning that is active, collaborative and fosters learning relationships" (p. 171). Thus, the redesign process focused on the development of small group activities, where students, working in groups of five - eight, were expected to complete work before class, to collaborate during the activity, and occasionally, to assess their peers. Unlike other courses, the mathematics course did not show a statistically significant improvement in active learning between the last traditional and first blended offerings. This is likely because the process of blending took place over a twoyear period: in the first "blended" year the only change was that lectures were videotaped and made available online as a supplement. The following year small group activities were incorporated into the design, coinciding with a significant improvement in scalelet scores.

Teachers are central to student engagement (Kuh et al., 2005), and student-faculty interaction scores appeared to support this. Despite the misgivings expressed by instructors regarding the reduction of face-to-face contact time in the blended format, usually from three to two hours per week, student-faculty interactions were perceived as being better in many courses. Since the CLASSE does not distinguish between faculty member instructors and teaching assistants, nor is it likely that student respondents would have drawn this distinction, it is probable that the interactions between the teaching assistants in the more intimate setting of the group activities would have influenced the results positively. In the case of the gender studies course, student-faculty interaction, and active learning during class scores were not significantly different from the last traditional to first blended offerings, something that may be attributable, in part, to staffing issues. While the other courses were developed and taught by the same instructor in their traditional and blended formats, gender studies course underwent unanticipated changes very late in the process when the instructor who had taught the traditional course and had completed most of the blended development was replaced by an instructor who had never taught the course before and had not previously been involved in the development.

The effectiveness of a course redesign is often measured by improvements in grades (e.g., Vaughan et al., 2011), and there was similar interest in grade changes in the Course Redesign Project. However, any improvement in grades could not be attributed to the course format alone since changes in assessment were also likely to have influenced the grades. For example, more low-stakes assessments into the redesigned psychology course, and started assigning attendance marks for the learning lab component of the blended course. Because results can be heavily conflated with other elements (e.g., different student abilities per year, changes in assessments and formats), changes in grades need to be interpreted with caution and are not always a dependable way to assess the effectiveness of course redesigns, and were, therefore, not included in this paper.

Structure and support policies

According to the framework of Graham et al. and colleagues' (2013) framework, the Course Redesign Project achieved "mature implementation/growth" (p. 7). All of the policies discussed above helped to encourage participation in the Course Redesign Project and to sustain it. One of the more unusual features was a requirement for departments to sign a memorandum of agreement, which laid out the expectations for everyone involved-including the roles of the instructor, the department, and the instructional designer-as well as providing details of intellectual property, compensation, and timelines for development and implementation. The memorandum was particularly useful in the case of the challenges experienced with the gender studies course. Since the agreement included a commitment to the redesigned blended course being offered in that format for at least five years and the blended course had already received curricular approval, the department was motivated to resolve the last-minute staffing issue by identifying a temporary replacement instructor at very short notice, and working to find a longer-term solution.

Curricular policies proved to be particularly influential. For example, the adoption of a catalog system in which courses were described in terms of student learning hours instead of contact time was initially only used for blended courses. Once tested, the system was gradually phased in over three years for all courses in the Faculty of Arts and Science, helping to establish a more progressive and student-centered approach.

Finally, although the focus of this paper is on redesigning courses as blended models, the Course Redesign Project itself had a broader scope. Specifically, each course was required to be redesigned in two ways, as a blended course (intended for on-campus students), and as a fully online course (intended primarily for distance students). By coupling the two initiatives, the Faculty of Arts and Science directed a single investment to achieving two different goals: first, to enhance student engagement in large on-campus classes, and, second, to expand the distance studies program. There are obvious efficiencies gained by pairing these initiatives, in terms of the financial cost of development, academic consistency, and effort: good quality online materials can be used for both the blended and the fully online versions of the course, with face-to-face classroom activities replaced in the fully online course by equivalent online activities designed to achieve the same learning outcomes.

Impact

Because of the focus on large courses in diverse subjects, the impact of the Course Redesign Project has been very high in the Faculty of Arts and Science. Every first-year student in this faculty is in at least one blended course; students in the first-year science cohort- approximately 1,000 of them-are in at least three. The total annual enrolment in blended courses is over 10,000.

Owing to the longevity of the project, the encouraging improvements in student engagement, and the project's high public profile, its impact has also extended beyond the Faculty. Other faculties have adopted similar active learning approaches and have emulated the institutional-level, project-based approach to course redesign. In addition, the university's infrastructure is gradually changing, with classroom spaces being renovated to enable active learning, and enhancements being made to the university's support of educational technology.

CONCLUSION

The case study presented in this paper examined a large-scale, faculty-level course redesign project in which high-enrolment, introductory courses were transformed into blended models to enhance student engagement. Key insights from this case study include the importance of having a comprehensive approach to evaluation and policies concerning structure and support. For example, by adopting a longitudinal approach to evaluation it may be possible to examine patterns over time and clarify whether the effectiveness of course redesigns is sustainable. Furthermore, while instructor turnover can be challenging when attempting to develop and sustain course redesign initiatives, having policies and agreements in place help to motivate academic departments to support such initiatives.

Brenda Ravenscroft is the Dean of the Schulich School of Music of McGill University (Quebec, Canada) and the former Associate Dean of Teaching and Learning in the Faculty of Arts and Science at Queen's University.

Ulemu Luhanga is the Education Researcher for Graduate Medical Education and Assistant Professor of Medicine at Emory University (USA).

References

REFERENCES

Ambrose, S. A., Bridges, M. W., DiPietro, M. Lovett, M. C., Norman, M. K., & Mayer, R. E. (2010). How Learning Works: Seven Research-Based Principles for Smart Teaching. San Francisco: Jossey-Bass.

Brown, M. G. (2016). Blended instructional practice: A review of the empirical literature on instructors' adoption and use of online tools in face-to-face teaching. Internet and Higher Education, 31, 1-10. https://doi.org/10.1016/Mheduc.2016.05.001

Foyle, H.C. (1995). Interactive Learning in the Higher Education Classroom. Washington, DC: National Education Association.

Freeman, S., Eddy, S. L., McDonough, M., Smiths, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111 (23), 8410-8415. https://doi.org/10.1073/pnas.1319030111

Fried, C. B. (2008). In-class laptop use and its effects on student learning. Computers & Education, 50(3), 906-914. https://doi.org/10.1016/i.compedu.2006.09.006

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7(2), 95-105. https://doi.org/10.1016/Uheduc.2004.02.001

Garrison, D. R., & Vaughan, N. D. (2008). Blended Learning in Higher Education: Framework, Principles and Guidelines. San Francisco: Jossey-Bass.

Garrison, D. R., & Vaughan, N. D. (2013). Institutional change and leadership associated with blended learning innovation: Two case studies. Internet and Higher Education 18, 24-28. http://dx.doi.org/10.1016/Uheduc.2012.09.001

Graham, C., Woodfield, W., & Harrison, J. B. (2013). A framework for institutional adoption and implementation of blended learning in higher education. Internet and Higher Education 18, 4-14. https://doi.org/10.1016/Uheduc.2012.09.003

Hutchings, P., Huber, M. T., & Ciccone, A. (2011). The Scholarship of Teaching and Learning Reconsidered: Institutional Integration and Impact. San Francisco: Jossey-Bass.

Kuh, G. D., Kinzie, J., Schuh, J. H., Whitt, E. J., & Associates (2005). Student Success in College: Creating Conditions that Matter. San Francisco: Jossey-Bass.

Leger, A., Godlewska, A., Adjei, J., Schaefli, L., Whetstone, S. Finlay, J., Roy, R., & Massey, J. (2013). Large First-Year Course Re-design to Promote Student Engagement and Student Learning. Toronto: Higher Education Quality Council of Ontario.

Mayer, R. E. (2008). Learning and Instruction. Upper Saddle River: Pearson/Merrill Prentice-Hall.

Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of Evidence-Based Practices in Online Learning: A Meta-analysis and Review ofOnline Learning. Washington, DC: U.S. Department of Education.

Niemiec, M. & Otte, G. (2009). An administrator's guide to the whys and hows of blended learning. Journal of Asynchronous Learning Networks, 13(1), 19-30. Retrieved from https://www.learntechlib.org/p/104025/

Ouimet, J. A., & Smallwood, R.A. (2005). CLASSE-The class-level survey of student engagement. Assessment Update, 17(6), 13-16. https://doi.org/10.1002/au.176

Pascarella, E. T., & Terenzini, P. T. (1991). How College Affects Students: Findings and Insights from Twenty Years of Research. San Francisco: Jossey-Bass.

Picciano, A. G. 2006. Blended learning: Implications for growth and access. Journal of Asynchronous Learning Networks, 10(3), 95-102. Retrieved from https://www.anthonvpicciano.com/articles.html

Pike, G. R. (2006). The convergent and discriminant validity of NSSE scalelet scores. Journal of College Student Development, 47(5), 550-563. Retrieved from http://muse.ihu.edu/article/201904

Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering EducationWashington, 93(3), 223-232. https://doi.org/10.1002/i.2168-9830.2004.tb00809.x

Ragan, E. D., Jennings, S. R., Massey, J. D., & Doolittle, P. E. (2014). Unregulated use of laptops over time in large lecture classes. Computers & Education, 78, 78-86. https://doi.org/10.1016/i.compedu.2014.05.002

Twigg, C. A. (2000). Course-readiness criteria: Identifying targets of opportunity for large-scale redesign. Educause Review, 35(3), 41-49. Retrieved from

https://er.educause.edu/~/media/files/articles/2000/5/erm0033.pdf?la=en Twigg, C. (2004). Improving Learning and Reducing Costs: Lessons Learned from Round Ill of the Pew Grant Program in Course Redesign. Troy: National Center for Academic Transformation. Retrieved from http://www.thencat.org/PCR/RdIIILessons.pdf

Vaughan, N. D. (2007). Perspectives on blended learning in higher education. International Journal on Elearning, 6(1), 81-94. Retrieved from https://www.learntechlib.org/primarv/p/6310/

Vaughan, N. D. (2010). A blended community of inquiry approach: Linking student engagement and course redesign. Internet and Higher Education, 73(1-2), 60-65. Retrieved from https://www.learntechlib.org/p/108367/

Vaughan, N. D., Zimmer, J., and Villamar, F. (2011). Student engagement and interactive technologies: What's the connection? International Journal of Excellence in E-Learning, 4(1). Retrieved from https://iournals.hbmsu.ac.ae/Pages/Articles.aspx?AlD=198&11D=46

Wallace, L., & Young, J. (2010). Implementing blended learning: Policy implications for universities. Online Journal of Distance Learning Administration, 73(4). Retrieved from https://www.learntechlib.org/p/52603/

Yin, R. K. (2009). Case Study Research: Design and Methods. Thousand Oaks: Sage.

Zepke, N., & Leach, L. (2010). Improving student engagement: Ten proposals for action. Active Learning in Higher Education, 7 7(3), 167-177. https://doi.org/10.1177%2F1469787410379680.