Effectiveness of Using Online Discussion Forum for Case Study Analysis

Abstract

Business schools are exploring new pedagogical approaches to learning in order to deal with challenges such as increased class sizes, limited funding support, and difficulties in facilitating and encouraging active participation and learning among a diverse cohort of students. This paper reports on a study of the effectiveness of a pedagogical approach that blends online discussion board and case study. Analysing quantity and quality of online postings and comparing accounting students' performance with previous cohort, this study observes a significant improvement in student learning. Appropriate design and delivery strategies and clear assessment criteria for assessment and use have provided an effective learning vehicle for students, helped them overcome their own language related barriers, and encouraged them to participate in a nonthreatening environment. This approach further complemented the benefits of peer-to-peer learning and case study pedagogy. Reported increase in workload for students and marking load for academics and measuring the value of learning, however, are some of the challenges that need further attention by researchers.

Academic Editor:Gwo-Jen Hwang

Discipline of Accounting, The University of Sydney Business School, Sydney, NSW 2006, Australia

Received 22 January 2014; Revised 6 May 2014; Accepted 9 May 2014; 26 May 2014

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

The benefits of using online discussion forums and peer to peer learning for enhancing student learning are well known. Other than in full online courses, their adoption in traditional learning environments complementing face-to-face teaching is increasing [1] and has now become a common educational strategy in higher education [2]. In fact, interest in using online discussion forum as an indicator of students' performance is increasing [3]. Online discussion forums are expected to enable flexible and independent learning and knowledge construction [4] and develop critical thinking skills [5]. There are, however, some problems such as lack of learners' focus and inadequate reflection [6], students not responding to the ideas of others, and surface level discussions [7]. Though there are many advantages of its use, students use of the online discussion forums in the context of their study [8] and the characteristics of an effective online discussion environment that facilitate effective learning need to be investigated [1].

Further, difficulties of devising an online assessment and incorporating the benefits of case study pedagogy into this online forum have posed further challenges to educators in the design and effective management of these activities [9]. Driven by the increasingly competitive higher educational environments, higher expectations of students, cultural diversity of the student population, and limited availability of resources, business schools are increasingly exploring new pedagogical approaches to learning [10]. This paper discusses the context, approach, and effectiveness of one such pedagogical approach in an Australian business school. It blends online discussion forum and case study analysis in teaching a postgraduate elective unit to accounting and other business students. The objective is to combine the benefits of case study method of teaching with online discussion forum to enhance the quality of learning while making this an assessment component in order to ensure active students' participation. This paper will first briefly review the literature on the strengths and weaknesses and the pedagogical use of online discussion forums in higher education context. It will then explain the approach and methodology adopted in designing and implementing a case study based assessment component that incorporates online discussion forum. It will then present an analysis and discuss the findings and conclusions.

2. Literature Review

Increasing class sizes, reducing resources, and widening diversity of students' cohorts have all placed demands on higher education and stimulated them to explore new pedagogies [11, 12]. Identifying the most successful use of technology tools available in higher education and deploying them effectively in the classroom in order to improve student learning are now an important research issue facing accounting education [13]. In that context, using an asynchronous medium for learning has now become a significant component of course delivery in higher education [14]. Higher educational institutions have started adopting hybrid or blended models of education that combines classroom and asynchronous and distributed learning environments [15]. Driven by increasingly competitive higher educational environment [16], these hybrid models have resulted in the convergence of the traditional classroom and online learning modes [17] combining the strengths of face-to-face and online learning modes of learning and have become one of the key responses to changing higher educational climate [10].

Online discussion forums are considered an extension of traditional learning that promotes dialogue, reflection, knowledge construction, and self-assessment [18]. Given their potential benefits, online discussion forums have been widely adopted as tools for online learning [13, 19]. Online discussions have the potential to improve students' critical thinking and problem solving skills, decision-making ability, and written communication skills and contribute to students' ability to organize and analyse information [20]. Combining the online discussion forum with a case study method, a common pedagogical technique used in many business schools [21], is considered to be an effective method that encourages students to relate theory to practice [22]. Many academics therefore have started using online discussion forums for teaching via the case method [23]. While learning goals of online case discussions and face-to-face case discussions are generally the same, there is relatively little research focusing on the online case-method pedagogy and assessment in business and accounting education [9]. Effects of these hybrid models of learning that combine traditional classroom case study pedagogy with the asynchronous media such as online discussion forums on learning outcomes and processes are, however, not well understood [24]. With the widespread adoption of course management systems (CMS) and learning management systems (LMS) by the academic institutions being a recent phenomenon [25], further studies to offer insights into the current and future directions of this field are necessary [26].

Two studies have examined how students perceive online case method pedagogy in business education. According to study by Jonassen and Kwon [27], perceived students' satisfaction and perceived quality of the discussion process and outcomes were higher for members of the online group when compared with face-to-face groups [27]. They observed that online groups went through more cycles of the ideal problem solving process which involves defining the problem, orienting the discussion, and developing a solution. Their study found that online groups produced less amount of verbal interaction than their face-to-face counterparts [27]. In another study, Webb et al. [28] compared groups working with difference "doses" of online component in case method instruction from purely online to purely face-to-face. According to them, members that were under heavy online conditions have more positive perceptions of peer interaction during the discussion than others, and they have outperformed the face-to-face students in terms of conceptual and factual knowledge about the case discussed [28]. These findings must be treated with caution because of the differences in the cases and topics taught to these different groups. Further, it is possible that the newly developed online skills rather than increasing dose of online discussion may have contributed to these positive findings.

In addition to the above, other studies have reported challenges of online learning. For example, Valaitis et al. [29] reported confusion about the navigation of online environment by nursing students that has overshadowed the learning [29], while Concannon et al. [30] observed tendencies of accounting students to overlook critical aspects of the online environment as they are not particularly fond of computer technologies [30]. Zhu's [31] analysis of topical discussions in several undergraduate, graduate, and professional courses found that only 15-25% of the content was judged as cognitively deep [31]. Similarly, in an analysis of education students' discussion of authentic cases, Angeli et al. [32] found that only 9% of contents posted were justified (theoretically based) claims or opinions, while 49% were unsupported or personal opinions questioning the cognitive engagement of students [32]. While Garrison and Vaughan [33] and de Wever et al. [34] found low levels of critical thinking [33, 34], Hara et al. [7] observed high levels of critical thinking in the online discussions [7]. Lambiase [35], for example, observed topic decay with the percentage of posts on the topic decreasing steadily from 65% to 33% during the first nine days of discussion. In general, online discussions in higher education learning environment are feasible and viewed positively by students and the process and outcomes are as good as its face-to-face version [36]. While some studies examined educational applications of online technologies in broader terms [36, 37], others have suggested best practices and strategies [9].

Threaded forums are one of the most commonly used environments for online discussion, and, though well supported by learning theories, several limitations are identified by researchers. Excessive focus on new posts, overlapping exchanges and hierarchical structure of discussion threads [38], and lack of timely feedback [39] are some of the problems reducing the opportunities for meaningful reflection, social interaction, and knowledge construction [1]. Researchers have suggested improvements to the guidelines and instructions to students [40], developing online activities to engage learners [41], adopting teaching or moderating strategies, [42] and enhancing the design of environments [1] to encourage good discussion.

Even though learning through online discussion forums was studied by several researchers as shown above, research that focuses on the online case-method pedagogy and assessment is limited. Further, findings on the effectiveness of online discussion forum are also inconsistent and depend on the nature of discussion environments. The experimental, quasiexperimental, and descriptive studies relevant to this question have not produced coherent knowledge about how and when online discussion groups perform better or worse [28]. As the review suggests, findings on the effectiveness of online discussion forum are mixed with some pointing to the difficulty of facilitating students' critical thinking and cognitive engagement. Online discussion forum is just another technology enabled tool and the principles of technology acceptance, usability, ease of use, and so forth, will be equally applicable here. The availability of tool does not necessarily mean that students (users) will use it or perceive it as useful, and the learning effectiveness is influenced by other external factors such as incentives to participate, perceived value of those tools, moderator's skills, quality of the discussion environment, and past experience of the students in using those tools. Factors such as structure of the online environment, prior ability of students, facilitator's style, and participation, however, may have some influence on the quality of the discussion and the outcomes. Though best practices and strategies [9] and effective design environments [1] were suggested, there were no studies that measured the effectiveness of the design and delivery of online case discussion and assessment in a business school context.

Lack of participation by a majority and dominance of a few handful students are the main challenge in asynchronous discussion forums. As assessment is the currency that students deal in [43], grading can be a workable strategy that will encourage participation. Past online experience (or lack of it) may not be a factor any more given that the current generation of students has a lot of online and social media experience. Therefore, it is possible to encourage their participation and thereby the learning by grading students for their contribution. The consequent fear of losing marks may constrain their interaction to some extent; grading may actually encourage students to participate in the discussion with due consideration and adequate preparation. By opening up too much, for example, student may be afraid of exposing their lack of knowledge that could count against them. An effective learning environment and assessment must be carefully designed without unwittingly discouraging the motivation for learning [44].

Even though collaborative learning is critically important, and that the technology is a powerful enabler that match what is needed for discussion and collaboration, the extent to which asynchronous discussion forums succeeded in enabling learning is not known and lacks sufficient empirical evidence where a case study assessment is incorporated. Rather than comparing face-to-face and online groups, unlike previous studies, this study directly investigates online groups engaged in case based discussions. The findings of this study will contribute to the literature on case study pedagogy in online context, an area where existing research findings are limited [28]. It addresses the impact of a blended model that combines the strengths of online discussion forum and case study method in a business school. It incorporates effective design principles advocated by Gao et al. [1], strategies suggested by Rollag [9], and a significant assessment weightage to encourage active and quality participation. The next section gives a background to this particular study and explains the methodology and approach.

3. Study Approach

3.1. Background to the Study

An online discussion of case study is designed as an important assessment component and incorporated in this unit using some of the strategies suggested by Rollag [9] and the recommendations by Gao et al. [1] for improving the effectiveness of online discussion. Case study pedagogy is an established method employed in many postgraduate classes in this university business school. An assessment is designed blending the online discussion forum and case study pedagogy. Recognizing the well-documented benefits of asynchronous media and taking advantage of the support and resources made available in the school, this blended model was introduced in a postgraduate unit. The postgraduate unit, titled "business process integration," aims to demonstrate the benefits of business process perspective and information, process, and technology integration to accounting and other students in business school. The focus in this unit is to develop a thorough understanding of accounting and other related business processes, their links with each other, integration, internal controls, information flows, and the role of integrated information systems technologies such as enterprise systems and business intelligence tools. Building on the basic knowledge of accounting and other business processes, information flows, and functional understanding (such as marketing and accounting) learnt in other units and drawing knowledge from sales/marketing, accounting, and logistics disciplines based subjects studied earlier, this unit aims to develop a multidisciplinary process-centric view of the business to accounting students. Importantly, it is designed to help accounting students understand the interdependencies and process and information flows between accounting and other business functions.

The conceptual content that is covered in this subject includes accounting processes including accounts receivable, accounts payable, expenditure cycles, and costing processes and other business processes including procurement and sales. In addition, it also includes making a business case for the selection and implementation of a packaged business software, that is, enterprise system. The unit incorporates variety of teaching methods that include lectures, case study discussions, laboratory demonstrations of the SAP software, and hands-on exercises on the SAP software and emphasizes both conceptual and theoretical content as well as procedural knowledge of the software.

3.2. Assessment Task and Criteria

This online case study discussion is one of the key assessment components and weighs 25% of the total assessment. It requires students to answer the questions that deal with the challenges in the selection, evaluation, and implementation of a packaged business enterprise system software and challenges in postimplementation. The objective was to help students demonstrate their ability to work collaboratively using online discussion forum moderated by the lecturer and resolve their differences in developing an optimal solution using synergies generated through collective wisdom and peer-to-peer learning. It requires students to participate in an online discussion forum on a weekly basis in a semester that spans 13 weeks. During this time, they are expected to demonstrate their individual ability to apply the concepts and theories of enterprise systems to a business organization and suggest workable solutions to the problems/issues raised in a comprehensive case study.

At the beginning of the semester, a sheet detailing the expectations of students and the assessment criteria was given to students. This assessment criterion was designed considering the learning objectives of this assessment component, practicality of administering the grading, and key principles of assessment design. The dimensions and their descriptors for grading purpose were developed from several sources such as Oxford Brookes University's business assessment criteria grid, Washington State University's guide to rating integrative and critical thinking, and this university business school graduate attributes. The criteria thus developed consist of five dimensions--(i) identification of issues and problems, (ii) consideration of context and application of theories and principles, (iii) analysis of data and evidence in the case study, (iv) effective written communication, and (v) responsive contribution and integration.

Students were given a grid that explains each of the dimensions in a scale of 1 to 6 (1 to 2 = Poor; 3 to 4 = Average; and 5 to 6 = Excellent) with some descriptors. For example, the dimension of "responsive contribution and integration" is described as "Poor" when the student "simply mentions other contributions but neither explains the relevance nor adds to it " and/or when there is "no evidence of integrating others' views and perspectives. " Similarly, it is termed as average when student "makes references to earlier works that are starting point for new ideas, but not much information is incorporated " and/or makes "rough comparison and integration of multiple viewpoints ." A response is rated "Excellent" when the student "links ideas posted by others to their own, responds to others' contribution by elaboration, critique, demonstration of linkages among earlier contributions, and/or utilization of an earlier contribution as a foundation to build his/her own, " and/or if there is "evidence of integration from a variety of sources and timeliness of posting ." Thus, the objective of this assessment grid was to provide students some information about the expectations and requirements in providing their responses and to encourage them to take advantage of the benefits of collaborative learning and case study pedagogy. As the focus is on the quality, the assessment grid does not contain any information about the number of postings.

Four questions were set on the case study. In order to narrow down their focus on one aspect at a time and to concentrate on the issue at hand, these questions were posted one after another in the online blackboard so that students do not have any idea of the questions and specific details that would appear subsequently. In addition, the objective was to synchronize the questions with the lecture materials and to manage the flow of the discussion. Students, however, can see all the responses to the earlier questions and see how it is building up concepts. Facilitator was monitoring the discussion and, depending upon the need, provided clarifications, asked leading questions, corrected major deviations, and kept the discussion focused on the main issues.

The first question seeks students' analysis on the challenges the case study firm faced in the selection and implementation of an enterprise system (ES) software solution and evaluation of implementation strategy and software-business fit. The second question deals with the evaluation of the firm's attempt to change business processes, problems it faced, and its relationship with the adoption of enterprise system and the strategies employed. The third question requires students to analyse the current organizational structure of the firm and develop appropriate mapping structure to be able to configure the enterprise system for the necessary processes and the fourth question involves analysis of the suitability and challenges in selecting and implementing extensions to the enterprise system.

Each student is expected to post up to a maximum of 4 responses for each question. Even though this limit is set, the number of postings per question is not a criterion in the assessment and focus is on the quality of postings. Opportunity to post four times is expected to encourage students to reflect on their responses, read others' responses, and build on them before posting the next one. In addition, they were encouraged to read widely outside the text book and lecture notes and provide references to their responses in the forum. Students were expected to post their responses within two weeks in the relevant thread created in the blackboard discussion forum for the first three questions and one week for the fourth question. Once the deadline has passed for a particular question, students were not allowed to edit their responses and/or submit additional response. They, however, are allowed to see the responses already posted by all the students and use that knowledge while answering the next question.

3.3. Administration and Timeline

A time line of the online discussion of case study is presented in Figure 1.

Figure 1: Timeline for the online case study discussion.

[figure omitted; refer to PDF]

3.4. Data Collection and Analysis

Students' responses for each of the questions were the primary data used in this study. This data was analysed by two persons independent of each other--the author/facilitator and a research assistant. This research assistant was a qualified educator, specialized in online learning, and was working in the teaching and learning unit of the business school. The objective of this analysis was to assess the quality of responses and online discussion interaction. Rather than positioning the discussion interaction as a dependent variable along with the learning outcomes, a post hoc measurement approach was used in this study [45, 46]. According to this, the discussion interaction and quality of the responses were considered independent variables in relation to learning outcomes. Barron [46] suggests two alternative ways of conducting content analysis. The first approach involves qualitative analysis of the discussion and the second approach focuses on the responses related to proposed solutions to the case study questions. This second approach was adopted in this study which involves qualitative analysis of the student responses.

Quality of responses by each of the students for each question was analysed for assessment purposes as well as for evidence to support learning. While the first approach helped in awarding assessment mark to each of the responses, the second approach had helped in identifying any evidence to peer-to-peer and collaborative learning, data analysis skills, using and citing other sources, pooling of different ideas, and building on others' contributions. A comparison of the analysis revealed 81% consistency in the rating of both evaluators (i.e., 155 responses, out of 192 responses, 4 questions * 48 students = 192 responses). The remaining 19% of the responses were subsequently reevaluated jointly in order to arrive at a consensus assessment mark and evidence.

In addition to students' responses, their perception of the entire process was also collected using semistructured interviews. The approach taken in the analysis was to analyse the responses of students to semistructured questions, to examine their feedback on the effectiveness of this online assessment task, and to assess its pedagogical benefits. Further in the interviews, participants were asked to give their views on the structure of the learning environment including the online discussion forum, appropriateness of the weightage assigned to this assessment task, timing and level of difficulty of the case study questions, and participation by the academic as a moderator and their perception of collaborative and peer-to-peer learning. From a total sample of 48 students, 12 students have volunteered to be interviewed and provided such qualitative feedback.

3.5. Demographics

Demographics of the 48 respondents participated in this study are presented in Table 1. As noted in the table, 68% of the participants were international students from countries such as Hong Kong, Singapore, Thailand, Indonesia, China, India, Germany, USA, and the Netherlands. This is an elective unit and 61% of the students enrolled in this unit are majoring in accounting, 35% majoring in information technology/systems discipline, and the remaining 4% majoring in the logistics discipline. On the employment indicator, data revealed that 38% of the students were currently employed while 42% of the students have some previous work experience. With more than 65% of the students below the age of 25 years, age was not considered a major differentiating variable.

Table 1: Demographics of respondents.

	Characteristic of respondents	Percentage
1	Proportion of international students	68%
2	Students from accounting discipline	61%
2	Students from other disciplines such as information technology, business information systems, and logistics	39%
3	Percentage of students currently employed	38%
4	Percentage of students below 25 years of age	65%
5	Percentage of students with previous work experience	42%
6	Percentage of students who have completed more than 4 business related units/subjects	53%
7	Percentage of students who have completed at least one IT/IS related subjects	88%

Previous knowledge of business functions, business processes, information systems, and technologies gained in other units is expected to determine the prior content knowledge student may have and therefore could have an influence on their learning experience in this unit. Therefore, data on the completion of previous units before enrolling in this unit was collected and presented in Table 1. With 53% of students completing more than 4 business related subjects and almost 90% completing at least one IT/IS related subject, it is fair to assume good level of understanding of core business functions and processes and information technology components among the cohort of students. An analysis of this qualitative data, findings, and anecdotal evidence collected in this study are presented in the next section.

4. Analysis and Findings

4.1. Quantity of Responses

Each participant is expected to submit responses to four questions posted in the online discussion forum. There were 268 valid responses for all the four questions, that is, on average, 67 responses per question or 1.40 responses per question per student. As shown in Table 2, the number of responses per student has gradually increased from 1.1 per student for question 1 to 1.63 per student for question 4 and demonstrates gradual increase in students' interest and intensity of their participation in online discussion. Even though increase in the number of submissions by itself may not be a good indicator of the quality of contributions and/or learning, it certainly indicates increase in interest in the global topic signalling increased focus of discussion on the topic.

Table 2: Number of responses posted by student.

Number of responses posted	Question 1	Question 2	Question 3	Question 4
No.	%	No.	%	No.	%	No.	%
Zero	1	2%	0	0%	0	0%	0	0%
One	42	88%	36	71%	28	60%	40	83%
Two	4	8%	8	17%	13	25%	8	17%
Three	1	2%	3	6%	7	11%	0	0%
Four	0	0%	1	2%	1	2%	0	0%

Except for the fourth question, all the remaining questions are content based and deal with subject specific issues such as enterprise systems implementation challenges, accounting related business processes, challenges of integration, and critical success factors. The fourth question also deals with the adoption of enterprise system and requires students to develop an artefact, a mapping structure for implementing accounting processes (accounts receivable, accounts payable, and general ledger). Even though each student is allowed to post up to four responses to each of the questions, very few posted more than two responses as shown in Table 2. Though this proportion has gone up from the first question to the third one (2% to 13%), a majority of students simply posted one response. Even though they are encouraged to read other responses and submit additional postings, very few have done this with 88%, 71%, 60%, and 83% posting just one response for questions 1, 2, 3, and 4, respectively. This observation confirms previous research reviewed by Rourke and Kanuka [48] where the participants primarily focused heavily on exploration and less on integration of ideas for solutions.

4.2. Timing of Responses

In terms of timing, about 50% of the respondents posted their response on the due date, while about 25% of students posted in the first two days, as shown in Table 3. Most of the students who have posted in the first two days have posted second response. The second response suggests an improvement of student's understanding probably based on the responses from other students and/or self-directed learning by student.

Table 3: Timing of responses posted by students.

Timing of responses posted by students	Q1	Q2	Q3	Q4
First response posted in the first two days after question is made available	20%	28%	37%	39%
First response posted on the due date	55%	40%	32%	32%
Response posted between day 3 and the last day	25%	32%	31%	30%
Total responses	100%	100%	100%	100%

Analysis of the responses suggests that students were hesitant to submit their comments in the initial stages and are generally waiting for a leader to post first. About 20% of the respondents submitted their first response in the first two days while about 55% of the students submitted their response on the last day for the first question. This, however, has improved as students become familiar with the process and started probably realizing the value of collaborative learning. For the third and fourth questions, number of students who submitted their response on the first day increased from 20% to 37%, while the percentage of students who submitted their response to the fourth question on the last day has improved from 55% to 32%. Thus, there is clear improvement in the frequency of participation and the intensity of participation in the online discussions. Importantly, there is no topic decay as observed in previous studies [35]. Even though it is not possible to conclude from this analysis that there is a collaborative peer-to-peer learning that took place, further analysis of the qualitative responses from students supported this observation.

4.3. Quality of Responses

Based on the criteria set out for assessment purposes, the quality of written responses was analysed and a summary is presented in Table 4. One of the objectives of online case discussion is to facilitate exchange of ideas, views, and information among students and peer-to-peer learning and develop the solutions using the collective wisdom. It is therefore important to check whether there is any evidence that students are reading and understanding each other's ideas/contributions. It is necessary to analyse whether students are building on each other's contribution by elaborating, criticizing, and/or integrating.

Table 4: Summary of analysis.

Content analysis	Q1	Q2	Q3	Q4
No evidence that the student read/understood others' ideas/contributions	56%	44%	54%	34%
Responding to others contribution by elaboration/critique or building-up	15%	21%	31%	36%
Evidence of good analysis of the case study data	32%	41%	39%	44%
Evidence of using and citing other sources/references in the response	17%	18%	21%	24%
Demonstrable understanding of the questions and identification of issues in case study	21%	27%	33%	39%
Effective written communication	64%	72%	56%	71%
Evidence of pooling different ideas, expertise, and information in the development of an artifact (related to question 3 only)	NA	NA	45%	NA

Analysis of the responses as shown in Table 4 revealed that about 56% of the students did not show any evidence of that for the first question. This has gradually improved and for the final question about 66% of the students have demonstrated this characteristic in their response. Similarly, while only 15% of the students have responded to others' contribution and developed a solution for the first question, about 36% demonstrated such skills for the fourth question. From the data, it appears that students are reading each other's views and responses and adding (though in a limited way) through collaborative learning. Because of the clear instructions at the beginning of the case study discussion, none of the responses is simple and irrelevant such as "I agree," "I disagree," or similar responses and every response is consistent with the criterion. Thus, there are no real and effective integration and synthesis of ideas observed in the study; there is some evidence of collaborative and peer-to-peer learning in online environment. On the effectiveness of the case study pedagogy, analysis revealed that there is good evidence of improvement from the first question to the last question (from 32% to 44%) on different aspects as shown in Table 3.

4.4. Improvement in Learning through Online Discussion

It is necessary to measure the impact of this initiative on learning. Academic performance, though, is considered one of the variables that could measure the effectiveness of learning; it is not possible to conclusively prove this without a control group. Though not the most appropriate method, grading of the current cohort of students in this assessment task (i.e., case study) is compared with the performance of a previous cohort of students in the same unit with the same case study. In the past, students were asked to submit a written analysis of the case study after face-to-face discussion in the classroom. With the characteristics of the cohort of students considered generally similar every semester, a comparison is made. Simple comparison reveals improvement in the performance of students in this assessment component. Table 5 gives an indication of their academic performance for each of the questions in the case study and a comparison with the previous cohort.

Table 5: Performance in the assessment task.

Performance in the assessment task	Current cohort of students (with online discussion forum)	Old cohort*
Q1	Q2	Q3	Q4	Overall
Average and below average performance	60%	46%	40%	38%	46%	77%
Credit performance	34%	40%	38%	36%	37%	16%
Distinction performance	6%	14%	22%	26%	17%	7%
Total	10	100	100	100	100	100

^* This cohort of students did not have online discussion forum for case study analysis and assessment.

Grading data shown in Table 5 reveals improvement from the first question to the fourth question. As pointed out earlier, in addition to the improvement in quantity and quality of responses, it is fair to conclude that there is some improvement in cognitive knowledge (measured by academic performance). While this may not be enough to confirm that the learning has improved, it is fair to suggest that the asynchronous media has contributed to learning. As pointed out by Romero et al. [3], it could potentially be used as a predictor for final performance. This case study assessment is a formative assessment and therefore could be used as a feedback to the lecturer as well as to students for identifying knowledge gaps and to improve teaching and learning.

4.5. Improvement in Overall Learning

In order to determine whether this model of learning (case study analysis through online discussion forum) has contributed to improvement in overall learning or not, a comparison was made. Given the absence of control group, a simple comparison with the previous cohorts of students where no such asynchronous medium was used is made, fully acknowledging that this is not the appropriate method. Except for this assessment component, all the remaining aspects of the unit were exactly similar to the current one in terms of the learning outcomes, content, delivery methods, and general assessment components. For the previous cohort of students, a case study analysis was used as an assessment component, while the same case study analysis is used to the current cohort of students using asynchronous medium for discussion.

Using independent samples t -test at 5% significance level, the actual performance of the students in this assessment task as well as the teaching evaluation is compared between these two groups of respondents. This comparison reveals improvement in the overall performance of students in the assessment task as well as in their feedback on the unit through student evaluations of unit at the end of the semester. It is therefore safe to conclude that the asynchronous medium used for case study analysis has not only improved student learning in terms of achieving the learning outcomes but also contributed to the effectiveness of teaching and learning measured in terms of the improved unit evaluations. A summary of the t -test results is presented in Table 6.

Table 6: t -test results: differences between two cohorts of students.

Variable (posttest)	Group	N	Mean	S.D.	t value	d
Performance in the case study task	Online assessment (experimental group)	48	65.94	10.193	5.065*	0.000
Normal assessment (control group)	42	56.43	7.096
Overall teaching evaluation score	Online assessment (experimental group)	48	4.50	0.505	3.698*	0.000
Normal assessment (control group)	42	4.10	0.532

^* P < 0.001 .

Subject evaluation score in this unit has marginally improved from 4.1 to 4.52 in a scale of 1 to 5. The curriculum and delivery of this unit are quite different from other standard units delivered by this school. This unit incorporates SAP, an enterprise system software solution, and teaches students the concepts of business process integration and enterprise systems along with SAP software skills. The curriculum incorporates hands-on laboratory sessions and these hands-on components typically made this unit interesting and contributed to good evaluations, that is, 4.10, before introducing this online discussion assessment. The introduction of online discussion forum has further contributed to improvement in learning as well as in students' satisfaction. In qualitative feedback, a significant number of students (62% of them) pointed out online discussion forum as an important learning component. Informal feedback and comments from students also reflect a generally positive endorsement to the online discussion. They have observed that the exchange of views and ability to asynchronously post their responses online have encouraged them to interact more and better and facilitated their learning. They have acknowledged the benefits of peer-to-peer learning which they believe would not have happened without this online discussion. This online discussion has also helped them better in deriving the benefits of case study pedagogy. Thus, significant improvement in the quality of responses and observed improvement in students' academic performance on this particular assessment task indicate the contribution of this strategy to deep learning, given the structure of the assessment and content.

4.6. Challenges

In spite of positive endorsement and observed improvement in the learning, some challenges are identified. They include reported increase in students' workload because of the online discussion forum, inability to freely express their ideas and views because of the assessment involved, restriction to the number of postings per student per question, inadequate number of responses by some students, poor integration and synthesis of others' ideas by students, insufficient moderation by the lecturer, and some subject related issues. About 85% of the participants believed that the weightage given to the assessment task was appropriate and motivated them to actively engage in the discussion and learning. Some students, however, viewed the weight given to this single assessment component (25% of the overall assessment weight) as a major constraint in freeing up the discussion. Further, academic administration of this online discussion forum has been a challenge. With each student posting on average 1.4 postings per question, the total number of postings has reached about 270 and enormously increased the marking load to the academic. In addition, the need to monitor the discussion forum, edit some unhelpful comments, redirect the debate to the issues relating to the subject content, and provide hints has added to the normal teaching and administrative load. In addition to this, some of the typical challenges in case study pedagogy have also resurfaced during the discussion. They include request for one single answer to the case study questions, inability to comprehend the case study materials by some, inadequate English language skills of students, inability to apply theoretical frameworks, and skills in scoping out, assumptions, and analysis.

5. Conclusions

Teaching in emerging disciplines such as business process management and enterprise systems in business schools is challenging because of the dynamic nature of the subject content, diverse nature of student cohorts, nonavailability of localized case studies, required multidisciplinary focus, and other institutional factors such as large class sizes and increased competition for scarce resources and students. Integrating e-learning technologies such as discussion forums with the traditional case method pedagogy has the potential to change educational processes and enhance the quality of learning in this environment. Combining online discussion forum and case study analysis offered students a high-quality learning environment. According to study, this blended model encouraged collaborative learning and contributed to improvement in cognitive learning. By transferring discussions to an online environment, opportunities for learning from each other have improved. Data suggests that students perform better in an environment where a mixture of classroom and online technologies is employed and there is no decay in the interest on the topic. In general, students perceived improvement in learning because of the online environment and appreciated the opportunities it has provided for deeper understanding of content based issues, appreciation of multiple views, and reflection of their own learning and perceptions. Though it is early days in understanding the effectiveness of these blended learning models, their contribution to the improvements in the quality of discussion and the depth of learning in a higher educational environment appear to be significant and can be further expanded by imparting better argumentation and integrative skills to students. The workload involved in setting up, monitoring, and assessing the online discussions, though, is significant; it is possible to make it less burdensome by efficient design and structure of the initiative using guidelines well established now and with appropriate faculty workload policies. Further studies on the influence of students' knowledge and skills, development of argumentation skills for students and moderators' skills, and lecturers and students' actual use of synchronous and asynchronous communications, interactions, and reflections would help in designing a more effective online discussion environment and assessment strategies in business schools.

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.

References

[1] F. Gao, T. Zhang, T. Franklin, "Designing asynchronous online discussion environments: recent progress and possible future directions," The British Journal of Educational Technology , vol. 44, no. 3, pp. 469-483, 2013.

[2] J. W. Gikandi, D. Morrow, N. E. Davis, "Online formative assessment in higher education: a review of the literature," Computers & Education , vol. 57, no. 4, pp. 2333-2351, 2011.

[3] C. Romero, M.-I. López, J.-M. Luna, S. Ventura, "Predicting students' final performance from participation in on-line discussion forums," Computers & Education , vol. 68, pp. 458-472, 2013.

[4] O. Noroozi, A. Weinberger, H. J. A. Biemans, M. Mulder, M. Chizari, "Facilitating argumentative knowledge construction through a transactive discussion script in CSCL," Computers & Education , vol. 61, no. 1, pp. 59-76, 2013.

[5] J. Zhang, M. Scardamalia, M. Lamon, R. Messina, R. Reeve, "Socio-cognitive dynamics of knowledge building in the work of 9- and 10-year-olds," Educational Technology Research and Development , vol. 55, no. 2, pp. 117-145, 2007.

[6] N.-S. Chen, Kinshuk, C.-W. Wei, C.-C. Liu, "Effects of matching teaching strategy to thinking style on learner's quality of reflection in an online learning environment," Computers & Education , vol. 56, no. 1, pp. 53-64, 2011.

[7] N. Hara, C. J. Bonk, C. Angeli, "Content analysis of online discussion in an applied educational psychology course," Instructional Science , vol. 28, no. 2, pp. 115-152, 2000.

[8] D. DeWitt, N. Alias, S. Siraj, A. R. Zakaria, "Interactions in online forums: a case study among first-year undergraduate students," Frontiers in Education , vol. 2, no. 1, pp. 6-13, 2014.

[9] K. Rollag, "Teaching business cases online through discussion boards: strategies and best practices," Journal of Management Education , vol. 34, no. 4, pp. 499-526, 2010.

[10] N. P. Napier, S. Smith, "Assessing blended learning: student outcomes and perceptions," in Proceedings of the 15th Americas Conference on Information Systems (AMCIS '09), San Francisco, Calif, USA, August 2009.

[11] D. Boud, N. Solomon, C. Symes, C. Smes, D. Solomon, "Learning for real: work-based education in universities," Work-Based Learning: A New Higher Education? , Open University Press, Bukingham, UK, 2000.

[12] R. O'Leary, "Online communication using discussion boards," http://www.economicsnetwork.ac.uk/handbook/discussionboards. The Handbook for Economics Lecturers , The Economics Network, University of Bristol, 2005.

[13] B. Apostolou, J. M. Hassell, J. E. Rebele, S. F. Watson, "Accounting education literature review (2006-2009)," Journal of Accounting Education , vol. 28, no. 3-4, pp. 145-197, 2010.

[14] C. A. Dykman, C. K. Davis, "Part one: the shift toward online education," Journal of Information Systems Education , vol. 19, no. 1, pp. 11-16, 2008.

[15] P. Cookson, "The hybridization of higher education: cross-national perspectives," International Review of Research in Open and Distance Learning , vol. 2, no. 2, pp. 1-4, 2002.

[16] W. C. Symmonds, "Colleges in crisis," Business Week , vol. 3830, pp. 72-78, 2003.

[17] D. Shale, "The hybridisation of higher education in Canada," International Review of Research in Open and Distance Learning , vol. 2, no. 2, pp. 1-11, 2002.

[18] M. A. Gerosa, D. Filippo, M. Pimentel, H. Fuks, C. J. P. Lucena, "Is the unfolding of the group discussion off-pattern? Improving coordination support in educational forums using mobile devices," Computers & Education , vol. 54, no. 2, pp. 528-544, 2010.

[19] S. Vonderwell, X. Liang, K. Alderman, "Asynchronous discussions and assessment in online learning," Journal of Research on Technology in Education , vol. 39, no. 3, pp. 309-328, 2007.

[20] A. M. Zalpaska, F. Falnegin, D. Rudd, "Student feedback on distance learning with the use of WebCT," Computers in Higher Education Economics Review , vol. 16, no. 1, pp. 10-14, 2004.

[21] A. M. Greenhalgh, "Case method teaching as science and art: a metaphoric approach and curricular application," Journal of Management Education , vol. 31, no. 2, pp. 181-194, 2007.

[22] R. Hackney, T. McMaster, A. Harris, "Using cases as a teaching tool in IS education," Journal of Information Systems Education , vol. 14, no. 3, pp. 229-234, 2003.

[23] S. L. Brooke, "Using the case method to teach online classes: promoting socratic dialogue and critical thinking skills," International Journal of Teaching and Learning in Higher Education , vol. 18, no. 2, pp. 142-149, 2006.

[24] L. J. Smith, "Content and delivery: a comparison and contrast of electronic and traditional MBA marketing planning courses," Journal of Marketing Education , vol. 23, no. 1, pp. 35-44, 2001.

[25] G. Z. Liu, J. Macalister, I. S. P. Nation, "The blended language learning course in Taiwan: issues & challenges of instructional design," Case Studies in Language Curriculum Design:Concepts and Approaches in Action Around the World , pp. 82-100, Routledge, New York, NY, USA, 2011.

[26] M. Loncar, N. E. Barrett, G.-Z. Liu, "Towards the refinement of forum and asynchronous online discussion in educational contexts worldwide: trends and investigative approaches within a dominant research paradigm," Computers & Education , vol. 73, pp. 93-110, 2014.

[27] D. H. Jonassen, H. Kwon II, "Communication patterns in computer mediated versus face-to-face group problem solving," Educational Technology Research and Development , vol. 49, no. 1, pp. 35-51, 2001.

[28] H. W. Webb, G. Gill, G. Poe, "Teaching with the case method online: pure versus hybrid approaches," Decision Sciences Journal of Innovative Education , vol. 3, no. 2, pp. 223-250, 2005.

[29] R. K. Valaitis, W. A. Sword, B. Jones, A. Hodges, "Problem-based learning online: perceptions of health science students," Advances in Health Sciences Education , vol. 10, no. 3, pp. 231-252, 2005.

[30] F. Concannon, A. Flynn, M. Campbell, "What campus-based students think about the quality and benefits of e-learning," The British Journal of Educational Technology , vol. 36, no. 3, pp. 501-512, 2005.

[31] E. Zhu, "Interaction and cognitive engagement: an analysis of four asynchronous online discussions," Instructional Science , vol. 34, no. 6, pp. 451-480, 2006.

[32] C. Angeli, N. Valanides, C. J. Bonk, "Communication in a web-based conferencing system: the quality of computer-mediated interactions," The British Journal of Educational Technology , vol. 34, no. 1, pp. 31-43, 2003.

[33] D. R. Garrison, N. D. Vaughan Blended Learning in Higher Education: Frameworks, Principles, and Guidelines , Jossey-Bass, San Francisco, Calif, USA, 2008.

[34] B. de Wever, M. van Winckel, M. Valcke, "Discussing patient management online: the impact of roles on knowledge construction for students interning at the paediatric ward," Advances in Health Sciences Education , vol. 13, no. 1, pp. 25-42, 2008.

[35] J. J. Lambiase, "Hanging by a thread: topic development and death in an online discussion of breaking news," http://www.languageatinternet.org/articles/2010/2814 Language@Internet , vol. 7, article 9, 2010.

[36] M. Hammond, "A review of recent papers on online discussion in teaching and learning in higher education," Journal of Asynchronous Learning Networks , vol. 9, no. 3, pp. 9-23, 2005.

[37] R. Luppicini, "Review of computer mediated communication research for education," Instructional Science , vol. 35, no. 2, pp. 141-185, 2007.

[38] J. Hewitt, "How habitual online practices affect the development of asynchronous discussion threads," Journal of Educational Computing Research , vol. 28, no. 1, pp. 31-45, 2003.

[39] A. Jeong, S. Frazier, "How day of posting affects level of critical discourse in asynchronous discussions and computer-supported collaborative argumentation," The British Journal of Educational Technology , vol. 39, no. 5, pp. 875-887, 2008.

[40] P. A. Ertmer, J. C. Richardson, B. Belland, D. Camin, P. Connolly, G. Coulthard, K. Lei, C. Mong, "Using peer feedback to enhance the quality of student online postings: an exploratory study," Journal of Computer-Mediated Communication , vol. 12, no. 2, pp. 412-433, 2007.

[41] M. D. Smet, H. V. Keer, B. D. Wever, M. Valcke, "Cross-age peer tutors in asynchronous discussion groups: exploring the impact of three types of tutor training on patterns in tutor support and on tutor characteristics," Computers & Education , vol. 54, no. 4, pp. 1167-1181, 2010.

[42] M. E. Bradley, L. R. Thom, J. Hayes, C. Hay, "Ask and you will receive: how question type influences quantity and quality of online discussions," The British Journal of Educational Technology , vol. 39, no. 5, pp. 888-900, 2008.

[43] K. Swans, P. Shea, E. E. Fredericksen, A. M. Picket, W. E. Pelz, "Course design factors influencing the success of online learning," ERIC Document Reproduction , no. ED448760, ERIC, 2000.

[44] W. Harlen, R. Deakin-Crick, "Testing and motivation for learning," Assessment in Education: Principles, Policy and Practice , vol. 10, no. 2, pp. 169-207, 2003.

[45] R. M. Marra, J. L. Moore, A. K. Klimczak, "Content analysis of online discussion forums: a comparative analysis of protocols," Educational Technology Research and Development , vol. 52, no. 2, pp. 23-40, 2004.

[46] B. Barron, "When smart groups fail," Journal of the Learning Sciences , vol. 12, no. 3, pp. 307-359, 2003.

[47] M. I. López, C. Romero, S. Ventura, J. M. Luna, "Classification via clustering for predicting final marks starting from the student participation in forums," in Proceedings of the 5th International Conference on Educational Data Mining, pp. 148-151, 2012.

[48] L. Rourke, H. Kanuka, "Learning in communities of inquiry: a review of the literature," Journal of Distance Education , vol. 23, no. 1, pp. 19-48, 2009.