Developing Summarizing Skills in 4th Grade Students: Intervention Programme Effects

Abstract

The aim of our study was to determine whether summarizing skills could be developed in 4th grade primary school students. We designed a 5 -month intervention programme as an experimental study, in which teachers trained students in the experimental group in their ability to summarize, which is one of the important strategies that enhance reading comprehension. 190 students in 4th grade from 8 primary schools in Slovenia participated in the study. We evaluated students' general reading competency, their metacognitive knowledge about reading and their ability to make summaries of two short and one longer expository text (pretest, posttest and follow-up test). The general reading competency explained the most variance in summarizing at pretest and posttest by experimental and control group of students. In the follow-up test, the summarizing from posttest was the strongest predictor in both groups and in the experimental group also the metacognitive knowledge about reading. The results showed that teachers can develop summarizing skills in students by systematically training them to use these skills, but the training effects decrease if the learning environment does not encourage students to use these skills.

Headnote

Abstract

The aim of our study was to determine whether summarizing skills could be developed in 4th grade primary school students. We designed a 5 -month intervention programme as an experimental study, in which teachers trained students in the experimental group in their ability to summarize, which is one of the important strategies that enhance reading comprehension. 190 students in 4th grade from 8 primary schools in Slovenia participated in the study. We evaluated students' general reading competency, their metacognitive knowledge about reading and their ability to make summaries of two short and one longer expository text (pretest, posttest and follow-up test). The general reading competency explained the most variance in summarizing at pretest and posttest by experimental and control group of students. In the follow-up test, the summarizing from posttest was the strongest predictor in both groups and in the experimental group also the metacognitive knowledge about reading. The results showed that teachers can develop summarizing skills in students by systematically training them to use these skills, but the training effects decrease if the learning environment does not encourage students to use these skills.

Keywords: Summarizing, younger students, intervention, reading competency, metacognitive knowledge

Introduction

Learning to read is an important activity in the lower grades of elementary school as it forms the basis for further learning and academic achievement of an individual (Pečjak, Kolić-Vehovec, Rončević Zubković, & Ajdišek 2009). When acquiring reading competency in the first three years of schooling, the focus is on developing vocabulary and fluent reading with good understanding of the material read (Chall, 1996; Gillet, Temple, Crawford, & Cooney 2003). After that, in the 4th grade, students enter a period of reading to learn (Gillet et al., 2003), in which students are expected to learn how to use reading for independent learning from textbooks. Students in the 4th grade are confronted with longer and more demanding texts and they are expected to read them independently, find the main ideas in them and combine these ideas into a meaningful summary. Since learning from longer texts causes great difficulties for many students, these (too) high expectations are known as "hitting the wall of the 4th grade" (Meltzer, 2007), therefore, summarizing is one of the key strategies for good reading comprehension in this period. Summarizing is a learning strategy, by which students find important information in a text and combine them into a short, coherent text - summary. To be able to do this, students have to analyse each of the sentences/paragraphs, search for important words in each paragraph, leave out the unimportant or specific information and then gather the important information in to a whole that makes sense (Westby, Culatta, Lawrence, & Hall-Kenyon, 2010).

Reading Comprehension and Summarizing Skill

Reading comprehension is a process of interaction between the characteristics of the text, the reader and the reading context. In a reader, numerous cognitive processes interactively contribute to comprehension (McCloskey & Perkins, 2013; Oakhill & Yuill, 1996; van den Broek & Espin, 2012). For understanding one sentence only, the reader must visually process each word, identify it, reach phonological, orthographical and semantic representation, and finally connect all these perceptions in order to understand the basic meaning of the sentence. It is similar in understanding the text as a whole: the reader has to identify individual ideas and form a coherent mental representation of the text. Summarizing is one of the reading strategies that enables students to more deeply understand the text and it is an indicator of understanding at the same time. By using the summarizing strategy, we assume that students are able to find important information and meaningfully connect them - with the words from the text or with their own words. Therefore, in this strategy, students first analyse each sentence/paragraph by searching for important words and important details, then they leave out unimportant information, and merge the important information into a meaningful whole (Westby, Culatta, Lawrence, & Hall-Kenyon, 2010). Summaries are shorter than the original text, but reflect the (so-called) macrostructure of it (Brown & Day, 1983).

By explaining the process of text comprehension, Kintsch (1974) proved that the number and the structure of the statements in the text are important in this process. The author distinguishes three levels of statements in a text with regard to their importance: the first level represents the most important statements (macrostructures), the second level represents the statements with more details and the third one represents statements with the most details (microstructures). Kintsch and van Dijk (1978) proposed three processes that are a part of summarizing-deletion, generalization and integration. They formed rules on the basis of which students connect individual statements at a lower level (the level of sentences, phrases and words) into macrostructures, and named them "macrorules". The first two rules include the process of deletion of unnecessary material: the exclusion of unnecessary (trivial) information and the additional removal of redundant information. The next two rules refer to the process of generalization-replacement of individu- al specific terms with broader concepts. Brown and Day (1983) named these two rules selection and Invention. The last two rules by Kintsch and van Dijk (1978) describe the process of integration, in which students connect microstatements in a joint general statement. Students do that by either choosing a statement with the main idea form the text or by forming a keynote statement with their own words, a process called superordination by Brown and Day (1983).

Kintsch (1974) found that 10-year-old students were most effective in applying the rule of deletion and selection, but were able to take into account only one rule at a time when creating a summary. When they decided to include an individual phrase or a sentence in the summary, they more or less copied it from the original text. The results of other studies also show that copy-delete strategy is the most common strategy in younger students (Brown, Day, & Jones, 1983; Brown & Smiley, 1978). They usually read sentence by sentence, each time deciding whether to include it in the summary or not. If they choose to include it, they more or less literally copy it from the text. The Reading Quest Organization (2017) also suggests that the most common difficulty students are faced with when acquiring the summarizing strategy is copying everything or a lot from the text or (literally) copying the whole statements. Based on these facts, authors (Brown et al., 1983; Brown & Smiley, 1978) conclude that the use of generalization and integration rules in summarizing increases with students' age and experience.

Studies show that the use of summarizing strategy affects reading comprehension and through reading comprehension also students' learning achievement in both very young and older students (Brown et al., 1983; Marzano, Pickering, & Pollock 2001; McCulley & Osman, 2015; Kolić-Vehovec, Bajšanski, & Rončević Zubković, 2011). Students who understood the text well included more first level statements in their summaries compared to students who did not understand the text well.

Relations Between Summarizing and Students' General Reading Competency

Along with perception and motivational factors, meta(cognitive) abilities are those, which define individual differences in students' reading abilities. They influence the processes of reading automation and reading comprehension (Borella, Carretti, & Pelegrina, 2010; Gerst, Cirino, Fletcher, & Yoshida, 2015). Among these abilities, we included the students' GRC and their MKR in our study.

We defined students' GRC as a composite variable, including the following reading dimensions: acquired reading technique, vocabulary and reading comprehension as the final output, since the prerequisite of creating a quality summary is to understand the text well. Additionally, in the early years of schooling, reading comprehension in students is predicted mostly by the automation of the reading technique, which is represented by reading fluency and well-developed vocabulary.

Reading comprehension starts with the process of word decoding (e.g., Altert, Schiefele, & Schneider, 2001; McKenna & Stahl, 2003; Oakhill & Cain, 2003; Pečjak, 2011) in which the reader recognizes individual visual symbols, transforms them into phonemes and connects them into words. In the initial reading phase (when students learn how to read), most of their mental attention from their working memory is focused on decoding, so they are less effective in storing and processing information, and consequently their reading comprehension is worse (Hintze, Mathews, Williams, & Tobin, 2002; Perfetti, 1985). With training, students develop fluency which is designated by the ability to read with speed, accuracy and proper expression (Barone, Mallette, & Xu, 2005; Rasinski, Homan, & Biggs, 2009). Studies prove that reading fluency increases reading comprehension in students (Droop, & Verhoeven, 2003; Nunes, Bryant, & Barros, 2012; Shiotsu, 2010; Verhoeven, 2000) or, as stated by Pikulski and Chard (2005), the fluency is the bridge between decoding and reading comprehension. Research shows that reading comprehension of 3rd grade students is still quite determined by their word decoding ability; in the 5th grade, however, students help themselves with the context (the remaining text), which is an important predictor of comprehension at their age (Saarnio, Oka, & Paris, 1990).

Reading vocabulary refers to comprehension of words, which students recognize and understand by reading. Numerous studies show that vocabulary is a factor that influences reading comprehension in the middle school (which the students include in our study attended) directly and indirectly. Indirectly by facilitating the process of decoding and releasing some of the capacities of the working memory for word processing or understanding (Nouwens, Groen, & Verhoeven, 2015; Pečjak, 2011; Rydland, Grøver Aukrust, & Fulland, 2012). However, Pečjak, Podlesek and Pirc (2009) found a moderate direct effect of reading vocabulary on reading comprehension (r = 0.51), which is consistent with the results of other studies (Elleman, Lindo, Morphy, & Compton, 2009; Nagy & Townsend, 2012). Readers with broader vocabulary determine the meaning of individual paragraphs faster compared to those who guess the meaning of the unknown words with the help of the remaining text (Ong, 2011). Nevertheless, Wanzek, Wexler, Vaughn and Cuillo (2010) warn in their meta-analysis of reading interventions for struggling readers that merely vocabulary training has a relatively weak effect on these students' reading comprehension.

The relation between vocabulary and reading comprehension also depends on the type of texts used to establish this connection (Diakidoy, Stylianou, Karefillidou, & Papageorgiou, 2005; Kelley & Clause-Grase, 2010). It is usually larger for expository texts than for narrative texts, the former containing more difficult words/concepts or academic vocabulary (Biemiller & Boote, 2006; Spiro & Taylor, 1980). Despite the acquired reading technique and vocabulary, reading comprehension does not "just happen"; instead, students have to learn different reading strategies, which enable them to better understand the content read. One of these strategies is summarizing. There are numerous intervention programs for training students in summarizing. In some of these programs, summarizing is the only strategy students are trained in (as was the case in our study) while in others it is only one of several strategies (e.g., in the CORI programme - Guthrie, Wigfield, Barbosa Perencevich, Taboada, Davis et al., 2004; programme McKown & Barnett, 2007; Reciprocal teaching - Palincsar & Brown, 1984). The meta-analyses of reading comprehension interventions, which included summarizing, show that in most cases the interventions are designed specifically for at-risk readers (whose reading achievement is below the 50th percentile) and reading-disabled readers, and have medium to large effects on reading comprehension (Richards-Tutor, Baker, Gersten, Baker, & Smith, 2016; Solis, Cuillo, Sharon Vaughn, Pyle, Hassaram, & Leroux 2011; Suggate, 2016).

Metacognitive Knowledge About Reading and Summarizing

Metacognition represents control structures of a higher order, which enable an individual to comprehend and to regulate one's own mental activity - also by reading (Demetrlou & Efklides, 1989).

Metacognitive knowledge refers to the knowledge of oneself as a reader and the knowledge about reading tasks and strategies, which are suitable for resolving different kinds of problems. These strategies comprise students' knowledge about the main goal of reading, their knowledge about reading the text several times to form a summary and their knowledge about trying to decipher the meaning of unknown words from the context, etc. (Pečjak, 2010).

However, the mere knowledge about how to read and knowledge about which strategies are most suitable to use does not influence comprehension by itself. Therefore the research results are mixed - from those which do not confirm direct connections between metacognitive knowledge and reading comprehension (Cromley & Azavedo, 2006 by ninth grade students; Pečjak et al., 2009 by fifth grade students) to the results that show significant correlations between both concepts (Kolić-Vehovec, Pečjak, & Rončević, 2009; Csíkos & Steklacs, 2010).

Students' age has to be considered when explaining these connections. Namely, metacognitive knowledge starts to evolve more intensively at the age between 8 and 10, when students start to encounter (longer) texts and more demanding academic tasks (Veenman & Spaans, 2005). It starts to show the greatest "power" in learning by the time they reach adolescence. Despite the inconsistent study results, an important finding by Walczyk (1994) is that metacognition-based reading intervention programs in primary education may be effective, especially for poorly performing students when there is compensation of deficiencies in lower-level of subcomponents of the reading process (fluency) through higher-level metacognitive processes.

The Problem of Research

Although summarizing is a basic reading (and learning) strategy, it is difficult for students to acquire it, because it requires them to monitor their comprehension of the text read and their understanding of the text structure. Therefore, we designed a 5-month intervention programme as an experimental study in which teachers in the EG trained students in the use of macrorules that are essential for summarizing (Kintsch & van Dijk, 1978). Students in the CG worked in accordance with the mandatory curriculum. We tried to increase the ecological validity of our study by integrating the programme into classroom settings of eG and by teachers implementing the program.

In the first part of the programme, students in the EG were trained to use deletion in texts proposed by the researchers - they were eliminating trivial and redundant information and/or maintaining important information. They were also trained in using macrorules of generalization and integration into coherent summaries. In the second part of the training, they rehearsed the skill of summarizing on textbook materials (from science and social studies).

We expanded previous studies in this field with some important aspects. First, there were only a few studies to explore the "pure" effects of the summarizing strategy, because it was usually developed in combination with other strategies (Sulak & Güneş, 2017). Second, the effects of summarizing training were most often studied in students with reading comprehension difficulties (Brown & Palincsar, 1984) or in students with learning disabilities (Solis et al., 2011), but not in a normative population - on all students of a classroom, which was the case in our study. Third, most interventions included older students (from 6th to 9th grade; Solis et al., 2011), while we included 4th grade students. At this age, students are expected to have relatively well-developed basic reading competency (fluent reading with comprehension) and some metacognitive knowledge, which should enable them to self-regulate when summarizing. From this point of view, such an intervention programme has a role as primary prevention programme - by starting to develop summarizing skills at this early age, it helps students to be more effective in independent learning.

In our study, we addressed the following research questions and examined the assumptions that were based on the results of previous empirical studies:

Does training in summarizing of EG students have an effect on their achievement in summarizing compared to CG students who were not trained? How high is the achievement in both groups of students' right after and three months after the completion of the programme. We presumed that EG students would have significantly better achievement in summarizing than CG students would right after training and three months after the training.

In which elements of summarizing would EG students progress most compared to CG students? We assumed that EG students would include more important and less unimportant information in the summary, which would be more coherent and have more appropriate titles right after the training and three months after the training.

How would some (meta)cognitive factors at starting pointstudents' reading competency, summarizing achievement and MKR, predict EG and CG students' achievement in summarizing at the beginning of the 4th grade, right after and three months after the finished programme? We assumed that all the mentioned factors would predict current achievement in summarizing.

Method

Participants

A total of 190 students in 4th grade from 8 primary schools in Slovenia participated in the experimental study. It was a convenient sample. There were 114 students in Eg (50.9% boys and 49.1% girls) from 4 schools and 5 of their teachers. The CG comprised 76 students (51.3% boys and 48.7% girls) from 4 other schools. There were no significant differences between EG and CG students regarding gender (X2(1) = 0.004, p = .535). The average age of students at the beginning of the study was 9.27 years (SD = 0.31).

Instruments

We used the same instruments for EG and CG, measuring the students' GRC and MKR before intervention and summarizing before, right after and three months after the intervention.

We used the Reading test (Pečjak, Potočnik, & Podlesek, 2011) and the Vocabulary test (Hershel, 1963) for measuring the students' GRC. The Reading test has two subtests: Reading fluency and Reading comprehension. Reading fluency has 25 items. Students filled in the missing words to complete a sentence meaningfully by selecting one out of four offered words. Time was limited to 7 minutes. Each correct answer was rated with 1 point and the maximum score was 25. Cronbach's a coefficient was .92. The subtest of Reading comprehension has 5 short texts (80 - 108 words), which students read and then answered 4 multiple choice questions for each text. Time was limited to 10 minutes. Maximum score was 20. Cronbach's a reliability coefficient was .85.

Vocabulary test from Herschel's (1963) Test of Reading (Level 3 - Elementary Form) was adapted for the Slovenian language by Zorman and Žagar (1974, in Toličič & Zorman, 1977). It measures the size of students' vocabulary and comprises 20 tasks. It is suitable for students from the third to the fifth grade. The students have to answer the questions by choosing the appropriate word from the pool of five choices. Time is limited to 5 minutes and each correct answer brings 1 point (for a maximum of 20 points). Cronbach's a was .86, in our study .88.

We merged the results of both tests in a composite variable of GRC, which represented the sum of all possible scores from both instruments. The maximum score was 75.

Metacognitive knowledge was measured with the adapted version of Metacognitive Knowledge Questionnaire (Kolić-Vehovec, Rončević Zubković, & Pahljina-Reinić, 2014). The original version has 14 items and measures students' MKR in general and academic reading. We used the first 9 items, which refer to MKR in general. Students had to choose an answer, which best describes the main purpose of reading and had to display knowledge of different reading strategies in reading comprehension in multiple-choice questions. The maximum score was 9 points, Cronbach's a was .68.

Summarizing was assessed three times: just before the intervention (summarizing 1), right after the intervention (summarizing 2) and three months after the end of the intervention (summarizing 3). In summarizing 1 and summarizing 3 we used short texts referring to water pollution, traffic and healthy food (from 99 to 120 words in summarizing 1) and subjects of holidays, Arctic and Antarctic and the life of noblemen (from 111-119 words in summarizing 3). After reading, students were asked to summarize the main ideas from each of the text and give the text a title. The criteria for the evaluation of the summaries were adapted from Friend (2001).

Each summary was evaluated with regard to: i) the number of main ideas in the summary (each text had three semantic units, which represented the main ideas of the text; the maximum score was 9 points); ii) coherence of the summary (sentences being meaningfully connected or not). The rating scale was: 0-incoherent summary; 0.5partly coherent summary and 1-coherent summary; (the maximum score was 3); iii) the title of the text (0-inappropriate title; 0.5-partly appropriate title; 1-appropriate title; the maximum score was 3). The maximum score for summarizing 1 and 3 was 15 points (number of points from all three texts). We also marked the number of unimportant/ specific ideas in the summaries.

In summarizing 2 we used only one longer text (237 words) about winds. After reading, students had to make a summary. Summaries in this phase were evaluated by the same criteria as the other two summaries: main ideas (maximum 9 points), text coherence (maximum 1 point) and the title of the text (maximum 1 point); the maximum score for the whole summary was 11. Internal consistency for summarizing 2 with two independent raters was .84. Two independent raters assessed the students' summaries. If their scores differed, it was necessary for them to reach consensus. Internal consistency for summarizing 1 was .86, for summarizing 3 .88 and for summarizing 2 .87. Description of The Intervention Program

Content and duration of the program

The five-month intervention programme, which lasted from the beginning of December until the end of May, consisted of two parts with different content.

The first part, which lasted for 7 weeks, comprised 14 sessions (30 minutes 2-times a week). Students received short expository texts (50 - 100 words) and were trained in: i) recognition of the main ideas in the texts - after reading short texts, students chose a title expressing the basic idea from different proposed titles; in the next step, they created titles which comprised the main ideas; ii) marking the main ideas (after reading, students deleted unimportant ideas - e.g., conjunctions, detailed information and the repeated information; after that they circled/coloured the important words; iii) they tried to meaningfully connect the circled/coloured words into 1-2 sentences (summary). The trainings were a combination of explicit teacher modelling, guided practice and independent practice of individual students or pairs of students. Afterwards, students examined their summaries, improved and corrected them together with their teachers (frontal and individual feedback).

The second part of the intervention programme lasted from February until May and comprised 20 sessions: in 12 sessions, students were given two short expository texts (60 - 120 words) twice a week from their textbooks for science and social studies with subjects they were currently dealing with. They made summaries of these texts in a way they learned in the first part of the program. The lessons lasted for 30 minutes. In the next 8 sessions, students received longer texts (from 150 - 250 words) from science and social studies. In these sessions, students worked mostly by themselves, with their teacher checking the accuracy of the summaries. These sessions lasted for 45 minutes. In total, the training of students lasted for 19 hours.

Implementation of the intervention

The programme was implemented by five 4th grade teachers, who taught students most of the subjects. Teachers attended a one-day training, in which they familiarised with the content and the course of implementation of the programme more thoroughly. They received a manual with precisely described schedule of content, along with prepared texts for the first part of the programme. Goals and didactic methods were defined for each learning session. For the first part of the programme, texts had been previously prepared for them. In the second part, teachers chose texts from their textbooks (for science and social studies). Students also received workbooks with texts, which they used during the entire course of the training. We had three meetings with teachers. In the first meeting, we simulated the use of summarizing strategy in workshops with teachers in the manner in which they were supposed to train it with students. We also supplied them with manuals. They took part in our training in order to achieve as standardized implementation of the intervention programme as possible. We had a meeting with teachers again after the implementation of the first part of the programme and after the end of intervention. In these meetings, teachers were asked to provide feedback on how the implementation progressed.

Procedure of Data Collection

After the schools agreed to take part in our study, we gathered the parents' written consents for their children to participate in our research. In each classroom, data collection took place three times: in October 2016 (pretest), in June 2017 (posttest) and in September 2017 (follow-up test). It took two school hours to apply the instruments each time. In the first hour we applied the Reading test and the Metacognitive Knowledge Questionnaire and in the second the Vocabulary test and Summarizing.

Results

The Results of EG and CG In Summarizing Before The Intervention, After The Intervention and Three Months After The End of The Intervention

We first determined whether EG and CG were comparable in terms of GRC, MKR and summarizing ability before the intervention (pretest) (Table 1).

With t-tests for independent samples, we confirmed that EG and CG did not differ significantly in their achievements at the starting point, although we found slightly better GRC and summarizing in CG and more MKR in general in EG. Nevertheless, we can conclude that both groups were similar in all variables before the intervention.

Next, we examined the possible differences in summarizing achievement between EG and CG right after the intervention programme (posttest) and three months later (follow-up test) using univariate ANOVA (Table 2).

The results in Table 2 show that EG had significantly higher achievement in summarizing than CG had after the intervention (Summ_2), but the effect size was small (Cohen, Miles, & Shevlin, 2001). There were no significant differences between EG and CG iln the follow-up measurement (Summ_3).

By examining the dynamics of both groups of students' achievement, taking into account the maximum scores (Table 1 and 2), we established that EG made an improvement from Summ_1 to Summ_3 by 1.92 points on average, which shows a significant progress (t(101)= 5.975; p< .001). There was also an improvement from Summ_1 to Summ_3 in CG, but it was smaller (0.79 point on average) and not significant (t(68)= 1.901; p= .0б1). Also in Summ_2, where students had to summarize a longer text, EG received significantly more points (53.2% of maximum score) than CG (43.8% of maximum score).

The Analysis of Differences Between EG and CG In Summarizing After The Intervention

Since we found significant differences between CG and EG in Summ_2, we wanted to find out which elements of summarizing determined these differences. We evaluated the following elements in Summ_2: number of main ideas, amount of unimportant/specific information, coherence of the text and appropriateness of the title (as an indicator of generalization competency).

Table 3 shows that CG and EG differed significantly in all elements defining the quality of the summary but the title. The effect sizes were small for main and unimportant ideas and moderate for coherence. Students in EG compared to their peers in CG stated significantly more important ideas and included fewer unimportant ideas in the summary, which was significantly more coherent.

Predictors of Achievement In Summarizing

Next, we were interested in how students' GRC, their MKR and their achievement in summarizing in the pretest predicted the summarizing achievement in all three measurements (pre- and posttest, and follow-up test). Since the correlations between individual variables were moderate (Table 4) and the condition of homoscedasticity (Field, 2009) was met, we used the hierarchical regression analysis (Table 5). In the proposed model, cognitive variables were included in the first step (GRC and previous summarizing achievement) and the MKR in the next steps.

It is evident from Table 5 that 33% of the variance of the achievement by summarizing 1 in EG could be explained by the proposed model and 25% of the variance in CG. In summarizing 2, 39% of the achievement could be explained in EG and 18% in CG. In addition, in summarizing 3, 45% of the variance in achievement could be explained in both groups. Since the differences between the values of adjusted R2 and R2 were very small in both groups, we can conclude that our model has a good cross-validity, which means that our results are generalizable across the 4thgrade students population. E.g., in summarizing 3, the model in EG would explain only 1% more variance of the dependent variable if applied in the whole population compared to the one applied in our sample, and 4% more in CG.

In EG and CG, students' achievement by summarizing 1 before the intervention programme was moderately predicted only by GRC (ß = .51 in EG vs. ß = .47 in CG). Students' achievement in summarizing 2 was still mostly determined by GRC in both groups, but with less power, which decreased somewhat less in EG (ß = .45) and more in CG (ß = .27). However, students' achievement in summarizing in the pretest showed to be an important predictor of summarizing 2 in EG. Finally, summarizing 2 was a moderate predictor of summarizing 3 in both groups and MKR was an additional significant predictor of summarizing 3 in the EG.

Discussion

In our study, we investigated the effects of an intervention programme for 4th grade primary school students. They were trained in their ability to summarize, which is one of the important strategies that enhance reading comprehension. The EG comprised students from the classrooms which were included in the intervention programme. The programme was implemented by their teachers who at- tended short one-day training beforehand with the aim of improving the ecological validity of the programme.

Intervention Programme Effect on Summarizing Achievement

First, we wanted to establish if a 5-month training of EG students in summarizing would have an effect on their summarizing achievement compared to CG students, who worked in accordance with the established curriculum - right after the completed programme and three months later. We expected significant differences between EG and CG in both measurements. The results showed that both groups were comparable in their GRC, MKR and summarizing ability before training (Table 1).

In the posttest we found significant improvement in summarizing achievement of EG of students compared to CG students (Table 2), but the effect size was small (n2 was .036). Since students had to make a summary of a longer text, we might conclude that EG students are on a track of using summarizing skills towards using summarizing as a strategy. They showed they were capable to use what they had learned also in different (longer) texts.

However, EG students were not able to keep this advantage after three months - in the follow-up test at the beginning of their 5th grade. Our assumption that EG students would have significantly better achievements in both -post- and follow-up test was only partially confirmed. The question is - why? It might be that this reading skill was still not trained enough in EG students or that these students were still not able to automatically transfer what they had learned into a new learning situation. Namely, students at this stage perceive learning of a certain skill useful in a specific subject, grade or by a certain teacher, where they learn(ed) to use it. It does not necessarily mean they can thoughtfully use it in other situations as well. By measuring summarizing in the follow-up test (Summ_3), we did not explicitly order them to remember what they were trained in the 4th grade and to make a summary with having rules of summarizing in mind. As some authors state (Afflerbach, Pearson, & Paris, 2008), a difference has to be made between reading skill and reading strategy. Reading skill may be employed tacitly, without deliberate thought or intention, whereas a strategy is a deliberately controlled process. In EG students, we were probably successful in developing summarizing as a skill, but not as a strategy, which students would be able to use flexibly in different texts and learning contexts. This is strongly connected with the shaping of an environment in which the use of these strategies is enhanced. Some authors emphasize, that students must first recognize the need for a strategy before they would use it (Paris, Lipson, & Wixson, 1983; Yang, 2006).

A more thorough examination of the progress of EG students showed significant improvement. Their achievement was nearly 2 points higher (on average) in the follow-up test Summarizing_3 compared to the pretest starting point (Summ_1) and the results were more homogeneous. On the other hand, this progress was not significant in CG and the results were more dispersed.

Since students in EG made significantly better summaries in summarizing_2 than students in CG, we were interested in finding the elements where this improvement was achieved. We evaluated the number of main ideas, the number of unimportant ideas, coherence and the appropriateness of the title in line with the adapted criteria by Friend (2001). The results showed (Table 3) that EG students included significantly more important and significantly less unimportant information in their summaries. This shows that students learned the process of deletion (Kintsch & van Dijk, 1978). Students in EG also made more coherent summaries than students in CG, which shows they learned the process of integrating individual statements into a meaningful unit as well (Brown & Day, 1983). Although the effect sizes were small, our assumption that EG students would make significant progress in individual elements of summarizing (compared to the CG students) was almost entirely confirmed. Namely, EG and CG of students were not significantly different in searching for the best title. Both groups were comparable in their ability to form appropriate or broad enough titles to represent the key message of a text. The reason for this result could be in both groups' numerous experiences in finding a title for narrative and expository texts. This is one of the rare reading skills students are trained in from the beginning of schooling.

Students in both groups had the most difficulties with the process of generalization (Kintsch & van Dijk, 1978). Only exceptionally did their summaries include superordinates or statements with key messages written with their own words. Our results are in line with the findings of other studies, which outline that copy - delete strategy is the most common one in younger students (Brown et al., 1983; Brown & Smiley, 1978; Kintsh, 1974). This was also the strategy that we started our programme with. Despite encouraging students to use superordinates where possible in the following sessions of the programme, most students kept using the copy - delete strategy.

Predictors of Achievement In Summarizing by EG and CG

In our first model, based on the students' results before the intervention programme, we entered the variables, which showed to be the strongest predictors of summarizing according to empirical studies: GRC and MKR (Borella et al., 2010; Efklides, 2014; Gerst et al., 2015; Kolić Vehovec et al., 2009; Csikos & Steklacs, 2010). Next to the GRC and MKR, we entered the currently developed summarizing skills in the second and the third model (in model 2 Summ_1 and in model 3 Summ_1 and Summ_2).

We assumed that GRC as a composite variable of reading fluency, vocabulary and reading comprehension would be the strongest predictor of summarizing in model 1. All abilities mentioned are a necessary, but not a sufficient condition to make a well-written summary. Namely, if students want to make a summary, they have to understand the content well. Good reading comprehension is enabled by fluent reading (showing automated reading skill by minimally loading their working memory) and broad vocabulary.

It was in fact confirmed that this variable was the strongest predictor of summarizing skill before the intervention programme (Table 5) - if GRC increased by 1 standard deviation, the students' achievement would improve for as much as .55/.51 SD in EG and a little less in CG (.49/.47). MKR was not a significant predictor of summarizing in any of the groups. Nevertheless, we were able to explain 33% of the variance in summarizing by EG students and 25% in CG students with model 1.

GRC was an important predictor of summarizing achievement in EG and CG of students also in model 2, although its predictive power was stronger for EG (ß was between .45 and .59) than for CG (ß was between .27 and .38). In this model, summarizing_1 was also an important predictor (ß = .26) accounting for additional 5% of the variance in students' achievement by Summ_2. Again, MKR was not an important predictor in either of the groups. Model 2 explained 39% of the variance in students' summarizing achievement in the EG, but only 18% in the CG.

In model 3, we were able to explain the same amount of variability by summarizing achievement in both groups with included variables - 45%, which is a large portion. GRC had less predictive power for summarizing achievement in both groups, while the strongest predictor in both groups turned out to be Summ_2 (ß was .39 in EG and .52 in CG), and in EG also MKR (ß .24). It is evident that EG students made important progress in their MKR. They were significantly more aware of the strategies to use in order to understand the text better (e.g., one must read difficult texts more slowly/twice, one must try to determine the meaning of an unknown word from the context etc.), which was apparently helpful for them in summarizing.

In our opinion, EG students became familiar with the strategies for comprehension monitoring when they were checking and analysing their summaries during the course of the intervention programme, which enabled them to write better summaries. We also assume that this knowledge will help EG students to progress from the acquired summarizing skill to summarizing strategy more quickly, i. e. that they will be able to use their skills flexibly in diverse texts and subjects. However, it has to be considered that the effects of programmes, which explicitly developed metacognitive strategies in students related to their reading achievement, were small to moderate (Csikos & Steklacs, 2010).

Conclusions with Implications

With our programme, we were able to demonstrate the possibility that teachers can develop summarizing skills in students by systematically training them to use these skills, as well as establish that the training effects decrease quickly if the learning environment does not enhance the use of these skills. We also determined that metacognitive knowledge development acquired by reflection made during the discussions about summaries (Schwannenflugel & Flanagan Knapp, 2016) helps with intentional use of summarizing in different contexts.

Since training was carried out in authentic situations (whole classrooms and their teachers), we were able to assure good ecological validity of the programme, which was one of its strongest advantages. Namely, Solis et al. (2011) pointed out in their meta-analysis that most intervention programmes for summarizing skills development were implemented by the researchers, with the effects of these programmes being stronger than those of the programmes carried out by the teachers. These differences raise a question about how to effectively transfer this training into the natural dynamics of a classroom.

Further, we were able to explain almost half of the variability in students' achievement by summarizing with the variables, included into regression analysis, which is a substantial amount. Also, our results are generalizable across population, since the differences between adjusted R2 and R2 were very small in both groups. Nevertheless, in further studies, students' summarizing achievement should be controlled also for their familiarity with the structure of informational texts, since this is connected with reading comprehension (Meyer & Ray, 2011). There were also some limitations in the implementation of the programme, which we suggest to be considered in further research. First, there was not enough formative monitoring of the EG teachers - large differences in relative progress appeared between classrooms of individual teachers, which probably reflect the differences in the teachers' engagement/effort in implementation of the intervention. Therefore, it would be sensible to monitor the implementation quality by teachers who have implemented the programme, which is an advise given also in current studies (e. g., Okkinga, van Steensel, van Gelderen, & Sleegers, 2018). The second major limitation was that we had no control over what the students in CG did - we do not know, whether they were familiarised with the summarizing strategy during regular instruction or how much they were trained in it or using it. The results of a meta-analysis by Scammaca, Roberts, Vaughn and Stuebing (2015) showed that for struggling readers new intervention programs had smaller effects (between 2005 and 2011) than the old ones (between 1980 and 2004), which authors attribute to general improvements in the school instruction. Finally, we would like to emphasize the need for controlling the students' writing ability, which is equally important for making a summary as the ability to read. One reassuring circumstance in our programme was the fact that writing a summary was not time limited. Therefore, the final output of the students' should not have been influenced by the automation of writing ability and the speed of writing.

Sidebar

Received: 14 March 2018

Revised: 21 May 2018

Accepted: 05 June 2018

References

References

Afflerbach, P., Pearson, P. D., & Paris, S. G. (2008). Clarifying differences between rerading skills and reading stratgies. The Reading Teacher, 61(5), 364-373. URL: http://www.jstor.org/stable/20204600.

Altert, C., Schiefele, U., & Schneider, W. (2001). Predictors of Reading Literacy. European Journal of Psychology of Education, 14(3), 363-383.

Barone, D. M., Mallette, M. H., & Xu, S. H. (2005). Teaching early literacy: Development, assessment, and instruction. New York London: The Guilford Press.

Biemiller, A., & Boote, C. (2006). An effective method for building meaning vocabulary in primary grades. Journal of Educational Psychology, 98, 44-62. DOI: 10.1037/0022-0663.98.1.44.

Borella, E., Carretti, B., & Pelegrina, S. (2010). The specific role of inhibition in reading comprehension in good and poor comprehenders. Journal of learning disabilities, 43(6), 541-552. DOI: 10.1177/0022219410371676.

Brown, A. L., & Day, J. D. (1983). Macrorules for summarizing text: the development of expertise. University of Notre Dame.

Brown, A. L., Day, J. D., & Jones, R. S. (1983). The development of plans for summarizing texts. Child Development, 54, 968-979. URL: http://www.jstor.org/ stable/1129901.

Brown, A. L., & Palincsar, A. S. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 2, 117-175. URL: http://www.jstor.org/ stable/3233567.

Brown, A. L., & Smiley, S. (1978). The development of strategies for studying texts. Child Development, 49, 1076-1088. URL: http://www.jstor.org/stable/1128747.

Chall, J. S. (1996). Stages of reading development. Orlando, FL: Harcourt Brace.

Cromley, J. G., & Azevedo, R. (2006). Self-report of reading comprehension strategies: What are we measuring? Metacognition and Learning, 1, 229-247. DOI 10.1007/s11409-006-9002-5.

Csikos, C., & Steklacs, J. (2010). Metacognition-based reading intervention programs among fourth-grade Hungarian students. In A. Efklides & P. Misailidi (Eds.), Trends and prospects in metacognition research (pp. 327-344). Springer Science+ Bussiness Media, LLC. DOI 10.1007/978-1-4419-6546-2_16.

Cohen, J., Miles, J., & Shevlin, M. (2001). Applying regression and correlation: A guide for students and researchers. London: Sage.

Demetriou, A., & Efklides, A. (1989). The person's conception of the structures of developing intellect: Early adolescence to middle age. Genetic, social and general psychology monographs, 115(3), 371-423.

Diakidoy, I. A. N., Stylianou, P., Karefillidou, C., & Papageorgiou, P. (2005). The relationship between listening and reading comprehension of different types of text at increasing grade levels. Reading Psychology 26(1), 55-80. DOI: 10.1080/02702710590910584.

Droop, M., & Verhoeven, L. (2003). Language proficiency and reading ability in first and second language learners. Reading research quarterly, 38(1), 78-103. URL: http://www.jstor.org/stable/4151690.

Efklides, A. (2014). How Does Metacognition Contribute to the Regulation of Learning? An Integrative Approach. Psychological Topics 23(1),1-30.

Elleman, A. M., Lindo, E., Morphy, P., & Compton, D. L. (2009). The impact of vocabulary instrucution on passage-level comprehension of school children: A meta-analysis. Journal of Research on Educational Effectiveness, 2, 1-44. DOI 10.1080/19345740802539200.

Friend, R. (2001). Effects of strategy instruction on summary writing of college students. Contemporary Educational Psychology, 26(1), 3-24. https://doi. org/10.1006/ceps.1999.1022.

Field, A. (2009). Discovering statistics using SPSS (third edition). SAGE publications Ltd.

Gerst, E. H., Cirino, P. T., Fletcher, J. M., & Yoshida, H. (2015). Cognitive and behavioral rating measures of executive function as predictors of academic outcomes in children. A Journal on Normal and Abnormal Development in Childhood and Adolescence, 23(4), 381-407.

Gillete, J. W., Temple, C., Crawford, A., & Cooney, B. (2003). Understanding reading problems: Assessment and instruction (5th edition). Boston: Allyn and Bacon.

Guthrie, J. T., Wigfield, A., Barbosa, P., Perencevich, K. C., Taboada, A. Davis, M. H., Scafiddi, N. T., & Tonks, S. (2004). Increasing Reading Comprehension and Engagement Through Concept-Oriented Reading Instruction. Journal of Educational Psychology, 96(3), 403- 423. DOI: 10.1037/0022-0663.96.3.403.

Hershel, T. M. (1963). Test of Reading Level 3 - Elementary Form. In T. M. Herschel (Ed.), The 20th Yearbook of the National Council on Measurement in Education. Ann Arbor, MI: SGSR.

Hintze, J. M., Matthews, W. J., Williams, S. A. S., & Tobin, K. G. (2002). Oral reading fluency and prediction of reading comprehension in African American and Caucasian elementary school children. School Psychology Review, 31, 540-553.

Kelley, M. J., & Clausen-Grase, N. (2010). Guiding students through expository text with text feature walk. The Reading Teacher, 64, 191-195. URL: http://www. jstor.org/stable/40961980.

Kintsch, W. (1974). The representation of meaning in memory. Hillsdale, NJ: Lawrence Erlabaum Associates.

Kintsch, W., & van Dijk, T. A. (1978). Toward a model of text comprehension and production. Psychological Review, 85, 363-394.

Klinger, J. K., & Vaughn, S. (1996). Reciprocal teaching of reading comprehension strategies for students with learning disabilities who use english as a second language. Elementary School Journal, 96(3), 275-293. URL: http://www.jstor.org/stable/1001758.

Kolić-Vehovec, S., Bajšanski, I., & Rončević Zubković, B. (2011). The role of reading strategies in scientific text comprehension and academic achievement of university students. Review of Psychology, 18(2), 81-90.

Kolić-Vehovec, S., Rončević Zubković, B., & Pahljina-Reinić, R. (2014). Development of metacognitive knowledge of reading strategies and attitudes toward reading in early adolescence: The effect on reading comprehension. Psihologijske teme, 23(1), 77-98.

Kolić-Vehovec, S., Pečjak, S., & Rončević Zubkovič, B. (2009). Spolne razlike u (meta)kognitivnim i motivacijskim čimbenicima razumijevanja teksta adolecenata u Hrvatskoj i Sloveniji [Gender differences in (meta) cognitive and motivational factors of text comprehension in adolescents in Croatia and Slovenia]. Suvremena psihologija, 12, 229-242.

Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2001). Classroom instruction that works: Researchbased strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development.

McCloskey, G., & Perkins, L. A. (2013). Essentials of executive functions assesment. John Wiley & Sons, Inc.

McCulley, L. V., & Osman, D.J. (2015). Effects of reading instruction on learning in social studies: A synthesis of quantitative research. The Journal of Social Studies Research, 39, 183-195.

McKenna, M. C., & Stahl, S. A. (2003). Assessment for Reading Instruction. New York: Guildford.

Meltzer, L. (2007). Executive functions in education. New York: The Guilford Press.

Meyer, B. J., & Ray, M. N. (2011). Structure strategy interventions: Increasing reading comprehension of expository text. International Electronic Journal of Elementary Education, 4(1), 127-152.

McKown, B. A., & Barnett, C. L. (2007). Improving Reading Comprehension through Higher-Order Thinking Skills. Online Submission.

Nagy, W. E. & Townsend, D. (2012). Words as tools: Learning academic vocabulary ans alnguage acquisition. Reading Research Quarterly, 47(1), 91-108. DOI: 10.1002/RRQ.011.

Nouwens, S., Groen, M. A., & Verhoeven, L. (2015). How storage and executive functions contribute to children's reading comprehension. Learning and Individual Differences, 47, 96-102.

Nunes, T., Bryant, P., & Barros, R. (2012). The development of word recognition and its significance for comprehension and fluency. Journal of Educational Psychology, 104(4), 959-973. DOI: 10.1037/a0027412.

Oakhill, J., & Cain, K. (2003). The development of comprehension skills. In T. Nunes & P. Bryant (Eds.), Handbook of children's literacy (pp. 155-180). Dordrecht: Kluwer Academic Publishers.

Oakhill, J., & Yuill, N. (1996). Higher order factors in comprehension disability: Processes and remediation. In C. Cornoldi & J. Oakhill (Eds.), Reading comprehension difficulties: Processes and intervention (pp. 69-92). Mahwah, NJ: Erlbaum.

Okkinga, M., van Steensel, R., van Gelderen, A. J., & Sleegers, P. J. (2018). Effects of reciprocal teaching on reading comprehension of low-achieving adolescents. The importance of specific teacher skills. Journal of Research in Reading, 41(1), 20-41.

Ong, J. (2011). Cognitive factors that affect reading comprehension. Education.

Pečjak, S. (2010). Psihološki vidiki bralne pismenosti: Od teorije k praksi. [Psychological aspects of reading literacy: From theory to practice]. Ljubljana: Znanstvena založba Filozofske fakultete.

Pečjak, S. (2011). Razvoj bralnih zmožnosti v prvem trilerju osnovne šole. [Reading competency development in the first three years of elementary school] In L. Marjanovič Umek, S. Pečjak & U. Fekonja Peklaj (Eds.). Govor in branje otrok: ocenjevanje in spodbujanje [Children's speech and reading: evaluating and enhancing] (pp. 45-76). Univerza v Ljubljani: Znanstvena založba Filozofske fakultete.

Pečjak, S., Kolić-Vehovec, S., Rončević Zubković, B., & Ajdišek, N. (2009). (Meta)kognitivni i motivacijski prediktori razumijevanja teksta u Hrvatskoj i Sloveniji. [(Meta)cognitive and motivational predictors of text comprehension in Croatia and Slovenia]. Suvremena psihologija, 12, 257-270.

Pečjak, S., Podlesek, A., & Pirc, T. (2009). Model of Reading comprehension for 5th grade students. Studia psychologica, 51(1), 53-68.

Pečjak, S., Potočnik, N., & Podlesek, A. (2011). Bralni test [Readingtest]. Ljubljana: Center za psihodiagnostična sredstva.

Perfetti, C. A. (1985). Reading ability. New York: Oxford University Press.

Pikulski, J. J., & Chard, D. J. (2005). Fluency: Bridge between decoding and reading comprehension. The Reading Teacher, 58, 510-519. URL: http://www.jstor. org/stable/20205516.

Rasinski, T., Homan, S., & Biggs, M. (2006). Teaching Reading Fluency to Struggling Readers: Method, Materials, and Evidence. Reading & Writing Quarterly, 25(2-3), 192-204. https://doi.org/10.1080/105735 6080268362204.

Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary educational psychology, 8(3), 293-316.

Reading Quest Organisation (2017). Strategies for reading comprehension. Summarizing. Retrieved January 12th 2018 from: http://www.readingquest.org/ strat/summarize.html.

Richards-Tutor, C., Baker, D. L., Gersten, R., Baker, S. K., & Smith, J. M. (2016). The effectiveness of reading interventions for english learners: A research synthesis. Exceptional Children, 82(2), 144-169.

Robinson, F. P. (1961). Study skills for superior students in secondray school. The Reading Teacher, 15(1), 29-37.

Rydland, V., Grøver Aukrust, V., & Fulland, H. (2012). How word decoding, vocabulary and prior topic knowledge predict reading comprehension. A study of language-minority students in Norwegian fifth grade classrooms. Reading and Writing, 25(2). 465482. DOI: 10.1007/s11145-010-9279-2.

Saarnio, D. A., Oka, E. R., & Paris, S. G. (1990). Developmental predictors of children's reading comprihension. In T. H. Carr & B. A. Levy (Eds.), Reading and it's development: component skills approaches (pp. 255-282). New York: Academic.

Scammaca, N. K., Roberts, G., Vaughn, S., & Stuebing, K. K. (2015). A meta-analysis of intervention for struggling readers in grades 4-12: 1980-2011. Journal of Learning Dissabilities, 48(4), 369-390.

Schwanenflugel , P. J., & Flanagan Knapp, N. (2016). The psychology of reading. New York, London: The Guilford Press.

Shiotsu, T. (2010). Components of L2 reading: Linguistic and processing factors in the reading test preformances of japanese EFA learners. Cambridge: Universitiy Press.

Solis, M., Ciullo, S., Vaughn, S., Pyle, N., Hassaram, B., & Leroux, A. (2011). Reading comprehension interventions for middle school students with learning disabilities: A synthesis of 30 Years of Research. Journal of Learning Disabilities, 45(4), 327-340. DOI: 10.1177/0022219411402691.

Spiro, R. J., & Taylor, B. M. (1980). On investigating children's transition from narrative to expository discourse: The multidimensional nature of psychological text classification (Tech. Rep. No. 195). Urbana: University of Illinois, Center for the Study of Reading.

Suggate, S. (2016). A meta-analysis of the long-term effects of phonemic awareness, fluency, and rerading comprehension interventions. Journal of Learning Disabilities, 49(1), 77-96.

Sulak, S. E., & Güneş, F. (2017). The The Effects of Teaching Informative Text Through Processual Model on Reading Comprehension Skills. International Electronic Journal of Elementary Education, 10(2), 265-271.

Toličič, I., & Zorman, L. (1977). Okolje in uspešnost učencev. [Environment and students' achievement]. Ljubljana: Državna založba Slovenije.

Van den Broek, P., & Espin, C. A. (2012). Connecting cognitive theory and assessment: Measuring individual differences in reading comprehension. School Psychology Review, 41(3), 315-325.

Veenman, M. V. J., & Spaans, M. A. (2005). Relation between intellectual and metacognitive skills: Age and task differences. Learning and Individual Differences, 15, 159-176. https://doi.org/10.1016/j. lindif.2004.12.001.

Verhoeven L. (2000). Components in early second language reading and spelling. Scientific Studies of Reading, 4(4), 313-330. DOI: 10.1207/S1532799XSSR0404_4.

Walczyk, J. J. (1994). The development of verbal efficiency, metacognitive strategies, and their interplay. Educational Psychology Review, 6, 173-189.

Wanzek, J., Wexler, J., Vaughn, S., & Cuillo, S. (2010). Rerading interventions for struggling readers in the upper elementary grades: A synthesis of 20 years of research. Reading and Writing, 23(8), 889-912. DOI 10.1007/s11145-009-9179-5.

Westby, C., Culatta, B., Lawrence, B., & Hall-Kenyon, K. (2010). Summarizing expository texts. Topics in Language Disorders, 30, 275-287.

Yang, I. F. (2006. Reading strategies or comprehension monitoring strategies? Reading Psychology, 27(4), 313343. https://doi.org/10.1080/02702710600846852.