1. Theoretical Framework
1.1. Sustainability: Utopia or Reality in Higher Education?
Planetary challenges such as the climate and environmental crisis, pollution, economic inequalities, and extreme poverty make it essential to raise the need to consider environmental, economic, political and social sustainability.
Consequently, sustainability is present in global and national educational agendas. At the international level, the declaration of the 2005–2014 decade of education for sustainable development [1] or The Global Goals stand out regarding Sustainable Development, SDG, for the 2030 Agenda [2].
In the educational field, the Association of University Leaders For A Sustainable Future [3] suggested a greater commitment of the University to Education for Sustainable Development through its decalogue: (1) Take advantage of every opportunity to raise awareness of government, industries, foundations, and universities in publicly expressing the need to move toward an environmentally sustainable future; (2) Encourage the university to engage in education, research, policy formation, and information exchange on issues related to population, environment, and development in order to achieve a sustainable future; (3) Establish programs that train experts in environmental management, sustainable development, demography, and related topics to ensure that university graduates receive environmental training and are responsible citizens; (4) Create programs that develop the university’s capacity to teach the environmental subjects to undergraduate and graduate students and professional institutes; (5) Be an example of environmental responsibility by establishing resource conservation, recycling, and waste reduction programs within the university; (6) Engage government (at all levels), foundations, and industry in supporting university research, education, policy formation, and information exchanges on sustainable development; extend this work to non-governmental organizations (NGOs) and thus find more comprehensive solutions to environmental problems; (7) Bring together environmental professionals to develop research programs, policy formation, and information exchanges to achieve an environmentally sustainable future; (8) Partner with elementary and middle schools to train their teachers to teach population, environment, and sustainable development issues; (9) Work with the United Nations Conference on Environment and Development, UNCED, the United Nations Environment Program, UNEP and other national and international organizations to promote a worldwide university effort that leads to a sustainable future; and (10) Establish a Steering Committee and Secretariat to continue this initiative and to inform and support each other in fulfilling this Declaration.
The World Conference on Education [4] also represents a clear commitment insofar as it states that:
“The fundamental missions of higher education systems (namely, to educate, train, conduct research and, in particular, contribute to sustainable development and the betterment of society as a whole) must be preserved, reinforced, and further promoted, especially in order to train highly qualified graduates and responsible citizens and to constitute an open space that promotes higher education and lifelong learning. In addition, higher education is performing unprecedented functions in today’s society, as an essential component of cultural, social, economic and political development, and as a key element in the strengthening of endogenous capacities, the consolidation of human rights, sustainable development, democracy and peace, within a framework of justice. Higher education must ensure that the values and ideals of the culture of peace prevail.”
[4] (p. 5).
However, the incorporation of these policies and strategies seems to have remained mere ‘good intentions’. Almost 20 years ago, [5] analyzed the evolution since 1990 of the statements on sustainability in higher education documents and concluded that they were a starting point in the sense that they acted as a utopian reference or a reminder of what is relevant. but they did not constitute a solid, operationalized proposal. Advances in issues such as the creation of an interdisciplinary curriculum that incorporates sustainability transversally, the ethical and moral responsibility of the university to contribute to sustainability, and social, and scientific transfer policies promoting open access or cooperation between institutions so that education for sustainable development is on the agendas should be able to be planned and evaluated with clear indicators. However, Wright feared that the statements were still promulgations of good intentions. The recent Berlin Declaration [6] represents a clear commitment to education for sustainable development, but we will have to wait to see if concrete actions are consolidated.
Within the Spanish state, the Conference of Rectors of Spanish Universities [7] also offered guidelines for institutional policies and educational practices in order to consider sustainability in teaching, but which have probably not progressed as expected.
Higher education institutions can make relevant contributions and commit to sustainability through: (a) Being sustainable and healthy universities themselves; (b) Contribute with their policies and strategies to the development of sustainable development objectives; (c) Train professionals in skills that allow them to respond to environmental and social needs, from a perspective that incorporates sustainability.
However, some studies indicate that the promotion of sustainability is not developing as would be desirable. [8] confirms this lack of consideration, especially in Latin America:
“The challenge of incorporating, introducing or integrating the dimensions of the SDGs in the university curriculum still does not seem sufficient (Albareda, et al., 2019; Paletta and Bonoli, 2019; Leal Filho et al., 2019; Valderrama Hernández et al., 2019; HESI, 2019). The development of sustainability competencies addressing the SDGs from a transversal or specific nature (Serrate et al., 2019; Segalàs and Sánchez, 2019) has complex implications that range from consideration of global and institutional purposes to aspects of a more general nature particular or technical that involve the expectations and actions of the main actors in the university education process, such as students and teachers?”
[9] (p. 93).
Within the classification established by [10], GUNI itself accepts that perhaps we would still be only in the first stage.
1.2. Higher Education Study Plans: Sustainability as a Transversal Competence
Sustainability has to do, on the one hand, with the very structure of study plans, which must have a flexible architecture that overcomes disciplinary fragmentation and systematically integrates changing social and professional demands, making degrees sustainable. On the other hand, cross-cutting content related to sustainability must be included in study plans. However, sustainability has not been present at the same level as other competences.
Competences can be defined as a combination of knowledge, skills, and attitudes appropriate to the context [11]. There are specific competences for each discipline and there are cross-curricular competences, which are useful for professional development and lifelong learning in all the knowledge areas. In this sense, the sustainability competency seems very important, but it was not initially included in Tuning Europe [12], although it is true that it could be linked to some of the initial 30 competences, such as ethical commitment or appreciation of diversity and multiculturalism. It does not appear in Tuning Latin America either, although it is implicit in several of the 27 competences, such as “Social responsibility and citizen commitment”, “Ability to identify, pose and solve problems”, “Commitment to the preservation of the environment”, “Commitment to their socio-cultural environment”, “Appreciation and respect for diversity and multiculturalism”, or “Ethical commitment”. This lack of explicit consideration of the initial competency frameworks of reference may have caused the delay in their incorporation into the curricula.
In the case of Spain, for example, the government agency ANECA produced a series of white papers for the development of competence-based university curricula. Reviewing the dozens of existing books, the competence of sustainability as such is practically only found in the studies of Chemical Engineering and Agricultural and Forestry Engineering. It also appears partially in Building Engineering (linked to the sustainability of buildings) and in Materials Engineering and Architecture, indirectly, as one of the necessary types of knowledge, as well as in Environmental Sciences, where it is considered a key content. There is, therefore, a reductionist vision of sustainability, closely linked to environmental sustainability. Economic or social sustainability are not considered.
Despite these difficulties, over time sustainability has been taken into account by many higher education institutions as a reference for the design of their institutional documents of cross-curricular, generic or transversal competences, which indicate the stamp that each institution wants to impress on its graduates. Finally, at the level of teaching, sustainability has to do with an inclusive teaching approach, which attends to the diversity of the participating students, applying the principles of universal design, valid and sustainable for all, with the alignment between discourse and practical activity.
“Sustainability competency is defined as the interlinked set of knowledge, skills, attitudes, and values that enable effective, embodied action in the world with respect to real-world sustainability problems, challenges, and opportunities, according to the context.”
[13] (p. 2).
Like all competences, it has to be embedded in the subjects, systematically incorporated into the tasks and assessment criteria, and intertwined with the content [13].
1.3. The Assessment of Transversal Competences
The link between assessment and sustainability can be approached from two perspectives: the way in which assessment is part of the competence-based curriculum design and the conceptualization of assessment itself.
In the first place, the development and acquisition of a competence is linked both to the type of learning tasks and to the assessment that is practiced. The learning tasks (and their assessment) have to be authentic, contextualized, real and integrate the competences to be promoted [14]. In other words, work with projects, problems, challenges, or cases must incorporate aspects that can be worked on and resolved by activating transversal skills. In fact, the quality of tasks has been revealed to be one of the elements that best explains competence development. In this sense, proposing challenging, deep, and transferable tasks that involve various communication skills on the part of the participating students (Table 1) is associated with a higher degree of competence development. On the other hand, the assessment process must be aligned with the tasks and include substantive assessment criteria that inform the development of competences [15]. In this sense, keeping in mind criteria that report on the sustainability of a proposal is essential.
Secondly, the conception of the assessment itself can suppose a value more or less close to sustainability. The already classic study by [17] allowed the expression “sustainable assessment” to be coined to refer to assessment as a capacity embedded in the day-to-day life of graduates and future professionals. It is the assessment “that meets the needs of the present and (also) prepares students to meet their own future learning needs” [17] (p. 151). For Carless and Boud [18], it is about the ability to judge the quality of one’s own work and that of others honestly and to act accordingly, seeking the resources to improve said executions. This requires evaluative judgment, which must be sought intentionally so that, at the end of the studies, all students are capable of assessing whether their learning processes and products have sufficient quality and/or deserve to be reviewed [19]. For this to occur, the feedback must not be given in a unidirectional way by the teachers and must be focused on the quality of the information given to the student so that it covers the gap between how an execution has been carried out and how the ideal execution would have been. Feedback must focus on the action carried out by participating students to give meaning to the information received and to decision making on subsequent actions, thus consolidating the idea of sustainable assessment, embedded into the student’s own profile [20].
In this sense, if sustainability competence was important, in general, in times of pandemic it took on a particularly relevant meaning [21]. The centers were closed, but education continued to be provided, moving to hybrid modalities. The digital divide, the difficulties in the digital training of teachers, and the lack of institutional infrastructure were some of the obstacles that could compromise teaching. Therefore, it was necessary to look for new formulas and think about their sustainability, given the lack of knowledge of the duration and intensity of the pandemic.
At this time, it also became clear that not only the content linked to the various subjects are important, but that the transversal competences (emotional, digital, linguistic) can constitute sets of skills with which to respond to particularly difficult situations, as was the confinement. In this sense, the competence-based designs made more sense, if possible, and the need to develop a sustainable assessment and strengthen evaluative judgment through assessment practices participated in by the participating students became more important.
1.4. Competence Assessment in Online Learning Environments
The main benefits usually attributed to online assessment are related to automatized processes, data management optimization or faster marking [22]. During COVID-19 the sudden change to online environments made teaching staff focused on preventing cheating [23] or avoiding a digital gap [24]. Nevertheless, the online assessment could be used for learning to be enhanced taking advantage of: (a) immediate feedback; (b) flexibility; (c) monitoring possibilities; and (d) autonomy to be fostered [25].
These specific challenges can be highlighted for competence-based online assessment practices:
Assessment strategies and tools should be used with a specific formative purpose in order for student progress in complex tasks to be monitored and dropout be avoided. In this sense, designing loops [18] for assignments to be progressively delivered and designing authentic tasks [16] help in a more authentic assessment [26].
The immediate feedback could help decision-making by teachers [27] as well as students, fostering their agency [28], which is a key feature of the reflective dimension of achieving competences.
There is the possibility of customizing the pace and level of assessment assignments, as well as the kind of outputs (including several format files), achieving a more inclusive evaluation [29]. This is also important because competence-based assessment tasks are open tasks and it is accepted that there are several ways for a task be done, due to the interest and cognitive style of each student.
To provide iterative opportunities for the competences be developed and for tracking the progress could be supported by technologies, which offer the possibility of collecting, keeping and managing data for decision-making by teachers as well as by students. Nevertheless, the challenge is not about technological tools but about digital literacy [30,31] and assessment literacy [32,33].
For this reason, we ask ourselves what assessment was made of the competences during this period, investigating both from the perspective of teachers and students in: (a) what assessment strategies were considered most useful and which were the most used to evaluate each of the transversal competences of the University of Barcelona; and (b) what were the characteristics that were associated with good assessment practice at this time?
2. Methodologies
A descriptive study has been conducted following two research questions (RQ):
RQ1.. What were the most useful and most used assessment strategies in the UB during the COVID-19 pandemic?
RQ2.. What were the main characteristics of the assessment strategies at the UB during COVID-19?
These 2 RQs led to 6 specific objectives: (1) to describe the assessment practices mostly used in line with the generic competencies from the perspective of the teachers and the students; (2) to know the main purposes of the assessment practices carried out in lockdown-forced online teaching environments; (3) to analyze the characteristics of the proposals that both teachers and students consider most useful to develop generic competencies; (4) to explore how and for what purpose both teachers and students use Learning Analytics resources available on the Virtual Campus; (5) to identify the use that teachers make of digital tools for competence assessment; and (6) to identify the perception of the usefulness of digital tools for the assessment of competencies by students.
All students and teachers from a selection of faculties of the University of Barcelona during the 2020–2021 academic year were invited to participate in the study. It has been considered necessary to have data on the various areas of knowledge that the University of Barcelona has, given that the various fields of knowledge may have their own assessment strategies, instruments and characteristics [34,35]. To guarantee the viability of the research, the research team includes members from various faculties of the University of Barcelona (Law, Pharmacy, Mathematics, History, Information and Audiovisual Communication, Psychology and Education), thus considering the diversity of areas of knowledge.
In the case of the University of Barcelona, among its 6 transversal competences it included sustainability, understood as the ability to assess the social and environmental impact of actions in its field and the ability to express integrated and systemic visions [36]. This meant a commitment to sustainability, placing it at the same level as teamwork, communication or ethical commitment.
A total of 394 subjects responded: 265 students (61.9 female, 35.8% male, 1.1% non-binary and 1.1% No answer/don’t know) and 129 teachers (61.2% female and 38.0% male). The response rate is low, but acceptable in the context affected by the pandemic. Table 2 and Table 3 present a basic description of these samples made up of teachers and students from various degree courses.
In the case of students, these come from the degrees of Mathematics (26.4%), Primary Education (19.2%), Pharmacy (18.1%), Computer Science (10.6%), Archeology (9.4%), Management and Public Administration (7.9%), Audiovisual Communication (5.3%) and Psychology (3.0%). For its part, the teachers’ sample is made up of teachers from the faculties of Education (33.3%), Psychology (16.3%), Geography and History (15.5%), Pharmacy (14.7%), Mathematics (8.5%), Law (7.0%) and Information and Audiovisual Media (4.7%). The teachers also indicated having previous experience of online teaching prior to the pandemic in 27.91%, while, for the students, declared previous experience drops to 20.38%.
2.1. Instrument
Once the theoretical framework was established, two ad hoc questionnaires were designed to explore the opinions of students and teachers. The information was collected online during the month of March 2021 through a questionnaire with differentiated versions for students and teachers. The answers collected refer to the second semester of the 2019–2020 academic year, and the first and second semester of the 2020–2021 academic year (during the application of the questionnaire). For its construction and application, all responsible research and innovation procedures were followed, as well as the Code of Good Practices in Research of the University of Barcelona; informed consent was requested from all participants; confidential data were stored on secure devices, and the results were returned to the participants.
A first section of the questionnaire collected informed consent and demographic interest data (grade and previous experience with online teaching-learning). Next, 21 Likert-type items were presented to investigate the perceptions of students and teachers on the following aspects (see Table 4):
Quality characteristics of the given evaluative practices. Nine characteristics are considered according to the previous theoretical review (range 1 to 5: 1 ‘not at all frequent’ and 5 ‘very frequent’).
Assessment strategies and instruments. The first one considered is methods teachers use to evaluate their students’ progress and plan the content in their courses, and the second is the documented activities developed to support the assessment method and used to collect the evidence of student competence. Twelve characteristics are considered according to the previous theoretical review (range 1 to 5: 1 ‘not at all frequent’ and 5 ‘very frequent’ and degree of utility).
The questionnaire includes closed multiple-choice questions to collect the opinion of the respondents regarding the degree of usefulness of the different dimensions for the development of transversal skills: ethical commitment (C1), learning capacity (C2), teamwork (C3), creative capacity (C4), sustainability (C5), and communication capacity (C6); but, particularly, in this contribution we present the results related to the transversal competence of sustainability (C5). In addition, each of the dimensions will be assessed based on the degree of frequency or use by teachers and students during the period of mixed teaching caused by the pandemic.
2.2. Data Collection and Analysis Procedure
Quantitative data collected were analyzed with the GraphPad Prism software package. In the first place, a descriptive exploration of the collected quantitative data was carried out to see how they behave (sample distribution), in order to analyze later how the results are distributed (minimum, maximum, mean, median, standard deviation), according to the perception of students and teachers.
Semi-structured interviews were carried out with teachers and students. A semi-structured interview is suggested based on a guide of start issues or questions, offering the interviewer the freedom to introduce additional questions to clarify concepts and obtain more information, adjusting the interview flow to the interviewees’ pace [37].
Specific interview scripts for students and teachers were designed grounded in the theoretical framework. The scripts started with preliminary questions to help the participants understand the objects of the study. The whole interdisciplinary research team contributed to the review process of the script’s design. Validation and contextual adjustment of the final scripts were warranted by sending items to one professor from each Degree (Pharmacy, Archaeology, Primary Education, Computer Engineering, Mathematics, Audio-visual communications and media studies, Psychology, Biology, Management and Public Administration) asking them to provide feedback to ensure intelligibility.
The interview script had different open questions (Table 5). These questions are related to the specific objectives of the research. Primary demographic and identification data were gathered from the participants: sex, degree and course (in the case of students); gender, years of teaching experience, degree/s, previous experience to COVID-19 with online teaching (in the case of teachers).
The interviews were carried out in May 2021. All interviews were conducted online through a video conference with Blackboard Collaborate (as provided on the virtual campus). The interviews were recorded, transcribed and sent back to the interviewees for the content’s validation.
The authors proceeded to a thematic analysis of the transcribed interviews [38] following the constant comparison model of Guba & Lincoln [39]. By constantly comparing ideas expressed in the interviews, analysts identified codes, which were grouped into categories. All co-authors were equally implied in the process of analysis, contrasting results until reaching full consensus of discrepancies. The content analysis happened in two basic steps. First, two researchers categorized open data according to classification criteria through peer-review. The third researcher was then responsible for reviewing these categorizations and obtaining a definitive classification in the event of discrepancies between the two start researchers. Second, a frequency count was done for each code. This frequency, as well as some paragraphs of these transcribed interviews (anonymized and identified with a letter of the degree they belong to and a number), will be shown below.
3. Results
As the main general result, we report the globally positive satisfaction of both groups with assessment practices in mixed teaching, given that the average of teachers is M = 3.6 (SD = 0.9) and of students M = 3 (SD = 1.1).
It is interesting to note that, in the interviews carried out, when referring to assessment and teaching in times of pandemic, neither the participating students nor the teachers did not allude to competences on their own initiative. 62% of teachers only referred to skills after being asked by the interviewer, especially regarding teamwork and communication.
“The competence of teamwork, I think that in general a lot of work is done (…) The ethical commitment, the critical and self-critical capacity in general, we have also tried to develop them. (…) but sustainability, I would say explicitly, no (…) It would not be the competition that would stand out the most.”
(EP15).
The results are then presented, broken down by study objectives and combining the information obtained from questionnaires and the interviews.
3.1. Characteristics of Assessment Practices
The results show that teachers perceive certain characteristic features of good assessment practices more frequently than students (see Figure 1 and Figure 2), in accordance with their responsibility for design and implementation.
As we can see, sustainability competence is the transversal competence valued less frequently in the nine characteristics identified as good practices of competence assessment aligned to formative assessment.
According to the interviews carried out, it is also confirmed that certain competences, such as sustainability, do not appear.
“A lot of the issue of teamwork or communication skills as well. For example, I don’t think they’ve worked on sustainability. Anyway, at least, I didn’t see it.”
(EC,12).
There is even some ignorance:
“Sustainability… I’m not sure how to refer to that.”
(EI,4).
There are also exceptions:
“Also, the competence of sustainability. I think that is also important, because we have realized that you don’t have to spend so much paper, for example, so many sheets, and it is not the same thing, but a lot of things can be done online or on your computer. And I also think that is very important, because you can also avoid unnecessary travel that involves CO2 emissions and everything. So, bottom line is that we’re really looking forward to it.”
(Eb,7).
Specifically, both groups agree that: 1—The assessment activities are productive (creative, application, design, decision-making…), this is the characteristic with the greatest link to the sustainability competence, but there are discrepancies in relation to the feature with the least link. Although teachers indicate that: 9—Students can use the support of various technological tools to give and receive feedback (7.0%) as the least frequent feature; the students contribute that: 2—The assessment activities are coherent with the competences of the Degree and the objectives of the subjects (8.3%); and 5—The students participate in self-assessment activities (8.3%).
Likewise, it is interesting to analyze that there are relevant differences between groups in other characteristics regarding the competence of sustainability, such as in 2— the assessment activities are consistent with the competences of the Degree and the objectives of the subjects (11.1 points of difference); and 7—Students have the opportunity to integrate the feedback received in future tasks or versions of the same task (6.8 points of difference), valued more frequently by the group of teachers.
In particular, the aforementioned competence shows an average of 14.7% on the part of the teachers and 9.8% on the part of the participating students (see Figure 3); while the other competences are in a range between 21.5% and 44.2% on average.
3.2. Assessment Strategies and Instruments
In the first place, we present the results, according to the perspectives of teachers and students, regarding the most frequently used assessment strategies and instruments during the period of mixed teaching, and their perception about their usefulness for the development of the transversal competence of sustainability. Secondly, a comparison is shown with what corresponds to the characteristics of the assessment practices, and perceptions about their usefulness for the development of the transversal competence of sustainability.
As we can see (Table 6), according to the perception of the two participating groups (teachers and students), there are coincidences in the assessment strategies and instruments most frequently used during the mixed teaching period: 7—Works and projects (M = 4.2, SD = 1.2 and M = 3.4, SD = 1.5, respectively) and 5—Long-answer tests (M = 3.2, SD = 1.5 and M = 3.3, SD = 1.3). However, teachers also indicated 9—Real and/or simulated performance assessment (M = 3.2, SD = 1.6); while participating students mentioned 3—Objective tests (M = 3.4, SD = 1.3). However, both 7—Works and projects and 9—Real and/or simulated performance assessment are the two items that show a higher mean difference (0.8 points), scored more frequently by teachers. On the other hand, the objective 3—Tests are the assessment strategy highlighted with the greatest difference between both groups (0.6 points), valued as more useful by the participating students than by the teachers.
On the other hand, there is also concurrence in those assessment strategies and instruments indicated as the least used (Table 7): 10—Attitude scales to collect values or social skills (M = 1.6, SD = 1.0 and M = 1.5, SD = 1.0 respectively), 11—Observation techniques with records, checklists… (M = 1.8, SD = 1.2 and M = 1.6, SD = 1.0) and 12—Portfolio or learning folder (M = 2.0, SD = 1.5 and M = 1.8, SD = 1.2).
On the other hand, the teachers indicates that 7—Works and projects is the strategy with the greatest degree of utility for the development of the transversal competence of sustainability, with 30.2% (15.5% by the participating students). This characteristic is where the greatest discrepancy is evident between the perspective of the teachers and the participating students with a difference of 14.7 points, followed by 12—Portfolio or Learning Folder (11.1 points), scored as more useful by teachers.
Both groups also show some agreement: 10—Scales of attitude to collect values or social skills are valued as useful to contribute to the development of the aforementioned competence with 18.6% by teachers and 23.0% according to students; followed by 9—Real and/or simulated performance assessment (18.6% and 18.1% respectively, the item that shows fewer points of difference between groups).
With this, there is also evidence of a discrepancy in relation to the frequency and usefulness, since the 10-Scales of attitude to collect values or social skills are valued as infrequent but, in turn, useful, both according to the perception of the teachers and the participating students. In addition, 9—real and/or simulated performance assessment are considered as frequent by teachers, but useful by both.
4. Discussion
In this work, results have been presented on the perception of a sample of professors and students at the University of Barcelona of the evaluative practices carried out during the period of mixed teaching affected by the pandemic, specifically the results related to the transversal competence of sustainability. This perception also refers in a certain way to the conceptions of some regarding evaluation.
The results show a scant assessment of the sustainability competence, and, in general terms, it seems that there is scant consideration of the competency itself and of opportunities for its development.
In the first place, if the nine characteristics of good assessment practice are analyzed (in the sense of training, participation by the participating students, transparency), sustainability is once again the transversal competence least linked to all the characteristics. In fact, when asked about the characteristics of good assessment practice, all of them were very weakly linked to the development of the sustainability competence (with an average of 10.4%). Sustainability does not appear as a competency aligned with learning objectives or the object of formative assessment practices, suggesting that there is still a long way to go in terms of teaching and learning this competency.
Second, regarding its frequency, students report that the sustainability competency is the least evaluated, regardless of the assessment strategy used. Their linkage to the different strategies is very low in all cases. According to teachers, it is also the competence that claims to have been evaluated the least insofar as it is the least linked to all assessment strategies. This fits with what was found [8] and may be due to a lack of teacher training regarding both sustainability and assessment. This scant evaluative literacy affects not only the competence of sustainability, and it is not exclusive to the teachers either, so that a greater training of teachers and students, in order to develop an assessment that is more sustainable in itself (following [17]), would benefit the development of skills and the graduation profile.
Thirdly, of the assessment strategies and instruments that are most closely related to this competence, ‘Works and projects’ clearly stand out. This strategy is the most used, both in the opinion of teachers and students. At the same time, it is the strategy that the teachers consider to be the most useful for promoting and evaluating competence in sustainability. However, when referring to usefulness, the assessment presents great discrepancies: 30.2% of teachers consider it useful compared to 15.5% of students. This leads to the question of what kind of work is included in the label ‘Works and projects’. Project work is probably more aligned with sustainability than some monographic works. Further research would be necessary to obtain more data regarding the conceptualization of teachers and students about what ‘Works and projects’ incorporates and to what extent they meet the characteristics indicated [16].
For the participating students, the most useful factor to evaluate the sustainability competence are the attitude scales (23.0%), while for the teachers, information about the development of said competence can practically be collected with attitude scales, reports/memoirs, and performance tests and execution of real tasks (the three strategies and instruments with a representation of 18.6%). However, it is admitted that the attitude scales have hardly been used (1.6 and 1.5 on average in both groups), so that said competence is simply no longer evaluated. In addition, in relation to conceiving it as an attitude, although it is true that sustainability can incorporate an attitudinal dimension, it can also imply conceptual learning and skills, abilities or procedures. Any of these can constitute an object of learning and development, including the attitudinal dimension, and must be evaluated in order to guide students to become aware of its importance [19] and to incorporate it in all their proposals for academic work and future professional work [9]. This refers to the importance of having assessment criteria embedded in rubrics or checklists that are used as records to evaluate the quality of a job [15]. It also leads to the need to carry out activities of appropriation of the criteria by the participating students, which help them to give meaning to the principles and characteristics of social, economic, and environmental sustainability and to incorporate them into their tasks as an element that has to do with their quality in a progressively autonomous way, as suggested by [19].
These activities aligned with sustainability competence should be carried out face-to-face and in blended or online learning environments. It seems important to highlight the benefits of assessment enhanced by digital technologies [26,29], which has been analyzed during the COVID-19 pandemic [24,25].
The collected data not only describe what happened during the pandemic but also seem to be consistent with previous findings stated in the literature. The use of digital technologies for marking for a better managed and faster return, and not for an authentic pedagogical innovation, was stated [22]. The need for assessment literacy has been systematically claimed [18,40]. Therefore, the findings during the pandemic appear to be quite similar but a deeper understanding of the underlying assessment processes has been achieved.
5. Conclusions
This work represents a first exploratory approximation of the evaluative practices carried out in mixed teaching during a pandemic in a basically ‘face-to-face’ institution with regard to the transversal competence of sustainability. As a whole, the results outline that it seems necessary to strengthen the evaluative literacy of the teachers so that they understand and participate in the purpose of a formative assessment and can put into practice assessment strategies that are progressively more participated in by the students, with more spaces for student participation in the assessment, with definition and reflection on their practices by themselves and the criteria that support them, which constitutes a challenge for the future in terms of strengthening evaluative literacy [18,41].
Learning experiences supported by adjusted technology have important implications for future hybrid teaching designs: finding a way to apply a more competence-based assessment supported by technology and aligned with competence-based university curricula and global and national educational agendas, such as the SDGs of the Agenda 2030 [2], continues to be a challenge for the future. It also draws implications for training in assessment and digital competence [24,25]. Assessment literacy is a must for the formative purpose to be considered and the complex task assessed with transparent criteria to be promoted, as well as a stronger students’ agency [28] to be fostered in the assessment processes.
The strengthening of sustainability and its integration in competence-based teaching practices depends on its integration as a criterion for assessing tasks and it still seems to be a pending challenge because results show that sustainability, as learning content, is weakly connected with formative assessment practices. Teachers and students consider attitude scales as the most useful assessment tool, but nobody uses these scales (in addition, considering them as an attitude and not a competence could affect their inclusion in the syllabus). Besides this, sustainability is not linked with the characteristics of formative assessment (students’ engagement with criteria and objectives; feedback to be included in further assignments; authentic assessment proposals) and digital technologies are not used for providing feedback and in current practices, Finally, projects and tests are common assessment assignments, but they are apparently disconnected from sustainability.
The main limitations of this study come from the composition and size of the sample and the contextual nature of the study. Future research would be necessary to understand the replicability and the reasons for these descriptive results beyond this specific institutional context, and for assessment practices not conditioned by the periods of mixed teaching triggered by the pandemic in relation to aspects related to sustainability, since this should not only be present at a theoretical level, but also in educational practices.
Conceptualization, E.C.G.; methodology, L.L.M.; software, L.L.M.; formal analysis, L.L.M.; writing—original draft preparation, E.C.G. and L.L.M.; writing—review and editing, E.C.G. and L.L.M.; funding acquisition, E.C.G. All authors have read and agreed to the published version of the manuscript.
Not applicable.
Informed consent was obtained from all subjects involved in the study.
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Enlarge this image.Figure 1. Characteristics of the formative assessment practices according to the transversal competences (from teachers’ perspective).
Enlarge this image.Figure 2. Characteristics of the formative assessment practices according to the transversal competences (from students’ perspective).
Enlarge this image.Figure 3. Average of the transversal competences linked with the characteristics of the formative assessment practices (from teachers’ and students’ perspective).
Constructs of a good assessment task. Source: Adapted from Ref. [
| Construct | Definition |
|---|---|
| Challenge | To design open and complex problems that require significant relationships and connections, decision-making, and solving authentic problems. |
| Depth | To use critical thinking in order to depth understanding to be shown. |
| Communication | To develop communication strategies in oral, written and symbolic products. |
| Transfer | To apply learning and experience in further courses, in professional performance, and in social reality. |
Reference courses. Frequency of students and teachers.
| Academic Course | Students Enrolled Course (%) | Teachers Higher Teaching Load (%) |
|---|---|---|
| 1st course | 77 (29.1) | 36 (27.9) |
| 2nd course | 70 (26.4) | 26 (20.2) |
| 3rd year | 49 (18.5) | 35 (27.1) |
| 4th grade | 69 (26.0) | 32 (24.8) |
Years of teaching experience.
| Teaching Experience | Teachers Years (%) |
|---|---|
| Up to 10 years | 42 (32.6) |
| Between 11 and 20 years of age | 27 (20.9) |
| Between 21 and 29 years old | 34 (26.3) |
| 30 years or older | 26 (20.2) |
Description of the dimensions of the questionnaire.
| Dimensions | Items (%) |
|---|---|
| 9 Characteristics Of Assessment Practices: | 15.52% |
| 1-Assessment activities are productive (creative, application, design, decision making…). | |
| 2-The assessment activities are consistent with the competences of the Degree and the objectives of the subjects. | |
| 3-The students participate in the definition and understanding of the objectives of the task. | |
| 4-Students participate in the definition and understanding of the assessment criteria of the task. | |
| 5-Students participate in self-assessment activities. | |
| 6-Students participate in peer assessment activities. | |
| 7-Students have the opportunity to integrate the feedback received in future tasks or versions of the same task. | |
| 8-Students can reflect on the feedback received. | |
| 9-Students can use the support of various technological tools to give and receive feedback. | |
| 12 Assessment Strategies Or Instruments: | 22.68% |
| 1-Self-assessment activities | |
| 2-Peer assessment activities | |
| 3-Objective tests | |
| 4-Short answer tests | |
| 5-Long answer tests | |
| 6-Oral tests | |
| 7-Works and projects | |
| 8-Reports/memories… | |
| 9-Real and/or simulated performance assessment | |
| 10-Attitude scales to collect values or social skills | |
| 11-Observation techniques with records, checklists… | |
| 12-Portfolio or Learning Folder |
Interview scripts’ dimensions aligned with the specific targets.
| Specific Objectives | Dimensions of the Students’ |
Dimensions of the Teachers’ |
|---|---|---|
| To describe the assessment practices mostly used in line with the generic competences from the perspective of the teachers and the students. | Type of assessment tasks undertaken. | Assessment tasks proposed. |
| Differences between the assessment task before and during the COVID-19 period. | Differences between the assessment task before and during the COVID-19 period. | |
| Knowledge about generic competences. | Information transferred to students about generic competences at the beginning of the course. | |
| Relationship between the assessment tasks and the development of generic competences. | Work on generic competences. | |
| To know the main purposes of the assessment practices carried out in lockdown-forced online teaching environments. | Main purposes of the assessment practices developed. | Main purposes of the designed assessment practices. |
| Differences between main purposes of the assessment practices before and during COVID-19 period. | Differences between main purposes of the assessment practices before and during COVID-19 period. | |
| Main concerns about the online assessment process. | Main concerns about the online assessment process. | |
| To analyze the characteristics of the proposals that both teachers and students consider most useful to develop generic competences. | Description of the different assessment tasks performed during the mixed teaching period. | Description of an assessment task considered especially good and successful during the mixed teaching period, and why it is considered most useful to develop generic competences. |
| Characteristics of the assessment tasks developed considered most useful to develop generic competences. | Characteristics of the assessment tasks designed considered the most useful to develop generic competences. | |
| To explore how and for what purpose, both teachers and students use Learning Analytics resources available on the Virtual Campus. | Knowledge of Learning Analytics. | Knowledge about Learning Analytics in Moodle–Virtual Campus and/or in external tools. |
| Use of Learning Analytics. | Use of Learning Analytics. | |
| Purpose of Learning Analytics used. | Purpose of Learning Analytics used. | |
| Usefulness of Learning Analytics. | ||
| To identify the use that teachers make of digital tools for competence assessment. | List of digital tools used for competence assessment. | |
| Digital tools (from Moodle–Virtual Campus or external pages) considered most useful for assessing generic competences. | ||
| To identify the perception of the usefulness of digital tools for the assessment of competences by students. | Digital tools (from Moodle–Virtual Campus and from external pages) considered most useful for developing and for assessing generic competences. |
Assessment strategies and instruments: how often they have been used to assess sustainability competence.
| Teachers | Students | |||
|---|---|---|---|---|
| Assessment Strategies and Instruments | M | SD | M | SD |
| 1-Self-assessment activities | 2.8 | 1.4 | 2.4 | 1.3 |
| 2-Peer assessment activities | 2.4 | 1.4 | 2.2 | 1.2 |
| 3-Objective tests | 2.8 | 1.6 | 3.4 | 1.3 |
| 4-Short answer tests | 2.8 | 1.4 | 2.7 | 1.3 |
| 5-Long answer tests | 3.2 | 1.5 | 3.3 | 1.3 |
| 6-Oral tests | 2.9 | 1.6 | 2.2 | 1.3 |
| 7-Works and projects | 4.2 | 1.2 | 3.4 | 1.5 |
| 8-Reports/memories… | 3.0 | 1.6 | 2.8 | 1.5 |
| 9-Real and/or simulated performance assessment | 3.2 | 1.6 | 2.4 | 1.4 |
| 10-Attitude scales to collect values or social skills | 1.6 | 1.0 | 1.5 | 1.0 |
| 11-Observation techniques with records, checklists… | 1.8 | 1.2 | 1.6 | 1.0 |
| 12-Portfolio or Learning Folder | 2.0 | 1.5 | 1.8 | 1.2 |
| Overall average | 2.7 | 2.5 | ||
Assessment strategies and instruments: degree of usefulness of the development of the competence of sustainability.
| Teachers | Students | |
|---|---|---|
| Assessment Strategies and Instruments | Degree of Usefulness (%) | Degree of Usefulness (%) |
| 1-Self-assessment activities | 14.7 | 8.3 |
| 2-Peer assessment activities | 7.8 | 9.4 |
| 3-Objective tests | 6.2 | 4.2 |
| 4-Short answer tests | 10.1 | 3.4 |
| 5-Long answer tests | 14.0 | 4.9 |
| 6-Oral tests | 11.6 | 5.3 |
| 7-Works and projects | 30.2 | 15.5 |
| 8-Reports/memories… | 18.6 | 8.3 |
| 9-Real and/or simulated performance assessment | 18.6 | 18.1 |
| 10-Attitude scales to collect values or social skills | 18.6 | 23.0 |
| 11-Observation techniques with records, checklists… | 14.7 | 9.4 |
| 12-Portfolio or Learning Folder | 17.1 | 6.0 |
| Overall average | 15.2 | 9.7 |
References
1. UNESCO. United Nations Decade of Education for Sustainable Development (2005–2014): International Implementation Scheme; UNESCO: París, France, 2005.
2. United Nations. Sustainable Development Goals. Agenda 2030; UN: New York, NY, USA, 2015.
3. University Leaders for a Sustainable Future. The Talloires Declaration; ULSF: Talloires, France, 1990; Available online: http://ulsf.org/wp-content/uploads/2015/06/Spanish_TD.pdf (accessed on 12 April 2022).
4. UNESCO. Higher Education in the Twenty-first Century Vision and Action. Proceedings of the World Congress on Higher Education; Paris, France, 5–9 October 1998; UNESCO: París, France, 1998; Available online: https://unesdoc.unesco.org/ark:/48223/pf0000117022_eng (accessed on 15 May 2022).
5. Wright, T. The Evolution of Sustainability Declarations in Higher Education. Higher Education and the Challenge of Sustainability; Corcoran, P.B.; Wals, A.E.J. Springer: Cham, Switzerland, 2004; pp. 7-19. [DOI: https://dx.doi.org/10.1007/0-306-48515-X_2]
6. UNESCO. Berlin Declaration on Education for Sustainable Development; UNESCO: Berlin, Germany, 2021.
7. Council of Rectors of Spanish Universities. Guidelines for the Introduction of Sustainability in the Curriculum; Council of Rectors of Spanish Universities CRUE: Madrid, Spain, 2005; Available online: https://www.crue.org/wp-content/uploads/2020/02/Guidelines_Sustainability_Crue2012.pdf (accessed on 14 April 2022).
8. Miranda, L.F.; Sánchez Buitrago, J.O.; Viloria, J. Environmental Sustainability in Higher Education: Mapping the Field. Electron. J. Educ. Res.; 2021; 23, e09. [DOI: https://dx.doi.org/10.24320/redie.2021.23.e09.4053]
9. Ramos Torres, D.I. Contribución De La educación Superior a Los Objetivos De Desarrollo Sostenible Desde La Docencia. Revista española de Educación Comparada; 2020; pp. 89-110. Available online: https://hdl.handle.net/11162/206804 (accessed on 2 May 2022).
10. Leal Filho, W. Sustainability at Universities: Opportunities, Challenges and Trends; Peter Lang Scientific Publishers: New York, NY, USA, 2010.
11. European Commission. Key Competences for Lifelong Learning; Publications Office of the European Union: Brussels, Belgium, 2019.
12. González, J.; Wagenaar, R. Tuning Educational Structures in Europe; Universidad de Deusto: Bilbao, Spain, 2014; Available online: http://tuningacademy.org/wp-content/uploads/2014/02/TuningEUI_Final-Report_SP.pdf (accessed on 20 April 2022).
13. Bianchi, G. Sustainability Competences. A Systematic Literature Review; Publications Office of the European Union: Luxemburg, 2020.
14. Fook, C.Y.; Sidhu, G.K. Authentic Assessment and Pedagogical Strategies in Higher Education. J. Soc. Sci.; 2010; 6, pp. 153-161. [DOI: https://dx.doi.org/10.3844/jssp.2010.153.161]
15. Dawson, P.; Carless, D.; Lee, P.P.W. Authentic Feedback: Supporting Learners to Engage in Disciplinary Feedback Practices. Assess. Eval. High. Educ.; 2021; 46, pp. 286-296. [DOI: https://dx.doi.org/10.1080/02602938.2020.1769022]
16. Ibarra-Sáiz, M.S.; Rodríguez-Gómez, G. Evaluando la evaluación. Validación mediante PLS-SEM de la escala ATAE para el análisis de las tareas de evaluación. RELIEVE; 2020; 26, [DOI: https://dx.doi.org/10.7203/relieve.26.1.17403]
17. Boud, D. Sustainable Assessment: Rethinking Assessment for the Learning Society. Stud. Contin. Educ.; 2000; 22, pp. 151-167. [DOI: https://dx.doi.org/10.1080/713695728]
18. Carless, D.; Boud, D. The Development of Student Feedback Literacy: Enabling Uptake of Feedback. Assess. Eval. High. Educ.; 2018; 43, pp. 1315-1325. [DOI: https://dx.doi.org/10.1080/02602938.2018.1463354]
19. Tai, J.; Ajjawi, R.; Boud, D.; Dawson, P.; Panadero, E. Developing Evaluative Judgement: Enabling Students to Make Decisions about the Quality of Work. High. Educ.; 2018; 76, pp. 467-481. [DOI: https://dx.doi.org/10.1007/s10734-017-0220-3]
20. Lodge, J.M.; Panadero, E.; Broadbent, J.; de Barba, P.G. Supporting self-regulated learning with learning analytics. Learning Analytics in the Classroom; Routledge: Oxfordshire, UK, 2018; pp. 45-55. [DOI: https://dx.doi.org/10.4324/9781351113038-4]
21. Navarro-Espinosa, J.A.; Vaquero-Abellán, M.; Perea-Moreno, A.-J.; Pedrós-Pérez, G.; Aparicio-Martínez, P.; Martínez-Jiménez, M.P. The Higher Education Sustainability before and during the COVID-19 Pandemic: A Spanish and Ecuadorian Case. Sustainability; 2021; 13, 6363. [DOI: https://dx.doi.org/10.3390/su13116363]
22. Bhagat, K.K.; Spector, J.M. International Forum of Educational Technology & Society Formative Assessment in Complex Problem-Solving Domains: The Emerging Role of Assessment Technologies. J. Educ. Technol. Soc.; 2017; 20, pp. 312-317.
23. Noorbehbahani, F.; Mohammadi, A.; Aminazadeh, M. A Systematic Review of Research on Cheating in Online Exams from 2010 to 2021. Educ. Inf. Technol.; 2022; [DOI: https://dx.doi.org/10.1007/s10639-022-10927-7] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35283658]
24. Baughan, P. Assessment and Feedback in a Post-Pandemic Era: A Time for Learning and Inclusion; Advance HE: London, UK, 2021.
25. Montenegro-Rueda, M.; Luque-de la Rosa, A.; Sarasola Sánchez-Serrano, J.L.; Fernández-Cerero, J. Assessment in Higher Education during the COVID-19 Pandemic: A Systematic Review. Sustainability; 2021; 13, 10509. [DOI: https://dx.doi.org/10.3390/su131910509]
26. Pitt, E.; Carless, D. Signature Feedback Practices in the Creative Arts: Integrating Feedback within the Curriculum. Assess. Eval. High. Educ.; 2021; pp. 1-13. [DOI: https://dx.doi.org/10.1080/02602938.2021.1980769]
27. Henderson, M.; Ajjawi, R.; Boud, D.; Molloy, E. The Impact of Feedback in Higher Education: Improving Assessment Outcomes for Learners; Springer: Cham, Switzerland, 2019; [DOI: https://dx.doi.org/10.1007/978-3-030-25112-3]
28. Winstone, N.; Pitt, E.; Nash, R. Educators’ Perceptions of Responsibility-Sharing in Feedback Processes. Assess. Eval. High. Educ.; 2021; 46, pp. 118-131. [DOI: https://dx.doi.org/10.1080/02602938.2020.1748569]
29. Tai, J.; Ajjawi, R.; Bearman, M.; Boud, D.; Dawson, P.; Jorre de St Jorre, T. Assessment for Inclusion: Rethinking Contemporary Strategies in Assessment Design. High. Educ. Res. Dev.; 2022; pp. 1-15. [DOI: https://dx.doi.org/10.1080/07294360.2022.2057451]
30. Sillat, L.H.; Tammets, K.; Laanpere, M. Digital Competence Assessment Methods in Higher Education: A Systematic Literature Review. Educ. Sci.; 2021; 11, 402. [DOI: https://dx.doi.org/10.3390/educsci11080402]
31. Wang, H.; Tlili, A.; Lehman, J.D.; Lu, H.; Huang, R. Investigating Feedback Implemented by Instructors to Support Online Competency-Based Learning (CBL): A Multiple Case Study. Int. J. Educ. Technol. High. Educ.; 2021; 18, 5. [DOI: https://dx.doi.org/10.1186/s41239-021-00241-6]
32. Ayalon, M.; Wilkie, K.J. Developing Assessment Literacy through Approximations of Practice: Exploring Secondary Mathematics Pre-Service Teachers Developing Criteria for a Rich Quadratics Task. Teach. Teach. Educ.; 2020; 89, 103011. [DOI: https://dx.doi.org/10.1016/j.tate.2019.103011]
33. Xu, Y.; He, L. How Pre-Service Teachers’ Conceptions of Assessment Change Over Practicum: Implications for Teacher Assessment Literacy. Front. Educ.; 2019; 4, 145. [DOI: https://dx.doi.org/10.3389/feduc.2019.00145]
34. De Miguel Díaz, M. Modalidades de Enseñanza Centradas En El Desarrollo de Competencias. Orientaciones Para Promover El Cambio Metodológico En El EEES; MEC/Universidad de Oviedo: Madrid, Spain, 2008.
35. Cano, E. Buenas Prácticas En La Evaluación Por Competencias. Cinco Casos; Laertes: Barcelona, Spain, 2011.
36. University of Barcelona. Transversal Competences of the UB; UB: Barcelona, Spain, 2008; Available online: http://www.ub.edu/cubac/sites/default/files/ct_de_la_universitat_de_barcelona_1.pdf (accessed on 25 April 2022).
37. Hernández-Sampieri, R.; Mendoza, C. Metodología de la Investigación. Las Rutas Cuantitativa, Cualitativa y Mixta; Editorial Mc Graw Hill: Ciudad de México, Mexico, 2018.
38. Mieles Barrera, M.D.; Tonon, G.; Alvarado Salgado, S.V. Investigación cualitativa: El análisis temático para el tratamiento de la información desde el enfoque de la fenomenología social. Univ. Hum.; 2012; 74, pp. 195-225.
39. Guba, E.G.; Lincoln, Y.S. Epistemological and methodological bases of naturalistic inquiry. Educ. Commun. Technol. J.; 1982; 30, pp. 233-252. [DOI: https://dx.doi.org/10.1007/BF02765185]
40. Winstone, N.E.; Nash, R.A.; Parker, M.; Rowntree, J. Supporting Learners’ Agentic Engagement with Feedback: A Systematic Review and a Taxonomy of Recipience Processes. Educ. Psychol.; 2017; 52, pp. 17-37. [DOI: https://dx.doi.org/10.1080/00461520.2016.1207538]
41. Gulikers, J.T.M.; Biemans, H.J.A.; Wesselink, R.; van der Wel, M. Aligning Formative and Summative Assessments: A Collaborative Action Research Challenging Teacher Conceptions. Stud. Educ. Eval.; 2013; 39, pp. 116-124. [DOI: https://dx.doi.org/10.1016/j.stueduc.2013.03.001]
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
