For a Healthy (and) Higher Education: Evidences from Learning Outcomes in Health Sciences

Abstract

Increased recognition of outcomes, or competency-based education, has evolved across higher education on health sciences. However, there is significant diversity in the current study of Portuguese programmes. Considering learning outcomes (LO) as indicators of knowledge, skills, abilities, attitudes and the understanding that the student will gain as a result of an educational experience, this study aims to explore which LO are emphasised on the study programmes of health sciences in Portugal. Through a qualitative methodology, carried out through MAXQDA software, all LO of all Portuguese health sciences study programmes submitted to quality accreditation to the Portuguese Agency for Assessment and Accreditation of Higher Education (A3ES) since 2009 until 2016 were analysed. Although specific knowledge was the most referenced LO, transversal skills were also emphasized, such as critical and reflexive analysis/critical thinking, research, ability to organize and plan and professional ethics. Significant differences were found between LO selection when the analysis was made by comparing the diverse study programmes. This required assortment of knowledge and skills seems to reflect not only the specificities of each health science programme but also the challenging demands on professionals in the 21st century, along with the necessary changes imposed by society, fostering intercultural understanding, tolerance, mutual respect and an ethic of global citizenship and shared responsibility, crucial enablers of educational development for all in the scope of the 2030 Agenda for Sustainable Development.

1. Introduction

Learning outcomes (LO), defined as statements of what a learner is expected to know, understand and is capable of doing at the end of a period of learning [1], serve as a vital tool for implementation of the Bologna principles [2]. European educational systems are required to assure their curricula comparability in terms of structure, study programmes and pedagogical methodologies. Hence, assessment methodologies, curricula design procedures, scientific contents, and LO need to be aligned, to allow a comparison not only between study programmes but also between European higher education institutions [3].

As highlighted by Adam [4], LO are generally described in terms of knowledge, skills, abilities, attitudes and the understanding that the student will gain as a result of an educational experience. Assessment of LO is of particular importance in the scope of the 2030 Agenda for Sustainable Development [5,6], considering the Sustainable Development Goals (SDG) to be achieved within a plan of action for people, planet and prosperity. Education, addressed in SDG 4, i.e., “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”, is the main pillar of the 2030 Agenda and has direct implications for the remaining goals. Accordingly, SDG 4, addressing education, needs to be effectively monitored to assess the results obtained in implementing an education with quality in the curricula of all disciplines. A path is needed to pursue SDG 4 and targets in a broader range, namely, in providing inclusive and equitable quality education at all levels through target 4.7, “…ensure that all learners acquire the knowledge and skills needed to promote … global citizenship and appreciation of cultural diversity and of culture’s contribution…”. This ambitious agenda brings something new that focuses on the relevance of LO both for the world of work, of particular importance due to the specific context addressed by each study programme, further explored in the text, as well as for citizenship in a global and interconnected world.

Emphasis is placed on what a student is expected to acquire regarding knowledge, skills and competences, according to each qualification level [7,8,9,10]. That is, more than describing the expected or intended learning outputs, LO should also point out how these achievements will be assessed. This approach helps students to identify in advance what they are expected to know, understand and accomplish, whether for a given study programme or for a specific class, as well as the assessment criteria that will be used.

The literature on this subject presents different descriptions and understandings of LO, rending the concept ambiguous and unclear [11,12,13] and embracing a wide diversity of typologies and categories [14]. Some European studies, such as the Assessment of Higher Education Learning Outcomes (AHELO) [15], have defined LO as a combination of generic skills and discipline-specific skills. Critical thinking, discipline knowledge, problem-solving, teamwork, communication, professional skills, ethics and values, creativity, and learning to learn are some examples of the most-mentioned LO in this study. Three types of generic competencies could be distinguished [16]: instrumental competences refer to cognitive, methodological, technological and linguistic abilities; interpersonal competencies integrate individual abilities such as social skills (i.e., social interaction and co-operation); and systemic competencies encompass abilities and skills concerning whole systems (i.e., a combination of understanding, sensibility and knowledge; prior acquisition of instrumental and interpersonal competencies as required). A list of 31 generic competences is assessed in the Tuning Project [16]. Some of them are related to the ability to communicate in a second language, the capacity to learn and stay up-to-date with learning, the ability to communicate both orally and through the written word in a first language, the ability to be critical and self-critical, the ability to plan and manage time, the ability to reveal awareness of equal opportunities and gender issues, the capacity to generate new ideas, the ability to search for, process and analyse information, the ability to identify, pose and solve problems, the ability to make reasoned decisions, the ability to undertake research at an appropriate level, the ability to work in a team or in an international context, the ability to act based on ethical reasoning, or the spirit of enterprise and the ability to take initiative. A similar perspective is presented by Adam [10,12], to whom an LO is a “mixture of knowledge, skills, abilities, attitudes and understandings that an individual will achieve” (p. 2), divided between subject-specific (i.e., related to the subject discipline) and generic (i.e., key transferable skills related to all disciplines) outcomes. Nusche [17] presents a distinct perspective on LO typologies: “What a learner knows or can do as a result of learning” (p. 7) could be categorized in terms of cognitive and non-cognitive outcomes. Cognitive LO range from more specific areas of knowledge to broader processes of thinking and problem solving [18]. A range of classifications of cognitive LO have been proposed in the literature, mainly based on Bloom’s taxonomy of learning objectives [19]. For Bloom [19], education should go further than simple factual knowledge and understanding. Academic outcomes such as practical application, synthesis, analysis and assessment should be considered learning objectives. Focusing on Nusche’s [17] perspective, cognitive LO could be sub-divided into knowledge and skills acquirement. Knowledge acquirement focuses on a general field of know-how (that is, basic curricula) or on a specific scientific domain. In terms of skills acquirement, cognitive LO integrate, for instance, verbal and quantitative thinking, understanding, analytical processing, critical thinking or problem solving: i.e., skills that are transferable across different disciplinary subjects. On the other hand, non-cognitive skills refer to changes in beliefs or the evolution of certain standards and civic values [20]. Psychosocial development encompasses self-development, namely, maturing identity and self-esteem growth, as well as building up of relations among peers, with teachers, and with the institutions they are inserted into. Interpersonal and intercultural know-how, as well as autonomy and maturity, are also connected with psychological development. In turn, behaviours and ethics, despite being intimately connected, are two distinct dimensions. Behaviours are beliefs directed towards a specific object, while ethics are general patterns of action that transcend behaviours [21]. Social awareness, learning motivations, and respect for diversity are examples of non-cognitive LO concerning behaviours and ethics [14,22].

Considering the literature review and LO diversity in health courses, there is a gap in the identification of those that are most valued by colleges. Thus, this study aims to explore which LO are specifically emphasised in study programmes submitted to quality accreditation in the Portuguese Agency for Assessment and Accreditation of Higher Education (A3ES) [23], i.e., what knowledge, skills, competences and attitudes are reinforced as expected LO in the health education competency profiles. The identification of these LO can reduce the gap identified by Skochelak and Stack [24] by determining whether these LO meet the needs of future health professionals.

2. Learning Outcomes in Health Sciences

Despite a large consensus on necessary changes in healthcare education, most authors, such as Skochelak and Stack [24], agree that the gap between how health students are trained and the future needs of the health care system continues. According to Simões et al. [25], it is essential to humanize medical education and to deconstruct the image of a profession merely related to social prestige and financial return, with medical students being those who most value the social prestige and the economic components of the profession [26]. Increased recognition of LO- or competency-based education has evolved across health professions’ education and training arenas [27]. However, there is a significant diversity among existing study programmes in terms of required courses, international field experiences, and thesis research projects [28]. Additionally, Metzel and Petty [29] advocate a pre-health major in medicine, health and society, which can incorporate structural competency concepts and skills into undergraduate courses. Therefore, Peluso et al. [30] presented the curriculum development working group, formed to assess the current state of curriculum design and implementation; the transformative learning working group, formed to examine how transformative learning theories could inform global health curricular recommendations; and the current status of general health education working group, which is using standardized questionnaire approaches to fill the knowledge gap, in line with Skochelak and Stack [24], who proposed the creation of medical schools of the future. Gonzalo et al. [31] also defined priority areas and potential solutions for the successful integration and sustainability of health systems science in undergraduate medical education: (1) partner with licensing, certifying, and accrediting bodies; (2) develop comprehensive, standardized, and integrated curricula; (3) develop, standardize, and align assessments; (4) improve the undergraduate medical education (UME) to graduate medical education (GME) transition; (5) enhance teachers’ knowledge and skills and incentives for teachers; (6) demonstrate value added to the health system; and (7) address the hidden curriculum.

Brewer [32] stated that there are three potentially paradigm-shifting opportunities for undergraduate global health education: (i) the chance to substantially expand the pool of practitioners; (ii) engaging experts from disciplines beyond the health sciences to pursue global health careers; (iii) more public knowledge about global health issues. The partnerships between medical schools and health systems that are commonplace today also use health systems as a substrate for learning [33].

Different authors have studied different skills in tutors (foundation skills; Strauss and Hunter [34]) and in health students, namely, preparation and skilfulness to care for culturally diverse patients [35]; communication [36] and active listening [37]; reflective skills [38]; cross-disciplinary learning, role preparation, demonstrated skills in delivering effective integrated care [39]; specific therapies, case conceptualization, ethics and professional guidelines, governing laws and rules pertinent to behavioural health practice, and professional communication and record-keeping [40]; adolescent health and communication and interdisciplinary approach skills [41]; clinical skills laboratories and specific skills [42]; sensitivity to environmental sustainability [43]; health intervention research [44,45]; discovery and creation of new knowledge, assessment of health service needs, application of research, and dissemination of research findings through publications and participation in local, national and professional societies, research and evaluation projects [46]. Combined together, these different skills greatly contribute to successful LO.

3. Materials and Methods 3.1. Research Question Considering the significant diversity among current Portuguese health sciences study programmes, will the LO (as indicators of knowledge, skills, abilities, attitudes and understanding that the student will gain as a result of an educational experience) emphasised in study programmes on health sciences in Portugal also differ? 3.2. Study Design

All LO from all health courses taught in Portugal were surveyed. These data are included in the description of each course at A3ES. After the data collection, LO were classified. A content analysis of LO was carried out, considering all LO included in the proposals of study programmes submitted to quality accreditation since 2009 until 2016. The analysis is based on the taxonomy of LO proposed by Dias and Soares [14], according to which LO are discriminated in terms of technical skills (specific knowledge) and generic skills (e.g., critical thinking, communication skills, leadership, or professional ethics). This technique of content analysis is more consistent with a theoretical thematic approach, in which “the researcher looks for themes with important messages inherent in the material” [47], trying to associate them with the taxonomy of LO. The number of incidences of each category mentioned in those study programmes was assessed, not only from an overall perspective but also reflecting each Health Science study programme.

3.3. Measures

At an initial stage of the accreditation process, higher education institutions submit an accreditation proposal to A3ES entitled “Request for Prior Accreditation of a New Study Cycle”. In the description of each study cycle that is submitted, institutions must describe what the “intended LO” expected to be achieved by students at the end of a specific learning period are. Data analysis focused on information included in this specific question, which is limited by A3ES to 1000 characters. All LO of health sciences study programmes submitted to accreditation in A3ES [23] (2009–2016) were explored: Nursing, Diagnostic and Therapeutic Technics, Therapy and Rehabilitation, Pharmaceutical Sciences, Medicine, Dentistry and other Health Sciences (e.g., Podiatry, Cardiovascular Physiopathology, Nutrition, Palliative Care, Health Communication, Biomedical Sciences, Tropical Diseases and Global Health or Primary Mental Health Care).

3.4. Data Analysis Data analysis was performed through MAXQDA (version 12), a software program designed for computer-assisted qualitative and mixed-methods data, text and multimedia analysis. 3.5. Sample Characterization

The sample included 140 health sciences study programmes (51 bachelor’s, 70 master’s and 19 Ph.D. degrees) from the Portuguese public (92) and private (48) higher education subsectors, as well as polytechnic (82) and university (58) higher education subsystems (Table 1), comprising all health sciences study programmes available.

Since, in Portugal, there are no specific standards to be followed regarding LO, the examination of each LO study programme was based on the taxonomy proposed by Dias and Soares [14], according to which LO are characterized in terms of technical skills and generic skills and specifically designed for the Portuguese reality. The number of incidences of each category mentioned on those study programmes was assessed, and the results, translated in percentages, are related to the total of codified excerpts (competence representativeness).

4. Results 4.1. General Results

Overall, specific knowledge is the most-mentioned LO in all categories. Critical and reflexive analysis/critical thinking, research and the ability to organize and plan are the next categories with the highest frequency. Next is professional ethics, followed by teamwork, written communication skills, troubleshooting, personal and social responsibility, concern about social, economic and environmental sustainability and development of autonomous work. Then, the LO of the studied courses value oral communication skills, ability in interpersonal relationships, general knowledge, motivation for excellence, motivation for continuous/long-life learning and decision making. In this alignment, practical knowledge, theoretical knowledge (i.e., innovation, leadership, ability to manage information, initiative and entrepreneurship), information technology, interdisciplinary teamwork and adaptation to new situations are also found. Finally, the less-mentioned categories were creativity, respect for diversity and multiculturalism, international teamwork, foreign language proficiency and negotiation skills (Table 2).

4.2. Results of the Five Most-Mentioned Learning Outcome Categories

Dentistry, as well as other healthcare study programmes, are the ones that make the most references to specific knowledge, followed by diagnostic and therapeutic technics and nursing study programmes and, finally, medicine, pharmaceutical sciences and therapy and rehabilitation programmes (Table 2).

Dentistry study programmes are also the ones that most mention critical and reflexive analysis/critical thinking in their LO. Therapy and rehabilitation, nursing, and other health sciences study programmes follow. Finally, medicine, diagnostic and therapeutic technics and pharmaceutical sciences appear. The research category is mainly referred to by the medicine and therapy and rehabilitation. Following this are courses in nursing and pharmaceutical sciences, followed by health, diagnostic and therapeutic technics and dentistry courses. Ability to organize and plan is mostly referred to by dentistry, medicine, therapy and rehabilitation, nursing and health. At a great distance from these courses are those of pharmaceutical sciences and diagnostic and therapeutic technics. As far as professional ethics are concerned, the courses that referred to it most were dentistry and nursing. Following these are courses on therapy and rehabilitation, diagnostic and therapeutic technics, medicine, health and pharmaceutical sciences (Table 2).

4.3. Results by Courses The LO by groups of courses will be analysed next, and their hierarchical categorical valorisation presented.

Starting this analysis with the medicine courses (Table 3), it is possible to find that the LO of these courses essentially value, in descending order, specific knowledge, research, ability to organize and plan, development of autonomous work and critical and reflexive analysis/critical thinking. Next, distantly, comes concern about social, economic and environmental sustainability; still more distant are the categories of general knowledge, troubleshooting, domain of information technologies, teamwork, ability in interpersonal relationships and professional ethics.

These courses do not make any reference to creativity, foreign language proficiency, ability to manage information, international teamwork, negotiation skills, conflict management, innovation, adaptation to new situations or respect for diversity and multiculturality.

The analysis of the nursing courses (Table 4) follows. The LO of these courses mainly value, in descending order, specific knowledge, research, critical and reflexive analysis, ability to organize and plan and professional ethics. More distantly, they value general knowledge, personal and social responsibility, teamwork, ability in interpersonal relationships, leadership, troubleshooting, motivation for excellence and concern about social, economic and environmental sustainability.

These courses do not refer to the foreign language proficiency, negotiation skills or conflict management categories.

Continuing the analysis with diagnostic and therapeutic technics courses (Table 5), it is possible to verify that the LO of these courses highlight, in descending order, specific knowledge, critical and reflexive analysis/critical thinking, research, troubleshooting, personal and social responsibility, professional ethics, and teamwork.

These courses do not make any reference to foreign language proficiency, interdisciplinary teamwork, negotiation capacity, conflict management or respect for diversity and multiculturality.

Next, the therapy and rehabilitation courses are analysed (Table 6). It can be verified that the LO of these courses essentially value, in descending order, specific knowledge, critical and reflexive analysis, research, ability to organize and plan, and teamwork. More distantly, they value professional ethics, motivation for lifelong learning, general knowledge, theoretical knowledge, written communication skills, ability in interpersonal relationships, and personal and social responsibility.

These courses do not make any reference to foreign language proficiency, international teamwork, conflict management or innovation.

The analysis of pharmaceutical sciences courses (Table 7) follows. The LO of these courses value, in descending order, specific knowledge, followed by motivation for excellence, concern about social, economic and environmental sustainability and research. Immediately following these, they value critical and reflexive analysis/critical thinking and oral communication skills and writing. Following at the same distance are troubleshooting and development of autonomous work.

These courses do not refer to general knowledge (practical and theoretical), decision making, foreign language proficiency, ability to manage information, domain of information technologies, international teamwork, negotiation capacity, conflict management, adapting to new situations, initiative and entrepreneurial spirit, leadership, personal and social responsibility, or respect for diversity and multiculturality.

Continuing the analysis with dentistry courses (Table 8), the LO of these courses essentially value specific knowledge and critical and reflexive analysis/critical thinking. Following these, they value the ability to organize and plan, ability in interpersonal relationships, concern about social, economic and environmental sustainability and professional ethics. More distantly, they value practical knowledge, decision making, the domain of information technologies, motivation for lifelong learning, the development of autonomous work, and research.

These courses do not make any reference to creativity, oral communication skills, foreign language proficiency, international teamwork, negotiation capacity, conflict management, initiative and entrepreneurial spirit, or leadership.

This is followed by the analysis of health courses (Table 9). It can be found that the LO of these courses value above all, specific knowledge, critical and reflexive analysis/critical thinking, and research. Following are skills to organize and plan, written communication skills, concern about social, economic and environmental sustainability, oral communication skills, professional ethics, teamwork, troubleshooting and motivation for excellence.

These courses do not refer to conflict management, negotiation capacity, international teamwork or interdisciplinary teamwork. 5. Discussion In the context of the last decade, to achieve the SDGs of the 2030 Agenda, namely, SDG 4, i.e., education with quality, and considering LO as major indicators of the knowledge, skills, abilities, attitudes and understanding that students will gain because of an educational experience, this study aims to explore the LO emphasized in analysing health sciences study programmes in Portugal (2009–2016).

The specific knowledge category is, by far, the most mentioned, followed by critical and reflexive analysis/critical thinking, research, ability to organize and plan and professional ethics. This means that the acquisition of specific knowledge is the most important expectation for the curricular designers of health sciences study programmes, according to the LO definition suggested by Marouchou [1]. This also reinforces that, from the healthcare student perspective, specific knowledge is the most expected and important LO [7], compared with all the other LO. Specific knowledge is thus not only seen as the major teaching aim to be achieved by academics but is, therefore, also seen by health students as the most important learning acquisition in higher education [7].

In terms of critical and reflexive analysis, Devi et al. [38] advocate reflection as an efficient tool to promote the engagement of students in thinking what they have learnt in a given context and how they will apply that learning in future. According to Jacob et al. [47], patient outcomes and safety are directly linked to healthcare workers’ critical thinking skills. Other studies, such as the one by Soares et al. [48], have also emphasized both critical thinking and specific knowledge as nuclear dimensions of future Portuguese graduates’ competency profiles. This contributes to contextualize and understand the relevance of the results obtained.

Regarding research skill, Basu et al. [49] described the transformation of the Cambridge Health Alliance internal medicine residency programme (2012) into a required social medicine and research-based health advocacy curriculum. One of the innovations consisted of a research-based health advocacy project group within the curricular year, where the students learn health services research methods. This project became a centrepiece of the residency programme, to “stimulate further synthesis of knowledge and skills and to provide residents with a first-hand experience of serving in the role of health advocate” (p. 518). This kind of research encourages the self-identification of the required knowledge [50]. Gonzalo et al. [31] recognized that health courses are encouraged to increase education and research into further integrating schools and student activities with academic health centres and community health programmes so that students could add value to the health system while they learn. This can be achieved by the LO to be valued in each course, thus contributing to a better education for all and a healthier education in health sciences. However, the perceived importance of clinical research skills by students, current residents, and residency programme directors is low [51] and, thus, the awareness of its importance must be better worked out near students.

Most authors include the ability to organize and plan in research-related competencies [31,33,51]. However, some authors considered this skill alone and applied it to other aspects of learning in health courses than research [52]. Safari and Meskini [53] considered that self-monitoring and planning are two important metacognitive skills for problem-solving.

Undergraduate medical students experienced ethical conflicts during their medical training, with a prevalence of conflict in the advanced years of the course [54]. Professional education programmes do not promote moral judgment development unless the programme contains an ethics curriculum [55]. This corroborates the data in this study. In fact, although there are health courses, namely dentistry and nursing, that value professional ethical issues, Portuguese medical courses seem to relegate these dimensions to a secondary plan, not contributing to a healthier higher education and to endowing future medical students with the necessary skills to deal with ethical issues.

The less-mentioned categories were creativity, respect for diversity and multiculturalism, international teamwork, foreign language proficiency and negotiation skills. It is surprising, however, that creativity is the least referenced skill, when one of the most pointed out is critical thinking. Is it possible to develop critical thinking without creativity? Regarding international teamwork and foreign language proficiency being two of the least referenced skills, this also seems to contradict the fact that research is one of the most-mentioned skills. Again, is it possible to perform research with high quality standards without international teamwork and foreign language proficiency? These inconsistencies seem to suggest that decision-makers understand the need for certain LO and include them in their courses. However, they show difficulty operationalizing them. This is in fact difficult to do, and one must be cautious with the operationalisation of LO, particularly the less-mentioned categories, which must be considered in the context of LO found to be more relevant. Concerning the respect for diversity and multiculturalism skills’ low focus, this result corroborates Crawford et al.’s [56] findings; i.e., health students’ self-perceptions of competence related to cultural diversity were low. Although cultural competence is increasingly included in medical curricula to counter related health issues, Lu et al. [57] suggest that the training often risks reinforcing negative stereotypes. This could be problematic, especially when the paradigm of person-centered care [58] is widely used and understood as a reference framework, which is based on the respect for the individual’s personal values and preferences. How is it possible to respect the other, even in a doctor–patient relationship, if respect for diversity is not recognized as an important health outcome is study programmes?

Regarding negotiation skills, almost all authors refer to them, although they were not relevant in the LO in this study. These results may suggest a huge gap between theory and practice with regard to the design of LO, which seems to be in line with reality.

Other health courses, as well as dentistry courses make the most references to specific knowledge. This can be explained, as related to the dentistry course, by the specific technology that is constantly changing and evolving in this area [59] and, concerning the other several courses included in the health category, for the same reason, given the specificity of some of the courses involved. The dentistry course is the one that most refers to critical and reflexive analysis/critical thinking in their LO. Technological development may justify a greater need for critical thinking skills in this course. The research category is mainly referred to by the medicine and therapy and rehabilitation courses, courses that need a greater investment in this area. This is one of the core competencies of medical schools in the employment of evidence-based practice, as medical information becomes quickly obsolete [60] and “does not prepare tomorrow’s doctors to practice and learn in an ever-changing clinical environment” (p. 16). Ability to organize and plan is mostly referred to by dentistry, medicine, therapy and rehabilitation, and nursing and health, that is, for almost all health sciences courses. As far as professional ethics is concerned, the courses that referred to it most were dentistry and nursing. In dentistry and nursing, touch mediates professionals’ interactions with patients [61], which may justify the concern of these courses with ethical questions. Some authors have called for touch to be included within formal curricula in nursing [62] and dentistry [63].

The results thus highlight that some courses do not value the most important LO to achieve SDG 4 of the 2030 Agenda. If respect for diversity and multiculturalism are neglected, this could mean that educational objectives and expected LO are not really universalist and do not support diversity in education. Education should be understood by all as not just a part of but also a key enabler of sustainable development. This mission, for a more equitable and fair society, is possibly the most relevant and ambitious part of SDG 4, focusing on LO both for the world of work as well as for better preparation for dealing with a diverse citizenship in a global and interconnected world. The LO of today’s world deal with this specific goal and its outcomes as a reflection of the values of the current society. Consequently, the reported required assortment of knowledge and skills seems to reflect not only the particularities of each health science study programme but also the challenging demands on the professionals of the 21st century, along with the necessary changes imposed by society, as seen, fostering intercultural understanding, tolerance, mutual respect and an ethic of global citizenship and shared responsibility: crucial enablers of educational development for all. 6. Conclusions This study aims to explore which LO are emphasised in health sciences study programmes in Portugal. The findings show that the specific knowledge category is, by far, the most-mentioned LO, followed by critical and reflexive analysis/critical thinking, research, ability to organize and plan and professional ethics. The less-mentioned categories were creativity, respect for diversity and multiculturalism, international teamwork, foreign language proficiency and negotiation skills. Future health professionals’ profiles are characterized, above all, by the centrality assumed by specific knowledge (as a technical skill), critical thinking and ability to plan and organize (as soft skills). Success in any venture requires basic skills and knowledge, and this necessary variety of knowledge and skills seems to reflect the demands to the professionals of the 21st century and the specific characteristics of these scientific areas in health sciences, as well as the growing search for information and education with quality (SDG 4) by society, based on a global partnership where aspects formerly ignored are now valued in the context of the Sustainable Development of Agenda 2030, in a direct reflection of how the LO of heath sciences study programmes must be properly addressed, since they will affect the future professional activities of present students with consequences in society. This article presents some limitations, namely, the non-distinction of LO in public and private universities. Furthermore, it was not possible to ascertain whether the proposed LO were in fact implemented and achieved.

Courses	N	%
Other health sciences	36	25.71
Nursing	35	25.00
Diagnostic and therapeutic technics	30	21.43
Therapy and rehabilitation	20	14.29
Pharmaceutical sciences	8	5.71
Medicine	7	5.00
Dentistry	4	2.86

	Medicine	Nursing	Diagnostic and Therapeutic Technics	Therapy and Rehabilitation	Pharmaceutical Sciences	Health	Dentistry	Total
Categories	%	%	%	%	%	%	%	%
Specific knowledge	85.71	91.43	96.67	85	87.5	98	100	92.86
Critical and reflexive analysis/critical thinking	57.14	65.71	56.67	85	50	62.73	100	67.14
Research	85.71	68.57	53.33	80	62.5	57.28	50	64.29
Ability to organize and plan	71.43	62.86	23.33	70	25	47.64	75	52.14
Professional ethics	28.57	60	36.67	50	12.5	27.09	75	42.14
Teamwork	28.57	40	33.33	60	12.5	27.09	25	36.43
Written communication skills	14.29	31.43	30	35	50	38.18	25	33.57
Troubleshooting	28.57	34.29	43.33	20	37.5	27.09	25	32.86
Personal and social responsibility	14.29	42.86	40	35	0	21.09	25	31.43
Concern about social, economic and environmental sustainability	42.86	34.29	13.33	15	62.5	36.18	75	30.71
Development of autonomous work	71.43	28.57	30	30	37.5	16.55	50	30
Oral communication skills	14.29	28.57	20	30	50	35.64	0	29.29
Ability in interpersonal relationships	28.57	40	16.67	35	25	14.55	75	27.86
General knowledge	28.57	42.86	3.33	35	0	26.55	25	27.14
Motivation for excellence	14.29	34.29	10	25	62.5	27.09	25	27.14
Motivation for continuous/long-life learning	14.29	22.86	20	45	25	12.55	50	23.57
Decision making	14.29	28.57	23.33	15	0	20.55	50	22.86
Practical knowledge	0	22.86	13.33	25	0	12	50	17.86
Theoretical knowledge	0	20	13.33	35	0	8	25	16.43
Innovation	0	25.71	16.67	0	25	14.55	25	16.43
Leadership	14.29	37.14	16.67	5	0	6	0	16.43
Ability to manage information	0	14.29	6.67	20	0	20.55	25	15
Initiative and entrepreneurial spirit	14.29	20	13.33	5	0	18.55	0	15
Domain of information technologies	28.57	20	3.33	10	0	6.55	50	11.43
Working in interdisciplinary teams	14.29	5.71	0	15	12.5	14	25	10.71
Adapting to new situations	0	17.14	10	10	0	6	25	10.71
Creativity	0	8.57	10	10	12.5	2	0	7.14
Respect for diversity and multiculturality	0	8.57	0	20	0	2	25	6.43
International teamwork	0	2.86	3.33	0	0	0	0	1.43
Foreign language proficiency	0	0	0	0	0	2	0	0.71
Negotiation capacity	0	0	0	5	0	0	0	0.71

Categories	%
Specific knowledge	85.71
Research	85.71
Ability to organize and plan	71.43
Development of autonomous work	71.43
Critical and reflexive analysis/critical thinking	57.14
Concern about social, economic and environmental sustainability	42.86
General knowledge	28.57
Troubleshooting	28.57
Domain of information technologies	28.57
Teamwork	28.57
Ability in interpersonal relationships	28.57
Professional ethics	28.57
Decision making	14.29
Oral communication skills	14.29
Written communication skills	14.29
Interdisciplinary teamwork	14.29
Motivation for excellence	14.29
Motivation for continuous/long-life learning	14.29
Initiative and entrepreneurial spirit	14.29
Leadership	14.29
Personal and social responsibility	14.29

Categories	%
Specific knowledge	91.43
Research	68.57
Critical and reflexive analysis/critical thinking	65.71
Ability to organize and plan	62.86
Professional ethics	60.00
General knowledge	42.86
Personal and social responsibility	42.86
Teamwork	40.00
Ability in interpersonal relationships	40.00
Leadership	37.14
Troubleshooting	34.29
Motivation for excellence	34.29
Concern about social, economic and environmental sustainability	34.29
Written communication skills	31.43
Decision making	28.57
Oral communication skills	28.57
Development of autonomous work	28.57
Innovation	25.71
Practical knowledge	22.86
Motivation for continuous/long-life learning	22.86
Theoretical knowledge	20.00
Domain of information technologies	20.00
Initiative and entrepreneurial spirit	20.00
Adapting to new situations	17.14
Ability to manage information	14.29
Creativity	8.57
Respect for diversity and multiculturality	8.57
Interdisciplinary teamwork	5.71
International teamwork	2.86

Categories	%
Specific knowledge	96.67
Critical and reflexive analysis/critical thinking	56.67
Research	53.33
Troubleshooting	43.33
Personal and social responsibility	40.00
Professional ethics	36.67
Teamwork	33.33
Written communication skills	30.00
Development of autonomous work	30.00
Decision making	23.33
Ability to organize and plan	23.33
Oral communication skills	20.00
Motivation for continuous/long-life learning	20.00
Ability in interpersonal relationships	16.67
Innovation	16.67
Leadership	16.67
Practical knowledge	13.33
Theoretical knowledge	13.33
Initiative and entrepreneurial spirit	13.33
Concern about social, economic and environmental sustainability	13.33
Creativity	10.00
Motivation for excellence	10.00
Adapting to new situations	10.00
Ability to manage information	6.67
General knowledge	3.33
Domain of information technologies	3.33
International teamwork	3.33

Categories	%
Specific knowledge	85.00
Critical and reflexive analysis/critical thinking	85.00
Research	80.00
Ability to organize and plan	70.00
Teamwork	60.00
Professional ethics	50.00
Motivation for continuous/long-life learning	45.00
General knowledge	35.00
Theoretical knowledge	35.00
Written communication skills	35.00
Ability in interpersonal relationships	35.00
Personal and social responsibility	35.00
Oral communication skills	30.00
Development of autonomous work	30.00
Practical knowledge	25.00
Motivation for excellence	25.00
Troubleshooting	20.00
Ability to manage information	20.00
Respect for diversity and multiculturality	20.00
Decision making	15.00
Interdisciplinary teamwork	15.00
Concern about social, economic and environmental sustainability	15.00
Creativity	10.00
Domain of information technologies	10.00
Adapting to new situations	10.00
Negotiation capacity	5.00
Initiative and entrepreneurial spirit	5.00
Leadership	5.00

Categories	%
Specific knowledge	87.50
Motivation for excellence	62.50
Concern about social, economic and environmental sustainability	62.50
Research	62.50
Critical and reflexive analysis/critical thinking	50.00
Oral communication skills	50.00
Written communication skills	50.00
Troubleshooting	37.50
Development of autonomous work	37.50
Ability to organize and plan	25.00
Ability in interpersonal relationships	25.00
Innovation	25.00
Motivation for continuous/long-life learning	25.00
Creativity	12.50
Teamwork	12.50
Interdisciplinary teamwork	12.50
Professional ethics	12.50

Categories	%
Specific knowledge	100.00
Critical and reflexive analysis/critical thinking	100.00
Ability to organize and plan	75.00
Ability in interpersonal relationships	75.00
Concern about social, economic and environmental sustainability	75.00
Professional ethics	75.00
Practical knowledge	50.00
Decision making	50.00
Domain of information technologies	50.00
Motivation for continuous/long-life learning	50.00
Development of autonomous work	50.00
Research	50.00
General knowledge	25.00
Theoretical knowledge	25.00
Troubleshooting	25.00
Written communication skills	25.00
Ability to manage information	25.00
Teamwork	25.00
Interdisciplinary teamwork	25.00
Motivation for excellence	25.00
Innovation	25.00
Adapting to new situations	25.00
Personal and social responsibility	25.00
Respect for diversity and multiculturality	25.00

Categories	%
Specific knowledge	98.00
Critical and reflexive analysis/critical thinking	62.73
Research	57.28
Ability to organize and plan	47.64
Written communication skills	38.18
Concern about social, economic and environmental sustainability	36.18
Oral communication skills	35.64
Professional ethics	27.09
Teamwork	27.09
Troubleshooting	27.09
Motivation for excellence	27.09
General knowledge	26.55
Personal and social responsibility	21.09
Decision making	20.55
Ability to manage information	20.55
Initiative and entrepreneurial spirit	18.55
Development of autonomous work	16.55
Ability in interpersonal relationships	14.55
Innovation	14.55
Working in interdisciplinary teams	14.00
Motivation for continuous/long-life learning	12.55
Practical knowledge	12.00
Theoretical knowledge	8.00
Domain of information technologies	6.55
Leadership	6.00
Adapting to new situations	6.00
Creativity	2.00
Respect for diversity and multiculturality	2.00
Foreign language proficiency	2.00

Author Contributions

Conceptualization, Â.L., D.S. and D.D.; methodology, D.S. and D.D.; software, D.S. and D.D.; validation, Â.L., D.S. and D.D.; formal analysis, D.S. and D.D.; investigation, D.S. and D.D.; data curation, D.S. and D.D.; writing-original draft preparation, Â.L., D.S. and D.D.; writing-review and editing, Â.L., D.S., H.F.P.eS., D.G.V., M.A.P.D. and D.D.; visualization, Â.L., D.S., H.F.P.eS., D.G.V., M.A.P.D. and D.D.; supervision, Â.L., D.S. and D.D.; project administration, D.S. and D.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

1. Marouchou, D.V. Can students' concept of learning influence their learning outcomes? High. Learn. Res. Commun. 2012, 2, 18-33.

2. Stanley, J. Learning outcomes: From policy discourse to practice. Eur. J. Educ. 2015, 50, 404-419.

3. Dias, D.; Amaral, A. Assessment of Higher Education learning outcomes (AHELO): Feasibility Study. In Quality Assurance in Higher Education. Contemporary Debates; Rosa, M.J., Amaral, A., Eds.; Palgrave Macmillan: New York, NY, USA, 2014; pp. 66-86. ISBN 978-94-6091-867-4.

4. Adam, S. An introduction to learning outcomes. In EUA Bologna Handbook; Froment, E., Kohler, J., Purser, L., Wilson, L., Eds.; Raabe: Berlin, Germany, 2006.

5. UNESCO. Education for Sustainable Development Goals: Learning Objectives; United Nations Educational, Scientific and Cultural Organization: Paris, France, 2017; ISBN 978-92-3-100209-0.

6. Vidal, D.G. Por uma sociologia do desenvolvimento sustentável: Uma reflexão sobre a criação de um novo campo disciplinar [For a sustainable development sociology: A reflection on the creation of a new disciplinary field]. Rev. Ambiente Sustent. 2019, 17, 115-125.

7. Wiliam, D. The role of formative assessment in effective learning environment. In The Nature of Learning: Using Research to Inspire Practice; OECD: Paris, France, 2010.

8. Gallavara, G.; Hreinsson, E.; Kajaste, M.; Zadeh, M.S. Learning Outcomes: Common Framework-Different Approaches to Evaluation Learning Outcomes in the Nordic Countries; European Association for Quality Assurance in Higher Education: Helsinki, Finland, 2008.

9. Tissot, P. Terminology of Education and Training Policy: A Multilingual Glossary; European Centre for the Development of Vocational Training: Luxembourg, 2014.

10. Adam, S. Using learning outcomes; A consideration of the nature, role, application and implications for European education of employing 'learning outcomes' at the local, national and international levels. In Proceedings of the United Kingdom Bologna Seminar, Edinburgh, UK, 1-2 July 2002.

11. Aamodt, P.O.; Hovdhaugen, E. Assessing Higher Education Learning Outcomes as a Result of Institutional and Individual Characteristics; OECD: Paris, France, 2008.

12. Adam, S. Learning Outcomes based Higher Education: The Scottish experiences. In Proceedings of the Bologna Seminar, Edinburgh, UK, 21-22 February 2008.

13. Scott, I. The Learning Ooutcome in Higher Education: Time to think again? Worcester J. Learn. Teach. 2011, 5, 1-8.

14. Dias, D.; Soares, D. Learning outcomes in Higher Education: Designing a conceptual map for portuguese academia. In Proceedings of the International Technology, Education and Development Conference, Valencia, Spain, 6-8 March 2017; pp. 9188-9194.

15. Tremblay, K.; Lalancette, D.; Roseveare, D.; Dias, D.; Amaral, A. Assessment of Higher Education learning outcomes (AHELO): Feasibility Study. Feasibil. Study Rep. 2013, 1, 66-86.

16. Gonzalez, J.; Wagenaar, R. Tuning Educational Structures in Europe. Final Report. Pilot Project-Phase 1; Deusto-Groningen: Bilbao, Spain, 2003.

17. Nusche, D. Assessment of learning outcomes in Higher Education. OECD Educ. Work. Pap. 2008, 15, 1-49.

18. Shavelson, R.J.; Huang, L. Responding responsibly to the frenzy to assess learning in higher education. Change 2003, 31, 10-19.

19. Bloom, B. Taxonomy of Educational Objectives; Addison-Wesley: Boston, UK, 1956.

20. Ewell, P. Enhancing alumni research: European and american perspectives. New Dir. Inst. Res. 2005, 126, 19-29.

21. Pascarella, E.; Terenzini, P. How College Affects Students: A Third Decade of Research; Jossey-Bass: Indianapolis, IN, USA, 2005.

22. Reason, R.D.; Hemer, K. Civic Learning and Engagement: A Review of the Literature on Civic Learning, Assessment, and Instruments; Association of American Colleges and Universities: Washington, DC, USA, 2012.

23. A3ES. Available online: https://www.a3es.pt/en (accessed on 2 May 2020).

24. Skochelak, S.E.; Stack, S.J. Creating the Medical Schools of the Future. Acad. Med. 2017, 92, 16-19.

25. Simões, J.D.A.; Augusto, G.F.; Hernández-Quevedo, C. Portugal: Health System Review; World Health Organization: Copenhagen, Denmark, 2017; Volume 19.

26. Nora, D.; Vidal, D.G.; Viterbo, L.M.F.; Dinis, M.A.P.; Sousa, H.F.P. Motivations Influencing the Surgeon's Healthcare Unit Choice to Perform Surgery: A Confirmatory Study in Portugal. Eur. J. Investig. Health Psychol. Educ. 2020, 10, 143-158.

27. Calhoun, J.G.; Wrobel, C.A.; Finnegan, J.R. Current state in U.S. public health competency-based graduate education. Public Health Rev. 2011, 33, 148-167.

28. Drain, P.K.; Mock, C.; Toole, D.; Rosenwald, A.; Jehn, M.; Csordas, T.; Ferguson, L.; Waggett, C.; Obidoa, C.; Wasserheit, J.N. The Emergence of Undergraduate Majors in Global Health: Systematic Review of Programs and Recommendations for Future Directions. Am. J. Trop. Med. Hyg. 2017, 96, 16-23.

29. Metzl, J.M.; Petty, J. Integrating and assessing structural competency in an innovative prehealth curriculum at Vanderbilt University. Acad. Med. 2017, 92, 354-359.

30. Peluso, M.J.; Kellett, A.; Davies, D.; Samaan, J.; Brewer, T.; Van Schalkwyk, S.; Garg, B.; Mayanja-Kizza, H.; Margolis, C.Z.; Rohrbaugh, R. Towards a Universal Medical Education Global Health Curriculum: Update on the Bellagio Global Health Education Initiative. Ann. Glob. Health 2017, 83, 49-50.

31. Gonzalo, J.D.; Baxley, E.; Borkan, J.; Dekhtyar, M.; Hawkins, R.; Lawson, L.; Starr, S.R.; Skochelak, S. Priority areas and potential solutions for successful integration and sustainment of Health Systems Science in Undergraduate Medical Education. Acad. Med. 2017, 92, 63-69.

32. Brewer, T.F. Undergraduate global health degrees: The time is right. Am. J. Trop. Med. Hyg. 2017, 96, 7-8.

33. Gonzalo, J.D.; Lucey, C.; Wolpaw, T.; Chang, A. Value-Added Clinical Systems Learning Roles for Medical Students That Transform Education and Health. Acad. Med. 2017, 92, 602-607.

34. Strauss, P.; Hunter, J. Uncertain pathways: Foundation skills tutors and marginalised students in neoliberal times. J. Furth. High. Educ. 2018, 42, 879-892.

35. Green, A.R.; Chun, M.B.J.; Cervantes, M.C.; Nudel, J.D.; Duong, J.V.; Krupat, E.; Betancourt, J.R. Measuring medical students' preparedness and skills to provide cross-cultural care. Health Equity 2017, 1, 15-22.

36. Bylund, C.L.; Alyafei, K.; Anand, A.; Al Marri, A.; Omer, W.; Sinha, T.; Alam, W.; Abdelrahim, H.; Al-Khal, A. Implementing and tailoring a western-developed communication skills training program for graduate medical trainees in Qatar. Int. J. Med. Educ. 2017, 8, 16-18.

37. Cantu, A.G. Impacting practice: Using a poverty Simulation to develop leadership skills in baccalaureate nursing students. In Proceedings of the Stigma Theta Tau International's 28th International Nursing Research Congress, Dublin, Ireland, 27-31 July 2017.

38. Devi, V.; Abraham, R.R.; Kamath, U. Teaching and assessing reflecting skills among undergraduate medical students experiencing research. J. Clin. Diagn. Res. 2017, 11, JC01-JC05.

39. Shea, J.M. Impact of an Integrated Care Clinical Experience on Skills and Confidence of Nurse Practitioner Students. In Proceedings of the Sigma Theta Tau International's 28th International Nursing Research Congress, Dublin, Ireland, 27-31 July 2017.

40. Callan, J.E.; Maheu, M.M.; Bucky, S.F. Crisis in the behavioral health classroom: Enhancing knowledge, skills, and attitudes in Telehealth Training. In Career Paths in Telemental Health; Springer International Publishing: Cham, Switzerland, 2017; pp. 63-80.

41. Fonseca, H. How an adolescent health curriculum was implemented in Portugal. In International Handbook on Adolescent Health and Development; Springer International Publishing: Cham, Switzerland, 2017; pp. 341-358.

42. McKenna, L.; Williams, B. The hidden curriculum in near-peer learning: An exploratory qualitative study. Nurse Educ. Today 2017, 50, 77-81.

43. Truong, S. Expanding Curriculum Pathways Between Education for Sustainability (EfS) and Health and Physical Education (HPE). In Reimagining Sustainability in Precarious Times; Springer: Singapore, 2017; pp. 239-251.

44. Paradis, G.; Hamelin, A.-M.; Malowany, M.; Levy, J.; Rossignol, M.; Bergeron, P.; Kishchuk, N. The University-Public health partnership for public Health research training in Quebec, Canada. Am. J. Public Health 2017, 107, 100-104.

45. Rangan, A.; Pitchford, J.; Williams, P.; Wood, B.; Robson, S. Standardisation of delivery and assessment of research training for specialty trainees based on curriculum requirements: Recommendations based on a scoping review. BMJ Open 2017, 7, e013955.

46. Nelson-Hurwitz, D.C.; Arakaki, L.-A.; Uemoto, M. Insights in Public Health: Training Today's Students to Meet Tomorrow's Challenges: Undergraduate Public Health at the University of Hawai'i at Manoa. Hawai'i J. Med. Public Health 2017, 76, 89-93.

47. Jacob, E.; Duffield, C.; Jacob, D. A protocol for the development of a critical thinking assessment tool for nurses using a Delphi technique. J. Adv. Nurs. 2017, 73, 1982-1988.

48. Soares, D.; Franco, A.; Dias, D. Translating critical thinking skills to higher education practices. In Proceedings of the International Technology, Education and Development Conference, Valencia, Spain, 6-8 March 2017; pp. 9195-9200.

49. Basu, G.; Pels, R.J.; Stark, R.L.; Jain, P.; Bor, D.H.; McCormick, D. Training Internal Medicine Residents in Social Medicine and Research-Based Health Advocacy. Acad. Med. 2017, 92, 515-520.

50. Schmidt, L.; Wray, A.; Miller, J.; Scott, J.; Pope, K. A Tool for Educating Global Health Practitioners: The Curriculum Design Compass. MedEdPublish 2017, 6, 1-14.

51. Anderson, H.D.; Saseen, J.J. The importance of clinical research skills according to PharmD students, first-year residents, and residency directors. Curr. Pharm. Teach. Learn. 2017, 9, 224-229.

52. Tibyampansha, D.; Ibrahim, G.; Kapanda, G.; Tarimo, C.; Minja, A.; Kulanga, A.; Muiruri, C.; Mteta, K.; Kessy, E.; Bartlett, J. Implementation of a Learning Management System for Medical Students: A Case Study of Kilimanjaro Christian Medical University College. MedEdPublish 2017, 6, 1-16.

53. Safari, Y.; Meskini, H. The Effect of Metacognitive Instruction on Problem Solving Skills in Iranian Students of Health Sciences. Glob. J. Health Sci. 2015, 8, 150-156.

54. Menezes, M.M.; Maia, L.C.; Rocha, C.U.; Sampaio, C.A.; Costa, S.D. Conflitos éticos vivenciados por estudantes de Medicina [Ethical conflicts experienced by medical students]. Rev. Bras. Educ. Med. 2017, 41, 162-169.

55. Verrinder, J.M.; Ostini, R.; Phillips, C.J.C. Differences in Moral Judgment on Animal and Human Ethics Issues between University Students in Animal-Related, Human Medical and Arts Programs. PLoS ONE 2016, 11, e0149308.

56. Crawford, S.Y.; Awé, C.; Tawk, R.H.; Simon Pickard, A. A cross sectional and longitudinal study of Pharmacy student perceptions of readiness to serve diverse populations. Am. J. Pharm. Educ. 2016, 80, 62.

57. Lu, P.-Y.; Tsai, J.-C.; Tseng, S.Y.H. Clinical teachers' perspectives on cultural competence in medical education. Med. Educ. 2014, 48, 204-214.

58. Rubashkin, N.; Warnock, R.; Diamond-Smith, N. A systematic review of person-centered care interventions to improve quality of facility-based delivery. Reprod. Health 2018, 15, 169.

59. Bobich, A.M.; Mitchell, B.L. Transforming dental technology education: Skills, knowledge, and curricular reform. J. Dent. Educ. 2017, 81, eS59-eS64.

60. Verma, S. What are the fundamentals for a Medical School Curriculum in the 21st Century? Rev. Med. 2016, 95, 15.

61. Kelly, M.A.; Nixon, L.; McClurg, C.; Scherpbier, A.; King, N.; Dornan, T. Experience of touch in health care: A meta-ethnography across the Health care professions. Qual. Health Res. 2018, 28, 200-212.

62. O'Lynn, C.; Krautscheid, L. "How should I touch You?": A qualitative study of attitudes on intimate touch in Nursing Care. AJN Am. J. Nurs. 2011, 111, 24-31.

63. Schifter, C.C.; Bogert, M.; Boston, D. Acquisition of a touching style. J. Dent. Educ. 1999, 63, 400-406.