Methodology for Analysis and Quantitative Evaluation of Teaching Quality in University Subjects

Abstract

This work describes a general method to quantitatively evaluate the teaching quality of a subject by taking into account the main factors that intervene in the educational process (companies, teachers, students, etc.). The proposed methodology, based on the concatenation of the four “Houses of Quality” of the “Quality Function Deployment” (QFD), is applicable to any university subject. Through an integrated matrix development of the four quality houses, a vector has been obtained ($$\overrightarrow{{\mathbf{q}}_{\mathbf{4}}}$$) that integrates in its definition the entire teaching process of a university subject. Therefore, from this vector, a representative parameter of the teaching quality has been defined, namely, the “Teaching Value”, which globally quantifies the teaching of a university subject. Likewise, and in a complementary manner, the overall effectiveness of teaching has been established ($${\mathsf{\eta }}_{\mathbf{g}\mathbf{l}\mathbf{o}\mathbf{b}\mathbf{a}\mathbf{l}}$$), which is defined as the geometric mean of the product of the efficiencies of the four Houses of Quality and allows the determination of the current performance of teaching and its margin for potential improvement. In particular, the use of the proposed methodology for the engineering course “Project Management” has allowed the teaching quality of this course to be determined (64.2 Megapoints with an efficiency of 48.60%).

1. Introduction

Continuous improvement in the quality of university teaching will define and select the universities of the future. To be competitive in the labor market, each university must analyze and improve the quality of their university’s studies in relation to companies’ real needs. The main reason for choosing university studies is the personal attraction to a career, [1] and in most cases, students would like to carry out their professional life in an environment related to their studies. However, it is difficult for all students to find a job related to the studies they have completed. Therefore, positive expectations of employability motivate students to choose some studies over others, according to a study conducted in Europe [2].

1.1. Literature Review

Many of the methods used to improve quality in higher education can be included in the so-called “Total Quality Management” (TQM), developed by Japanese industries between 1950 and 1970 [3]. Bilen [4] provides a review of the TQM methods applied to improving the quality of university studies. Kwarteng [5] measured from external customers’ perspectives the quality performance concerning TQM implementation at Higher Education. Hawi and Al-zyadat [6] measured the TQM through tree points of view, participation, development and evaluation, showing the participation variable was the stronger one. Vykydal et al. [7] presents the approaches of quality management systems at universities. They point out three concepts (ISO 9000, the branch standard and organization’s excellence) as a suitable direction for applying within higher education environment. Author’s argued that the European Foundation for Quality Management (EFQM) excellence model could serve as an example for universities organization.

In addition, other researchers have proposed other methods for improving quality in higher education. Thus, Plaza et al. [8] use the “Sheward cycle” or “ Deming quality circle” (plan-do-check-act or plan-do-check-adjust) to increase the satisfaction of students (clients in this case) through the cycle of continuous improvement. Likewise, there is a large number of publications [9,10,11,12,13,14,15,16,17,18,19,20,21] seeking quality improvement through the use of the so-called “Quality Function Deployment” (QFD), a Japanese methodology developed by Yoji Akao [22]. There are also different quality studies [23,24,25,26,27] using variations from the classic QFD approach.

One of the main difficulties in applying a quality method is defining the client. For example, Karapetrovic [28] considers the industry, community, society, alumni, students and professional or accreditation organizations to be the client, since the author applied ISO 9000 standard terms to engineering education and research. Wiklund and Wiklund [29] used QFD and conjoint analysis for transforming student needs into quantified course attributes. They were focused on student satisfaction and learning. Alves et al. [30] studied a work integrated learning (WIL) program in a master’s degree of two years. They tried to join companies needs with student engagement through student’s labor practices in companies, supervised and coordinated. Quality needs a target customer for the optimization a process, product or service. Many publications speak only of students as a single client [8,12,14,16,17,19,29,30]. Other publications consider companies as customers [13,20]. There are also a number of authors who support the view that the client is not unique. Thus, with the aim of improving teaching quality in engineering, Köksal and Eğїtman [15] consider the needs of three clients (students, companies and faculty members) and generate the weight of each client’s needs by means of a geometric mean of the valuation of each client using a procedure based on the analytic hierarchy process (AHP) [31]. Other authors [9,32] address this question regarding the identification of the client through the Triple Role of Juran, where students, teachers, colleagues and the industry (society) can act, in one way or another, as a client, process or supplier.

Most researchers who use the QFD technique as a tool to improve quality in higher education use a single house of quality; even if researchers use more than one house of quality, they analyse these houses independently [14]. However, Brian and Teo [9] are an exception because they use three concatenated houses of quality (house-to-house translation) that are obtained from an adaptation of the basic theory of the QFD. These authors applied this methodology in three cases: improvement of a course in NUS Business School (students are considered clients), an online registration system for a course and an application for research grants. In this line, Authors [33] present a preliminary investigation to evaluate teacher quality through three sequential QFD matrices in which students were the only clients.

1.2. Objectives and Hypothesis

However, these works do not take into account all the sectors that participate in the educational process and do not perform a quantitative assessment of the quality of the teaching provided. Therefore, this paper aims to propose a method to quantify the teaching quality of an engineering course considering the main factors that intervene in the educational process: society (through companies), university (as an institution), teachers and students.

The proposed methodology is based on QFD matrices with sequential dependence, and it is applicable to any university subject. The main hypothesis is that this methodology will allow the quantitatively evaluation of the global teaching of a subject, highlighting the strengths and weaknesses of the teaching, and facilitating a proposal of actions to improve teaching within a framework of the continuous improvement of educational quality.

2. Methodology

The proposed methodology for assessing the teaching–learning process of a university subject taking into account the participation of students, professors, companies and universities will be presented in this section.

This methodology is based on the concatenation of matrices called “Houses of Quality” (QFD) [22,34,35]. The QFD technique uses decision matrices to relate and weigh the client’s needs (WHATs) with the attributes that allow the satisfaction of these needs (HOWs). In the proposed method, four concatenated matrices are considered that relate the main aspects of the teaching process of a subject two to two (Table 1). In this way, the WHATs will be different in each of the proposed matrices, and the weighted HOWs of a matrix will be the WHATs of the following matrix.

Figure 1 shows the workflow of the four concatenated matrices. In this scheme, matrices are numbered, indicating in each matrix, the WHATs, the HOWs and who participates and where each factor participates in each part of the teaching process (WHOs and WHEREs).

2.1. The Proposed “House of Quality”: Description and Characteristics

Figure 2 shows the structure and detailed composition of each house of quality. Thus, the first column (W₁, W₂, …, W_n) is constituted by the “n” WHATs. The second column $\left({\mathrm{P}}_1,{\mathrm{P}}_2,\dots ,{\mathrm{P}}_{\mathrm{n}}\right)$ corresponds to the weights assigned to the WHATs. The first row $\left({\mathrm{H}}_1,{\mathrm{H}}_2,\dots ,{\mathrm{H}}_{\mathrm{n}}\right)$ is formed by the “m” HOWs.

On the rooftop of the house of quality, the compatibility between the different HOWs is collected (“+” positive correlation, “−” negative correlation and “or” without correlation). The relationship of each WHAT with each HOW must be evaluated according to a numerical scale. For example, on a scale of “1” to “9”, the value “1” would indicate the minimal correlation and “9” the maximal correlation (values c_ij of Figure 2). The following matrix shows the correlation (relationship matrix) between the WHATs and the HOWs (matrix C of dimensions n x m):

(1)$\mathrm{C}=\left[\begin{array}{cccccc}{\mathrm{c}}_{11}& {\mathrm{c}}_{12}& \dots & {\mathrm{c}}_{1\mathrm{j}}& \dots & {\mathrm{c}}_{1\mathrm{m}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{c}}_{\mathrm{i}}& {\mathrm{c}}_{\mathrm{i}2}& \dots & {\mathrm{c}}_{\mathrm{ij}}& \dots & {\mathrm{c}}_{\mathrm{im}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{c}}_{\mathrm{n}}& {\mathrm{c}}_{\mathrm{n}2}& \dots & {\mathrm{c}}_{\mathrm{nj}}& \dots & {\mathrm{c}}_{\mathrm{nm}}\end{array}\right]$

In each box of the correlation zone in Figure 2, in the lower triangle, each element d_ij of the correlation matrix is calculated as the product of the weighting of row i, P_i, by the corresponding value of the correlation matrix, c_ij, that is, d_ij = P_i · c_ij.

Thus, satisfaction of each WHAT could be defined as the sum of the weighted terms of its corresponding row. That is, the degree of satisfaction of WHAT located in row “i” (S_i) would be obtained by the following expression:

(2)${\mathrm{S}}_{\mathrm{i}}={\mathrm{P}}_{\mathrm{i}}·\left({{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{j}=\mathrm{m}}{\mathrm{c}}_{\mathrm{ij}}\right)$

The last column of Figure 2 shows the satisfaction degrees of the “n” WHATs (S₁, S₂, …, S_n).

Similarly, satisfaction of each HOW could be defined by adding the weighted terms of its corresponding column. That is, the degree of satisfaction of the HOW located in the “j” column (q_j) would be obtained by the following expression:

(3)${\mathrm{q}}_{\mathrm{j}}={{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{n}}{\mathrm{c}}_{\mathrm{ij}}·{\mathrm{P}}_{\mathrm{i}}$

Row “n + 1” of Figure 2 shows the satisfaction degrees of the “m” HOWs (q₁, q₂, …, q_m).

The satisfaction of the WHATs and HOWs could be written in a matrix form in the following way:

• Satisfaction matrix of WHATs (row vector of dimension 1 × n):

  (4)$\overrightarrow{\mathrm{s}}=\left({\mathrm{S}}_1,{\mathrm{S}}_2,\dots ,{\mathrm{S}}_{\mathrm{n}}\right)$

• Satisfaction matrix of HOWs (row vector of dimension 1 × m):

  (5)$\overrightarrow{\mathrm{q}}=\left({\mathrm{q}}_1,{\mathrm{q}}_2,\dots ,{\mathrm{q}}_{\mathrm{m}}\right)$

If a row vector is defined for the weights in the following way:

(6)$\overrightarrow{\mathrm{p}}=\left({\mathrm{P}}_1,{\mathrm{P}}_2,\dots ,{\mathrm{P}}_{\mathrm{n}}\right)$

The vector $\overrightarrow{\mathrm{q}}$ could be written in the form of the following matrix product:

(7)$\overrightarrow{\mathrm{q}}=\left({\mathrm{q}}_1,{\mathrm{q}}_2,\dots ,{\mathrm{q}}_{\mathrm{m}}\right)=\overrightarrow{\mathrm{p}}·\mathrm{C}$

where C is the relationship matrix between the WHATs and the HOWs.

The efficiency values η (q_j) of each HOW H_j have been added in the “House of Quality” of Figure 2. These values relate the degree of satisfaction of each HOW with the maximum that would be possible to obtain; that is, when the corresponding column of the relationship matrix had the maximum value in all its boxes. Thus, if the maximum obtainable valuation in each box C_ij of the relationship matrix defined in Equation (1) is Δ, then, η(q_j) would be obtained in the following way:

(8)$\mathsf{\eta }({\mathrm{q}}_{\mathrm{j}})=\frac{{\mathrm{q}}_{\mathrm{j}}}{\Delta ·{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{n}}{\mathrm{P}}_{\mathrm{i}}}$

Figure 2 also shows the general efficiency of the matrix, $\mathsf{\eta }\left(\mathrm{C}\right)$, which relates the degree of satisfaction of all the HOWs ${{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{j}=\mathrm{m}}{\mathrm{q}}_{\mathrm{j}}$ with the maximum that would be possible to obtain; that is, when all the squares of the relationship matrix had their maximum value (Δ). Thus, the general efficiency of the matrix, η(C), would be obtained in the following way:

(9)$\mathsf{\eta }(\mathrm{C})=\frac{{{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{j}=\mathrm{m}}{\mathrm{q}}_{\mathrm{j}}}{\Delta ·\mathrm{m}·{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{n}}{\mathrm{P}}_{\mathrm{i}}}=\frac{{{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{j}=\mathrm{m}}\mathsf{\eta }({\mathrm{q}}_{\mathrm{j}})}{\mathrm{m}}$

2.2. Analysis and Matrix Development

By applying Equation (7) to the four matrices considered in the proposed methodology, we obtain the following matrix development that characterizes the teaching process of a university subject:

(10)$\begin{array}{c}\overrightarrow{{\mathrm{q}}_1}\\ 1\times \mathrm{m}\end{array}=\begin{array}{c}{\mathrm{P}}_1\dots \dots {\mathrm{P}}_{\mathrm{n}}\\ 1\times \mathrm{n}\end{array}·\begin{array}{c}\left[\begin{array}{c}\begin{array}{cccccc}{\mathrm{c}}_{11}& {\mathrm{c}}_{12}& \dots & {\mathrm{c}}_{1\mathrm{j}}& \dots & {\mathrm{c}}_{1\mathrm{m}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{c}}_{\mathrm{i}1}& {\mathrm{c}}_{\mathrm{i}2}& \dots & {\mathrm{c}}_{\mathrm{i}\mathrm{j}}& \dots & {\mathrm{c}}_{\mathrm{i}\mathrm{m}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{c}}_{\mathrm{n}1}& {\mathrm{c}}_{\mathrm{n}2}& \dots & {\mathrm{c}}_{\mathrm{nj}}& \dots & {\mathrm{c}}_{\mathrm{nm}}\end{array}\end{array}\right]\\ \mathrm{n}\times \mathrm{m}\end{array}=\begin{array}{c}\overrightarrow{\mathrm{p}}·{\mathrm{C}}_1\\ \end{array}$

(11)$\begin{array}{c}\overrightarrow{{\mathrm{q}}_2}\\ 1\times \mathrm{r}\end{array}=\begin{array}{c}\overrightarrow{{\mathrm{q}}_1}\\ 1\times \mathrm{n}\end{array}\begin{array}{c}\left[\begin{array}{cccccc}{\mathrm{d}}_{11}& {\mathrm{d}}_{12}& \dots & {\mathrm{d}}_{1\mathrm{j}}& \dots & {\mathrm{d}}_{1\mathrm{r}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{d}}_{\mathrm{i}1}& {\mathrm{d}}_{\mathrm{i}2}& \dots & {\mathrm{d}}_{\mathrm{i}\mathrm{j}}& \dots & {\mathrm{d}}_{\mathrm{i}\mathrm{r}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{d}}_{\mathrm{m}1}& {\mathrm{d}}_{\mathrm{m}2}& \dots & {\mathrm{d}}_{\mathrm{mj}}& \dots & {\mathrm{d}}_{\mathrm{mr}}\end{array}\right]\\ \mathrm{m}\times \mathrm{r}\end{array}=\begin{array}{c}\overrightarrow{{\mathrm{q}}_1}·{\mathrm{C}}_2\\ \end{array}$

(12)$\begin{array}{c}\overrightarrow{{\mathrm{q}}_3}\\ 1\times \mathrm{s}\end{array}=\begin{array}{c}\overrightarrow{{\mathrm{q}}_2}\\ 1\times \mathrm{r}\end{array}\begin{array}{c}\left[\begin{array}{cccccc}{\mathrm{e}}_{11}& {\mathrm{e}}_{12}& \dots & {\mathrm{e}}_{1\mathrm{j}}& \dots & {\mathrm{e}}_{1\mathrm{s}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{e}}_{\mathrm{i}1}& {\mathrm{e}}_{\mathrm{i}2}& \dots & {\mathrm{e}}_{\mathrm{i}\mathrm{j}}& \dots & {\mathrm{e}}_{\mathrm{i}\mathrm{m}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{e}}_{\mathrm{r}1}& {\mathrm{e}}_{\mathrm{r}2}& \dots & {\mathrm{e}}_{\mathrm{rj}}& \dots & {\mathrm{e}}_{\mathrm{rs}}\end{array}\right]\\ \mathrm{r}\times \mathrm{s}\end{array}=\begin{array}{c}\overrightarrow{{\mathrm{q}}_2}·{\mathrm{C}}_3\\ \end{array}$

(13)$\begin{array}{c}\overrightarrow{{\mathrm{q}}_4}\\ 1\times \mathrm{t}\end{array}=\begin{array}{c}\overrightarrow{{\mathrm{q}}_3}\\ 1\times \mathrm{s}\end{array}\begin{array}{c}\left[\begin{array}{cccccc}{\mathrm{f}}_{11}& {\mathrm{f}}_{12}& \dots & {\mathrm{f}}_{1\mathrm{j}}& \dots & {\mathrm{f}}_{1\mathrm{t}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{f}}_{\mathrm{i}1}& {\mathrm{f}}_{\mathrm{i}2}& \dots & {\mathrm{f}}_{\mathrm{i}\mathrm{j}}& \dots & {\mathrm{f}}_{\mathrm{i}\mathrm{t}}\\ \dots & \dots & \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots & \dots & \dots \\ {\mathrm{f}}_{\mathrm{s}1}& {\mathrm{f}}_{\mathrm{s}2}& \dots & {\mathrm{f}}_{\mathrm{sj}}& \dots & {\mathrm{f}}_{\mathrm{st}}\end{array}\right]\\ \mathrm{s}\times \mathrm{t}\end{array}=\begin{array}{c}\overrightarrow{{\mathrm{q}}_3}·{\mathrm{C}}_4\\ \end{array}$

Substituting Equation (10) to (12) in Equation (13), we have to:

(14)$\begin{array}{c}\overrightarrow{{\mathrm{q}}_4}\\ 1\times \mathrm{t}\end{array}=\begin{array}{c}\overrightarrow{\mathrm{p}}\\ 1\times \mathrm{n}\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_1\right]\\ \mathrm{n}\times \mathrm{m}\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_2\right]\\ \mathrm{m}\times \mathrm{r}\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_3\right]\\ \mathrm{r}\times \mathrm{s}\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_4\right]\\ \mathrm{s}\times \mathrm{t}\end{array}$

where ${\mathrm{C}}_1,{\mathrm{C}}_2,{\mathrm{C}}_3{\mathrm{y}\mathrm{C}}_4$ are the relationship matrices of each phase of the proposed methodology. In general, for $\mathrm{k}$ concatenated matrices, the weight vector of the HOWs could be obtained in the following way:

(15)${\overrightarrow{\mathrm{q}}}_{\mathrm{k}}=\overrightarrow{\mathrm{p}}·{{\displaystyle \prod }}_{\mathrm{j}=1}^{\mathrm{j}=\mathrm{k}}{\mathrm{C}}_{\mathrm{j}}$

Thus, given that the vector $\begin{array}{c}\overrightarrow{{\mathrm{q}}_4}\end{array}$ integrates the valuation of the companies, the conditioning factors of the university, the teaching work of the professors and the opinion of the students in its definition, it is possible to propose a representative parameter of the teaching quality ($Teaching Value$) from this vector that globally quantifies the teaching of a university subject as follows:

(16)$Teaching Value={{\displaystyle \sum }}_{j=1}^{j=t}\overrightarrow{{\mathrm{q}}_4}\left(\mathrm{j}\right)={\mathrm{q}}_4\left(1\right)+\cdots +{\mathrm{q}}_4\left(\mathrm{i}\right)+\cdots +{\mathrm{q}}_4\left(\mathrm{t}\right)$

The “$Teaching Value$” obtained by applying Equation (16) usually reaches a high numerical score, since it comes from the multiplication of four matrices. For this reason, it is preferable to give the “$Teaching Value$” in units of Megapoints ($1\mathrm{Mp}={10}^6 \mathrm{points}$) so that it has fewer significant digits, and its practical use is facilitated.

Thus, according to Equations (14) and (16), to calculate the “$Teaching Value$” corresponding to the teaching of a subject, it is necessary to know the weights of the competencies that must be acquired with their teaching (vector $\overrightarrow{\mathrm{p}}$) and the relationship matrices between the different WHATs and HOWs (matrices ${\mathrm{C}}_1,{\mathrm{C}}_2,{\mathrm{C}}_3{\mathrm{and}\mathrm{C}}_4$). To obtain the vector $\overrightarrow{\mathrm{p}}$, surveys will be carried out with managers and technical directors of companies in sectors related to the subject, where they weigh the importance of each competency in the professional development from 1 to 10. The first three relationship matrices are determined by the consensus assessment of professors who teach the subject. The fourth matrix, ${\mathrm{C}}_4$, will be obtained from a survey of the students of the subject on the means and teaching resources used (each element of the matrix corresponds to the average value of the surveys conducted).

The overall effectiveness of the teaching of a subject is a complementary parameter to the “$Teaching Value$” that allows for the evaluation the teaching performance and its margin of potential improvement. The overall efficiency relates the “$Teaching Value$” to the maximum that would be possible to obtain, (that is, when all the squares of the four relationship matrices had their maximum value, Δ), and it is defined as the geometric mean of the product of the efficiencies of the four matrices considered. Thus, the overall effectiveness of the teaching of a subject (${\mathsf{\eta }}_{\mathrm{global}}$) would be obtained in the following way:

(17)${\mathsf{\eta }}_{\mathrm{global}}=\sqrt[4]{\mathsf{\eta }\left({\mathrm{C}}_1\right)·\mathsf{\eta }\left({\mathrm{C}}_2\right)·\mathsf{\eta }\left({\mathrm{C}}_3\right)·\mathsf{\eta }\left({\mathrm{C}}_4\right)}={\left[\frac{{{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{j}=\mathrm{t}}{\mathrm{q}}_4\left(\mathrm{j}\right)}{\mathrm{m}·\mathrm{r}·\mathrm{s}·\mathrm{t}·{\mathsf{\Delta }}^4\left({{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{n}}{\mathrm{P}}_{\mathrm{i}}\right)}\right]}^{1/4}$

2.3. Internal and External Variables of the “Teaching Value”

Some research papers show the existence of internal and external variables in the assessment of the teaching process of a subject [4,37]. In this study, the university institution, teachers and students are considered internal variables while companies would be the external variable.

To highlight the differential contribution of each of these variables (internal and external) in the teaching assessment of a subject, a process of analytical decomposition of the “Teaching Value” has been developed that makes it possible to express this value as a sum of products of external and internal factors.

First, the vector $\overrightarrow{\mathrm{p}}$ is expressed $\overrightarrow{}$ as a linear combination of unit vectors as follows:

(18)$\overrightarrow{\mathrm{p}}={\mathrm{P}}_1·\left(1,0\dots ,0\right)+\dots +{\mathrm{P}}_{\mathrm{i}}·\left(0,.,1,..,0\right)+\dots +{\mathrm{P}}_{\mathrm{n}}·\left(0,.,0,..,1\right)$

If we call ${\mathrm{C}}_{\mathrm{g}}$ the product of four matrices, we have:

(19)$\begin{array}{c}\left[{\mathrm{C}}_{\mathrm{g}}\right]=\left[{\mathrm{C}}_1\right]\\ \mathrm{n}\times \mathrm{t}\mathrm{n}\times \mathrm{m}\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_2\right]\\ \mathrm{m}\times \mathrm{r}\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_3\right]\\ \mathrm{r}\times \mathrm{s}\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_4\right]\\ \mathrm{s}\times \mathrm{t}\end{array}$

Substituting Equations (18) and (19) in Equation (14), we have

(20)$\overrightarrow{{\mathrm{q}}_4}={\mathrm{P}}_1·\left(1,0\dots ,0\right)\left[{\mathrm{C}}_{\mathrm{g}}\right]+\dots +{\mathrm{P}}_{\mathrm{i}}·\left(0,.,1,..,0\right)\left[{\mathrm{C}}_{\mathrm{g}}\right]+\dots +{\mathrm{P}}_{\mathrm{n}}·\left(0,.,0,..,1\right)\left[{\mathrm{C}}_{\mathrm{g}}\right]$

(21)$\begin{array}{c}\overrightarrow{{\mathrm{q}}_4}={\mathrm{p}}_1\\ \end{array}\begin{array}{c}\left({\mathrm{C}}_{{\mathrm{g}}_{11}}\dots {\mathrm{C}}_{{\mathrm{g}}_{1\mathrm{t}}}\right)\\ 1\times \mathrm{t}\end{array}\begin{array}{c}+{\mathrm{p}}_2\\ \end{array}\begin{array}{c}\left({\mathrm{C}}_{{\mathrm{g}}_{21}}\dots {\mathrm{C}}_{{\mathrm{g}}_{2\mathrm{t}}}\right)\\ 1\times \mathrm{t}\end{array}\begin{array}{c}+\cdots \cdots \cdots +{\mathrm{p}}_{\mathrm{n}}\\ \end{array}\begin{array}{c}\left({\mathrm{C}}_{{\mathrm{g}}_{\mathrm{n}1}}\dots {\mathrm{C}}_{{\mathrm{g}}_{\mathrm{nt}}}\right)\\ 1\times \mathrm{t}\end{array}$

As seen in Equation (21), each initial “k” weighting of the competencies (external variable defined by the companies) multiplies row “k” of the resulting global matrix ${\mathrm{C}}_{\mathrm{g}}$. On the other hand, the influence of the other factors (internal variables) in the “Teaching Value” can be grouped into a single parameter. This parameter, called “Subject Value” (${\mathrm{S}}_{{\mathrm{v}}_{\mathrm{k}}}$), takes into account, for the “k” competency, the assessment of learning goals, topics, techniques and resources used in the teaching of a subject. The following equation expresses this concept:

(22)${\mathrm{S}}_{\mathrm{vk}}={\mathrm{C}}_{{\mathrm{g}}_{\mathrm{k}1}}+{\mathrm{C}}_{{\mathrm{g}}_{\mathrm{k}2}}+\cdots \cdots \cdots +{\mathrm{C}}_{{\mathrm{g}}_{\mathrm{kt}}}={{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{j}=\mathrm{t}}{\mathrm{C}}_{\begin{array}{c}{\mathrm{g}}_{\mathrm{kj}}\\ \end{array}}$

Thus, Equation (22) provides the “Subject Value” or internal contribution of the k-th competency to the “Teaching Value”.

By developing the products indicated in Equation (21) and substituting in Equation (16), the “Teaching Value” can be rewritten according to the following equation:

(23)$\begin{array}{c}Teaching Value=\\ \\ \begin{array}{c}\\ \end{array}\end{array}\begin{array}{c}{\mathrm{p}}_1·{\mathrm{C}}_{{\mathrm{g}}_{11}}+\cdots \cdots \cdots +{\mathrm{p}}_1·{\mathrm{C}}_{{\mathrm{g}}_{1\mathrm{t}}}+\\ {\mathrm{p}}_2·{\mathrm{C}}_{{\mathrm{g}}_{21}}+\cdots \cdots \cdots +{\mathrm{p}}_2·{\mathrm{C}}_{{\mathrm{g}}_{2\mathrm{t}}}+\\ \begin{array}{c}⋮\\ {\mathrm{p}}_{\mathrm{n}}·{\mathrm{C}}_{{\mathrm{g}}_{\mathrm{n}1}}+\cdots \cdots \cdots +{\mathrm{p}}_{\mathrm{n}}·{\mathrm{C}}_{{\mathrm{g}}_{\mathrm{nt}}}\end{array}\end{array}$

Taking as a common factor p₁, p₂, ..., p_n in the above equation, and taking into account the definition of the “Subject Value” (22), we have:

(24)$Teaching Value={\mathrm{p}}_1·{\mathrm{S}}_{\mathrm{v}1}+{\mathrm{p}}_2·{\mathrm{S}}_{\mathrm{v}2}+\cdots \cdots +{\mathrm{p}}_{\mathrm{n}}·{\mathrm{S}}_{\mathrm{vn}}={{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{n}}{\mathrm{P}}_{\mathrm{i}}·{\mathrm{S}}_{{\mathrm{v}}_{\mathrm{i}}}$

Equation (24) defines the “Teaching Value” as a sum of products of internal variables (${\mathrm{S}}_{{\mathrm{v}}_{\mathrm{i}}}$) by external variables (${\mathrm{P}}_{\mathrm{i}}$). Likewise, it can be deduced from Equation (24) that each summand (${\mathrm{p}}_{\mathrm{i}}·{\mathrm{S}}_{\mathrm{vi}}$) contributes a certain percentage (x_i) to the total value of the “Teaching Value” (TV); that is:

(25)$\begin{array}{c}{\mathrm{p}}_1·{\mathrm{S}}_{\mathrm{v}1}={\mathrm{x}}_1·\mathrm{TV}\\ {\mathrm{p}}_2·{\mathrm{S}}_{\mathrm{v}2}={\mathrm{x}}_2·\mathrm{TV}\\ \begin{array}{c}\dots \\ {\mathrm{p}}_{\mathrm{n}}·{\mathrm{S}}_{\mathrm{vn}}={\mathrm{x}}_{\mathrm{n}}·\mathrm{TV}\end{array}\end{array}$

By moving the second member terms of internal variables (${\mathrm{S}}_{{\mathrm{v}}_{\mathrm{i}}}$) and the percentages (x_i) the first member and adding all the Equation (25) the following expression will result:

(26)$\frac{{\mathrm{p}}_1}{{\mathrm{x}}_1}+\frac{{\mathrm{p}}_2}{{\mathrm{x}}_2}+\dots +\frac{{\mathrm{p}}_{\mathrm{n}}}{{\mathrm{x}}_{\mathrm{n}}}=\mathrm{TV}\left(\frac{1}{{\mathrm{S}}_{\mathrm{v}1}}+\frac{1}{{\mathrm{S}}_{\mathrm{v}2}}+\dots +\frac{1}{{\mathrm{S}}_{\mathrm{vn}}}\right)$

Defining the following internal and external equivalent variables:

(27)${\mathrm{p}}_{\mathrm{eq}}=\frac{{\mathrm{p}}_1}{{\mathrm{x}}_1}+\frac{{\mathrm{p}}_2}{{\mathrm{x}}_2}+\dots +\frac{{\mathrm{p}}_{\mathrm{n}}}{{\mathrm{x}}_{\mathrm{n}}}$

(28)${\mathrm{S}}_{\mathrm{veq}}={\left(\frac{1}{{\mathrm{S}}_{\mathrm{v}1}}+\frac{1}{{\mathrm{S}}_{\mathrm{v}2}}+\dots +\frac{1}{{\mathrm{S}}_{\mathrm{vn}}}\right)}^{-1}$

Equation (26) can be put in the following way:

(29)${\mathrm{p}}_{\mathrm{eq}}·{\mathrm{S}}_{\mathrm{veq}}=\mathrm{TV}$

Thus, Equation (29) defines the “Teaching Value” of a subject as the product of an equivalent external variable ${\mathrm{p}}_{\mathrm{eq}}$ (which takes into account the general assessment of companies on teaching competencies) by an equivalent internal variable ${\mathrm{S}}_{\mathrm{veq}}$ (which globally value the intrinsic factors to the university environment).

3. Results and Discussion

3.1. Project Management Subject

The methodology described in the previous section was applied to the subject “Project Management” during the 2018–2019 academic year. This subject is taught in the fourth engineering course of the School of Engineering and Industrial Design of the Polytechnic University of Madrid.

The progressive harmonization of university systems required by the process of the construction of the European Higher Education Area (EHEA), initiated in 1999 with the Bologna Declaration and the subsequent interaction between these systems by the various national regulations successively promulgated, has promoted change in European universities. Therefore, the universities of the EHEA define a series of general and specific competencies that students must acquire to obtain the corresponding degree. The competencies, according to the definition of the Polytechnic University of Madrid, represent a dynamic combination of attributes that describe the learning goals of a given program, or how students will be able to perform at the end of the educational process. A distinction is made between general competencies (common to a set of subjects) and specific competencies (specific to each subject). The subject “Project Management” has seven general competencies (GCi) and a specific competency (SCi) (Table 2).

The learning goals ($L{G}_i$) are the explicit description of what the student must know, understand and/or be able to apply after the completion of the learning process. The subject “Project Management” considers twelve learning goals (Table 3).

The themes of the program (Ti) are the set of differentiated contents in which the subject is divided. The subject “Project Management” considers twelve themes (Table 4).

Teaching techniques are the different pedagogical ways of transmitting the contents of the subject to the students and, thus, achieve the learning goals and associated competencies. In “Project Management”, three main techniques were used, which are described in Table 5.

Teaching resources are those material resources that facilitate communication between teachers and students. The six teaching resources used in “Project Management” are shown in Table 6.

3.2. Characterization of Teaching Matrix

The proposed matrix analysis in Section 2 is applied in this section to the subject “Project Management” in order to calculate its “Teaching Value” and to determine the most important areas for improvement in teaching.

To accomplish this goal, first, the vector $\overrightarrow{\mathrm{p}}$ weighting of competencies by company executives and managers through opinion polls is obtained. The results of these surveys on the eight competencies are shown below:

(30)$\overrightarrow{\mathrm{p}}=\left(\begin{array}{cccccccc}8.08& 8.83& 7.17& 9.25& 9.42& 8.0& 8.33& 8.58\end{array}\right)$

Second, the four correlation matrices corresponding to the subject “Project Management” must be carried out. The rating scale for each element will be the same in all matrices: 1, 3, 5, 7 and 9 (“1” indicates minimum correlation and “9” maximum correlation).

Thus, the relationship matrix of the learning targets (Table 3) with the competencies (Table 2) is obtained by consensus assessment of the teachers of the subject. The obtained results (matrix C₁) are indicated below:

(31)$\begin{array}{c}{\mathrm{C}}_1=\begin{array}{c}\left[\begin{array}{cccccccccccc}3& 3& 3& 5& 5& 7& 7& 5& 1& 1& 1& 1\\ 5& 5& 3& 7& 7& 7& 5& 7& 1& 9& 9& 9\\ 9& 5& 5& 9& 9& 7& 3& 1& 1& 1& 3& 1\\ 1& 5& 5& 1& 1& 3& 3& 1& 1& 3& 3& 9\\ 1& 1& 1& 1& 1& 1& 1& 1& 9& 7& 1& 1\\ 9& 5& 3& 3& 5& 9& 5& 9& 1& 7& 1& 1\\ 9& 9& 9& 9& 7& 7& 3& 9& 1& 3& 5& 3\\ 9& 9& 9& 9& 9& 9& 9& 9& 1& 1& 7& 3\end{array}\right]\\ 8\times 12\end{array}\end{array}$

The interpretation of the meaning of each element of the matrix C₁ requires the use of Table 3 and Table 4. Thus, for example, the term C₁ (3,4) (row 3, column 4), relates the general competency GC3 from Table 2 (“Understanding the impact of industrial engineering on the environment, the sustainable development of society and the importance of working in a professional and responsible environment”) with the LG4 learning goal from Table 3 (“Understanding the implications of total quality management for industrial design and manufacturing projects “). In this case, as C₁ (3,4) = 9, there was a high relation between GC3 and LG4.

Then, the matrix relating C₂ subjects (Table 4) with learning outcomes (Table 3) by consensus assessment of teachers of the subject was obtained. The obtained results were the following:

(32)${\mathrm{C}}_2=\begin{array}{c}\left[\begin{array}{cccccccccccc}9& 9& 9& 9& 1& 1& 1& 1& 1& 1& 1& 1\\ 9& 9& 9& 9& 1& 1& 1& 1& 1& 1& 1& 1\\ 9& 9& 9& 9& 1& 1& 1& 1& 1& 1& 1& 1\\ 1& 1& 1& 1& 1& 9& 1& 1& 1& 1& 1& 1\\ 1& 1& 1& 1& 9& 7& 9& 9& 7& 9& 1& 1\\ 1& 1& 1& 1& 5& 7& 5& 7& 7& 7& 1& 1\\ 1& 1& 1& 1& 1& 1& 1& 1& 9& 1& 9& 1\\ 1& 1& 1& 1& 1& 1& 1& 1& 1& 1& 1& 9\\ 5& 5& 5& 5& 5& 5& 5& 5& 5& 5& 5& 5\\ 1& 1& 1& 1& 5& 5& 9& 9& 3& 9& 1& 1\\ 9& 1& 1& 1& 9& 1& 1& 1& 3& 7& 1& 3\\ 9& 1& 1& 1& 9& 3& 1& 3& 1& 3& 1& 1\end{array}\right]\\ 12\times 12\end{array}$

In this case, for example, the term ${\mathrm{C}}_2\left(4,1\right)$ (row 4, column 1), relates the LG4 learning goal of Table 3 (“Understanding the implications of total quality management for design projects and industrial manufacturing”) with theme T1 of Table 4 (“Project Methodology”). In this case, since ${\mathrm{C}}_2\left(4,1\right)=$ 1, there was a low relation between LG4 and T1.

The next step consists of the realization of the matrix ${\mathrm{C}}_3$ that relates teaching techniques (Table 5) with themes (Table 4). This matrix was also constructed through the consensus opinion of the teachers. The obtained results are indicated below:

(33)$\begin{array}{c}{\mathrm{C}}_3=\begin{array}{c}\left[\begin{array}{c}\begin{array}{ccc}1& 9& 7\\ 1& 9& 7\\ 1& 7& 7\end{array}\\ \begin{array}{ccc}1& 5& 9\\ 3& 9& 7\\ 5& 1& 9\end{array}\\ \begin{array}{c}\begin{array}{ccc}7& 5& 7\\ 7& 5& 7\\ 5& 3& 7\end{array}\\ \begin{array}{ccc}3& 7& 7\\ 7& 5& 9\\ 7& 5& 7\end{array}\end{array}\end{array}\right]\\ 12\times 3\end{array}\end{array}$

In this case, for example, the term ${\mathrm{C}}_3\left(1,3\right)$ (row 1, column 3), relates theme T1 from Table 4 (“Project Methodology”) with the educational technique GGT1 from Table 5 (“Expository technique”). In this case, as ${\mathrm{C}}_3\left(1,3\right)=7$, there was a medium-high relationship between T1 and GGT1.

Finally, we obtained the matrix ${\mathrm{C}}_4$ that relates the teaching resources (Table 6) with the teaching techniques (Table 5). This matrix was constructed taking into account the opinion of the students through a survey. Each element of the matrix includes the average value of the opinion of the 117 students who studied the subject “Project Management” in the 2018–2019 academic year (rating scale: 0 to 9, the null value supposes the non-existence of a relationship). The obtained results are indicated below:

(34)${\mathrm{C}}_4=\begin{array}{c}\left[\begin{array}{cccccc}0& 0& 0& 6.1& 5.2& 6.1\\ 6.6& 0& 5.9& 5.8& 5.8& 6.0\\ 6.6& 6.9& 6.6& 0& 0& 5.9\end{array}\right]\\ 3\times 6\end{array}$

In this case, for example, the term ${\mathrm{C}}_4\left(1,6\right)$) (row 1, column 6), relates the Teaching Technique IT1 from Table 5 (“Directed Study”) to the Teaching Resource dIR1 from Table 6 (“Documentation of the subject”). In this case, as ${\mathrm{C}}_4\left(1,6\right)=6.1$, there is a medium-high relationship between IT1 and dIR1.

Once known ${\mathrm{C}}_1,{\mathrm{C}}_2,{\mathrm{C}}_3{\mathrm{y}\mathrm{C}}_4$, the vector $\overrightarrow{{\mathrm{q}}_4}$, the “Teaching Value” and the overall teaching efficacy of the subject are obtained by applying Equations (14), (16) and (17):

(35)$\begin{array}{c}\overrightarrow{{\mathrm{q}}_4}\\ 1\times 6\end{array}=\begin{array}{c}\overrightarrow{\mathrm{p}}\\ 1\times 8\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_1\right]\\ 8\times 12\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_2\right]\\ 12\times 12\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_3\right]\\ 12\times 3\end{array}\begin{array}{c}·\\ \end{array}\begin{array}{c}\left[{\mathrm{C}}_4\right]\\ 3\times 6\end{array}\begin{array}{c}={10}^6·\\ \end{array}\begin{array}{c}\left(13.07.512.48.47.915.0\right)\\ 1\times 6\end{array}$

(36)$Teaching Value={{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{j}=6}\overrightarrow{{\mathrm{q}}_4}\left(\mathrm{j}\right)=64.2·{10}^6\mathrm{p}=64.2\mathrm{Mp}$

(37)${\mathsf{\eta }}_{\mathrm{global}}={\left[\frac{{{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{j}=\mathrm{t}}{\mathrm{q}}_4\left(\mathrm{j}\right)}{\mathrm{m}·\mathrm{r}·\mathrm{s}·\mathrm{t}·{\mathsf{\Delta }}^4\left({{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{n}}{\mathrm{P}}_{\mathrm{i}}\right)}\right]}^{1/4}={\left[\frac{64.2·{10}^6}{12·12·3·6·9^4\left(67.66\right)}\right]}^{1/4}=0.4860$

Thus, the overall assessment corresponding to the teaching of the subject “Project Management” is 64.2 Megapoints with a global efficacy of 48.60%.

3.3. Compacted QFD Design

To improve the interpretation of the traditional QFD and show the results of the previous analysis in more detail, a new design of the QFD has been developed that reflects the scheme from Figure 1 but with the four houses of quality attached (Figure 3). This compacted design of the QFD gathered the most relevant elements of each quality house (described in Figure 2) and allowed the strengths and weaknesses of the teaching of a subject to be highlighted. The most relevant characteristics of this new QFD are the following:

• (a). Each quality house has a matrix of numbers that are obtained from the product of the weights of the WHATs by the corresponding relationship matrix. Specific:

  • The House of Quality 1 relates the competencies (with weights p₁, p₂, …, p_n) with the learning goals. Therefore, the values of the numerical boxes are obtained as the product of the weights by the elements of the matrix C1. For example, the value to be entered in box d1₂₃ (row 2, column 3 of House 1) is the product of p₂ by C1₂₃; that is, d1₂₃ = 8.83 · 3 = 26.5. Additionally, the sum of each of the columns in House 1 provides each of the components of the vector ${\overrightarrow{q}}_{\mathbf{1}}$ (375.8, 352.1,…, 246.1).

  • The House of Quality 2 relates the weighted learning goals (vector ${\overrightarrow{q}}_{\mathbf{1}}$ 1 of House 1) with the themes of the subject. Therefore, the values of the numerical boxes are obtained as the product of the components of the vector ${\overrightarrow{q}}_{\mathbf{1}}$ by the elements of the matrix C2. For example, in box d2₄₆ (row 4, column 6 of House 2) the product of q₁(4) will be entered by C2₄₆; that is, d2₄₆ = 361.6 · 9 = 3254.4 = 32.54 · 10² (32.5 is recorded in the box since 10² is extracted as a common factor of all the boxes in House 2). Additionally, the sum of each of the columns of House 2 provides each of the components of the vector ${\overrightarrow{q}}_{\mathbf{2}}$ (167.2, 127.0,…, 76.6).

  • The House of Quality 3 relates the themes of the subject weighted in House 2 (vector ${\overrightarrow{q}}_{\mathbf{2}}$) with the teaching techniques. Therefore, the values of the numerical boxes were obtained as the product of the components of the vector ${\overrightarrow{q}}_{\mathbf{2}}$ by the elements of the matrix C3. For example, in box d3₆₃ (row 6, column 3 of House 3) the product of q₂(6) will be entered by C3₆₃; that is, d3₆₃ = 134.7 · 10² · 9 = 121,230 = 12.1 · 10⁴ (in the box 12.1 is noted since 10⁴ is extracted as a common factor of all the boxes in House 3). Additionally, the sum of each of the columns in House 3 provides each of the components of the vector ${\overrightarrow{q}}_{\mathbf{3}}$ (53.4, 88.1, 109.2).

  • The House of Quality 4 relates the weighted teaching techniques (vector ${\overrightarrow{q}}_{\mathbf{3}}$ of House 3) with the teaching resources. Therefore, the values of the numerical boxes are obtained as the product of the components of the vector ${\overrightarrow{q}}_{\mathbf{3}}$ by the elements of the matrix C4. For example, in box d4₂₃ (row 2, column 3 of House 4) the product of q₃(2) will be entered by C4₂₃; that is, d4₂₃ = 88.1 · 10⁴ · 5.9 = 5,197,900 = 5.19 · 10⁶ (in the box 5.2 is given since 10⁶ is extracted as a common factor of all the boxes in House 4). Additionally, the sum of each of the columns of House 4 provides each of the components of the vector ${\overrightarrow{q}}_{\mathbf{4}}$ (13.0, 7.5,…, 15.0).

• (b). Each quality house has a roof at the top where the compatibility between each two HOWs is noted by symbols. Thus, “+” indicates positive correlation, “−” indicates negative correlation and “or” indicates no correlation.

• (c). In the lower part of the houses of quality a table is included with the values of the teaching efficiencies (partial and global) of the subject. The following considerations have been taken into account for its calculation:

  • The efficiency of each HOW, ${\mathsf{\eta }}_i\left(q_j\right)$, relates the degree of satisfaction obtained, q_i(j), with the maximum that would be possible to obtain. Therefore, the calculation of the effectiveness of each HOW will be different in each of the four quality houses:

    • -. Efficiency of HOW “j” of House 1:

      (38)${\mathsf{\eta }}_{\mathbf{1}}\left(q_j\right)=\frac{q_{\mathbf{1}}\left(j\right)}{\Delta ·{{\displaystyle \sum }}_{i=\mathbf{1}}^{i=n}{p}_{\mathbf{i}}}$

    • -. Efficiency of HOW “j” of House 2:

      (39)${\mathsf{\eta }}_{\mathbf{2}}\left(q_j\right)=\frac{q_{\mathbf{2}}\left(j\right)}{\Delta ·{{\displaystyle \sum }}_{i=\mathbf{1}}^{i=m}{q}_{\mathbf{1}}\left(i\right)}$

    • -. Efficiency of HOW “j” of House 3:

      (40)${\mathsf{\eta }}_{\mathbf{3}}\left(q_j\right)=\frac{q_{\mathbf{3}}\left(j\right)}{\Delta ·{{\displaystyle \sum }}_{i=\mathbf{1}}^{i=r}{q}_{\mathbf{2}}\left(i\right)}$

    • -. Efficiency of HOW “j” of House 4:

      (41)${\mathsf{\eta }}_{\mathbf{4}}\left(q_j\right)=\frac{q_{\mathbf{4}}\left(j\right)}{\Delta ·{{\displaystyle \sum }}_{i=\mathbf{1}}^{i=s}{q}_{\mathbf{3}}\left(i\right)}$

  • The efficiency of each house of quality, ${\mathsf{\eta }}_i\left(C{C}_i\right)$, relates the degree of satisfaction of all its HOWs, ${{\displaystyle \sum }}^{}{q}_i\left(j\right),$ to the maximum that would be possible to obtain. Therefore, the calculation of this effectiveness will be different in each of the four quality houses:

    • -. House Efficiency 1:

      (42)${\mathsf{\eta }}_{\mathbf{1}}\left(C{C}_{\mathbf{1}}\right)=\frac{{{\displaystyle \sum }}_{j=\mathbf{1}}^{j=m}{q}_{\mathbf{1}}\left(j\right)}{\Delta ·m·{{\displaystyle \sum }}_{i=\mathbf{1}}^{i=n}{P}_i}=0.5144$

    • -. House Efficiency 2:

      (43)${\mathsf{\eta }}_{\mathbf{2}}\left(C{C}_{\mathbf{2}}\right)=\frac{{{\displaystyle \sum }}_{j=\mathbf{1}}^{j=r}{q}_{\mathbf{2}}\left(j\right)}{\Delta ·r·{{\displaystyle \sum }}_{j=\mathbf{1}}^{j=m}{q}_{\mathbf{1}}\left(j\right)}=0.3612$

    • -. House Efficiency 3:

      (44)${\mathsf{\eta }}_{\mathbf{3}}\left(C{C}_{\mathbf{3}}\right)=\frac{{{\displaystyle \sum }}_{j=\mathbf{1}}^{j=s}{q}_{\mathbf{3}}\left(j\right)}{\Delta ·s·{{\displaystyle \sum }}_{j=\mathbf{1}}^{j=r}{q}_{\mathbf{2}}\left(j\right)}=0.6315$

    • -. − House Efficiency 4:

      (45)${\mathsf{\eta }}_{\mathbf{4}}\left(C{C}_{\mathbf{4}}\right)=\frac{{{\displaystyle \sum }}_{j=\mathbf{1}}^{j=t}{q}_{\mathbf{4}}\left(j\right)}{\Delta ·t·{{\displaystyle \sum }}_{j=\mathbf{1}}^{j=s}{q}_{\mathbf{3}}\left(j\right)}=0.4743$

  • The sum of all the components of the vector ${\overrightarrow{q}}_{\mathbf{4}}$ (13.0, 7.5, …, 15.0) allows the “Teaching Value” to be determined (64.2 Megapoints). The overall effectiveness of the teaching of the subject, ${\mathsf{\eta }}_{\mathbf{g}\mathbf{l}\mathbf{o}\mathbf{b}\mathbf{a}\mathbf{l}}$, can be determined by direct application of Equation (17) or as a geometric mean of the product of the efficiencies of the four houses of quality; that is:

    (46)${\mathsf{\eta }}_{\mathbf{g}\mathbf{l}\mathbf{o}\mathbf{b}\mathbf{a}\mathbf{l}}=\sqrt[\mathbf{4}]{{\mathsf{\eta }}_{\mathbf{1}}·{\mathsf{\eta }}_{\mathbf{2}}· {\mathsf{\eta }}_{\mathbf{3}}· {\mathsf{\eta }}_{\mathbf{4}}}={\left[\frac{{{\displaystyle \sum }}_{j=\mathbf{1}}^{j=t}{q}_{\mathbf{4}}\left(j\right)}{m·r·s·t·{\Delta }^{\mathbf{4}}\left({{\displaystyle \sum }}_{i=\mathbf{1}}^{i=n}{P}_i\right)}\right]}^{\mathbf{1}/\mathbf{4}}= \mathbf{0.4860}$

• (d). The boxes in the picture of the efficiencies are shown with a background color whose tone varied according to the value of the effectiveness. Thus, strong tones indicate that the efficiency was relatively high (greater than 50%) while light tones highlight that the effectiveness was low (less than 50%) and that it should be improved.

The analysis of the results shown in Figure 3 allowed us to draw the following conclusions:

• (a). The House of Quality 1 had adequate efficiency (51.44%). However, the low efficiency of the HOW “LG9: autonomous learning” (23%) indicates that it does not have, in the opinion of the companies, an adequate correspondence with the competencies of the subject established in the curriculum. Therefore, this HOW would be an aspect to reconsider in future modifications of the curriculum.

• (b). The House of Quality 2 had low efficiency (36.12%). In general, themes related to industrial facilities projects are better valued than those related to industrial manufacturing projects (in particular, Topics 11 and 12 had very low efficiencies: 20% and 23%, respectively). Thus, one aspect to improve would be the focus and content of the issues related to industrial manufacturing projects so that they are better suited to the learning goals considered.

• (c). The House of Quality 3 had good efficiency (63.15%) that indicates an adequate relationship between the themes of the subject and the teaching techniques proposed for its teaching. However, the technique “IT1: directed study” (40%) should be enhanced in the first four topics and the advisability of introducing teaching techniques complementary to those currently used should be analyzed.

• (d). The House of Quality 4 has an average value efficiency (47.43%) that could be improved by enhancing the teaching resources “ocR2: blackboard” (33%), “trR1: Basic bibliographic endowment” (37%) and “trR2: access to document databases (35%)” to better adapt to the teaching techniques used. Likewise, the use of other teaching resources that are accepted by students and improve their learning should also be considered.

• (e). The overall effectiveness of the teaching of the subject had an average value (48.60%) and, therefore had room for improvement. In this sense, two priority areas of action must be considered: improving the focus and content of the themes related to industrial manufacturing projects (House of Quality 2) and promoting current teaching resources and incorporating new complementary resources (House of Quality 4).

3.4. Analysis of the Internal and External Variables of the “Teaching Value”

By applying Equation (24) to the subject “Project Management”, its “Teaching Value” can be expressed as a sum of products of internal variables (related to university, professors and students) by external variables (related to opinion of companies). Likewise, by applying the Equations (27) and (28) the values of the internal and external equivalent variables of the “Teaching Value” can be determined. Table 7 lists these results and shows the values of the external and internal variables for each of the competencies of the subject “Project Management”, the contributions of these variables to the total “Teaching Value” and the equivalent variables.

The analysis in Table 7 allowed us to draw the following conclusions:

• (a). The GC4 and GC5 competencies were valued as the best by the companies (9.25 and 9.42, respectively). However, these competencies were relatively undervalued (6.26·10⁵ and 5.43·10⁵, respectively) by internal variables (university institution, professors and students).

• (b). The specific competency SC8 was highly valued by both internal variables (8.58) and external variables (13.88·10⁵).

• (c). The CG2 and SC8 competencies provide, through the product of the corresponding external and internal variables, the highest contributions to the “Teaching Value” (17.04% and 18.55%, respectively). However, the lowest contribution was provided by the GC5 competition (7.96%). However, it should be noted that only three competencies (SC8, GC2 and GC7) contributed more than 50% to the total value of the “Teaching Value”.

• (d). In the composition of the “Teaching Value” the internal factor (equivalent internal variable: 1.073·10⁵) had a much higher range (approximately 10²) than the external factor (equivalent external variable: 598.41).

4. Conclusions

In the present work, a general method was proposed that allows the quantification of the teaching quality of an engineering subject taking into account the main factors involved in the educational process: society (through companies), university (as an institution), professors and students. The proposed methodology is based on the concatenation of four matrices called “Quality Houses” (QFD) and is applicable to any university subject.

Through an integrated matrix development of the four quality houses, a vector was obtained ($\begin{array}{c}\overrightarrow{q_{\mathbf{4}}}\end{array}$) that integrates in its definition the entire teaching process of a university subject. Therefore, from this vector, a representative parameter of the teaching quality has been defined, namely, the “Teaching Value”, which globally quantifies the teaching of a university subject. Likewise, and in a complementary manner, the overall effectiveness of teaching has been established (${\mathsf{\eta }}_{\mathbf{g}\mathbf{l}\mathbf{o}\mathbf{b}\mathbf{a}\mathbf{l}}$), which is defined as the geometric mean of the product of the efficiencies of the four houses of quality and allows the determination of the current performance of teaching and its margin for potential improvement.

Additionally, and in order to highlight the differential contribution that the internal (relative to the university, professors and students) and external (related to the companies) variables have in the teaching assessment of a subject, an analytical decomposition of the “Teaching Value” was used. This approach has allowed, on the one hand, the determination of the values of the external and internal variables for each competition and their percentage of contribution to the total “Teaching Value”, and, on the other hand, the expression the “Teaching Value” as a product of an equivalent external variable $p_{\mathrm{eq}}$ (which takes into account the general valuation of companies on teaching competencies) by an equivalent internal variable ${\mathrm{S}}_{\mathrm{veq}}$ (which globally values the factors intrinsic to the university environment).

The proposed general methodology has been applied to the subject “Project Management” in order to show its suitability and usefulness for the analysis and improvement of the teaching quality of university subjects.

Thus, the matrix characterization of the subject “Project Management” has made it possible to determine its “Teaching Value” (64.2 Mp) and its overall effectiveness (48.60%). To show the results of the matrix analysis in more detail, a new compacted design of the QFD (with the four quality houses attached) has been developed that collects the most relevant elements of each quality house and has highlights the strengths and weaknesses of the teaching of the subject. In the first case (strengths), an adequate relationship between the themes of the subject and the teaching techniques proposed for teaching (partial effectiveness of 63.15%) stood out. In the second case (weak points), it was concluded that two priority areas of action should be considered: improving the focus and content of the issues related to industrial manufacturing projects (partial efficiency of 36.12%) and strengthening current teaching resources and incorporating new complementary resources (partial efficiency of 47.43%).

Additionally, the analytical decomposition of the “Teaching Value” of the “Project Management” course allowed us to conclude that only three competencies (SC8, GC2 and GC7) contribute more than 50% to the total “Teaching Value”. Likewise, it was determined that in the “Teaching Value” composition, the internal factor (equivalent internal variable: 1.073·10⁵) had a much higher range (approximately 10²) than the external factor (equivalent external variable: 598.41).

Thus, the proposed method proved appropriate to quantitatively assess the overall teaching of a subject, to highlight its strengths and weaknesses and to facilitate the proposal for actions to improve teaching in a framework of the continuous improvement of educational quality.

Author Contributions

Conceptualization, J.M.A.R. and J.D.C.-M.; methodology, J.M.A.R. and J.D.C.-M.; supervision, J.M.A.R. and J.D.C.-M.; validation, All authors; investigation, All authors; data curation, F.V.S.M. and M.I.M.; writing—original draft preparation, J.D.C.-M.; writing—review and editing, J.M.A.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. Workflow of the four concatenated matrices.

Figure 2. Structure and composition of the house of quality.

Figure 3. Compacted design of the quality function deployment (QFD) (“terraced houses”).

Table 1

The WHATs and HOWs of the four concatenated matrices [36].

Matrix	WHATs	HOWs	WHOs
1	“n” Competencies	Learning Goals (m)	University, companies and teachers
2	“m” Learning Goals	Themes of the subject (r)	Teachers
3	“r” Themes of the subject	Teaching Techniques (s)	Teachers
4	“s” Teaching Techniques	Teaching resources (t)	Students and teachers

Table 2

General and specific competencies of the subject “Project Management” [36].

Code	Description
GC1	Know and apply knowledge of basic sciences and technologies to the practice of industrial engineering
GC2	Apply the knowledge acquired to identify, formulate and solve problems within broad and multidisciplinary contexts, being able to integrate knowledge, working in interdisciplinary teams
GC3	Understand the impact of industrial engineering on the environment, the sustainable development of society and the importance of working in a professional and responsible environment
GC4	Know how to communicate the knowledge and conclusions, orally, written and graphically, to specialized and non-specialized audiences in a clear and unambiguous way.
GC5	Possess learning skills that allow students to continue studying throughout their lives for their adequate professional development.
GC6	Incorporate new technologies and tools of industrial engineering in their professional activities.
GC7	Organization and planning in the field of the company, and other institutions and organizations of projects and human teams.
SC8	Knowledge and skills to organize and manage projects. Know the organizational structure and functions of a project office.

Table 3

Learning goals “Project Management” subject [36].

Code	Description
LG 1	Apply the methodology for the management and legal processing of projects of facilities or industrial plants of a singular character
LG 2	Differentiate the main functions of the project manager in the field of his relationship with the client and the planning of the project of an installation or industrial plant
LG 3	Enumerate the characteristic activities in the procedures for implementation, control, monitoring and project closure of a facility or industrial plant.
LG 4	Understand the implications of total quality management for industrial design and manufacturing projects
LG 5	Analyze the activities, production system and plant layout necessary for the development of the design and industrial manufacturing project
LG 6	Evaluate design and manufacturing processes to introduce improved methods
LG 7	Apply techniques for the determination of manufacturing times and costs
LG 8	Practice the methodology to streamline work and schedule batch design and manufacturing projects
LG 9	Learn autonomously
LG 10	Have an open attitude and constructive criticism when analyzing technical issues
LG 11	Work as a team in an organized way
LG 12	Communicate effectively to the public

Table 4

Themes of the subject “Project Management” [36].

Code	Description
T1	Project Methodology
T2	Basic Documentation of the Project
T3	Planning, programming and execution
T4	Legislation and legal processing of Projects
T5	Design of products
T6	Quality management
T7	Process and capacity strategies
T8	Analysis of processes with machines and labor
T9	Measure of Work
T10	Plant layout
T11	Manufacturing costs and budget
T12	Planning and programming of work

Table 5

Teaching techniques applied in the subject “Project Management” [36].

Code	Description
IT1	Directed study
GT3	Project method
GGT1	Expository technique

Table 6

Teaching resources of the subject “Project Management” [36].

Code	Description
ocR1	The classroom: sufficient capacity, good lighting, both natural and artificial, and, for group work, have mobile tables.
ocR2	Blackboard (presentation of the lessons and the resolution of the proposed exercises).
ocR3	Presentation system: Projector, screen and PC with presentation software (you could even consider the use of electronic whiteboards connected to the computer)
trR1	Basic bibliographic endowment (follow-up and learning of the lessons, exercises and group work).
trR2	Access to document databases: university library, access to databases of electronic documents (magazines, catalogs, etc.) and consultation of AENOR regulations.
dlR1	Documentation of the subject (lessons, exercises, bibliography, links of interest)

Table 7

Internal and external variables and contribution to “Teaching Value”.

		Subject Competencies								Equival. Variables
		GC1	GC2	GC3	GC4	GC5	GC6	GC7	SC8
Variablesexternal	Weighingcompetitionspi	8.08	8.83	7.17	9.25	9.42	8.00	8.33	8.58	598.41
Variablesinternal	Subject value${\mathrm{S}}_{\mathrm{vi}}$(×105)	6.77	12.39	9.51	6.26	5.43	9.86	12.34	13.88	1.073
Contribution to“Teaching Value”	${\mathrm{p}}_{\mathrm{i}}·{\mathrm{S}}_{\mathrm{vi}}$(×106)	5.47	10.94	6.82	5.79	5.11	7.89	10.28	11.91	64.21
	Contribution percentageXi%	8.52	17.04	10.62	9.02	7.96	12.28	16.01	18.55	100%