Fuzzy Decision-Making and Resource Management Model of Performance Evaluation Indices

Abstract

The Performance Evaluation Matrix (PEM) is an excellent decision-making tool for assessment and resource management. Satisfaction Index and Importance Index are two important evaluation indicators of construction and PEM. Managers can decide whether the service item needs to be improved based on the Satisfaction Index of the service item. When resources are limited, managers can determine the priority of improving the service item based on the Importance Index. In order to avoid the risk of misjudgment caused by sample errors and meet the needs of enterprises’ rapid decision-making, this study proposed a fuzzy test built on the confidence intervals of the above two key indicators to decide whether essential service items should be improved and determine the priority of improvement. Since the fuzzy test was relatively complex, this study further came up with fuzzy evaluation values and fuzzy evaluation critical values of service items following fuzzy testing rules. Besides, evaluation rules were established to facilitate industrial applications. This approach can be completed with any common word processing software, so it is relatively convenient in application and easy to manage. Finally, an application example was presented in this paper to explain the applicability of the proposed approach.

1. Introduction

The Performance Evaluation Matrix (PEM) is an outstanding evaluation and improvement tool for various service operating systems [1,2,3,4]. Many papers have been devoted to conducting research into the PEM, aiming to evaluate the performance of various service operating systems and determine whether they have reached the required level [5,6,7]. The PEM method, mainly based on the service items provided by the service operating systems and then designed into questionnaire scales, can be employed to investigate customers’ or users’ satisfaction and importance for each service item as well as to set the Satisfaction Index and the Importance Index [8,9,10].

Additionally, a few studies have used confidence intervals of indicators to create evaluation coordinate points of the Satisfaction Index and the Importance Index for each service item [11,12]. Observing where the evaluation coordinate points of all service items are located in the service quality zones of PEM can help determine which service item needs improvement or whether resource transfer is required [13,14]. The service item that falls into the service quality improvement zone has high customer importance and low customer satisfaction, so it needs improvement. The service item that falls into the service quality maintenance zone has equal customer importance and satisfaction, so it needs maintenance. The service item that falls into the resource transfer zone has low customer importance and high customer satisfaction, showing that customers are fully satisfied with the service item, but its importance is not high; therefore, this item must be reviewed, and a resource transfer must be considered to increase the overall satisfaction of the entire service operating system [15,16].

Yu et al. [17] have indicated that the above-mentioned method of performance evaluation may fail to identify improvement points due to customers’ different cultures and mindsets. With the spirit of continuous improvement and total quality management, PEM is divided into four quadrants by the average values of the Satisfaction Index and the Importance Index. Service items in quadrants 2 and 4, in principle, are those whose values are lower than the average as well as the items which require improvement. It seems that the evaluation is directly conducted by statistics calculated from the sample data, but the evaluation method has taken sampling errors into account. To solve the problem of sampling errors, some studies have also made statistical inferences through the confidence intervals of the above two indices [11,12]. However, considering cost and effectiveness, Chen and Yu [18] have suggested that the number of samples is usually not too large in practice for making decisions quickly and accurately in a short time, thereby affecting the accuracy of statistical inferences. Obviously, sampling errors, assessment accuracy, and limited resources are issues that need to be considered and solved in the development of PEM. Aiming to solve the problem concerning the maintenance of evaluation accuracy in the case of small samples, this study, based on Chen and Yu [18] and the confidence interval proposed by some studies [11,12,19], develops a complete fuzzy testing method to evaluate which service item needs improvement. Meanwhile, when resources are limited, this method helps decide which service item should be a top priority for improvement. Next, following the fuzzy testing rules, this study derives the fuzzy decision-making value for satisfaction improvement and the fuzzy decision-making value for improvement priority of importance, so as to facilitate managers’ decision-making [18,19].

In the PEM method, first, based on the service system which needs to be evaluated, we need to design a corresponding questionnaire and corresponding questions which are called service items by this study. In order not to lose generality, this study, like other studies, assumes that the number of service items provided by the service system is k; then k questions are designed to conduct a survey targeted at learners about satisfaction and importance for k service items [20].

Lin et al. [19] let random variable $X_h$ represent the hth service item of satisfaction, then random variable $X_h$ is distributed as a Beta distribution with the first parameter ${\alpha }_h$ and the second parameter ${\beta }_h$, denoted by $X_h~Beta\left({\alpha }_h,{\beta }_h\right)$, h $=$ 1, 2, …, k. Furthermore, let random variable $Y_h$ indicate the hth service item of importance, then random variable $Y_h$ is distributed with the first parameter $a_h$ and the second parameter $b_h$, denoted by $Y_h~Beta\left(a_h,b_h\right)$; the Beta distribution, denoted by $Y_h~Beta\left({\delta }_h,{\gamma }_h\right)$, is also displayed. Thus, these two indices can be shown as follows:

(1)$I_{Sh}=\frac{{\alpha }_h}{{\alpha }_h+{\beta }_h}(\mathrm{Satisfaction} \mathrm{Index});$

(2)$I_{Ih}=\frac{a_h}{a_h+b_h}(\mathrm{Importance} \mathrm{Index}).$

For the convenience of explanation, according to the characteristics of the Beta distribution, we set the level of satisfaction higher than 50% as high satisfaction and the level of satisfaction lower than 50% as low satisfaction. When $I_{Sh}=1/2$, we have ${\alpha }_h/\left({\alpha }_h+{\beta }_h\right)=1/2$, and then we make the conclusion ${\alpha }_h={\beta }_h$, showing that the level of high satisfaction is equal to the level of low satisfaction. When $I_{Sh}>1/2$, we have ${\alpha }_h/\left({\alpha }_h+{\beta }_h\right)>1/2$, and then we make the conclusion ${\alpha }_h>{\beta }_h$; at this time, the level of high satisfaction is higher than that of low satisfaction. When $I_{Sh}<1/2$, we have ${\alpha }_h/\left({\alpha }_h+{\beta }_h\right)<1/2$ and then we make the conclusion ${\alpha }_h<{\beta }_h$, meaning that the level of high satisfaction is lower than that of low satisfaction. When the value of Satisfaction Index $I_{Sh}$ is lower, the level of high satisfaction is also lower. In addition, the Importance Index has the same property as the Satisfaction Index. The higher the value of Importance Index $I_{Ih}$, the higher the level of high importance. Similarly, as the value of Importance Index $I_{Ih}$ is lower, the level of high importance is lower as well.

Obviously, Satisfaction Index and Importance Index are two important elements of the performance evaluation matrix. The purpose of this paper is to use the unilateral confidence intervals of these two important indicators to conduct a fuzzy test, so that the fuzzy evaluation criteria of the performance evaluation matrix can be developed. Next, according to the suggestion made by Lin et al. [19], a fuzzy performance evaluation chart is created to assist businesses with their managment and decision-making.

The remainder of this paper is organized as follows. In Section 2, we derive the 100$\left(1-\alpha \right)$% confidence interval of Satisfaction Index and develop its fuzzy evaluation rules. In Section 3, we derive the 100$\left(1-\alpha \right)$% confidence interval of Importance Index and make its fuzzy evaluation rules. In Section 4, we use a case study to illustrate an application of the model proposed by this study, demonstrating how to identify service items requiring improvement as well as how to prioritize them for improvement as resources are limited. In Section 5, conclusions, research limitations, and future research directions are presented and explored.

2. Fuzzy Evaluation Rules for Satisfaction Index

As mentioned earlier, this study, built on existing literature, assumes that both distributions of customer satisfaction and importance follow the Beta distribution [19]. Let $\left(X_{ h,1},\dots ,X_{ h,j},\dots ,X_{ h,n}\right)$ be the sample data of customer satisfaction for Service Item h with a size of n, where h $=$ 1, 2,…, k. Then, the unbiased estimator of Satisfaction Index $I_{Sh}$ is expressed as follows:

(3)${\widehat{I}}_{Sh}=\frac{1}{n}\times {\displaystyle \sum _{j=1}^n{X}_{h,j}}.$

The expected value of the unbiased estimator ${\widehat{I}}_{Sh}$ is equal to $I_{Sh}$, denoted by $E\left[{\widehat{I}}_{Sh}\right]=I_{Sh}$. Let random variable $Z_{Sh}$ be defined as follows:

(4)$Z_{Sh}=\frac{{\widehat{I}}_{Sh}-I_{Sh}}{S_{Xh}/\sqrt{n}},$

where $S_{Xh}$ is the sample standard deviation, given by:

(5)$S_{Xh}=\sqrt{\frac{1}{n-1}\times {\displaystyle \sum _{j=1}^n{\left(X_{h,j}-{\overline{X}}_h\right)}^2}}.$

Plenty of studies have revealed that the fuzzy test method based on the confidence interval must be able to derive the two-tailed confidence interval of the indicator, so that the subsequent fuzzy testing procedure can be completed [11,12,17,18]. Accordingly, if the two-tailed confidence interval of the indicator cannot be derived, then it must be completed by the Central Limit Theorem. Nevertheless, when the sample size is not large enough, it will lead to larger sampling errors. Considering the customer satisfaction questionnaire survey, the sample size is relatively large. Thus, in this paper, on the basis of the Central Limit Theorem, the distribution of random variable $Z_{Sh}$ approximates the standard normal distribution for large sample size n [21], expressed as follows:

(6)$Z_{Sh}=\frac{{\widehat{I}}_{Sh}-I_{Sh}}{S_{Xh}/\sqrt{n}}\approx N\left(0,1\right).$

Based on the above-mentioned, we have $1-\alpha =p\left(-z_{\alpha /2}\le Z_{Sh}\le z_{\alpha /2}\right)$, where $z_{\alpha }$ is the upper $\alpha$ quantile of the standard normal distribution. As noted by Lin et al. [19], Satisfaction Index $I_{Sh}$ is set as the x-axis to form the PEM. Let $\left(x_{ h,1},\dots ,x_{ h,j},\dots ,x_{ h,n}\right)$ be the observed value of $\left(X_{ h,1},\dots ,X_{ h,j},\dots ,X_{ h,n}\right)$. Then ${\widehat{I}}_{Sh0}$ is the observed value of ${\widehat{I}}_{Sh}$, written as follows:

(7)${\widehat{I}}_{Sh0}=\frac{1}{n}\times {\displaystyle \sum _{j=1}^n{x}_{h,j}}.$

The average of ${\widehat{I}}_{Sh0}$ is expressed as follows:

(8)$I_{S0}=\frac{1}{k}{\displaystyle \sum _{h=1}^k{\widehat{I}}_{Sh0}}.$

According to Yu et al. [17], when the value of Satisfaction Index for Service Item h is lower than the average value ($I_{Sh}\le I_{S0}$), then Service Item h must be improved. The hypotheses of the statistical test for the Satisfaction Index of Service Item h are written as follows:

(9)$\mathrm{null} \mathrm{hypothesis} H_0: I_{Sh}\ge I_{S0};$

(10)$\mathrm{alternative} \mathrm{hypothesis} H_1: I_{Sh}<I_{S0}.$

The significance level of the test is $\beta$ and the critical region can be defined as $C{R}_{Sh}=\left\{{\widehat{I}}_{Sh0}<C_{h0}\right\}=\left\{Z_{Sh}<\sqrt{n}\left(C_{h0}-I_{S0}\right)/S_{Xh}\right\}$. Therefore, the critical value of $C_{h0}$ is determined by

(11)$p\left\{Z_{Sh}<\frac{C_{h0}-I_{S0}}{S_{Xh0}/\sqrt{n}}\right\}=\beta ,$

where $S_{Xh0}$ is the observed value of $S_{Xh}$, written as follows:

(12)$S_{Xh0}=\sqrt{\frac{1}{n-1}\times {\displaystyle \sum _{j=1}^n{\left(x_{h,j}-{\overline{x}}_h\right)}^2}}.$

Thus, the critical value is denoted by $C_{h0}=I_{S0}-z_{\beta }{S}_{Xh0}/\sqrt{n}$. Obviously, we have $p\left\{{\widehat{I}}_{Sh}>C_{h0} \left|I_{Sh}=I_{S0}\right.\right\}=$ $p\left\{Z_{Sh}<-z_{\beta }\left|I_{Sh}=I_{S0}\right.\right\}$ $=\beta$. Let the observed value of $Z_{Sh}$ be $Z_{Sh0}$. Then

(13)$Z_{Sh0}=\frac{{\widehat{I}}_{Sh0}-I_{S0}}{S_{Xh0}/\sqrt{n}}.$

Thus, we can replace ${\widehat{I}}_{Sh0}$ with $Z_{Sh0}$ as the testing statistic and replace $C_{h0}$ with $z_{\beta }$ as the critical value. The $\alpha \text{-}cuts$ of the quasi-triangular fuzzy number ${\tilde{z}}_{\beta }$ is expressed as follows:

(14)${\tilde{z}}_{\beta }\left[\alpha \right]=\left\{\begin{array}{l}\left[z_{\beta 1}\left(\alpha \right),z_{\beta 2}\left(\alpha \right)\right]=\left[-z_{\beta }-z_{\alpha /2},-z_{\beta }+z_{\alpha /2}\right], 0.01\le \alpha \le 1\\ \left[z_{\beta 1}\left(\alpha \right),z_{\beta 2}\left(\alpha \right)\right]=\left[-z_{\beta }-z_{0.005},-z_{\beta }+z_{0.005}\right], 0\le \alpha \le 0.01\end{array}\right..$

Obviously, when $\alpha =1$, then $z_{\alpha /2}=0$ and $z_{\beta 1}(1)=z_{\beta 2}(1)=-z_{\beta }$. The fuzzy number is ${\tilde{z}}_{\beta }=\left(-z_{\beta L},-z_{\beta M},-z_{\beta R}\right)=\left(-z_{\beta }-z_{0.005},-z_{\beta },-z_{\beta }+z_{0.005}\right)$. In fact, the fuzzy evaluation technique proposed by this study based on the atypical fuzzy evaluation method suggested by Buckley [22] belongs to the type-2 of fuzzy sets [23]. A quasi-triangular fuzzy membership function is mainly constructed by the two-tailed confidence interval of the parameters that need to be evaluated. The quasi- triangular fuzzy membership function of ${\tilde{z}}_{\beta }$ is as follows:

(15)$\eta (x)=\left\{\begin{array}{l}0, if x<-z_{\beta }-z_{0.005}\\ 2\times \left(1-\Phi (-z_{\beta }-x)\right), if-z_{\beta }-z_{0.005}\le x<-z_{\beta }\\ 1, if x=-z_{\beta }\\ 2\times \left(1-\Phi (x+z_{\beta })\right), if-z_{\beta }<x\le -z_{\beta }+z_{0.005}\\ 0, if-z_{\beta }+z_{0.005}<x\end{array}\right..$

Based on Equation (15), membership function $\eta \left(x\right)$ with the vertical line of $x$ $=$ ${\widehat{I}}_{Sh0}$ is depicted in Figure 1.

Based on the concept of Yu et al. [24], let $d_{hR}=Z_{Sh0}-z_{\beta R}=Z_{Sh0}-(-z_{\beta }+z_{0.005})$ and $d_T=z_{\beta R}-z_{\beta L}=2\times z_{0.005}$. Then $d_{hR}/d_T$ is expressed as follows:

(16)$d_{hR}/d_T=\frac{-z_{\beta }+z_{0.005}-Z_{Sh0}}{2\times z_{0.005}}.$

Let the decision value be $d_{Sh0}$, such that

(17)$d_{hR}/d_T=\frac{-z_{\beta }+z_{0.005}-d_{Sh0}}{2\times z_{0.005}}=\varphi .$

Therefore, we have

(18)$d_{Sh0}=\left(1-2\varphi \right)z_{0.005}-z_{\beta }.$

According to Chen et al. [25], we let 0 $<$ $\varphi$ $<$ 0.5, and the decision rules of the fuzzy two-tailed testing model are listed below:

• (1). If $Z_{Sh0}<d_{Sh0}$, then $d_{hR}/d_T<\varphi$. Therefore, we reject $H_0$ and draw the conclusion $I_{Sh}<I_{S0}$. Thus, Service Item h needs to be improved.

• (2). If $Z_{Sh0}\ge d_{Sh0}$, then $d_{hR}/d_T\ge \varphi$. Thus, we do not reject $H_0$ and draw the conclusion $I_{Sh}\ge I_{S0}$. Thus, Service Item h does not need to be improved.

3. Fuzzy Evaluation Rules for Importance Index

Let $\left(Y_{ h,1},\dots ,Y_{ h,j},\dots ,Y_{ h,n}\right)$ be the sample data of customer importance for Service Item h with a size of n, where h $=$ 1, 2,…, k. Then, the unbiased estimator of Important Index $I_{Ih}$ is defined below:

(19)${\widehat{I}}_{Ih}=\frac{1}{n}\times {\displaystyle \sum _{j=1}^n{Y}_{h,j}}.$

The expected value of unbiased estimator ${\widehat{I}}_{Ih}$, equal to $I_{Ih}$, is denoted by $E\left[{\widehat{I}}_{Ih}\right]=I_{Ih}$. Let random variable $Z_{Ih}$ be defined as:

(20)$Z_{Ih}=\frac{{\widehat{I}}_{Ih}-I_{Ih}}{S_{Yh}/\sqrt{n}},$

where $S_{Yh}$ is the sample standard deviation, written as:

(21)$S_{Yh}=\sqrt{\frac{1}{n-1}\times {\displaystyle \sum _{j=1}^n{\left(Y_{h,j}-{\overline{Y}}_h\right)}^2}}.$

According to the Central Limit Theorem, the distribution of the random variable $Z_{Ih}$ approximates the standard normal distribution, expressed as follows:

(22)$Z_{Ih}=\frac{{\widehat{I}}_{Ih}-I_{Ih}}{S_{Yh}/\sqrt{n}}\stackrel{n\to \infty }{\to }N\left(0,1\right).$

Based on the above-mentioned, we have $1-\alpha =p\left(-z_{\alpha /2}\le Z_{Ih}\le z_{\alpha /2}\right)$. Similarly, the Important Index $I_{Ih}$ is set as the y-axis to form the PEM. Let $\left(y_{ h,1},\dots ,y_{ h,j},\dots ,y_{ h,n}\right)$ be the observed value of $\left(Y_{ h,1},\dots ,Y_{ h,j},\dots ,Y_{ h,n}\right)$. Then ${\widehat{I}}_{Ih0}$ is the observed value of ${\widehat{I}}_{Ih}$, expressed as follows:

(23)${\widehat{I}}_{Ih0}=\frac{1}{n}\times {\displaystyle \sum _{j=1}^n{y}_{h,j}}.$

Then, the average of ${\widehat{I}}_{Ih0}$ is defined as follows:

(24)$I_{I0}=\frac{1}{k}{\displaystyle \sum _{h=1}^k{\widehat{I}}_{Ih0}}.$

According to Yu et al. [17], when the Importance Index of Service Item h is smaller than the average value ($I_{Ih}\le I_{I0}$), the improvement priority of Service Item h is low. On the contrary, when the Importance Index of Service Item h is greater than the average value ($I_{Ih}>I_{I0}$), the improvement priority of Service Item h is high. Then, the hypotheses of the statistical test for Important Index h are written as follows:

(25)$\mathrm{null} \mathrm{hypothesis} H_0: I_{Ih}\le I_{I0};$

(26)$\mathrm{alternative} \mathrm{hypothesis} H_1: I_{Ih}>I_{I0}.$

The significance level of the test is ${\beta }^{\prime }$, and the critical region is defined as $C{R}_{Ih}=\left\{{\widehat{I}}_{Ih}>C_{h0}^{\prime }\right\}=\left\{Z_{Ih}>\sqrt{n}\left(C_{h0}^{\prime }-I_{Ih}\right)/S_{Yh0}\right\}$, where $C_{h0}^{\prime }$ is determined by

(27)$p\left\{Z_{Ih}>\frac{C_{h0}^{\prime }-I_{I0}}{S_{Yh0}/\sqrt{n}}\right\}={\beta }^{\prime }.$

Therefore, the critical value is written as $C_{h0}^{\prime }=I_{I0}+z_{{\beta }^{\prime }}{S}_{Yh}/\sqrt{n}$. Obviously, $p\left\{{\widehat{I}}_{Ih}>C_{h0}^{\prime } \left|I_{Ih}=I_{I0}\right.\right\}=$ $p\left\{Z_{Ih}>z_{{\beta }^{\prime }}\left|I_{Ih}=I_{I0}\right.\right\}$ $={\beta }^{\prime }$. Let the observed value of $Z_{Ih}$ be $Z_{Ih0}$, written as follows:

(28)$Z_{Ih0}=\frac{{\widehat{I}}_{Ih0}-I_{I0}}{S_{Yh0}/\sqrt{n}},$

where $S_{Yh0}$ is the observed value of $S_{Yh}$ as follows:

(29)$S_{Yh0}=\sqrt{\frac{1}{n-1}\times {\displaystyle \sum _{j=1}^n{\left(Y_{h,j}-{\overline{Y}}_h\right)}^2}}.$

Thus, we can then replace ${\widehat{I}}_{Ih0}$ with $Z_{Ih0}$ as the testing statistic and replace $C_{h0}^{\prime }$ with $-z_{{\beta }^{\prime }}$ as the critical value. The $\alpha \text{-}cuts$ of the triangular fuzzy number ${\tilde{z}}_{{\beta }^{\prime }}$ is

(30)${\tilde{z}}_{{\beta }^{\prime }}\left[\alpha \right]=\left\{\begin{array}{l}\left[z_{{\beta }^{\prime }1}\left(\alpha \right),z_{{\beta }^{\prime }2}\left(\alpha \right)\right]=\left[z_{{\beta }^{\prime }}-z_{\alpha /2},z_{{\beta }^{\prime }}+z_{\alpha /2}\right], 0.01\le \alpha \le 1\\ \left[z_{{\beta }^{\prime }1}\left(\alpha \right),z_{{\beta }^{\prime }2}\left(\alpha \right)\right]=\left[z_{{\beta }^{\prime }}-z_{0.005},z_{{\beta }^{\prime }}+z_{0.005}\right], 0\le \alpha \le 0.01\end{array}\right..$

As noted by Chen and Yu [18], when $\alpha =1$, then $z_{{\beta }^{\prime }1}(1)=z_{{\beta }^{\prime }2}(1)=z_{{\beta }^{\prime }}$. Therefore, the fuzzy number is ${\tilde{z}}_{{\beta }^{\prime }}=\left(z_{{\beta }^{\prime }L}, z_{{\beta }^{\prime }M}, z_{{\beta }^{\prime }R}\right)=\left(z_{{\beta }^{\prime }}-z_{0.005}, z_{{\beta }^{\prime }}, z_{{\beta }^{\prime }}+z_{0.005}\right)$, and the membership function of ${\tilde{z}}_{{\beta }^{\prime }}$ is

(31)${\eta }^{\prime }(x)=\left\{\begin{array}{l}0 ,if x\le z_{{\beta }^{\prime }}-z_{0.005}\\ 2\times \left(1-\Phi (z_{{\beta }^{\prime }}-x)\right) ,if z_{{\beta }^{\prime }}-z_{0.005}<x<z_{{\beta }^{\prime }}\\ 1 ,if x=z_{{\beta }^{\prime }}\\ 2\times \left(1-\Phi (x-z_{{\beta }^{\prime }})\right) ,if z_{{\beta }^{\prime }}<x<z_{{\beta }^{\prime }}+z_{0.005}\\ 0 ,if z_{{\beta }^{\prime }}+z_{0.005}\le x\end{array}\right..$

Following Equation (31), the diagram of membership function ${\eta }^{\prime }\left(x\right)$ with the vertical line of $x$ $=$ $Z_{Ih0}$ is presented in Figure 2.

According to Yu et al. [24], let $d_{hR}^{\prime }=z_{{\beta }^{\prime }R}-Z_{Ih0}=(z_{{\beta }^{\prime }}+z_{0.005})-Z_{Ih0}$ and $d_T^{\prime }=z_{{\beta }^{\prime }R}-z_{{\beta }^{\prime }L}=2\times z_{0.005}$. Then $d_{hR}^{\prime }/d_T^{\prime }$ is defined as follows:

(32)$d_{hR}^{\prime }/d_T^{\prime }=\frac{Z_{Ih0}-z_{{\beta }^{\prime }}+z_{0.005}}{2\times z_{0.005}}.$

Let $d_{Ih0}$, such that

(33)$d_{hR}^{\prime }/d_T^{\prime }=\frac{d_{Ih0}-z_{{\beta }^{\prime }}+z_{0.005}}{2\times z_{0.005}}=\varphi .$

Thus, we have

(34)$d_{I0}=\left(2\varphi -1\right)z_{0.005}+z_{{\beta }^{\prime }}.$

On the basis of the study of Chen et al. [25], we let 0 $<$ $\varphi$ $<$ 0.5, and the decision-making rules of the fuzzy two-tailed testing model are displayed below:

• (1). If $Z_{Ih0}>d_{Ih0}$, then $d_{hR}^{\prime }/d_T^{\prime }<\varphi$. Therefore, we reject $H_0$ and draw the conclusion $I_{Ih}>I_{I0}$. Consequently, Service Item h has a high priority for improvement.

• (2). If $Z_{Ih0}\le d_{Ih0}$, then $d_{hR}^{\prime }/d_T^{\prime }\ge \varphi$. Therefore, we do not reject $H_0$ and draw the conclusion $I_{Ih}\le I_{I0}$. Consequently, Service Item h has a low priority for improvement.

4. An Application Example

In order to explain the application of the above model, this paper adopted a foreign language teaching satisfaction questionnaire made by Yu et al. [26], including five dimensions to reflect the services provided by foreign language teaching. Among them, Dimension 1 is Teaching Preparation, containing 4 teaching service items; Dimension 2 is Teaching Attitude, including 5 teaching service items; Dimension 3 is Teaching Capability, containing 2 teaching service items; Dimension 4 is Teaching Management, containing 3 teaching service items; finally, Dimension 5 is Coursework and Evaluation, containing 2 teaching service items. These five dimensions include a total of 16 question items, and each question item has two subquestions about importance and satisfaction. Therefore, the entire questionnaire has a total of 32 questions that need to be answered. A total of 16 questions (k = 16) in these 5 dimensions are depicted as follows:

Dimension 1: Teaching Preparation

• The course material that the teacher has prepared is at an adequate level of complexity (X₁, Y₁).

• The quantity of the material is appropriate (X₂, Y₂).

• The content of the material helps improve my foreign language proficiency (X₃, Y₃).

• The teacher has prepared thoroughly for the class (X₄, Y₄).

Dimension 2: Teaching Attitude

• 5.. The teacher emphasizes conversation practice in a foreign language (X₅, Y₅).

• 6.. The teacher values the opinions of students (X₆, Y₆).

• 7.. Student-teacher interaction in class is intensive (X₇, Y₇).

• 8.. The teacher is happy to help students solve problems (X₈, Y₈).

• 9.. The teacher treats all students fairly (X₉, Y₉).

Dimension 3: Teaching Capability

• 10.. The teacher speaks the foreign language clearly (X₁₀, Y₁₀).

• 11.. The teacher expresses himself/herself logically (X₁₁, Y₁₁).

Dimension 4: Teaching Management

• 12.. The teacher uses a variety of teaching methods (X₁₂, Y₁₂).

• 13.. The teacher interests me in learning the foreign language (X₁₃, Y₁₃).

• 14.. The teacher adequately controls the pace of teaching (X₁₄, Y₁₄).

Dimension 5: Coursework and Evaluation

• 15.. The coursework or evaluation helps me improve my foreign language proficiency (X₁₅, Y₁₅).

• 16.. Evaluation is at an adequate level of complexity (X₁₆, Y₁₆).

As mentioned above, the performance of running foreign language learning curriculums is the foundation for students who intend to increase their foreign language proficiency since foreign language proficiency is often one of the key indicators for recruitment in the corporate world [27,28,29]. It can not only help students improve their learning efficiency in other professional subjects but also enhance their competitiveness for more advanced studies or employment. Therefore, foreign language learning curriculums are listed in important teaching enhancement plans promoted by various universities [26].

Based on the above-mentioned 16 questions in the foreign language teaching satisfaction questionnaire, a total of 350 copies of questionnaire were given to students in the case-study school and returned on the spot; in total, 324 copies of questionnaire were collected, yielding a response rate of 92.5%.

First, we calculate observed values ${\widehat{I}}_{Sh0}$ and $S_{Xh0}$ for each service item according to Equations (7) and (12). Following Equation (8), we calculate the average of ${\widehat{I}}_{Sh0}$ as follows:

$I_{S0}=\frac{1}{16}{\displaystyle \sum _{h=1}^{16}{\widehat{I}}_{Sh0}}=0.645.$

Therefore, the hypotheses of the fuzzy test for Satisfaction Index h are written as follows:

$\mathrm{null} \mathrm{hypothesis} H_0: I_{Sh}\ge 0.645;$

$\mathrm{alternative} \mathrm{hypothesis} H_1: I_{Sh}<0.645.$

This study sets the significance level $\beta$ as 0.05, then the membership function $\eta \left(x\right)$ with significance level $\beta$ $=$ 0.05 is expressed as follows:

$\eta (x)=\left\{\begin{array}{l}0, if x<-4.221\\ 2\times \left(1-\Phi (-1.645-x)\right), if-4.221\le x<-1.645\\ 1, if x=-1.645\\ 2\times \left(1-\Phi (x+1.645)\right), if-1.645<x\le 0.931\\ 0, if 0.931<x\end{array}\right..$

Let $\varphi$ $=$ 0.4. Then the decision value is $d_{S0}=\left(1-2\varphi \right)z_{0.005}-z_{0.05}=$ 0.6 $\times$ 2.576 $-$ 1.645 $=$ $-$0.10. Following Equation (15), the values of $Z_{Sh0}$ for all service items are calculated and shown in Table 1.

Similar to the Satisfaction Index, we calculate observed values ${\widehat{I}}_{Ih0}$ and $S_{Yh0}$ for each service item according to Equations (23) and (29). Following Equation (24), we calculate the average of ${\widehat{I}}_{Ih0}$ as follows:

$I_{I0}=\frac{1}{16}{\displaystyle \sum _{h=1}^{16}{\widehat{I}}_{Ih0}}=0.682.$

Therefore, the hypotheses of the fuzzy test for Important Index of Service Item h are written as follows:

$\mathrm{null} \mathrm{hypothesis} H_0^{\prime }: I_{Ih}\le 0.682;$

$\mathrm{alternative} \mathrm{hypothesis} H_1^{\prime }: I_{Ih}>0.682.$

This study sets the significance level ${\beta }^{\prime }$ as 0.05 and $\varphi$ $=$ 0.4, then the decision value is $d_{I0}=\left(2\varphi -1\right)z_{0.005}+z_{0.05}$ $=$ $-$0.6 $\times$ 2.576 $+$ 1.645 $=$ 0.10. According to Equation (28), the values of $Z_{Ih0}$ for all service items are calculated and displayed in Table 2.

In the fuzzy improvement decision table for satisfaction of service items, the five service items requiring improvement are as follows: “The teacher has prepared thoroughly for the class” (Item 4), “The teacher values the opinions of students” (Item 6), “The teacher is happy to help students solve problems” (Item 8), “The teacher interests me in learning a foreign language” (Item 13), and “The teacher adequately controls the pace of teaching” (Item 14). In the fuzzy decision list of improvement prioritization, improvement priority of Service Items 4, 8, and 13 is low, whereas that of Service Items 6 and 14 is high. Improvement involves various concerns, such as the cost of hiring high-quality teachers, teaching training time and methods, expenses related to formulating various reward and punishment systems, and other associated costs. Consequently, it is recommended that Service Items 6 and 14 should be listed in as the top priority for improvement within the constraints of limited resources and time to enhance the effectiveness of improvement efforts.

As highlighted by a number of studies, the fuzzy test based on the confidence interval tends to be more practical than the statistical test in real-world scenarios [11,12]. The value of $Z_{Sh0}$ for Service Item 13 is $-$0.11 in the above-mentioned case, which exceeds the critical value $-$$z_{0.05}$. Had the statistical test been employed, then null hypothesis would not have been rejected, potentially leading us to miss opportunities for improvement. Furthermore, this model takes the prioritization of service item improvements into account, offering a basis for decision making when resources are limited. Lastly, with standardized decision-making values and critical values, compared with other existing methods based on fuzzy tests and confidence intervals, this model is more convenient for decision makers [30,31].

5. Conclusions, Research Limitations and Future Research

The performance evaluation matrix can evaluate all service items of the service operating system simultaneously. The Importance Index and Satisfaction Index of the service items serve as two significant evaluation indicators of the performance evaluation matrix. To cater to enterprises’ needs for rapid decision-making, this paper initially derived the expected value and standard deviation of the Satisfaction Index estimation formula and set the random variable $Z_{Sh}$ equal to the standardized statistic of Satisfaction Index for Service Item h. According to the Central Limit Theorem, $Z_{Sh}$ followed the standard normal distribution, and then the $\beta$ lower quantile of the standard normal distribution was used as the critical value to establish the fuzzy test of the Satisfaction Index. Given the complexity of the fuzzy test, this paper obtained the fuzzy critical value $d_{S0}$ following the fuzzy testing rules. Managers only need to compare the $Z_{Ih}$ value and fuzzy critical value $d_{I0}$ of Service Item h to make decisions on whether to make improvements. Subsequently, this paper derived the expected value and standard deviation of the Important Index estimate and set the random variable $Z_{Ih}$ equal to the standardized statistic of the Important Index. Similarly, following the Central Limit Theorem, $Z_{Ih}$ also followed the standard normal distribution, and then the fuzzy test of the Important Index was established using the ${\beta }^{\prime }$ upper quantile of the standard normal distribution as the critical value. Since the fuzzy test was relatively complex, this paper received the fuzzy critical value $d_{I0}$ following the fuzzy testing rules. Managers only need to compare the values of $Z_{Ih}$ and $d_{I0}$ for each service item to determine whether the service item should be prioritized for improvement. In fact, the fuzzy evaluation model proposed in this paper can maintain evaluation accuracy in cases of small samples by incorporating past accumulated data experience. In addition, the fuzzy critical value can be derived by the fuzzy testing rules, and the decision-making rules can be established by the fuzzy critical value, which can facilitate industrial applications.

The fuzzy evaluation model presented in this paper is built on the confidence intervals of indices. Moreover, its importance lies in its ability to integrate past accumulated data and experts’ experiences to make the evaluation more authentic in practical settings [18,24]. However, this study has its limitations, including insufficient accumulation of past data and immature analysis techniques for experts’ experiences. In future research, we can focus on the management’s need for sophisticated techniques of data analysis as well as develop analysis and decision-making models based on the accumulated data. Additionally, the method proposed in this paper must be capable of deriving the two-tailed confidence interval of the indicator to complete the subsequent fuzzy testing process. If the two-tailed confidence interval of the indicator cannot be obtained, it may require a larger sample size and application of the Central Limit Theorem to reduce sampling errors caused by insufficient sample size. Therefore, future research could also explore the incorporation of other decision-making methods, such as the Multi-Criteria Decision-Making Fuzzy Methods proposed by Al-shami and Mhemdi [32].

Author Contributions

Conceptualization, K.-S.C. and C.-P.C.; methodology, K.-S.C., C.-P.C.; software, T.-H.H. (Tsung-Hua Hsieh); validation, K.-C.Y. and T.-H.H. (Tsun-Hung Huang); formal analysis, K.-S.C. and C.-P.C.; resources, T.-H.H. (Tsung-Hua Hsieh); data curation, T.-H.H. (Tsung-Hua Hsieh); writing—original draft preparation, K.-S.C., T.-H.H. (Tsung-Hua Hsieh), C.-P.C., K.-C.Y. and T.-H.H. (Tsun-Hung Huang); writing—review and editing, K.-S.C. and C.-P.C.; visualization, T.-H.H. (Tsung-Hua Hsieh); supervision, K.-S.C.; project administration, C.-P.C.; All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Membership function [Forumla omitted. See PDF.] with the vertical line of [Forumla omitted. See PDF.] [Forumla omitted. See PDF.] [Forumla omitted. See PDF.].

Figure 2. Membership function [Forumla omitted. See PDF.] with the vertical line of [Forumla omitted. See PDF.] [Forumla omitted. See PDF.] [Forumla omitted. See PDF.].

Table 1

The fuzzy improvement decision table for satisfaction of service items.

Dimensions/Item	${\widehat{\mathit{I}}}_{\mathit{S}\mathit{h}\mathbf{0}}$	${\mathit{Z}}_{\mathit{S}\mathit{h}\mathbf{0}}$	Remark
Dimension 1: Teaching Preparation
The course material that the teacher has prepared is at an adequate level of complexity.	0.72	1.95
2. The quantity of the material is appropriate.	0.69	0.89
3. The content of the material helps improve my foreign language proficiency.	0.76	2.24
4. The teacher has prepared thoroughly for the class.	0.52	−2.79	Improve
Dimension 2: Teaching Attitude
5. The teacher emphasizes conversation practice in a foreign language.	0.67	0.45
6. The teacher values the opinions of students.	0.49	−2.76	Improve
7. Student-teacher interaction in class is intensive.	0.68	0.54
8. The teacher is happy to help students solve problems.	0.51	−3.27	Improve
9. The teacher treats all students fairly.	0.69	0.82
Dimension 3: Teaching Capability
10. The teacher speaks the foreign language clearly.	0.71	1.22
11. The teacher expresses himself/herself logically.	0.73	1.61
Dimension 4: Teaching Management
12. The teacher uses a variety of teaching methods.	0.69	0.73
13. The teacher interests me in learning a foreign language.	0.65	−0.11	Improve
14. The teacher adequately controls the pace of teaching.	0.54	−2.37	Improve
Dimension 5: Coursework and Evaluation
15. The coursework or evaluation helps me improve my foreign language proficiency.	0.72	1.40
16. Evaluation is at an adequate level of complexity.	0.69	0.92

Remark: If $Z_{Sh0}<d_{Sh0}$ = −0.10, then Service Item h needs to be improved.

Table 2

Fuzzy decision list of improvement prioritization.

Dimensions/Item	${\widehat{\mathit{I}}}_{\mathit{I}\mathit{h}0}$	${\mathit{Z}}_{\mathit{I}\mathit{h}0}$	Priority
Dimension 1: Teaching Preparation
The course material that the teacher has prepared is at an adequate level of complexity.	0.75	1.68
2. The quantity of the material is appropriate.	0.68	−0.05
3. The content of the material helps improve my foreign language proficiency.	0.73	2.03
4. The teacher has prepared thoroughly for the class.	0.59	−2.15	Low
Dimension 2: Teaching Attitude
5. The teacher emphasizes conversation practice in a foreign language.	0.58	−3.00
6. The teacher values the opinions of students.	0.78	1.23	High
7. Student-teacher interaction in class is intensive.	0.72	0.84
8. The teacher is happy to help students solve problems.	0.56	−2.86	Low
9. The teacher treats all students fairly.	0.57	−2.84
Dimension 3: Teaching Capability
10. The teacher speaks the foreign language clearly.	0.82	2.80
11. The teacher expresses himself/herself logically.	0.71	0.56
Dimension 4: Teaching Management
12. The teacher uses a variety of teaching methods.	0.66	−0.52
13. The teacher interests me in learning a foreign language.	0.61	−1.83	Low
14. The teacher adequately controls the pace of teaching.	0.80	2.73	High
Dimension 5: Coursework and Evaluation
15. The coursework or evaluation helps me improve my foreign language proficiency.	0.74	1.32
16. Evaluation is at an adequate level of complexity.	0.61	−1.86

References

1. Li, Y.; Wang, L.; Li, F. A data-driven prediction approach for sports team performance and its application to national basketball association. Omega; 2021; 98, 102123. [DOI: https://dx.doi.org/10.1016/j.omega.2019.102123]

2. Jeng, M.Y.; Yeh, T.M.; Pai, F.Y. A Performance Evaluation Matrix for Measuring the Life Satisfaction of Older Adults Using eHealth Wearables. Healthcare; 2022; 10, 605. [DOI: https://dx.doi.org/10.3390/healthcare10040605] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35455784]

3. Kim, J.; Kim, E.; Hong, A. Ott streaming distribution strategies for dance performances in the post-COVID-19 age: A modified importance-performance analysis. Int. J. Environ. Res. Public Health; 2022; 19, 327. [DOI: https://dx.doi.org/10.3390/ijerph19010327] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35010585]

4. Wong, C.P.; Yang, L.; Szeto, W.Y. Comparing passengers’ satisfaction with fixed-route and demand-responsive transport services: Empirical evidence from public light bus services in Hong Kong. Travel Behav. Soc.; 2023; 32, 100583. [DOI: https://dx.doi.org/10.1016/j.tbs.2023.100583]

5. Askari, S.; Peiravian, F.; Tilahun, N.; Yousefi Baseri, M. Determinants of users’ perceived taxi service quality in the context of a developing country. Transp. Lett.; 2021; 13, pp. 125-137. [DOI: https://dx.doi.org/10.1080/19427867.2020.1714844]

6. Mustafa, H.; Omar, B.; Mukhiar, S.N.S. Measuring destination competitiveness: An importance-performance analysis (IPA) of six top island destinations in south east asia. Asia Pac. J. Tour. Res.; 2020; 25, pp. 223-243. [DOI: https://dx.doi.org/10.1080/10941665.2019.1687534]

7. Kumar, S.; Janardhanan, A.K.; Khanna, S.; William, R.M.; Saha, S. Students’ Satisfaction with Remote Learning During the COVID-19 Pandemic: Insights for Policymakers. Prabandhan Indian J. Manag.; 2023; 16, pp. 43-60. [DOI: https://dx.doi.org/10.17010/pijom/2023/v16i9/173166]

8. Nam, S.; Lee, H.C. A text analytics-based importance performance analysis and its application to airline service. Sustainability; 2019; 11, 6153. [DOI: https://dx.doi.org/10.3390/su11216153]

9. Sumrit, D.; Sowijit, K. Winning customer satisfaction toward omnichannel logistics service quality based on an integrated importance-performance analysis and three-factor theory: Insight from Thailand. Asia Pac. Manag. Rev.; 2023; 28, pp. 531-543. [DOI: https://dx.doi.org/10.1016/j.apmrv.2023.03.003]

10. Wu, J.; Yang, T. Service attributes for sustainable rural tourism from online comments: Tourist satisfaction perspective. J. Destin. Mark. Manag.; 2023; 30, 100822. [DOI: https://dx.doi.org/10.1016/j.jdmm.2023.100822]

11. Chen, H.Y.; Lin, K.P. Fuzzy supplier selection model based on lifetime performance index. Expert Syst. Appl.; 2022; 208, 118135. [DOI: https://dx.doi.org/10.1016/j.eswa.2022.118135]

12. Lo, W.; Yang, C.M.; Lai, K.K.; Li, S.Y.; Chen, C.H. Developing a novel fuzzy evaluation model by one-sided specification capability indices. Mathematics; 2021; 9, 1076. [DOI: https://dx.doi.org/10.3390/math9101076]

13. Lin, C.J.Y. General education competencies from students’ perspectives: A case study of a sports university in Taiwan. Hum. Soc. Sci. Commun.; 2023; 10, 848. [DOI: https://dx.doi.org/10.1057/s41599-023-02344-z]

14. Ramírez-Hurtado, J.M.; Hernández-Díaz, A.G.; López-Sánchez, A.D.; Pérez-León, V.E. Measuring online teaching service quality in higher education in the COVID-19 environment. Int. J. Environ. Res. Public Health; 2021; 18, 2403. [DOI: https://dx.doi.org/10.3390/ijerph18052403] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33804546]

15. Liu, S.H.; Tsai, C.H. Service quality of ski group package tours: A modified importance-performance analysis. Int. J. Serv. Oper. Manag.; 2023; 46, pp. 35-59. [DOI: https://dx.doi.org/10.1504/IJSOM.2023.133413]

16. Oey, E.; Librianne, G.; Elvira, E.; Irawan, D.A. Comparing two ways of integrating fuzzy Kano and importance performance analysis—With a case study in a beauty clinic. Int. J. Product. Qual. Manag.; 2023; 38, pp. 285-311. [DOI: https://dx.doi.org/10.1504/IJPQM.2023.129604]

17. Yu, C.M.; Li, S.Y.; Yu, C.H.; Chen, Y.P. The Performance Evaluation Model of Bus APP. Int. J. Reliab. Qual. Saf. Eng.; 2023; 30, 2350023. [DOI: https://dx.doi.org/10.1142/S0218539323500237]

18. Chen, K.S.; Yu, C.M. Fuzzy test model for performance evaluation matrix of service operating systems. Comput. Ind. Eng.; 2020; 140, 106240. [DOI: https://dx.doi.org/10.1016/j.cie.2019.106240]

19. Lin, T.C.; Chen, H.H.; Chen, K.S.; Chen, Y.P.; Chang, S.H. Decision-Making Model of Performance Evaluation Matrix Based on Upper Confidence Limits. Mathematics; 2023; 11, 3499. [DOI: https://dx.doi.org/10.3390/math11163499]

20. Yu, C.M.; Zhuo, Y.J.; Lee, T.S. A Novel Social Media App Performance Evaluation Model. J. Econ. Manag.; 2020; 16, pp. 69-81.

21. Pishro-Nik, H. Introduction to Probability, Statistics, and Random Processes; Kappa Research LLC: Sunderland, MA, USA, 2014; Available online: https://www.probabilitycourse.com (accessed on 1 March 2024).

22. Buckley, J.J. Fuzzy statistics: Hypothesis testing. Soft Comput.; 2005; 9, pp. 512-518. [DOI: https://dx.doi.org/10.1007/s00500-004-0368-5]

23. Dereli, T.; Baykasoglu, A.; Altun, K.; Durmusoglu, A.; Türksen, I.B. Industrial applications of type-2 fuzzy sets and systems: A concise review. Comput. Ind.; 2011; 62, pp. 125-137. [DOI: https://dx.doi.org/10.1016/j.compind.2010.10.006]

24. Yu, C.M.; Lai, K.K.; Chen, K.S.; Chang, T.C. Process-quality evaluation for wire bonding with multiple gold wires. IEEE Access; 2020; 8, pp. 106075-106082. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.2998463]

25. Chen, K.S.; Wang, C.H.; Tan, K.H. Developing a fuzzy green supplier selection model using Six Sigma quality indices. Int. J. Prod. Econ.; 2019; 212, pp. 1-7. [DOI: https://dx.doi.org/10.1016/j.ijpe.2019.02.005]

26. Yu, C.M.; Chang, H.T.; Hsu, S.Y. An assessment of quality and quantity for foreign language training course to enhance students’ learning effectiveness. Int. J. Inf. Manag. Sci.; 2017; 28, pp. 53-66.

27. Li, R. Research trends of blended language learning: A bibliometric synthesis of SSCI-indexed journal articles during 2000–2019. ReCALL; 2022; 34, pp. 309-326. [DOI: https://dx.doi.org/10.1017/S0958344021000343]

28. Li, X.; Huang, X. Improvement and Optimization Method of College English Teaching Level Based on Convolutional Neural Network Model in an Embedded Systems Context. Comput. Aided Des. Appl.; 2024; 21, pp. 212-227. [DOI: https://dx.doi.org/10.14733/cadaps.2024.S8.212-227]

29. Graham, K.M.; Yeh, Y.F. Teachers’ implementation of bilingual education in Taiwan: Challenges and arrangements. Asia Pacific Educ. Rev.; 2023; 24, pp. 461-472. [DOI: https://dx.doi.org/10.1007/s12564-022-09791-4]

30. Chiou, K.C. Building up of Fuzzy Evaluation Model of Life Performance Based on Type-II Censored Data. Mathematics; 2023; 11, 3686. [DOI: https://dx.doi.org/10.3390/math11173686]

31. Huang, C.C.; Chang, T.C.; Chen, B.L. Fuzzy assessment model to judge quality level of machining processes involving bilateral tolerance using crisp data. J. Chin. Inst. Eng.; 2021; 44, pp. 1-10. [DOI: https://dx.doi.org/10.1080/02533839.2020.1831966]

32. Al-shami, T.M.; Mhemdi, A. Generalized Frame for Orthopair Fuzzy Sets: (m,n)-Fuzzy Sets and Their Applications to Multi-Criteria Decision-Making Methods. Information; 2023; 14, 56. [DOI: https://dx.doi.org/10.3390/info14010056]