Reliability Assessment of Cloud Computing Platform Based on Semiquantitative Information and Evidential Reasoning

Abstract

A reliability assessment method based on evidential reasoning (ER) rule and semiquantitative information is proposed in this paper, where a new reliability assessment architecture including four aspects with both quantitative data and qualitative knowledge is established. The assessment architecture is more objective in describing complex dynamic cloud computing environment than that in traditional method. In addition, the ER rule which has good performance for multiple attribute decision making problem is employed to integrate different types of the attributes in assessment architecture, which can obtain more accurate assessment results. The assessment results of the case study in an actual cloud computing platform verify the effectiveness and the advantage of the proposed method.

Academic Editor:Zhijie Zhou

School of Computer Science and Technology, Harbin University of Science and Technology, Harbin, Heilongjiang, China

Received 31 July 2016; Accepted 14 September 2016

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Cloud computing, as the Pay-Per-Use computing service model which has the capacity for large-scale resource scheduling, is attracting more and more attention. Currently, scheduling strategy of cloud computing can be divided into 3 types: performance-driven scheduling strategy [1, 2], profit-driven scheduling strategy [3, 4], and reliability-driven scheduling strategy [5, 6]. With the upsurge of commercial operation, the requirements for the cloud computing systems reliability are more stringent than ever. The cloud systems are more dangerous because the data, services, and computing resource are highly clustered, which is more easily to be broken by the outside hackers or internal users with unstable operation. Recently, the failure events of cloud service frequently occurred. For instance, Azure cloud services of Microsoft had two times interrupt service due to the system failure in March 2015, which affected most parts of the central and eastern United States service areas. In Apple Cloud Services, about 11 iCloud-related features had experienced serious technical issues on May 20, 2015, including iCloud Account and iCloud mail, by which 2 million iCloud users were affected. Therefore, the first 2 strategies are not practicable because the reliability is not considered. Thus, as one of the significant factors of the task scheduling, the reliability assessment should be deeply considered in the cloud computing platforms [7].

Current assessment methods for cloud computing reliability mainly focus on the security assessment method [8], the risk assessment method [9], and the service assessment method [10]. But these approaches make the reliability assessment only in a narrow aspect point of view. The cloud computing system with huge computing process and complex structure is hard to establish the comprehensive assessment model because of the high heterogeneity in the infrastructure nodes. In cloud computing platform, a large number of reliability factors need to be considered, and the information between these factors is closely linked and mutually restricted. More intractably, most reliability factors include uncertainty information, which increase the difficulty for reliability assessment [11].

The essence of reliability assessment is to obtain a reasonable security state through the rational integration of multiattributes [12]. But in actual cloud computing operation, the available knowledge is incomplete and uncertain due to the limitation of information acquisition and resolution. Thus, the corresponding assessment models must have capability to deal with this uncertain information in assessment process. The current methods of reliability assessment include factor analysis [13], utility theory analysis [14], analytic hierarchy process [15, 16], Bayesian inference [17, 18], fuzzy logic reasoning [9, 19], hybrid method [20], and artificial intelligence reasoning [21-23]. The factor analysis can express the relationship between multiple attributes by a few weighted factors, which aims to reflect the comprehensive information of the system. But this method has a strict standard on the testing data. The utility theory analysis establishes the utility maximum function by analysis various factors. But the capability of information analysis is not good, and the solution of the utility function is too complicated to deal with a large amount of attributes. The analytic hierarchy process can effectively use the combination knowledge which includes both qualitative and quantitative information. After the quantification of the qualitative attributes, the weight vectors are determined to make corresponding assessment. But this method could not give the appropriate results when dealing with the incomplete information. Bayesian inference is a classical method of probabilistic reasoning, which can deal with uncertain problems. A statistical conclusion is obtained by using prior knowledge and sample information, but a large number of repeated trials are needed to obtain the probabilities; thus the training difficulty is increased. Fuzzy logic reasoning has a significant effect on the processing of a large number of inaccurate data, and it is suitable for the complex fuzzy systems. However, the precise membership function is hard to establish, which would produce large deviation of the results. Hybrid method is the combination of two or more computational approaches which can provide better advantages than single approaches. But it is hard to solve the conflict of the attributes in cloud computing systems, because of its parameter adjustment mode. D-S theory [24, 25] not only can deal with both qualitative knowledge and quantitative data very well, but also can deal with various types of uncertain information. Compared with Bayesian inference, the D-S theory can deal with the uncertainty due to the limitation of expert abilities or prior knowledge. After that, Wang et al. [26, 27] proposed evidential reasoning (ER) rule in late 1990s. ER rule is developed as a multicriteria decision analysis (MCDA) approach on the basis of belief decision and D-S theory [28, 29]. It has been widely employed in different fields [30, 31]. Compared to the D-S theory of evidence, the calculation process of the ER rule is linear, which reduces the computational complexity [32]. Moreover, the most important thing is that it can deal with the conflict of evidence attributes which cannot be solved by using D-S theory [33]. The conflict of evidence refers to the basic probability assignment of the two empty focal elements when the evidence is synthesized. The reliability assessment of cloud computing has different types of information needed to be integrated, which is the advantage of ER rule.

According to above description, the innovations of this paper are shown: (1) ER rule is first employed to establish the reliability assessment model of the cloud computing platform in this paper; (2) a new reliability assessment architecture including four aspects is proposed. A case study is given in the last section, and it demonstrates the good performance on assessing the reliability of a real cloud computing platform by using the proposed method.

The paper is organized as follows. In Section 2, the principle of ER rule is described. In Section 3, the reliability assessment of cloud computing platform is described, and the new reliability assessment architecture is proposed. In Section 4, a case study for assessing the reliability of an actual cloud computing platform is given, and the simulation results are analyzed. Finally, this paper is concluded in Section 5.

2. The Evidential Reasoning Rule: Basics

As mentioned above, ER rule can integrate different types of information, which include qualitative knowledge and quantitative data. It can also express various types of uncertainties such as fuzzy uncertainty, probabilistic uncertainty, and ignorance uncertainty [34]. The basic principle of the ER rule is introduced in this section.

Assume that there are M basic attributes (_r1 ,_r2 ,...,_ri ,...,_rM ) of a general attribute R in a two-level hierarchy, and (_w1 ,_w2 ,...,_wi ,...,_wM ) denotes the weights of the basic attributes, where 0<=_wi <=1. There are N assessment grades; then the basic step of the ER rule can be concluded as follows.

( 1 ) The belief degree should be converted into the basic probability mass. The process is shown as follows: [figure omitted; refer to PDF] where _Pi,j denotes the basic probability mass which refers to the degree of the ith basic attribute supporting the hypothesis that the attribute is assessed to the jth grade. _Pi,Θ denotes the rest of probability mass which has not been assigned to any consequent according to the ith basic attribute. _P¯i,Θ denotes the unassigned basic probability mass which refers to the unimportant degree of the ith basic attribute. _P~i,Θ denotes the unassigned basic probability mass which refers to the incomplete degree of the ith basic attribute.

( 2 ) ER rule is used to combine the first i basic attributes; the detailed process is described as follows: [figure omitted; refer to PDF] where _PI(i),j denotes the probability mass which refers to the degree of the first i basic attributes supporting the hypothesis that the attribute is assessed to the jth grade. _KI(i+1) can be calculated as [figure omitted; refer to PDF]

( 3 ) Thus, the final belief degree to the jth consequent and the remaining belief degree can be calculated as [figure omitted; refer to PDF]

3. Reliability Assessment of Cloud Computing Platform with ER Rule

3.1. Problem Formulation

In this section, the assessment attributes can be divided into 2 categories: quantitative data and qualitative knowledge. The quantitative data represents the data that can be expressed as certain quantity, amount, or range, such as Mean Time to Failures (MTTF) of central units, frequency of network attack, fault tolerant rate of cloud computing platform, and Mean Time to Repair (MTTR) of service. These attributes can be collected by the data monitoring systems. However, the qualitative knowledge is associated with the subjective quality of a situation or phenomenon, such as scalability of cloud system, controllability of access terminal, completeness of service, and stability of service. Qualitative knowledge is abstract and it either does not require measurement or cannot be measured because the reality they represent can only be approximated. Thus the decision making may be associated with uncertainty. Because the knowledge of these aspects is gained through observation combined with interpretative understanding, the uncertainty usually occurs because information is not clearly described or only described by partial and imprecise evidence, such as ability of personalized service and capability of disaster recovery [35]. ER rules can make full use of qualitative knowledge and quantitative data and can also express various types of uncertainties. In this section, a new reliability assessment architecture including four aspects is established, and then the ER rule is used to assess the reliability of the cloud computing platform.

3.2. The Basic Attributes of Cloud Computing Platform Reliability

A four-level reliability attributes structure of cloud computing platform is established, including both quantitative data and qualitative knowledge. Reliability of cloud computing platform is categorized in 4 distinct aspects: network reliability, hardware reliability, software reliability, and service reliability. For each attribute, the symbol "r" is numerically labeled according to the hierarchy, while the symbol "w" is valued as the weight which is one of the important parameters in the process of ER rule. In the aspect of the network reliability, all the attributes are measured as the quantitative data. In particular, considering the significance of network attack frequency in assessing the network reliability, the 4th level attributes are added. In the aspect of hardware reliability, most attributes are scalability of the cloud system and fault tolerance rate of the cloud platform. Scalability is the ability of cloud system to continue to run well when it is changed in volume in order to meet a requirement. The controllability of access terminal is the only qualitative attributes which is defined as an ability of access terminal in aspect of software reliability. Service reliability is an assessment for the Quality of Service (QoS); thus most of attributes are defined as qualitative knowledge. In this paper, we introduce 4-grade assessment levels as the frame of discernment of the reliability assessment model. The basic attributes of cloud computing platform reliability are shown in Table 1.

Table 1: The basic attributes of cloud computing platform reliability.

Reliability	1st level	2nd level	3rd level	4th level
Reliability of cloud computing platform (R)	Network reliability (r1 ), (w1 =0.25)	Network equipment reliability (r11 ), (w11 =0.5)	MTTF of central units (r111 ), (w111 =0.6)
MTTF of common units (r112 ), (w112 =0.4)
Network system reliability (r12 ), (w12 =0.5)	Frequency of network attack (r121 ), (w121 =0.6)	Dos attack (r1211 ), (w1211 =0.4)
Probe attack (r1212 ), (w1212 =0.3)
U2R attack (r1213 ), (w1213 =0.2)
R2L attack (r1214 ), (w1214 =0.1)
Vulnerability of network system (r122 ), (w122 =0.4)
Reliability of cloud computing platform (R)	Hardware reliability (r2 ), (w2 =0.25)	Cloud infrastructure reliability (r21 ), (w21 =0.5)	MTTF of cloud server (r211 ), (w211 =0.4)
MTTF of virtual machine (r212 ), (w212 =0.3)
Scalability of cloud system (r213 ), (w213 =0.3)
Cloud platform reliability (r22 ), (w22 =0.5)	MTTF of cloud platform I/O (r221 ), (w221 =0.5)
Fault tolerant rate of cloud platform (r222 ), (w222 =0.5)
Reliability of cloud computing platform (R)	Software reliability (r3 ), (w3 =0.25)	Application reliability (r31 ), (w31 =0.4)	Failure rate of web application (r311 ), (w311 =0.5)
Controllability of access terminal (r312 ), (w312 =0.5)
Cloud operating system reliability (r32 ), (w32 =0.6)	MTTF of cloud operating system (r321 ), (w321 =0.6)
Vulnerability of cloud operating system (r322 ), (w322 =0.4)
Reliability of cloud computing platform (R)	Service reliability (r4 ), (w4 =0.25)	Performance reliability (r41 ), (w41 =0.3)	Capacity of throughput (r411 ), (w411 =0.2)
Average response time of the service (r412 ), (w412 =0.3)
Failure rate of service (r413 ), (w413 =0.2)
MTTR of service (r414 ), (w414 =0.3)
Service availability (r42 ), (w42 =0.3)	Completeness of service (r421 ), (w421 =0.3)
Access success rate (r422 ), (w422 =0.4)
Ability of personalized service (r423 ), (w423 =0.3)
Service stability (r43 ), (w43 =0.4)	Service continuity and stability (r431 ), (w431 =0.4)
Disaster recovery capability (r432 ), (w432 =0.4)
Service transparency (r433 ), (w433 =0.2)

3.3. The Assessment Grades of the Attributes

In this paper, the assessment grades of the attributes of cloud computing platform are established. For the quantitative attributes, the data range is divided into 4 grades, determined as "excellent, good, common, and bad." The setting up of the interval of quantitative attributes is obtained by the expert experience and practical investigation, which ensure the data precision and traceability. For the qualitative attributes, the grade of assessment can be determined by experts according to the experience or investigation. The assessment rules can be established through the assessment grades, as shown in Table 2.

Table 2: The reference values of the assessment grades.

	E	G	C	B
r 111	90/kh	60/kh	30/kh	0/kh
r 112	60/kh	40/kh	20/kh	0/kh
r 1211	0 times/h	10 times/h	20 times/h	30 times/h
r 1212	0 times/h	10 times/h	20 times/h	30 times/h
r 1213	0 times/h	10 times/h	20 times/h	30 times/h
r 1214	0 times/h	10 times/h	20 times/h	30 times/h
r 122	0 times/mon	50 times/mon	100 times/mon	150 times/mon
r 211	30/kh	20/kh	10/kh	0/kh
r 212	15/kh	10/kh	5/kh	0/kh
r 213	Given by experts
r 221	9/kh	6/kh	3/kh	0/kh
r 222	Given by experts
r 311	0/fit	20/fit	40/fit	60/fit
r 312	Given by experts
r 321	9/kh	6/kh	3/kh	0/kh
r 322	0 times/mon	200 times/mon	400 times/mon	600 times/mon
r 411	Given by experts
r 412	0/ms	100/ms	200/ms	300/ms
r 413	0/fit	30/fit	60/fit	90/fit
r 414	0/h	10/h	20/h	30/h
r 421	Given by experts
r 422	100%	90%	80%	70%
r 423	Given by experts
r 431	Given by experts
r 432	Given by experts
r 433	Given by experts

3.4. The Belief Degree of the Attributes

When the value of an attribute of the assessment is determined, the corresponding belief degree should be calculated in order to obtain the basic probability mass in ((2)-(1)), as shown in the following: [figure omitted; refer to PDF] where U(_ri ) denotes the value of the attribute _ri and _Ri,j denotes the reference value of the jth grade for attribute _ri .

4. Case Study

In order to illustrate the detailed process of the reliability assessment, a case which uses the proposed reliability assessment architecture and ER rule to assess the reliability of an actual cloud computing platform is studied in this section.

4.1. The Actual Cloud Computing Platform

In this case, a cloud computing platform of Innovation Lab in HUST is investigated. The architecture of the platform can be divided into 4 layers and 2 centers. The layers consist of cloud infrastructure layer, cloud platform layer, cloud bus layer, and cloud application layer. The centers include cloud management center and cloud data center. The cloud infrastructure layer consists of hardware level and virtual level, including computers, servers, storage units, and network units. Employing technology of virtualization, the virtual level is a solution of IaaS service, which forms a virtual cluster from underlying hardware resources. The cloud platform layer includes service level and computing level. The purpose of the service layer is to provide the basic data or information to the upper layer through the processing of the form of service. The computing level includes distributed computing engine, concurrent computing engine, and utility computing engine. The cloud bus design includes service adapter level, bus level, and application adapter level. The service adapter provides an adapter for the service and bus communication in the cloud platform. The cloud bus is a kind of basic structure that interacts with the service through the service adapter. The application adapter provides an adapter for application software and bus communication in the cloud platform. The cloud application layer can be categorized into three types, such as inner application, external application, and platform of secondary development interface.

The cloud management center includes supercomputing center and security center. The supercomputing center can achieve rapid deployment, automation installation, and upgrade. The security center is responsible for user registration, user authentication, access control, and so forth. The cloud data center is in charge of processing and storage of data in the cloud platform. The architecture of the platform is shown in Figure 1.

Figure 1: The architecture of the cloud platform.

[figure omitted; refer to PDF]

4.2. The Process of the Reliability Assessment

The corresponding attributes described in Table 1 are collected from the cloud computing platform described above. Taking the frequency of network attack (_r121 ) as an example, assume that Dos attack _r1211 =2 times/h, Probe attack _r1212 =13 times/h, U2R attack _r1213 =8 times/h, and R2L attack _r1214 =21 times/h. Thus, the belief degrees of these attributes which are assessed to a grade can be calculated by (8): [figure omitted; refer to PDF]

The above belief degrees should be converted into the basic probability mass by (1)-(4) and the corresponding weights given in Table 1: [figure omitted; refer to PDF]

Then, in order to integrate the above probability masses and get the upper attribute, _KI(i+1) =_KI(1212) should be calculated at first: [figure omitted; refer to PDF]

Thus, the integration masses can be obtained by (5): [figure omitted; refer to PDF]

Next, in order to integrate the third probability mass, _KI(1213) should be calculated: [figure omitted; refer to PDF]

Then, the new integration masses can be obtained: [figure omitted; refer to PDF]

Then, _KI(1214) should be calculated: [figure omitted; refer to PDF]

Thus, the new integration masses can be obtained: [figure omitted; refer to PDF]

Finally, the belief degrees of frequency of network attack (_r121 ) can be calculated by (7): [figure omitted; refer to PDF]

Through the above conclusions, when _r1211 =2 times/h, _r1212 =13 times/h, _r1213 =8 times/h, and _r1214 =21 times/h, the final belief degrees of _r121 can be expressed by (E(70.24%),G(29.76%),C(0%),B(0%)), which means that the frequency of network attack (_r121 ) can be considered as "good."

The other attributes in Table 1 can be integrated by the method described in above process, and the final conclusion of reliability assessment of cloud computing platform can be obtained.

5. Conclusion

Considering the complex attributes, this paper first employed ER rule to make a comprehensive assessment for the reliability of cloud computing platform. ER rule has a good performance in multiple attribute decision making. It can process the evidence with high conflict and complete conflict and then make a more accurate assessment result. Moreover, a new reliability assessment structure of cloud computing for multiple attribute decision making is proposed in this paper. The 4-level reliability attributes mainly include four aspects, where the views of cloud computing servers and users are both taken into account, and the actual situation of reliability in cloud computing platform can be completely expressed. In conclusion, the innovation points of this paper can be concluded: (1) ER rule is first employed to establish the reliability assessment model of the cloud computing platform in this paper; (2) a new reliability assessment architecture including four aspects is proposed. Thus, the above two innovative researches cause a good result for reliability assessment in the complex dynamic cloud computing platform.

References

[1] W. Tian, Y. Zhao, M. Xu, Y. Zhong, X. Sun, "A toolkit for modeling and simulation of real-time virtual machine allocation in a cloud data center," IEEE Transactions on Automation Science and Engineering , vol. 12, no. 1, pp. 153-161, 2015.

[2] M. Abdullahi, M. A. Ngadi, S. M. Abdulhamid, "Symbiotic Organism Search optimization based task scheduling in cloud computing environment," Future Generation Computer Systems , vol. 56, pp. 640-650, 2016.

[3] X. L. Xu, L. Cao, X. Wang, X. Xu, L. Cao, X. Wang, "Resource pre-allocation algorithms for low-energy task scheduling of cloud computing," Journal of Systems Engineering & Electronics , vol. 27, no. 2, pp. 457-469, 2016.

[4] Y. Sun, J. White, S. Eade, D. C. Schmidt, "ROAR: a QoS-oriented modeling framework for automated cloud resource allocation and optimization," Journal of Systems and Software , vol. 116, pp. 146-161, 2016.

[5] A. I. Awad, N. A. El-Hefnawy, H. M. Abdel_Kader, "Enhanced particle swarm optimization for task scheduling in cloud computing environments," Procedia Computer Science , vol. 65, pp. 920-929, 2015.

[6] J. Wang, B. Gong, H. Liu, S. Li, "Multidisciplinary approaches to artificial swarm intelligence for heterogeneous computing and cloud scheduling," Applied Intelligence , vol. 43, no. 3, pp. 662-675, 2015.

[7] P. Qi, L. Li, "A fault recovery-based scheduling algorithm for cloud service reliability," Security and Communication Networks , vol. 8, no. 5, pp. 703-714, 2015.

[8] S. H. Albakri, B. Shanmugam, G. N. Samy, N. B. Idris, A. Ahmed, "Security risk assessment framework for cloud computing environments," Security and Communication Networks , vol. 7, no. 11, pp. 2114-2124, 2014.

[9] N. Ahmed, A. Abraham, "Neuro-fuzzy model for assessing risk in cloud computing environment," Proceedings of the Second International Afro-European Conference for Industrial Advancement AECIA 2015 , vol. 427, of Advances in Intelligent Systems and Computing, pp. 137-145, Springer International Publishing, Basel, Switzerland, 2016.

[10] H. Khazaei, J. Misic, V. B. Misic, S. Rashwand, "Analysis of a pool management scheme for cloud computing centers," IEEE Transactions on Parallel and Distributed Systems , vol. 24, no. 5, pp. 849-861, 2013.

[11] X. Qiu, Y. Dai, Y. Xiang, L. Xing, "A hierarchical correlation model for evaluating reliability, performance, and power consumption of a cloud service," IEEE Transactions on Systems, Man, and Cybernetics: Systems , vol. 46, no. 3, pp. 401-412, 2016.

[12] B. Larsen, P. Ingwersen, B. Lund, "Data fusion according to the principle of polyrepresentation," Journal of the American Society for Information Science and Technology , vol. 60, no. 4, pp. 646-654, 2009.

[13] J. C. de Winter, D. Dodou, "Factor recovery by principal axis factoring and maximum likelihood factor analysis as a function of factor pattern and sample size," Journal of Applied Statistics , vol. 39, no. 4, pp. 695-710, 2012.

[14] M. Sundaram, M. J. Smith, D. A. Revicki, L.-A. Miller, S. Madhavan, G. Hobbs, "Estimation of a valuation function for a diabetes mellitus-specific preference-based measure of health: the diabetes utility index^® ," PharmacoEconomics , vol. 28, no. 3, pp. 201-216, 2010.

[15] Z. Ruo-Xin, X.-J. Cui, S.-J. Gong, H.-K. Ren, K. Chen, "Model for cloud computing security assessment based on AHP and FCE," in Proceedings of the 9th International Conference on Computer Science and Education (ICCCSE '14), pp. 197-204, August 2014.

[16] N. Badie, A. R. C. Hussin, A. H. Lashkari, "Cloud computing data center adoption factors validity by fuzzy AHP," International Journal of Computational Intelligence Systems , vol. 8, no. 5, pp. 854-873, 2015.

[17] Y. Tamura, M. Kawakami, S. Yamada, "Reliability modeling and analysis for open source cloud computing," Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability , vol. 227, no. 2, pp. 179-186, 2013.

[18] D. Marudhadevi, V. N. Dhatchayani, V. S. S. Sriram, "A trust evaluation model for cloud computing using service level agreement," The Computer Journal , vol. 58, no. 10, pp. 2225-2232, 2014.

[19] Z. Pei, "Intuitionistic fuzzy variables: concepts and applications in decision making," Expert Systems with Applications , vol. 42, no. 22, pp. 9033-9045, 2015.

[20] Z. Xuejie, W. Zhijian, X. Feng, "Reliability evaluation of cloud computing systems using hybrid methods," Intelligent Automation & Soft Computing , vol. 19, no. 2, pp. 165-174, 2013.

[21] M. Khaki, I. Yusoff, N. Islami, "Application of the artificial neural network and neuro-fuzzy system for assessment of groundwater quality," Clean--Soil, Air, Water , vol. 43, no. 4, pp. 551-560, 2015.

[22] O. Hussain, H. Dong, J. Singh, "Semantic similarity model for risk assessment in forming cloud computing SLAs," On the Move to Meaningful Internet Systems, OTM 2010: Confederated International Conferences: CoopIS, IS, DOA and ODBASE, Hersonissos, Crete, Greece, October 25-29, 2010, Proceedings, Part II , vol. 6427, of Lecture Notes in Computer Science, pp. 843-860, Springer, Berlin, Germany, 2010.

[23] J. M. Garibaldi, T. Ozen, "Uncertain fuzzy reasoning: a case study in modelling expert decision making," IEEE Transactions on Fuzzy Systems , vol. 15, no. 1, pp. 16-30, 2007.

[24] X. Fan, M. J. Zuo, "Fault diagnosis of machines based on D-S evidence theory. Part 1: D-S evidence theory and its improvement," Pattern Recognition Letters , vol. 27, no. 5, pp. 366-376, 2006.

[25] R. Sun, H.-Z. Huang, Q. Miao, "Improved information fusion approach based on D-S evidence theory," Journal of Mechanical Science and Technology , vol. 22, no. 12, pp. 2417-2425, 2008.

[26] Y.-M. Wang, J.-B. Yang, D.-L. Xu, K.-S. Chin, "The evidential reasoning approach for multiple attribute decision analysis using interval belief degrees," European Journal of Operational Research , vol. 175, no. 1, pp. 35-66, 2006.

[27] Z.-G. Zhou, F. Liu, L.-C. Jiao, Z.-L. Wang, X.-P. Zhang, X.-D. Wang, X.-Z. Luo, "An evidential reasoning based model for diagnosis of lymph node metastasis in gastric cancer," BMC Medical Informatics and Decision Making , vol. 13, no. 1, article 123, 2013.

[28] Z.-G. Liu, Q. Pan, J. Dezert, "A new belief-based K-nearest neighbor classification method," Pattern Recognition , vol. 46, no. 3, pp. 834-844, 2013.

[29] Z.-G. Liu, Q. Pan, J. Dezert, A. Martin, "Adaptive imputation of missing values for incomplete pattern classification," Pattern Recognition , vol. 52, pp. 85-95, 2016.

[30] Z. Zhou, M. Folkert, N. Cannon, P. Iyengar, K. Westover, Y. Zhang, H. Choy, R. Timmerman, J. Yan, X. Xie, S. Jiang, J. Wang, "Predicting distant failure in early stage NSCLC treated with SBRT using clinical parameters," Radiotherapy & Oncology , vol. 119, no. 3, pp. 501-504, 2016.

[31] Z.-G. Zhou, F. Liu, L.-L. Li, L.-C. Jiao, Z.-J. Zhou, J.-B. Yang, Z.-L. Wang, "A cooperative belief rule based decision support system for lymph node metastasis diagnosis in gastric cancer," Knowledge-Based Systems , vol. 85, pp. 62-70, 2015.

[32] Z.-G. Liu, Q. Pan, J. Dezert, G. Mercier, "Credal c-means clustering method based on belief functions," Knowledge-Based Systems , vol. 74, pp. 119-132, 2015.

[33] J.-B. Yang, D.-L. Xu, "Evidential reasoning rule for evidence combination," Artificial Intelligence , vol. 205, pp. 1-29, 2013.

[34] Z. J. Zhou, C. H. Hu, G. Y. Hu, X. X. Han, B. C. Zhang, Y. W. Chen, "Hidden behavior prediction of complex systems under testing influence based on semiquantitative information and belief rule base," IEEE Transactions on Fuzzy Systems , vol. 23, no. 6, pp. 2371-2386, 2015.

[35] Z. J. Zhou, L. L. Chang, C. H. Hu, X. X. Han, Z. G. Zhou, "A new BRB-ER-based model for assessing the lives of products using both failure data and expert knowledge," IEEE Transactions on Systems, Man and Cybernetics: Systems , vol. 46, no. 11, pp. 1529-1543, 2016.