Comparison of GUM and Monte Carlo methods for the uncertainty estimation in hardness measurements

Abstract

Monte Carlo Simulation (MCS) and Expression of Uncertainty in Measurement (GUM) are the most common approaches for uncertainty estimation. In this work MCS and GUM were used to estimate the uncertainty of hardness measurements. It was observed that the resultant uncertainties obtained with the GUM and MCS without correlated inputs for Brinell hardness (HB) were ±0.69 HB, ±0.67 HB and for Vickers hardness (HV) were ±6.7 HV, ±6.5 HV, respectively. The estimated uncertainties with correlated inputs by GUM and MCS were ±0.6 HB, ±0.59 HB and ±6 HV, ±5.8 HV, respectively. GUM overestimate a little bit the MCS estimated uncertainty. This difference is due to the approximation used by the GUM in estimating the uncertainty of the calibration curve obtained by least squares regression. Also the correlations between inputs have significant effects on the estimated uncertainties. Thus the correlation between inputs decreases the contribution of these inputs in the budget uncertainty and hence decreases the resultant uncertainty by about 10%. It was observed that MCS has features to avoid the limitations of GUM. The result analysis showed that MCS has advantages over the traditional method (GUM) in the uncertainty estimation, especially that of complex systems of measurement. MCS is relatively simple to be implemented.

1 Introduction

The guide to the expression of measurement uncertainty (GUM, JCGM 100) and the propagation of distributions by a Monte Carlo method (GUMS1, JCGM 101) are two of the most widely used documents concerning measurement uncertainty evaluation in metrology. Both documents describe three phases: (a) the construction of a measurement model, (b) the assignment of probability distributions to quantities, and (c) a computational phase that specifies the distribution for the quantity of interest. The two approaches described in these two documents agree in the first two phases but employ different computational approaches, with the GUM using linearization to simplify the calculations [1]. The guide to the Expression of Uncertainty in Measurement (GUM) requires, “that the results of a measurement have been corrected for all recognized significant systematic effects and that every effort has been made to identify such effects”, before the issue of evaluating their uncertainty is tackled [2]. GUM provides a framework and procedure for evaluating and expressing measurement uncertainty. GUM procedure has two main limitations. Firstly, the way that the coverage interval is constructed to contain values of the measurand with a stipulated coverage probability is approximate. Secondly, insufficient guidance is given for the multivariate case in which there is more than one measurand [3]. Figure 1 shows the schematic to explain the main idea of GUM which mainly depends on the uncertainty propagations [4].

Monte Carlo Simulation (MCS) is the second method to estimate the uncertainty of measurements, where evaluating the measurement uncertainty by the MCS method can be carried out by establishments of the model equation for the measurand in function of the individual parameters of influence, then selecting the significant sources of uncertainty, identification of the probability density functions corresponding to each source of uncertainty selected, and selecting the number Monte Carlo trials, and finally calculating the M results by applying the equation that was defined for the measurand [5].

MCS process is illustrated in Figure 2 a and b in comparison with the propagation of uncertainties used by the GUM.

Figure 2 a shows an illustration representing the propagation of uncertainties. In this case, three input quantities are presented x1, x2 and x3 along with their respective uncertainties u(x1), u(x2) and u(x3). As can be noted, only the main moments (expectation and standard deviation) of the input quantities are used in the propagation. In Figure 2 b, while propagating density (PDFs), no approximations are made and the whole information contained on the input distributions is propagated to the output [5].

The aims of this present research article are to investigate the differences between GUM and MCS as methods of uncertainty estimation of hardness and to detect the effects of inputs correlation on the uncertainty budget.

Fig. 1 GUM and propagation of uncertainties.

Fig. 2 Schematic to the main idea of MCS.

2 Theoretical principle

2.1 Uncertainty classifications

To investigate the difference between GUM and MCS, the resultant budget uncertainties of Brinnell and Vickers hardness (HV) measurements were used as an application. For statistical analysis normal PDFs were proposed. The uncertainty estimated from normal distribution is believed to lie in the interval defined by U with a level of confidence of approximately 95%. For normal distribution, μ is the expectation or mean of the distribution, and the shaded areas represent ±1.96 standard uncertainty ustnd about the mean, and ±ustnd, encompasses about 95% of the distribution (see Fig. 3 ).

Each input source of the uncertainty should be classified as type A or type B where type A evaluation of standard uncertainty is based on any valid statistical method for treating data and can be calculated by the following equations:

(1)

(2)where Xi is an input quantity whose value is estimated from n independent observations Xi,k of Xi obtained under the same conditions of measurement, and the standard uncertainty u(xi) to be associated with xi is the estimated standard deviation of the mean. For type B standard uncertainty, the evaluation of standard uncertainty is usually based on scientific judgment using all of the relevant information available, that is, the uncertainty is either obtained from an outside source, or obtained from an assumed distribution.

Fig. 3 The normal distribution, and standard deviation with the confidence level.

2.2 Uncertainty in measurements with and without correlation

If the functional relationship between the measurand Y and the input quantities X in a measurement process is given by [6];

(3)The function f is used to calculate the output estimates: y, of the measurand; Y, using the estimates of X1, X2, X3, … , XN for the values of the N input quantities.

(4)The conventional estimation can be illustrated using a simple equation with y as a continuous function of x1, x2. y is approximated using a polynomial approximation or a 2nd order Taylor's series expansion about the means:

(5)where

are mean values and Z is the remainder:

(6)As the partial derivatives are computed at the mean values

they are the same for all i = 1, 2, …, N. All the higher terms are normally neglected at Z = 0, so equation (5) becomes;

(7)If f(x1, x2) is a linear function, then the second order partial derivatives in equation (6), are zero, so Z = 0. Both linearity and small uncertainty are prerequisites of conventional method of uncertainty estimation described below. The standard deviations σ(X1), σ(X2) are referred by GUM [7], as the standard uncertainties associated with the input estimates X1, X2. The standard uncertainty in y can be obtained by Taylor equation [8];

(8)This equation gives the uncertainty as a standard deviation irrespective of whether or not the measurements of x1, x2 are independent and of the nature of the probability distribution. In the case of correlated inputs then equation (3) can be written in terms of the correlation coefficient

;

(9)

The correlation coefficient can be calculated by the following equation [9]:

(10)As a direct consequence of its definition, the correlation coefficient r is restricted to the range ‒1 through zero to +1. When r = 0, the correlation is zero, when r = +1, the correlation is perfect and positive, and when r = −1 the correlation is perfect and negative.

The partial derivatives are called sensitivity coefficients, which give the effects of each input quantity on the final results (or the sensitivity of the output quantity to each input quantity). The term, expanded uncertainty is used in GUM to express the % confidence interval about the measurement result within which the true value of the measurand is believed to lie and is given by:

(11)where k is the coverage factor on the basis of the confidence required for the interval,

(12)If it was considered that y = x1 + x2 is a linear operation it was reported that there is compatibility between MCS and GUM and it was demonstrated that at correlated or uncorrelated inputs MCS gives identical results as given in equations (5) and (9). The results of y are linear functions in terms of x1 and x2. As the first order partial derivatives are all equal to ±1, the square is equal to unity. The second order partial derivatives are both equal to zero.

To quantify the distribution of the results of Skewness and Kurtosis, where Skewness quantifies how symmetrical the distribution, it can be calculated from the following equation:

(13)where

* Positive Skewness indicates a long right tail;

* Negative Skewness indicates a long left tail;

* Zero Skewness indicates a symmetry around the mean.

Kurtosis quantifies whether the shape of the data distribution matches the Gaussian distribution and it can be calculated from the following equation:

(14)

* Positive excess Kurtosis indicate flatness (long, fat tails);

* Negative excess Kurtosis indicates peakedness.

2.3 Brinell hardness (HB) results

This test is executed by applying a load on a sphere made of a hard material over the surface of the test sample (Fig. 4 ) [10].

During the test the sphere will penetrate through the sample leaving an indented mark upon unloading. The diameter of this indentation is inversely proportional to the hardness of the material of the sample. The model used for the HB is represented in equation (15) [11].

(15)where F is the applied load (N), D is the indenter diameter (mm) and d is the diameter of the indentation mark (mm).

Fig. 4 Schematic represents the Brinell hardness test.

2.4 HV results

HV is a measure of the hardness of a material, calculated from the size of an impression produced under load by a pyramid-shaped diamond indenter (Fig. 5 ) [12].

During the test the square pyramid indenter will penetrate through the sample leaving an indented mark upon unloading. The two diagonals of this indentation are measured and the mean of them was calculated.

The model used here for the HV is represented in equation (16) [12].

(16)where F is the applied load (kgf), and d is the mean of d1 and d2 of the indentation mark (mm).

Fig. 5 Schematic represents Vickers hardness test.

3 Application results

3.1 Procedure to estimate the uncertainty of HB

To estimate the uncertainty it is required to detect all the input sources of uncertainty of HB measurements. These sources can be summarized as shown in Figure 6 .

After detecting these parameters, measurements were performed and the results analysis was conducted taking into account the calculation of these parameters as normal distribution, which is can be summarized in Table 1.

Fig. 6 The cause–effect diagram for HB.

[Table Omitted - See PDF]

3.1.1 GUM procedure to estimate HB uncertainty with and without correlated inputs

To estimate the budget uncertainty in accordance with GUM, sensitivity coefficient should be calculated in accordance with the following equations:

(17)

(18)

(19)The uncertainty budget of HB is summarized in Tables 2 and 3.

[Table Omitted - See PDF]

[Table Omitted - See PDF]

3.1.2 MCS procedure to estimate HB uncertainty and without correlated inputs

10 000 iterations were performed on each parameter contributing the uncertainty budge using equation (13) to estimate the budget uncertainty (see Tab. 4; Fig. 7 ).

[Table Omitted - See PDF]

Fig. 7 Results of HB by MCS.

3.1.3 Summary of the results of uncertainty budget estimation of HB by MCS

MCS for HB without correlated inputs

Mean value: 100.1 HB.

Expanded uncertainty: 0.67 HB.

MCS for HB with correlated inputs

Mean value: 100.1 HB.

Expanded uncertainty: 0.59 HB.

3.2 Procedure to estimate the uncertainty of HV

The detected input sources of uncertainty of HV measurements can be summarizes as shown in Figure 8 and Table 5.

Fig. 8 The cause–effect diagram for HV.

[Table Omitted - See PDF]

3.2.1 GUM procedure to estimate HV uncertainty with and without correlated inputs

To estimate the uncertainty budget of HV sensitivity coefficient should be identified by the following equations (see Tabs. 6 and 7).

(20)

(21)

[Table Omitted - See PDF]

[Table Omitted - See PDF]

3.2.2 MCS procedure to estimate HV uncertainty with and without correlated inputs

10 000 iterations were performed on each parameter contributing the uncertainty budge using equation (16), to estimate the budget uncertainty (see Tab. 8; Fig. 9 ).

[Table Omitted - See PDF]

Fig. 9 Result of the Monte Carlo simulation for HV.

3.2.3 Summary of the results of uncertainty budget estimation by MCS

MCS for HV without correlated inputs

Mean value: 401.9 HB.

Expanded uncertainty: ±6.5 HV.

MCS for HV with correlated inputs

Mean value: 401.9 HB.

Expanded uncertainty: ±5.8 HV.

4 Discussion

From the previous figures and tables it was noted that the resultant expanded uncertainties (at 95% confidence levels) obtained with the GUM and MCS without correlated inputs for HB were ±0.69 HB, ±0.67 HB and for HV were ±6.7 HV, ±6.5 HV, respectively. The estimated expanded uncertainties with correlated inputs by GUM and MCS were ±0.6 HB, ±0.59 HB and for HV were ±6 HV, ±5.8 HV, respectively. GUM Framework overestimates a little bit the MCS estimated uncertainty. The main cause of this difference is the approximation used by the GUM Framework in estimating the budget uncertainty of the calibration curve produced by least squares regression.

The correlations between inputs have significant effects on the estimated uncertainties. At GUM procedure the estimated uncertainty of HB without correlated inputs was 0.69 HB and 0.6 HB at correlated inputs. For HV the estimated uncertainty without correlated inputs was 6.7 HV and 6 HV at correlated inputs. The same results were obtained in the case of MCS investigation.

Skewness calculation showed small value and hence a symmetrical distribution for the obtained results.

The shape of Kurtosis quantifications showed that the data distribution matches the Gaussian distribution.

Also, it was observed that MCS has features to avoid the limitations and assumptions of the GUM framework. The resulting analysis shows that MCS has many advantages over conventional method (GUM) in uncertainty estimation, especially that of complex measurement systems. MCS is relatively simple to implement; there is no need for complex mathematics related to calculating sensitivity coefficient by partial differentiation and also it was demonstrated that the MCS is relatively compatible with the conventional uncertainty estimation methods of linear systems and systems that have small uncertainties.

5 Conclusions

From this research article it was concluded that:

* the expanded uncertainty results estimated with the GUM Framework and the MCS showed no significant differences. In all the cases the estimated uncertainty using the GUM approach slightly overestimated the results obtained with the MCS;

* the correlations between inputs have significant effects on the estimated uncertainties. Thus the correlation between inputs decreases the contribution of these inputs in the budget uncertainty and hence decreases the resultant uncertainty by about 10%, and this value is depending on the correlation coefficient value;

* the result analysis shows that the MCS has numerous advantages over the traditional method (GUM) in the estimation of uncertainty, especially that of complex systems of measurements. There is no need for complex mathematics related to calculating sensitivity coefficient by partial differentiation;

* it was demonstrated that the MCS is relatively compatible with the GUM as a conventional uncertainty estimation methods of linear systems and systems that have small uncertainties.

References

1. A.B. Forbes, Approaches to evaluating measurement uncertainty, Int. J. Metrol. Qual. Eng. 3, 71–77 (2012)

2. F. Pavese, Corrections and input quantities in measurement models, Int. J. Metrol. Qual. Eng. 3, 155–159 (2012)

3. P.M. Harris, M.G. Cox, On a Monte Carlo method for measurement uncertainty evaluation and its implementation, Metrologia 51, S176–S182 (2014)

4. JCGM 101:2008, Evaluation of Measurement Data – Supplement 1 to the “Guide to the Expression of Uncertainty in Measurement” – Propagation of Distributions Using a Monte Carlo Method (Joint Committee for Guides in Metrology, 2008)

5. ISO/Guide, Guide to the Expression of Uncertainty in Measurement (GUM) – Supplement 1: Numerical Methods for the Propagation of Distributions (1998)

6. M. Basil, C. Papadopoulos, D. Sutherland, H. Yeung, Application of probabilistic uncertainty methods (Monte Carlo Simulation) in flow measurement uncertainty estimation, in Flow Measurement International Conference (2001)

7. ISO/GUM, Guide to the Expression of Uncertainty in Measurement – ISO (1995), ISBN 92-67-10188-9

8. J.R. Taylor, An Introduction to Error Analysis/The Study of Uncertainties in Physical Measurements, 2nd edn. (University Science Books, Sausalito, CA, 1997), ISBN 0-935702-42-3

9. I. Farrance, R. Frenkel, Uncertainty of measurement: a review of the rules for calculating uncertainty components through functional relationships, Clin. Biochem. Rev. 33, 49–75 (2012)

10. EUROLAB Technical Report 1/2006, Guide to the Evaluation of Measurement Uncertainty for Quantitative Test Results (2006)

11. ISO 6506-2:2014, Metallic Materials – Brinell Hardness Test – Part 2: Verification and Calibration of Testing Machines

12. ISO 6507-2:2005, Metallic Materials – Vickers Hardness Test – Part 2: Verification and Calibration of Testing Machines