Performance Analysis of Otsu-Based Thresholding Algorithms: A Comparative Study

Abstract

Image thresholding is a widely used technology for a lot of computer vision applications, and among various global thresholding algorithms, Otsu-based approaches are very popular due to their simplicity and effectiveness. While the usage of Otsu-based thresholding methods is well discussed, the performance analyses of these methods are rather limited. In this paper, we first review nine Otsu-based approaches and categorize them based on their objective functions, preprocessing, and postprocessing strategies. Second, we conduct several experiments to analyze the model characteristics using different scene parameters both on synthetic images and real-world cell images. We put more attention to examine the variance of foreground object and the effect of the distance between mean values of foreground and background. Third, we explore the robustness of algorithms by introducing two typical kinds of noises under different intensities and compare the running time of each method. Experimental results show that NVE, WOV, and Xing’s methods are more robust to the distance of mean values of foreground and background. The large foreground variance will cause a larger threshold value. Experiments on cell images show that foreground miss detection becomes serious when the intensities of foreground pixels change drastically. We conclude that almost all algorithms are significantly affected by Salt&Pepper and Gaussian noises. Interestingly, we find that ME increases almost linearly with the intensity of Salt&Pepper noise. In terms of algorithms’ time cost, methods with no preprocessing and postprocessing steps have more advantages. All these findings can serve as a guideline for image thresholding when using Otsu-based thresholding approaches.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Image segmentation which extracts objects of interest from background is a fundamental technology for various image processing tasks. It has been widely used in various kinds of applications including text detection [1], medical image processing [2], document binarization [3], remote sensing [4], and object detection [5]. Thresholding method is one of the most well-known segmentation algorithms due to its simplicity and effectiveness [6]. The main goal of the image thresholding is to divide the pixels of an image into several subsets by selecting some intensity values. While we can choose more than one threshold values for multilevel segmentation, in this paper, we only focus on a single threshold segmentation which separates the whole image into two parts, and each corresponds to the background and foreground. In the past decades, there were many threshold selection methods presented including histogram shape-based methods [7–9], clustering-based methods [10–12], and entropy-based methods [13, 14]. The thresholding algorithms can be generally categorized into the global thresholding and the local thresholding. In the global thresholding, all pixels of the processed image use the same threshold value. Among all existing global thresholding algorithms, Otsu’s technique [6] which was first proposed by Otsu in 1979 is still one of the most frequently used clustering-based global thresholding methods [15]. Due to simplicity and effectiveness, Otsu’s technique is usually used as the preprocessing step in complex applications [16, 17]. The main idea behind the Otsu’s algorithm is to maximize the between-class variance to determine the optimal threshold value. When meeting the assumption that the gray level distributions of both the foreground and background are Gaussian distributions with equal variance, Otsu’s method can produce satisfactory results on real-world images [18, 19]. However, most real-world images do not satisfy this applicable condition. Therefore, the performance of Otsu’s method may degrade when the gray level histogram of the image is close to unimodal or the variances of the foreground and background are significantly different. Figure 1 shows segmentation results of Otsu’s method on real cell images with a unimodal histogram and large between-class variance.

[figures omitted; refer to PDF]

Due to the limitations mentioned above, a lot of research efforts have been put into analyzing and improving the Otsu’s method. Ng [20] introduced a valley emphasis term into the objective function of Otsu’s method and proposed a valley emphasis algorithm (VE) to ensure the threshold value locating at the valley of the histogram. Based on the fact that histogram valley corresponds to a low grayscale probability, Ng took the grayscale probability as the valley metric and demonstrated the effectiveness of Ng’s method on some test images. However, its valley metric does not take the neighborhood information into consideration. Fan and Lei [21] modified the valley metric using smoothed grayscale probability and proposed a new method named NVE to improve Otsu’s method. The valley metrics used in VE and NVE depend on the grayscale probability. To better represent the histogram valley, Xing et al. [22] proposed an improved valley emphasis method using second-order derivative-based valley metric.

Hu and Gong [23] proposed a two-stage method to further expand the application scenarios of Otsu’s method. In the first stage, the method detects the peaks of the histogram close to left and right boundary and smooths the image pixels whose intensities are between the corresponding intensities of the two peaks. In the second stage, Otsu’s method is applied on the preprocessed image. The experimental results showed that Hu’s method can provide better segmentation results than Otsu’s algorithm. To better understand the limitations of Otsu’s method, Xu et al. [24] studied its characteristics and proposed a two-round Otsu’s algorithm. The main conclusion of Xu’s work is the threshold value computed using Otsu’s method tends to bias toward the class with a larger within-class variance. Inspired by Xu’s work, Yuan et al. [25] introduced a weighted object variance (WOV) parameter to the Otsu’s objective function and proposed an improved Otsu’s method for detection. Also motivated by Xu’s work, Yang et al. [26] analyzed the relationship between pixel intensity and cumulative pixel number and gave a postadjusting strategy for the threshold value acquired by Otsu’s method. Not only the improved one dimensional Otsu’s methods were developed which mentioned above but also many two-dimensional Otsu’s algorithms have been proposed [27–30]. Cao et al. [31] claimed that the existing improved Otsu’s methods could not process images with a broad histogram or a flat valley. Many of the existing methods, especially two-dimensional Otsu’s methods, are not parameter free. To overcome this challenge, Cao et al. [31] proposed a novel parameter-free method which can achieve a more accurate and robust performance. In addition, there are also studies trying to improve the performance of Otsu’s method in the case of multiple thresholds [32]. A typical way is to use bionic algorithms (e.g., artificial bee colony algorithm [32, 33]) to search optimal multiple thresholds instead of using brute force methods. In this paper, we are mainly focusing on the bilevel image thresholding, and we will not further review the multilevel thresholding methods.

While a large number of improved Otsu’s methods have been proposed, there are a few studies focusing on the performance analysis of the algorithms in-depth. Goh et al. [34] studied the performance of Otsu’s technique using Monte Carlo statistical method. However, the performance comparison among different Otsu-based algorithms is still understudied, and also the test images used in existing studies are significantly different. To the best of our knowledge, there is no existing study on the categorization strategy of a various Otsu-based algorithms. In this paper, we first give a categorization method for different Otsu-based methods based on the objective functions, preprocessing, and postprocessing strategies. Second, a real-world image dataset and a Monte Carlo-based image synthetic method are employed for the evaluation of Otsu-based thresholding algorithms. Furthermore, several typical corruptions are adopted to test different algorithms. The main contributions of the paper are as follows:

(1) We propose a categorization method for different improved Otsu-based algorithms based on their objective functions, preprocessing, and postprocessing strategies

(2) The characteristics of each Otsu-based method are analyzed in-depth. More specifically, the effect of variance and distance between mean values of foreground and background, the ratio of foreground object in the whole image are discussed using a Monte Carlo-based image synthetic method

(3) The robustness of each Otsu-based method to two typical kinds of corruptions is studied on a real-world cell image dataset, and the performance of each algorithm corresponding to different noise intensities is analyzed

(4) We compare the time cost of all test algorithms and find that preprocessing or postprocessing steps may significantly increase algorithm’s consuming time when implementing by Matlab if the preprocessing or postprocessing does not meet the language characteristics

The remainder of the paper is organized as follows. Section 2 introduces the basic principle of Otsu’s method and reviews the recent improved algorithms based on the Otsu’s method using a categorization strategy. Section 3 presents the characteristics and performance analytics for the compared Otsu-based algorithms by conducting experiments on the synthetic images and real-world cell images. Section 4 presents the discussion and conclusion.

2. Overview of Otsu-Based Thresholding Methods

Otsu’s method is one of the most frequently used automatic thresholding algorithms. In this section, we introduce the basic principle of Otsu’s algorithm and give a brief review of recently improved Otsu-based thresholding methods.

Suppose $I$ is a grayscale image of size $m\ast n$, and the intensities of pixels of $I$ are ranging from 0 to $L-1$. All pixels of $I$ can be divided into two sets $C_0$ and $C_1$ when giving a specific intensity value $t$ as a threshold. If the mean intensity of $C_0$ and $C_1$ is denoted as ${\mu }_{0}t$ and ${\mu }_{1}t$, and the probability of $C_0$ and $C_1$ are denoted as $P_{0}t$ and $P_{1}t$, the between class variance of Otsu’s method can be defined as: $$\begin{array}{}\text{(1)}& P_{0}t=\sum _{i=0}^t{p}_i,\text{(2)}& P_{1}t=\sum _{i=t+1}^{L-1}{p}_i=1-P_{0}t,\text{(3)}& \sigma t=P_{0}t·{{\mu }_{0}t-\mu }^2+P_{1}t·{{\mu }_{1}t-\mu }^2,\end{array}$$where $p_i$ is the probability of intensity $i$, which can be described as $p_i=fi/m\ast n$ if $fi$ represents the number of pixels with intensity $i$. $\mu$ is the mean intensity of the whole image $I$, which can be expressed as $\mu =P_{0}t·{\mu }_{0}t+P_{1}t·{\mu }_{1}t$.

With the above definitions, we can compute the optimal threshold $t^{\ast }$ of Otsu’s method by solving the following optimization problem: $$\begin{array}{}\text{(4)}& t^{\ast }=\underset{t}{\arg \max }\sigma t=\underset{t}{\arg \max }{P}_{0}t·{{\mu }_{0}t-\mu }^2+P_{1}t·{{\mu }_{1}t-\mu }^2.\end{array}$$

The objective function described in equation (4) indicates that the best threshold of Otsu’s method maximizes the weighted sum of the square of distance between mean intensity of foreground and the whole image and the square of distance between mean intensity of background and the whole image, which is also known as the interclass variance.

There are two main improvement strategies for the original Otsu’s method. One is to modify the objective function to make the threshold value more reasonable, and the other is to add preprocessing and/or postprocessing steps. Table 1 lists algorithms and their corresponding objective functions for the performance comparison purpose in this paper. Following the two main improvement strategies, we can roughly divide the eight methods in Table 1 into two categories. Methods including VE [20], NVE [21], Xing’s, WOV, and Cao’s are only focusing on improving the objective function, and Hu’s, Xu’s, and Yang’s methods introduce preprocessing and/or postprocessing steps. In the following, we will give a brief introduction to the other eight Otsu-based algorithms.

Table 1

A description of each Otsu-based algorithm discussed in this paper.

Method	Objective function	Preprocessing	Postprocessing
Otsu’s [6]	$$\sigma t$$	—	—
VE [20]	$$\sigma t·w_{v}t$$	—	—
NVE [21]	$$\sigma t·w_{v}t$$	—	—
Xing’s [22]	$$\sigma t·w_{v}t$$	—	—
WOV [25]	$$P_{0}t·P_{0}t·{{\mu }_{0}t-\mu t}^2+P_{1}t·{{\mu }_{1}t-\mu t}^2$$	—	—
Cao’s [31]	$$P_{0}t·P_{1}t·{{\mu }_{0}t-{\mu }_{1}t}^2+{{\mu }_{0}t-\mu t}^2+{{\mu }_{1}t-\mu t}^2$$	—	—
Hu’s [23]	$$\sigma t$$	√	—
Xu’s [24]	$$\sigma t$$	√	—
Yang’s [26]	$$\sigma t$$	—	√

As described in Table 1, the first improved algorithm VE modified the objective function of Otsu’s method by adding a new valley metric term $w_{v}t$ to ensure the selected threshold value is more likely located at the valley of the histogram. The valley metric used in VE algorithm is defined as below: $$\begin{array}{}\text{(5)}& w_{v}t=1-p_t,\end{array}$$where $p_t$ represents the probability of occurrence of pixels with intensity $t$. The valley metric does not take neighborhood information into consideration, and in NVE the metric is modified as: $$\begin{array}{}\text{(6)}& w_{v}t=1-{\overline{p}}_t,\end{array}$$where ${\overline{p}}_t={\sum }_{i=-m}^m{p}_{t+i}$ and the value of $m$ is suggested to be 5 according to Fan and Lei’s study [21].

A new second-order derivative based valley metric was proposed by Xing et al. [22] in which the valley metric proposed was defined as equation (7) below: $$\begin{array}{}\text{(7)}& w_{v}x=\frac{{△}^2{p}_x-\min {△}^2{p}_{·}}{\max {△}^2{p}_{·}-\min {△}^2{p}_{·}},\end{array}$$where $p_{·}$ represents the probability of all useable gray levels.

Besides the three valley emphasis methods above, Cao’s method and WOV algorithm also belong to the first category which are focusing on modifying the objective function. In WOV, the influence of the foreground variance is increased, and the weight of the first term of Otsu’s method is changed from $P_{0}t$ to $P_{0}t·P_{0}t$. To maximize the distance between mean values of foreground and background, Cao et al. added a distance term into the objective function, which is defined as: $$\begin{array}{}\text{(8)}& {{\mu }_{0}t-{\mu }_{1}t}^2.\end{array}$$

The second category is based on adding preprocessing and/or postprocessing steps. In Hu’s study, a preprocessing scheme is applied on the processed gray level image before implementing standard Otsu’s method. The image preprocessing can be described as: $$\begin{array}{}\text{(9)}& Ix,y=\begin{array}{cc}0.25\ast Ix-1,y-Ix,y+Ix+1,y-Ix,y+Ix,y-1-Ix,y+Ix,y+1-Ix,y+Ix,y\hfill & Ix,y>T_0\mathit{\text{and}}Ix,y<T_1\hfill \\ Ix,y\hfill & others\hfill \end{array},\end{array}$$where $Ix,y$ represents the pixel intensity at $x,y$, $T_0$ and $T_1$ are gray levels corresponding to the first and last peaks of the histogram. The main idea of formula (9) is to take neighborhood information into consideration for pixels whose intensities belong to $T_0,T_1$, and then the Otsu’s method is implemented on the smoothed image.

Different from Hu’s preprocessing strategy, Xu et al. proposed a two-round Otsu’s method which applied a preprocessing step on intensity probability of the processed gray image. Xu et al. assumed that the pixel intensity of foreground in an image is larger than background, the interclass variance of foreground is significantly large, and the real optimal threshold value should be smaller than Otsu’s result. To make the threshold value sounder, Xu et al. proposed to apply the following preprocessing strategy on pixel intensity probability and calculate the optimal threshold value using Otsu’s algorithm based on new pixel intensity probabilities in. $$\begin{array}{}\text{(10)}& \begin{array}{c}{P}_{0}t=\frac{{\sum }_{i=0}^t{p}_i}{{\sum }_{i=0}^T{p}_i},P_{1}t=\frac{{\sum }_{i=t+1}^T{p}_i}{{\sum }_{i=0}^T{p}_i},\\ {\mu }_{0}t=\frac{{\sum }_{i=0}^{t}i\ast p_i}{{\sum }_{i=0}^T{p}_i},{\mu }_{1}t=\frac{{\sum }_{i=t+1}^{T}i\ast p_i}{{\sum }_{i=0}^T{p}_i}.\end{array}\end{array}$$

Yang et al. also proposed a postprocessing strategy for threshold tuning based on the relationship between $PT$ and $Pt$ defined as below: $$\begin{array}{}\text{(11)}& \begin{array}{cc}PT=\sum _{i=0}^T{p}_i,Pt=\sum _{i=0}^t{p}_i\hfill & \mu <T,\hfill \\ PT=\sum _{i=T+1}^{L-1}{p}_i,Pt=\sum _{i=t}^{L-1}{p}_i\hfill & \mu \ge T,\hfill \end{array}\end{array}$$where $T$ represents the threshold value obtained by Otsu’s method, $L$ and $\mu$ are gray levels and mean intensity of the whole image defined as above. For a given threshold value $\beta$, the new optimal threshold value can be described as the solution of the following optimization problem. $$\begin{array}{}\text{(12)}& t^{\ast }=a\underset{t}{rg\min }abs\frac{Pt}{PT}-\beta .\end{array}$$

As we discussed, two different improvement ideas are employed in the eight improved Otsu’s methods. The first category of improved Otsu’s methods is aimed at forming a more suitable objection function by adding various kinds of constraint components, which provides some helpful clues for the optimal threshold value searching. Another kind of improved algorithms does preprocessing or postprocessing either on the input image or on its histogram instead of directly modifying the objection function, which, as a matter of fact, is a hybrid technology. In the following sections, experiments will be conducted to compare the performance of each algorithm introduced above.

3. Performance Analysis of Otsu-Based Thresholding Methods

In this section, the performance of each Otsu-based algorithm will be evaluated. All experiments are implemented using Matlab R2012b on a PC with Windows 7. The experiments are conducted on two kinds of images which are synthetic images and real world images. The following influenced factors will be discussed on synthetic images and real world images:

(1) Variance and Distance between Mean Values of Foreground and Background. We will generate different synthetic images whose pixel intensity distributions of foreground and background are approximately normal distribution, and the effect of variance and distance between mean values will be discussed

(2) The Ratio of Foreground Object in the Whole Image. The foreground object size is another important factor that will influence the thresholding results. Images with different foreground ratios will be generated and used to study the effect of foreground ratio on segmentation results

(3) Image corruption noises including Salt&Pepper noise and Gaussian noise are applied to test the robustness of each algorithm

3.1. Testing on Synthetic Images

In this section, we will study the relationship between algorithms’ performance and variances and distance of mean values of foreground and background. The segmentation ratio measurement is adopted as the evaluation metric [34], which is defined as: $$\begin{array}{}\text{(13)}& \zeta =\frac{\text{no}.\text{of}\text{segmented}\text{object}\text{pixel}}{\text{no}.\text{of}\text{true}\text{object}\text{pixel}}.\end{array}$$

From the definition, it is obvious that if the segmentation is completely correct, the segmentation ratio will be 1, other values are corresponding to under segmentation $\zeta <1$ and oversegmentation $\zeta <1$, respectively.

To study the first two items of the above three affecting factors, we generate appropriate images using different parameters. The intensities of foreground and background of the generated images are all approximately normally distributed as follows: $$\begin{array}{}\text{(14)}& \begin{array}{c}Ix,y\sim N{\mu }_1,{\sigma }_1^2,Ix,y\in background,\hfill \\ Ix,y\sim N{\mu }_2,{\sigma }_2^2,Ix,y\in foreground,\hfill \end{array}\end{array}$$where ${\mu }_1,{\mu }_2$ are mean values of background and foreground, and ${\sigma }_1^2,{\sigma }_2^2$ denote their respective variances. For an image with $m$ rows and $n$ columns, we denote the ratio of foreground object in the whole image as $\gamma$, which is defined as: $$\begin{array}{}\text{(15)}& \gamma =\frac{\text{no}.\text{of}\text{foreground}\text{pixel}}{m\ast n}.\end{array}$$

Figure 2 shows the binarization results of each algorithm on two synthetic images with different parameters. The foreground ratios of the two images are both 20%, and the variances of foreground and background are fixed as 10. The first synthetic image is almost bimodal, and the foreground and background distributions of the second synthetic image are significantly overlapped. From Figure 2, we can see that most methods obtain good segmentation results on the first synthetic image except Xing’s, Xu’s, and Yang’s. However, the segmentation results become worse on the second synthetic image. The segmentation results indicate that the distance of mean values of foreground and background will significantly affect the algorithms’ performance, and more details will be discussed next.

[figures omitted; refer to PDF]

Figure 3 demonstrates the relationship between $\zeta$ and $\eta$ which is defined as formula (16), where $\eta$ represents the distance between mean values of foreground and background. The trend is obvious that a small $\eta$ will result in a larger segmentation ratio and lead to a more serious over-segmentation as shown in Figure 2. Combining the results of Figures 2 and 3, we conclude that distance between foreground and background is an import factor that affects the segmentation results. Among all the compared algorithms, Xing’s method seems more robust, and VE is the most influenced scheme. $$\begin{array}{}\text{(16)}& \eta ={\mu }_2-{\mu }_1.\end{array}$$

[figure omitted; refer to PDF]

To test the influence of foreground object size, we further conduct experiments to verify the performance of algorithms on synthetic images with different foreground ratio $\gamma$ defined as formula (13). Through this experiment, we fix ${\mu }_1=30,{\mu }_2=150$ and ${\sigma }_1^2={\sigma }_2^2=10$. Figure 4 shows the segmentation ratio of each algorithm on four synthetic images with foreground ratio ranging from 0.2 to 0.8 with step length of 0.2. From Figure 4, we can see that the segmentation ratio of most algorithms except Xing’s and Xu’s methods are stable, which means the foreground object ratio has little influence on the segmentation results. Xing’s and Xu’s methods tend to oversegment according to the value of $\zeta$ (where $\zeta >1$ indicates oversegmentation as described above). However, the foreground seems to be properly segmented when $\gamma$ is large enough, which may indicate that a larger foreground ratio seems to reduce the difficulty of the segmentation task.

[figure omitted; refer to PDF]

In this part, synthetic images with different foreground variances are generated and tested to verify the influence of different foreground variances. The mean values of foreground and background are set as ${\mu }_1=30,{\mu }_2=150$, respectively, and the background variance is fixed as ${\sigma }_1^2=10$. The generated images and their corresponding histograms are shown in Figure 5. As introduced above, the part of histogram corresponding to the background of each synthetic image is extremely similar to each other, and the part corresponding to the foreground becomes more flat as ${\sigma }_2^2$ increases from 10 to 50.

[figure omitted; refer to PDF]

Table 2 illustrates the threshold values of each algorithm on synthetic images as shown in Figure 5. From Table 2, we find that the threshold value increases as the foreground variance increases for most algorithms, which is consistent with the conclusion in Xu’s work that the threshold value tends to be close to the class with larger variance.

Table 2

Threshold values of each algorithm on different synthetic images.

${\sigma }_2^2$	Otsu	Cao’s	VE	NVE	WOV	Xing’s	Xu’s	Hu’s	Yang’s
10	92	91	90	92	91	46	36	90	53
20	93	97	88	89	94	45	36	90	54
30	94	98	89	90	94	119	37	90	65
40	96	101	96	93	97	44	38	90	85
50	99	106	96	96	100	58	39	90	89

Figure 6 demonstrates the segmentation ratio of each method under different foreground variances on synthetic images. Although the foreground variance can influence the threshold value, it seems the segmentation ratio is not affected for most algorithms. One reason could be the distance between the mean values of foreground and background is large enough.

[figure omitted; refer to PDF]

Next, two kinds of noises are tested. First one is Salt&Pepper noise which is one of the most common noises. We test the influence of Salt&Pepper noise on each method on cell images. The ME values are calculated on original images and noise corrupted images with the noise intensity ranging from 0.1 to 0.7, and the relationship between average ME value of each algorithm and noise intensity can be found in Figure 8. It is obvious that all algorithms are significantly affected by Salt&Pepper noise, and the ME value grows almost linearly with increasing noise intensity $\delta$. Since Salt&Pepper noise can be effectively removed by using median filter, it is necessary to add a filtering step before applying thresholding algorithms.

[figure omitted; refer to PDF]

Gaussian noise is another type of common noise. In this study, we evaluate the performance of all methods on Gaussian corrupted cell images. The mean value of Gaussian is always set to 0 in this section, and the variance changes from 0 to 0.1. Figure 9 shows the relation curves between ME and ${\delta }^2$ of each algorithm. Unlike Salt&Pepper noise, the ME values grow quickly if we add Gaussian noise with a small variance. However, the influence of noise decreases rapidly with the increase of variance. Among all methods, Xu’s method performs best on Gaussian noise.

[figure omitted; refer to PDF]

3.3. Algorithm Time Cost Testing

In this section, we compare the time cost of all tested algorithms. Following the analysis method introduced in Xing’s study [22], it is not difficult for us to draw a conclusion that the time complexity of all test methods in this work is $On$, where $n$ represents the number of pixels of the image. However, it is important to note that we need to pay attention to the influence of the characteristics of the Matlab language on algorithm time cost. Taking Hu’s method as an example, it is difficult to write the preprocessing step (defined as equation (9)) in matrix form, and this will result in an increase in algorithm’s execution time. To compare the actual execution time of each algorithm, we conduct experiments using Matlab R2012b on a PC with Intel Core 2.30 GHz CPU and 8.0 GB memory.

Figure 10 shows the relationship between the consuming time of each method and the image size, and the results are the average of ten runs. It is obvious that the consuming time of methods with no preprocessing and postprocessing changes slowly with increasing image size. On the other hand, Hu’s, Xu’s, and Yang’s methods consume more time than other methods, and this becomes increasing significant with the increase of image size. As each algorithm has the same time complexity, the significant difference in time cost is related to whether the implementation code meets the language characteristics of Matlab.

[figure omitted; refer to PDF]

4. Conclusion

In this study, the performance of nine Otsu-based thresholding algorithms is analyzed. Experiments on synthetic images indicate that the distance between the mean values of foreground and background $\eta$ can significantly affect the algorithms’ performance. Among the discussed algorithms above, NVE, WOV, and Xing’s methods are more robust to $\eta$. The variance of foreground can also affect the threshold and the segmentation result. The threshold value of almost all Otsu-based algorithms tends to increase when the foreground variance increases. Experimental results on cell images indicate that foreground miss detection becomes serious when the intensity of foreground pixels changes drastically, and all the nine algorithms are not robust enough to both Salt&Pepper and Gaussian noises. In addition, preprocessing or postprocessing steps which do not meet the language characteristics will significantly increase algorithm’s consuming time when implementing the algorithms by Matlab. These facts can serve as a guideline for Otsu-based thresholding applications in the future.

Acknowledgments

This work is partly supported by the National Natural Science Foundation of China (Grant no. 61866031) and the Science Technology Foundation for Middle-aged and Young Scientist of Qinghai University (2018-QGY6).

References

[1] B. Epshtein, E. Ofek, Y. Wexler, "Detecting text in natural scenes with stroke width transform," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 13-18, pp. 2963-2970, .

[2] L. Lin, W. Yang, C. Li, J. Tang, X. Cao, "Inference with collaborative model for interactive tumor segmentation in medical image sequences," IEEE Transactions on Cybernetics, vol. 46, pp. 2796-2809, 2017.

[3] S. He, L. Schomaker, "DeepOtsu: document enhancement and binarization using iterative deep learning," Pattern Recognition, vol. 91, pp. 379-390, DOI: 10.1016/j.patcog.2019.01.025, 2019.

[4] B. Han, Y. Wu, "A novel active contour model based on modified symmetric cross entropy for remote sensing river image segmentation," Pattern Recognition, vol. 67, pp. 396-409, DOI: 10.1016/j.patcog.2017.02.022, 2017.

[5] R. Cucchiara, C. Grana, M. Piccardi, A. Prati, "Detecting moving objects, ghosts, and shadows in video streams," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25 no. 10, pp. 1337-1342, DOI: 10.1109/TPAMI.2003.1233909, 2003.

[6] M. Sezgin, B. Sankur, "Survey over image thresholding techniques and quantitative performance evaluation," Journal of Electronic Imaging, vol. 13, pp. 146-168, 2004.

[7] R. Azreal, D. Pilar, "Histogram concavity analysis as an aid in threshold selection," IEEE Transactions on Systems, Man, and Cybernetics, vol. 13, pp. 231-235, 1983.

[8] R. Guo, S. Pandit, "Automatic threshold selection based on histogram modes and a discriminant criterion," Machine Vision Applications, vol. 10 no. 5-6, pp. 331-338, DOI: 10.1007/s001380050083, 1998.

[9] M. J. Carlotto, "Histogram analysis using a scale-space approach," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-9 no. 1, pp. 121-129, DOI: 10.1109/TPAMI.1987.4767877, 1987.

[10] T. W. Ridler, "Picture thresholding using an iterative selection method," IEEE Transactions on Systems, Man, and Cybernetics, vol. 8, pp. 630-632, 1978.

[11] N. Otsu, "A threshold selection method from gray-level histograms," IEEE Transactions on Systems, Man, and Cybernetics, vol. 9 no. 1, pp. 62-66, DOI: 10.1109/TSMC.1979.4310076, 1979.

[12] C. V. Jawahar, P. K. Biswas, A. K. Ray, "Investigations on fuzzy thresholding based on fuzzy clustering," Pattern Recognition, vol. 30 no. 10, pp. 1605-1613, DOI: 10.1016/S0031-3203(97)00004-6, 1997.

[13] H. D. Cheng, Y. Cheng, Y. Sun, "A novel fuzzy entropy approach to image enhancement and thresholding," Signal Processing, vol. 75 no. 3, pp. 277-301, DOI: 10.1016/S0165-1684(98)00239-4, 1999.

[14] P. K. Sahoo, C. Wilkins, J. Yeager, "Threshold selection using Renyi's entropy," Pattern Recognition, vol. 30 no. 1, pp. 71-84, DOI: 10.1016/S0031-3203(96)00065-9, 1997.

[15] Z. Hou, Q. Hu, W. L. Nowinski, "On minimum variance thresholding," Pattern Recognition Letters, vol. 27 no. 14, pp. 1732-1743, DOI: 10.1016/j.patrec.2006.04.012, 2006.

[16] M. Zortea, E. S. Flores, J. Scharcanski, "A simple weighted thresholding method for the segmentation of pigmented skin lesions in macroscopic images," Pattern Recognition, vol. 64, pp. 92-104, DOI: 10.1016/j.patcog.2016.10.031, 2017.

[17] L. He, X. Ren, Q. Gao, X. Zhao, B. Yao, Y. Chao, "The connected-component labeling problem: a review of state-of-the-art algorithms," Pattern Recognition, vol. 70, pp. 25-43, DOI: 10.1016/j.patcog.2017.04.018, 2017.

[18] J. Kittler, J. Illingworth, "On threshold selection using clustering criteria," IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-15 no. 5, pp. 652-655, DOI: 10.1109/TSMC.1985.6313443, 1985.

[19] T. Kurita, N. Otsu, N. N. Abdelmalek, "Maximum likelihood thresholding based on population mixture models," Pattern Recognition, vol. 25 no. 10, pp. 1231-1240, DOI: 10.1016/0031-3203(92)90024-D, 1992.

[20] H. Ng, "Automatic thresholding for defect detection," Pattern Recognition Letters, vol. 27 no. 14, pp. 1644-1649, DOI: 10.1016/j.patrec.2006.03.009, 2006.

[21] J. L. Fan, B. Lei, "A modified valley-emphasis method for automatic thresholding," Pattern Recognition Letters, vol. 33 no. 6, pp. 703-708, DOI: 10.1016/j.patrec.2011.12.009, 2012.

[22] G. W. Xing, P. Yang, L. T. Qingge, "Automatic thresholding using a modified valley emphasis," IET Image Processing, vol. 14 no. 3, pp. 536-544, 2019.

[23] B. Hu, N. Gong, "An improved Otsu’s thresholding segmentation method," Microelectronics and Computer, vol. 26, pp. 153-155, 2009.

[24] X. Y. Xu, S. Z. Xu, L. H. Jin, E. M. Song, "Characteristic analysis of Otsu threshold and its applications," Pattern Recognition Letters, vol. 32 no. 7, pp. 956-961, DOI: 10.1016/j.patrec.2011.01.021, 2011.

[25] X. C. Yuan, L. S. Wu, Q. J. Peng, "An improved Otsu method using the weighted object variance for defect detection," Applied Surface Science, vol. 349, pp. 472-484, DOI: 10.1016/j.apsusc.2015.05.033, 2015.

[26] P. Yang, W. Song, X. B. Zhao, R. Zheng, L. T. Qingge, "An improved Otsu threshold segmentation algorithm," International Journal of Computational Science and Engineering, vol. 22, pp. 146-153, DOI: 10.1504/IJCSE.2020.107266, 2020.

[27] J. Z. Liu, W. Q. Li, "The automatic thresholding of gray-level pictures via two-dimensional Otsu method," Acta Automatica Sinica, vol. 19, pp. 101-105, 1993.

[28] J. L. Fan, F. Zhao, "Two-dimensional Otsu's curve thresholding segmentation method for gray-level images," Acta Electronica Sinica, vol. 35, pp. 751-755, 2007.

[29] Q. Chen, L. Zhao, J. Lu, G. Kuang, N. Wang, Y. Jiang, "Modified two-dimensional Otsu image segmentation algorithm and fast realisation," IET Image Processing, vol. 6 no. 4, pp. 426-433, DOI: 10.1049/iet-ipr.2010.0078, 2012.

[30] C. S. Sha, J. Hou, H. X. Cui, "A robust 2D Otsu's thresholding method in image segmentation," Journal of Visual Communication and Image Representation, vol. 41, pp. 339-351, DOI: 10.1016/j.jvcir.2016.10.013, 2016.

[31] X. H. Cao, T. H. Li, H. L. Li, S. R. Xia, F. J. Ren, Y. Sun, X. Y. Xu, "A robust parameter-free thresholding method for image segmentation," IEEE Access, vol. 7, pp. 3448-3458, DOI: 10.1109/ACCESS.2018.2889013, 2019.

[32] H. Gao, Y. Shi, C. M. Pun, S. Kwong, "An improved artificial bee colony algorithm with its application," IEEE Transactions on Industrial Informatics, vol. 15 no. 4, pp. 1853-1865, 2019.

[33] H. Gao, Z. Fu, C. M. Pun, J. Zhang, S. Kwong, "An efficient artificial bee colony algorithm with an improved linkage identification method," IEEE transactions on cybernetics,DOI: 10.1109/TCYB.2020.3026716, 2020.

[34] T. Y. Goh, S. N. Basah, H. Yazid, M. J. Safar, F. S. Saad, "Performance analysis of image thresholding: Otsu technique," Measurement, vol. 114, pp. 298-307, DOI: 10.1016/j.measurement.2017.09.052, 2018.