A Two Phase Ultrasound Image De-speckling Framework by Nonlocal Means on Anisotropic Diffused Image Data

Abstract

Image de-speckling is one of the most challenging issues in multimedia imaging systems. All of the available speckle noise reduction filters are nearly noise reduction capable, but they fail to restore subtle features such as low grey level edges and fine details against a low contrast background. Nonlocal mean filtering and anisotropic diffusion are two popular and effective methods for image despeckling while preserving detail. This paper presents a two-phase ultrasound image de-speckling framework by utilizing the capability of the non-local mean filtering method for de-speckling and edge preservation on anisotropic diffused images. The prior image smoothing along with edge preservation and contrast enhancement by anisotropic diffusion is carried out in the first phase, which is then followed by the non-local means method for de-speckling and edge sharpening in the next phase. The degree of speckle noise attenuation is measured on low-contrast standard and ultrasound images and compared to state-of-the-art and advanced anisotropic diffusion techniques and non-local means methods. The percentage improvement of PSNR over the existing methods is found to be in the range of 2.06% to 46.68%. The experimental results show that the proposed method is capable of reducing noise and preserving edges better than existing speckle reduction filters.

Headnote

Keywords: anisotropic diffusion, non-local mean filter, image de-speckling, contrast enhancement

Image de-speckling is one of the most challenging issues in multimedia imaging systems. All of the available speckle noise reduction filters are nearly noise reduction capable, but they fail to restore subtle features such as low grey level edges and fine details against a low contrast background. Nonlocal mean filtering and anisotropic diffusion are two popular and effective methods for image despeckling while preserving detail. This paper presents a two-phase ultrasound image de-speckling framework by utilizing the capability of the non-local mean filtering method for de-speckling and edge preservation on anisotropic diffused images. The prior image smoothing along with edge preservation and contrast enhancement by anisotropic diffusion is carried out in the first phase, which is then followed by the non-local means method for de-speckling and edge sharpening in the next phase. The degree of speckle noise attenuation is measured on low-contrast standard and ultrasound images and compared to state-of-the-art and advanced anisotropic diffusion techniques and non-local means methods. The percentage improvement of PSNR over the existing methods is found to be in the range of 2.06% to 46.68%. The experimental results show that the proposed method is capable of reducing noise and preserving edges better than existing speckle reduction filters.

Povzetek: Predstavljen je dvofazni pristop za odstranjevanje šuma iz ultrazvočnih slik, ki se je v primerjavi z obstoječimi filtri za zmanjšanje šuma izkazal za boljšega v ohranjanju robov in zmanjševanju šuma.

(ProQuest: ... denotes formulae omited.)

1 Introduction

Today, in the world of computer vision and artificial intelligence, a massive number of images are now required to train a deep learning model. Database management in block-chain technologies is another emerging multimedia application that stores a large number of videos and images. However, the performance of all these technologies is mostly dependent on the quality and natural aspects of the images. These images are mainly degraded due to the defects that arise during the image sensing and acquisition processes. This illposed problem affects ultrasound imaging and MRI in medical science, SAR images in remote sensing, and a variety of other scientific and industrial images [1] [40] [41]. Most of the methods available in the literature aim to remove these degradations from images, but doing so without disturbing the image's essential features has always been difficult. The effect of noise in ultrasound images with poor contrast is a serious issue that has been addressed in the proposed work. The most difficult type of noise in real world medical imaging is speckle noise [2][3] which is due to the multiplicative behaviour of unwanted pixels introduced into the images under poor illumination or environmental conditions. The multiplicative nature of the speckle tends to vary the mean value of pixels in a local area of the image, which degrades its visual information.

A generalized model of speckle noise [2] [3] is given as:

... (1)

where gij is the speckled noisy image, Tij is the original image at the pixel position (i,j) that has been perturbed by speckle noise nij . This is mathematically represented as a white Gaussian noise of zero mean and variance at position (i,j). Ultrasound imaging suffers from this multiplicative noise, which degrades its usefulness in medical diagnosis and modality. The major problem in such speckle reducing filters is inaccurate localization of edges under low contrast regions and thus the loss of image information during the filtering process, which is called "over-filtering" or "over-smoothing." Thus, it has always been observed that it is very difficult to despeckle such a low contrast image without disturbing the essential features like edges and boundaries.

Traditionally, the problem of image de-speckling had been tackled with the help of order statistic filtering like the mean, median, or Gaussian filter [4] [5] [6] for the reduction of Gaussian noise. Later on, some stochastic and wavelet-based methods are also developed for effective Gaussian noise reduction [7] [8] [45] [46] [47]. However, the blurring of edges, lines, and boundaries degrades the image's perception quality. Classical despeckling filters [9] [10] [11] [12] inhibit smoothing near edges by using adaptive filters that calculate coefficients of variation, but these filters do not possess directional properties, which creates the problem of poor reconstruction of edges and boundaries.

Recently, the use of partial differential equation based anisotropic diffusion has become a very convincing edge-preserving smoothing scheme since the earlier work of Perona and Malik [13] [27] [38]. The anisotropic diffusion is a non-linear and space-variant transformation of the original image that aims to smooth the noise adaptively. However, the performance of anisotropic diffusion in image de-noising is dependent on a more robust mathematical exploration of the diffusion equation, which deals with the image structures consisting of homogeneous as well as heterogeneous characteristics.

The Non-Local Means (NLM) method is another well-known and effective technique for de-speckling ultrasound images [28] [36][37]. The NLM method reduces the impact of noise in an image by computing a weighted average of all the pixels in the non-local areas of the image. The mechanism of the NLM filter is to compute the weight based on the surrounding pixels of the test pixel in a local window along with the other windows of similar appearance in the same image. Consequently, the similar-looking patches or regions in an image would influence the weight assignment more for the test pixel as compared to the regions that are different from the test pixel. This filter, unlike local mean filtering, produces a better smoothed image and tends to recover the pixels representing edges. The major drawback of non-local mean filtering is the computational cost, which includes selecting the number and size of similar patches in the image.

The proposed work presented here is a hybrid framework for de-speckling ultrasound images along with sufficient recovery of edges and boundaries by utilizing the properties of anisotropic diffusion and nonlocal mean filtering. As it has been observed, the two techniques have been independently applied for noise removal and edge preservation, but achieving both targets equally by either of the two methods is not satisfactory. Therefore, the proposed method has been framed to adopt the properties of the two methods to achieve the goal of image enhancement along with sufficient speckle reduction.

The contributions of the proposed method are as follows:

* The proposed method is a two-phase process where anisotropic diffusion is used as a prefiltering step to enhance the contrast by reformulating the diffusion coefficient function, taking speckle noise characteristics into consideration.

* The resulting diffused image is further passed through the NLM filter in the next phase in order to de-speckle the image with sufficient edge preservation.

* The use of the anisotropic diffusion process prior to applying the NLM filtering has been shown to be quite effective in synthetic and ultrasound images.

This two-phase hybrid algorithm maintains an optimal balance between speckle removal, fine details, and edge preservation, along with contrast improvement and less computational complexity.

The organization of the paper is followed by Section 2 which describes a brief understanding of ultrasound de-speckling with related anisotropic diffusion and NLM filtering methods. The proposed twophase method has been presented in Section 3. The experimental analysis and discussion are made in Section 4. Finally, the paper is concluded in Section 5.

2 Related ultrasound de-speckling methods

Many efficient methods have been developed so far for ultrasound de-speckling. The two very popular methods, anisotropic diffusion and non-local mean filters, have been briefly explained in this section.

2.1 Anisotropic diffusion

The anisotropic diffusion [13][27][38] introduced by Perona and Malik, due to its directional smoothing properties is quite effective in the images affected by Gaussian noise and now it has become quite popular in medical imaging especially in the enhancement and despeckling of ultrasound images. Anisotropic diffusion model as suggested by Perona and Malik [13] is given by:

... (2)

... (3)

wheret is iteration, I is input image, VI is image gradient, C(V/) is diffusion coefficient function and k is edge threshold parameter. The two equations (Eq. (2) and (3)) show that the diffusion process is controlled by С (V/) which is varied with respect to the gradient magnitude at each pixel of the image. The gradient magnitude specifies the direction of smoothing as per the image structures. The edge threshold parameter k and no. of iterations t are two important parameters which affects the performance of anisotropic diffusion. The values of k and t are chosen depending on the particular image application. If V/ > k at a pixel, then the smoothing stops and the pixels at higher gradient are preserved otherwise, if V/ < k, then anisotropic diffusion become isotropic and behave like Gaussian smoothing. This concept of smoothing was first used in ultrasound speckling by and Acton [14] where nonhomogeneous diffusive heat phenomenon was utilized. This method has been named as "speckle reducing anisotropic diffusion" (SRAD) where the diffusion coefficient function has been modified as follows:

... (4)

... (5)

where, q(x,y, t) is named as instantaneous coefficient of variation which is dependent on V/ and is determined as:

... (6)

and ... is speckle scale function. The edge preservation sensitivity of this method was further examined [15] and presented as detail preserving anisotropic diffusion where the orientation of edges was made to stabilize while removing speckle noise as indicated in Eq (7).

... (7)

The q(x,t) in Eq (7) denotes the edge stabilizing function with a, ß and у are the regularization parameters. However, the improper selection of these regularization parameters in the diffusion equation suffers from over-filtering and blurring of edges. Fernandez and Carlos [16] provide the estimation of these parameters using local statistics of the image features for better anisotropic diffusion de-speckling. A fuzzy optimization algorithm as suggested by Puvanathasan and Bizheva [17] has been used to properly select the parameters in order to compute the edginess of a pixel, which is effective for optical coherence tomography images specifically. Wu and Tang [18] suggested a new selective degenerate diffusion model using fidelity and speed functions based upon ENI (edge, noise, interior pixels) for impulse noise reduction. A similar type of improved edge-enhancing diffusion approach was developed by Febrinni et al. [19] to minimize noise in homogeneous regions while keeping weak edges. The statistical properties of speckle noise have been investigated for proper selection of parameters in the diffusion equation by Ramos et al. [20]. A double degenerated nonlinear diffusion model [21] was developed by reframing the diffusion coefficient function as shown in Eq (8).

... (8)

The above diffusion coefficient function is calibrated by taking a > 0,0 < ß < 1 and M indicates the maximum intensity range of the image. This work was extended by introducing a gray level indicator [22] in the diffusion coefficient function as given below:

... (9)

where g(T) is defined as ... This promotes the de-noising process with gray level extraction of the image. k-means clustering has also been used to filter out the noisy pixels from the image during the diffusion process [23]. In this method, the cluster-based speckle scale function and the homogeneous sample region are recursively chosen based on the previous clustering results. In order to control the diffusion process separately along the edges and across the edges, Mishra et al. [24] use the probability density function of edge and pixel relativity information, which is quite effective in ultrasound speckle filtering. Gao et al. [25] decompose the divergence term of the diffusion equation and modify the iteration stopping criteria to meet the requirements of speckle filtering along with edge preservation. This method diffuses erroneous pixels that appear in a uniform background. This method, however, is ineffective in low-contrast ultrasound images. Xu et al. [26] suggested Gabor-based anisotropic diffusion, supporting the advantages of the Gabor edge detector on edge preservation and the advantages of the Lattice Boltzmann method on rapid parallel implementation of the diffusion equation. Very recently, a fuzzy based approximated anisotropic diffusion [38] has been presented, which is quite effective for impulse noise removal from standard scientific and medical images. A detailed overview of various anisotropic diffusion techniques [48] [49][50] [51] [52] [53] used for image enhancement and de-noising is provided in [39]. In all the above anisotropic diffusion filters, the smoothing results in ultrasound images are effective but fail to restore fine details. In addition to this, the low contrast of the ultrasound image after diffusion produces false artifacts that lead to wrong medical diagnoses. The reason for this problem is that the presence of speckled, noisy pixels in the image exhibits high gradient magnitudes, which get restored at the output image due to the slow diffusion process. Therefore, there is a need for a more robust physical and mathematical interpretation of the anisotropic diffusion equation to deal with speckled, noisy images.

2.2 Non-local mean filtering

The non-local mean (NLM) [28][32][33] filter utilizes the pattern redundancy in the image content by analyzing large pixel data and collecting information from the whole image, looking for similar features. NLM filters are successful in filtering the noise and preserving edges, but their performance degrades at higher noise levels. Basically, the NLM algorithm [28] estimates a new value for a pixel in a given noisy image. The computation estimates the new value for a test pixel based on the surrounding pixels of the test pixel in a local window as well as other windows of similar appearance. Mathematically, the new estimated value at a pixel i can be computed as the weighted average of all the pixel values of the image I or some predefined region of the image space as given below:

... (10)

where, is the weight which corresponds to the similarity between the neighborhoods around the pixel i and the pixel j located at some similar patch of the image. The weight w(i, j) is computed as:

... (11)

The above calculation of the weight is exponentially decreasing function of weighted Euclidean distance ... where N(T) and lV(/)are the fixed size neighborhoods centered at pixel i and j respectively. <r is the standard deviation of Gaussian kernel. The weight obtained through above calculation indicates the similarity between the pixels i and j which must satisfy the following conditions:

...

The term z(i) defined in Eq (11) is a normalizing constant which is stated as:

... (12)

The weights calculated are a decaying exponential function of Euclidean distances. This decay in the exponential function is controlled by a parameter h in Eq (12), which is termed as smoothing parameter of the filtering operation. A small value of h leads to inefficient noise filtering, whereas a high value tends to smooth the entire image. Thus, the proper tuning is required to preserve the image details while filtering the noise. Coupe et al. [29] adapted this NLM algorithm for ultrasound de-speckling by using a Bayesian framework (OBNLM), where a block-wise approach for weight computation was used. This method computes the weighted average of patches instead of the weighted average of all the pixels in the image. This tends to reduce the computational complexity, but the preservation of low contrast edges is not satisfactory. A shape-adaptive patch in NLM was developed using Stein's unbiased risk estimate [30] to further decrease the computational burden.

Recently, the KS-NLM filter [31] has become very popular in NLM filtering because it is based on the evaluation of Kolmogorov-Smirnov distance between the image pixels instead of Euclidean distance. This tends to compute the filtered value from the pixels of the patches, which are very similar to the target pixel, and the results are completely free from the ghost effect that has been observed in Euclidean distance-based NLM approaches. A three-stage hybrid algorithm (HSR) for de-speckling ultrasound images was recently introduced by Singh et al. [32], where the NLM filter is used in conjunction with guided filtering and bilateral filtering. A maximum likelihood-based method [33] and a fuzzy logic-based computational model (FSR) [34] have also been developed to attain the same goal. However, maintaining the balance between noise removal and contrast improvement with low computational cost is still a challenging issue in NLM filtering.

Speckle noise's multiplicative nature has a significant impact on SAR image signals, which intercept information retrieval from SAR imaging systems. During the monitoring and investigation of target elements, the backscattered echo captured by the system produces interference because of sunlight and different weather conditions, resulting in speckles in the images. These speckles introduce a granular appearance in images, which intercepts the information retrieval from SAR systems. The use of NLM filtering in SAR image despeckling is extremely effective [42] [43] [44]. A recent study of implementing the NLM filter for SAR images was presented by Penna et al. [42], where stochastic distances have been embedded in the NLM filter in place of Euclidean distance in the wavelet domain. The method works well in homogeneous portions of real SAR images, but fails to restore the fine details and yields a slower computation time. Various Deep Learning (DL) and Convolutional Neural Network (CNN) based methods have also been utilized in computer vision tasks for achieving fast computation. Many of them are used to deal with SAR image de-speckling. Recently, a CNNbased self-supervised SAR image de-noising method was developed [43], with the restored images exhibiting poor contrast and loss of low grey level edges and fine details due to the unknown speckle distribution. Mullissa et al. [44] provide a DL based image de-speckling model that estimates the de-speckling behaviour in accordance with the speckle noise distribution. This method is effective in speckle noise removal, but the major issue is its nonadaptability to different regions of SAR images. In addition to this, the computational requirements for estimating and evaluating the unknown de-speckling parameters make this method not convincing.

3 Proposed method

The proposed two-phase hybrid algorithm applies the anisotropic diffusion process in the first phase for contrast improvement of the image. The NLM filtering is then applied in the next phase for de-speckling and edge preservation. The functional block diagram of the proposed filtering method is given in Fig. 1. The two phases of the proposed hybrid algorithm are explained below.

3.1 Phase I: Pre-processing by anisotropic diffusion

Most of the ultrasound images used in medical diagnosis are of low contrast, where all the gray level intensities are scattered towards the darker side of the histogram. This makes distinguishing speckle defects and low grey level edges in ultrasound images extremely difficult. In the first phase of the proposed approach, a modified anisotropic diffusion is used to improve the contrast of the ultrasound image, along with low gray level edge preservation. This is achieved by employing gray level variance to control the diffusion process. The diffusion coefficient function has been updated so that it will vary in accordance with the gradient magnitude as well as the gray level variance of image pixels.

Let r be the gray level of a pixel of an image whose gray levels are varying in the range I which is 0 to (L - 1) and p(r;) be an estimated histogram component corresponding to the gray leveli). The mean of the gray levels is then calculated as:

... (13)

The nth moment of r about mg can be written as:

... (14)

In particular, the second moment of r as computed using Eq (14) can be considered as gray level variance of the image denoted by og2 which can be defined as:

... (15)

The mean mg and the variance <гд2 as calculated through Eq. (13) and Eq. (15) respectively are termed as global gray level mean and global gray level variance of the entire image. The mg and ... of a pixel at any point can also be calculated directly from the discrete image of size M x N as given in Eq (16) and Eq (17) respectively.

... (16)

... (17)

The assumption made here is that the pixel (i, j) in the image needs to be enhanced if gray level variance of that pixel in its neighborhood (sub-image) is comparatively lower than the gray level variance of the entire image. For this, we computed local gray level variances (tw2 at each pixel in their neighborhood W and measmed their differences with the global gray level variance <гд2 of the entire image. The local gray level variance (tw2 of a pixel at location (i, j) at a neighborhood W of size m x n is calculated as:

... (18)

where mw is the mean of the pixels in W and the size m xn of W is very small as compared to that used in Eq.(17) for calculation of og2. The difference between the two variances og2 and ow2 at a pixel of the image indicates about the contrast information of that particular pixel area. It has been assumed here that at a pixel location(i,j), ifuw2 « og2, then that particular pixel needs to get preserved in the output image. The overall contrast of the image also gets enhanced if all such pixels get preserved. This assumption is utilized in the proposed approach to enhance the low gray level inter-region edges with overall contrast enhancement.

The edge threshold parameter k in Eq (3) of Perona - Malik anisotropic diffusion has been reformulated in this work so as to smooth the ultrasound image in accordance with gradient as well as gray level variance of each pixel. The k in Perona - Malik anisotropic diffusion is replaced by edge threshold function k(o2) in the proposed approach which is defined as follows:

... (19)

The above edge threshold function k(o2) controls the diffusion process in accordance with the ratio where <r0 is a scale factor with values ... for low contrast areas whereas <70 > 1 for high contrast areas. If at any pixel position, ... thenfc(<72) goes down which tends to stop the diffusion and preserves the low gray level inter-region edges. In reverse manner if, ..., then /c(<72) rises to increase the diffusion. This variation of controlling function in anisotropic diffusion is summarized as follows:

... (20)

The parameter q(l < q < 5) has been tuned here through experiments for best result. The revised diffusion coefficient function by substituting k(o2) in place of k in Eq. (3) is now expressed as:

... (21)

The above diffusion coefficient function varies in accordance with the combined effect of .... The following four cases have been observed:

1) Case 1: If at a pixel position, ..., then the high value of |V/| and low value of k (σ2) tends to stop the diffusion and preserve the low gray level inter-region edges.

2) Case 2: If at a pixel position, ..., then both the .... are high and diffusion undergoes as per the gradient magnitude and preserve the high gradient edges.

3) Case 3: If at a pixel position, ...., then both the ... are low and diffusion takes place with respect to variations in fc(<72). This tends to recover the low gradient and low gray level inert-region edges.

4) Case 4: If at a pixel position, ... is low but k(σ2) becomes high which makes the diffusion faster to smooth out the entire homogeneous area.

The variation of diffusion coefficient function as per the above four cases controls the performance of smoothing and preserving low gradient as well as low gray level edges in low contrast ultrasound images.

3.2 Phase II: post-processing by non-local mean filter

In this phase, the image obtained from phase I undergoes NLM filtering. As discussed in Section 2(B), the performance of NLM filtering is mostly dependent on the estimation of weight function which in turn depends on the measurement of patch similarity. The image obtained from Phase I of the proposed method is contrast enhanced image along with edge preservation. The purpose of NLM filtering as a post processing phase is to recover the original de-specked image with reduced ghost effects and well-located fine details. This has been attained by utilizing well popular "Kolmogorov - Smirnov" (KS) distance [31] in comparing the similar patches in search window. Unlike the conventional NLM filtering where the distance between similar patches have been measured, the proposed method measures the distances between the similar pixels. For this, the cumulative distribution functions (CDFs) are estimated for each pixel and the distances between their curves are calculated. The pixels which are very similar to the test pixel are merged to calculate the weight for the test pixel. The estimation of CDF of a pixel is probability based statistical measurement where we consider Pf(r) asa probability of occurrence of a pixel of gray level r in a discrete image f and is defined as:

... (22)

This is an estimated histogram component of /corresponding to gray level r in the range 0 to L - 1. Herenr and n are the number of occurrences of rth gray level and total number of pixels in / respectively. The CDF corresponding to pf(r)canbe defined as:

... (23)

The above Eq. (23) is also termed as image's accumulated normalized histogram where s denotes another gray level with less probability of occurrence compared to r. The CDF of any random variable represents a monotonically increasing curve. The KS test provides the vertical difference between the CDF curves of two pixel's gray levels lying at different locations of /. In the proposed case, the KS test for distance measurement can be taken as:

... (24)

where, rand s are two gray levels at position (tl,/l) and (í2,j2) respectively. Based on the distance D measured in Eq (24) with the help of CDFs estimated at each point, the points closer to the test pixel are merged to calculate the corresponding weight. The weight calculation process is same as given in Eq. (11) and (12) except the weights computed are decaying exponential function of the distance measured based on CDFs as per the Eq (24).

4 Experimental results and analysis

The proposed two-phase method was run in MATLAB R2015a on a Windows 7 computer with 4 GB of RAM and a 2.50 GHz Intel(R) Core (TM) ¡5-3210 processor. We tested the proposed method on varieties of ultrasound images taken from internet resources. However, for the purposes of this paper's demonstration, three real and one synthetic ultrasound images of 8 - bit length and of size 256 x 256 were used. All the test images have been corrupted by additive noise of variance <r2 varying from 0.2 to 0.8. Through experiments, the optimal parameters of the two phases of the proposed method were selected and, accordingly, the performance was evaluated. The criteria for parameter selection are explained in the following section.

4.1 Selection of parameters

In Phase I of the proposed method, explained in Section 3.1, it has been observed that the performance of the proposed method is dependent on two parameters <r0 and q of the diffusion coefficient function defined in Eq. (21). The two parameters have been tuned to a particular value based on the image under test. As per the assumptions made in Eq, (21) explained in Section 3.1, <70 < 1 and 1 < q < 5 are taken for initial experimentations. Initially, the value of <r0 has been kept fixed and the different values of qhave been varied in between 1 and 5. The results have been compared in terms of two well-known quality assessment parameters PSNRs and SSIMs at varying noise densities of <r2 = 0.2 to 0.8 respectively. In the similar fashion, the value of q has been kept fixed and different values of <r0 varying between 0 and 1 are used and again the results are compared with respect to PSNRsand SSIMs at the noise densitiescr2 = 0.2 to 0.8 respectively. The experiments shows that the best result is obtained at cr0 = O.Sandq = 2. Therefore, we kept cr0 = 0.8 and q = 2 in Eq. (21) of the proposed diffusion coefficient function for all the other test images used in this experimentation. The size of the neighborhood W for calculation of local gray level variance (tw2 as used in Eq. (18) and the no. of diffusion iterations are other important parameters which affects the result of the proposed method. We compared the three smallest standard local window sizes 3 x 3,5 x 5 and 7 x 7 in terms of PSNRs and SSIMs and the best results are obtained at W = 3 x Swhich has been taken in the entire experimentation. Similarly, the results are compared at iterations 30, 50,70 and 100 where the best performance was achieved at 50 iterations.

In Phase II of the proposed method explained in Section 3.2, the parameters in NLM filtering which need to be set for optimal performance are the patch size N(i) for a pixel i and the smoothing parameter h during weight assignment. Through experimentations, the patch size N(i) for a pixel i is decided by using all the standard sizes. However, the best results are obtained when the image is divided in to patches of size 3x3 and as per KS-NLM algorithm [31], the CDFs are estimated for each point of the 3 x 3window and their KS-distance is measured as given in Eq. (24). Similarly, during the weight assignment in Eq. (11) and (12), we have chosen the range of h to be between 4 to 10. At h = 5, the result is quite acceptable but if we increase the value of h towards 10, then restored image become over-smoothed. Therefore, finally we decided to keep h = 5 in weight calculation of NLM filtering for the entire experimentations.

All the optimal parameters chosen in the proposed method are tabulated in Table 1.

4.2 Simulation results

The performance of the proposed two-phase hybrid method has been tested on varieties of real and synthetic ultrasound images. However, for the sake of demonstration in this paper, the two real ultrasound images of 8 - bit length and of size 256 x 256 have been used. The two original images are shown in Fig. 2 where the first image is Skull Osteoma ultrasound image and second one is Thyroid Lesion ultrasound image showing hypo-plastic left thyroid lobe and a benign cystic nodule in the right lobe. The two original images have been cormpted by additive noise of variance <r2 = 0.2,0.4 and 0.6 which are shown in the left

columns of Fig. 3 and Fig. 4 respectively. The middle column is showing the results of phase I where as the corresponding restored images obtained by phase II of the proposed two-phase hybrid method are demonstrated in the right columns of Fig. 3 and Fig. 4 respectively.

As seen in the restored images in Figs. 3 and 4, noise has been easily reduced while low gradient edges and fine details have been preserved. The behaviom of the proposed method in contrast enhancement can also be observed through the experimental results. The Skull Osteoma image (Fig. 2 (left)) is of quite low contrast, which then shows improvement in overall contrast after processing by the proposed method as shown in the right column of Fig. 3. The contrast of the Thyroid Lesion image (Fig. 2 (right)) is slightly better than that of the other test image, and the results obtained using the proposed method show significant contrast improvement, noise reduction, and detail preservation, as shown in Fig. 4.

In order to evaluate the performance of the proposed method more robustly, the two well-known image quality assessment parameters, PSNR and SSIM [35] of the restored images have been computed. Table 2 demonstrates the values of PSNRs and SSIMs of the two noisy test images as well as the corresponding two restored images shown in Fig. 3 and 4 respectively at noise densities of 0.2,0.4 and 0.6.

It can be observed from Table 2 that there is a considerable amount of increment in the values of PSNRs and SSIMs of the restored images as compared to the noisy images and the intermediate results obtained by phase I of the proposed method.

4.3 Performance comparison

The proposed method has been validated by comparing its performance with the existing state-of-theart ultrasound de-speckling methods as well as some recently developed methods available in the literature. In the demonstration of comparison analysis, we used one synthetic and one real ultrasound image of 8 - bit length and of size 256 x 256 each. The synthetic image includes oval, cardioid, line, triangle and a rectangle whereas the real ultrasound image is a kidney ultrasonic image generated by linear acoustic program [32] [34] [36]. The two original images are shown in Fig. 5 and both are contaminated with different levels of speckle noise ranging from variance <r2 = 0.2 to 0.8. The restored images are compared perceptually as well as quantitatively with existing techniques: SRAD [14], OBNLM [29], FSR [34] and INLM [36]. Fig. 6 demonstrates the results obtained by the above four existing methods and the proposed method when applied to the synthetic image corrupted by speckle noise of variance <r2 = 0.6.

Table 3 and Table 4 respectively shows the average values of PSNRs and SSIMs of the restored synthetic and ultrasound images obtained by the all the five comparative methods. It can be seen from Fig 6 that the proposed method has shown the significant improvement in contrast of the input image in addition to the despeckling as compared to other existing methods.

Moreover, Table 3 indicates that the PSNR values of the proposed method are much better than SRAD [14], OBNLM [29] and FSR [34]. However, if we compare the values with INLM [36], the PSNR obtained by the proposed method at higher noise variances are larger than that of the INLM [36] which shows that average improvement of PSNR at increasing noise densities is more in the proposed method as compared to INLM [36] and even other methods.

The similar observation can be seen in Table 4 where the SSIM values of all the comparative methods have been represented at different noise variances, which again validates that the proposed method has outperformed the existing methods at higher noise levels. Figure 7 depicts the restored results of the kidney ultrasonic image that had been corrupted by speckle noise of variance <r2 = 0.6. This indicates, however, that while the PSNR and SSIM values obtained on the real ultrasound image are lower than those obtained on the synthetic ultrasound image, the values obtained by the proposed method are still better than those obtained by the existing methods. The average PSNR and SSIM values of the proposed method in comparison to the existing methods at varying noise densities on the two images are graphically represented in Figs. 8 and 9, which validate the effectiveness of the proposed method's de-speckling capability.

4.4 Computational complexity

The computational complexity of the proposed method depends upon two individual phases. In the Phase I of the proposed method, the computational complexity depends on the computation of gradient and local gray level variance of the image. If image is of size N x N. then the complexity of calculating gradient in four main directions of the test pixel is O(1V2) whereas the calculation of local gray level variance in the neighborhood size w x w yields the complexity of O(w2). Accordingly, the overall computational complexity of the first phase of the proposed method is O(N2w2'). The NLM filter in Phase II of the proposed method has computational complexity of O(/V2M2L2)for image of size N x N. search area M x M and local window size is Lx L. The proposed algorithm has been implemented in the system with Intel (R) Core (TM) İ5 - 3210M processor of speed 2.50 GHz and 4 GB RAM. The overall average computation time of the proposed algorithm on an image of size 256x256 is found to be 36.13 seconds that is comparatively better than the existing methods.

The key feature of the proposed method is to despeckle the low-contrast ultrasound image along with edge and fine detail preservation, which makes the proposed method different from the existing comparable methods, as can be observed from the above experimental analysis. There are. however, numerous opportunities to investigate the properties of anisotropic diffusion and non-local mean filtering in order to deal with high density speckle noise in edge-abundant images, as can be observed from the above experimental analysis.

4.5 Discussions

The performance analysis of the proposed two-phase method shown above demonstrates the effectiveness of ultrasound de-speckling in terms of better noise removal, acceptable edge preservation, easy implementation, and affordable computation requirements under the speckle noise variance level of 0.8. The proposed method does not require tuning of control parameters as used in existing anisotropic diffusion and NLM filtering. Because of this computational advantage, the proposed method is simple to implement and has a shorter execution time. However, the proposed method has a great deal of future scope. Tire main research issue in this work is to explore the properties of diffusion equations available widely in the literature to deal with discrete non-negative real image vector spaces for enhancing low gray level image pixels. The behaviom of the NLM filter can also be examined by the use of stochastic distances for a particular distribution in place of Euclidean distances.

The optimized mechanism of weight estimation in accordance with non-local self-similarity for a particular pixel requires parameter tuning through exhaustive experiments. This needs to be explored with the help of some optimization techniques for better perfonnance. The real noisy images, like ultrasound and scenes from real SAR systems, need effective speckle reduction along with adequate information preservation, where this proposed method can be used as a better noise limiting tool. However, the selection of the sizes of the local patch and search area in the NLM filter and the properties of anisotropic diffusion need to be explored in future work for better compactness with a variety of images.

5 Conclusion

In tliis work, a hybrid framework for de-speckling low-contrast ultrasound images is presented. The anisotropic diffusion technique is used in conjunction with the KS distance-based non-local mean filtering mechanism. The proposed method is a two-phase method where prior contrast improvement of the ultrasound image is done by modified anisotropic diffusion, which is then followed by non-local mean filtering for despeckling. The experiments are conducted on varieties of synthetic and real ultrasound images. The results show that the proposed method not only significantly reduces speckle noise but also preserves fine details and low grey level edges in the image, as well as an overall improvement in contrast. The above features of the proposed method make it useful in real-time medical applications to assist medical experts in the analysis and interpretation of ultrasonic images.

The proposed method can be further improved by investigating parameter optimization for better edge preservation and removal of high-level speckle noise in more complex ultrasound images. The exploration of the properties of anisotropic diffusion and researching the mechanism of non-local mean filtering are possible research issues in this work.

Sidebar

Received: September 6, 2022

References

References

[1] Goyal, B., Agrawal, S., &Sohi, B. S. (2018). Noise Issues Prevailing in Various Types of Medical Images. Biomedical & Pharmacology Journal, vol. 11, no. 3,pp. 12-27, 2018

[2] Tur, M., Chin, K. C., & Goodman, J. W. (1982). When is speckle noise multiplicative? Applied optics, 21(7), 1157-1159.

[3] R. F. Wagner, S. W. smith, J. M. Sandrik and H. Lopez (1983), "Statistics of speckle in ultrasound B-scans", IEEE Transactions on Sonics and Ultrasonics, Vol 30(3), pp. 156 - 163

[4] Xin Wang (1992) Adaptive Multistage Median Filter. IEEE Transactions on Signal Processing, 1992, 40(4): 1015-1017

[5] Kaur J., Kaur J., and Kaur M.: 'Survey of despeckling techniques for medical ultrasound images', Int. Journal of Computer Technology and Applications, 2011, 2, pp. 1003- 1007

[6] HakanGüray Senei, Richard Alan Peters, Benoit Dawant (2002), Topological median filter, IEEE Transactions on Image Processing, 11 (2)

[7] F. Jin, P. Fieguth, L. Winger, E. Jemigan (2003), Adaptive Wiener filtering of noisy images and image sequences, IEEE Conference on Image Processing (ICIP 2003)

[8] S. Sudha, G.R. Suresh, R. Sukanesh (2009), Speckle noise reduction in ultrasound images by wavelet thresholding based on weighted variance, International Journal of Computer Theory and Engineering 1(1): 1793-8201

[9] Lee, J. S. (1980). Digital image enhancement and noise filtering by use of local statistics. IEEE transactions on pattern analysis and machine intelligence, (2), 165-168.

[10] Frost, V. S., Stiles, J. A., Shanmugan, K. S., &Holtzman, J. C. (1982). A model for radar images and its application to adaptive digital filtering of multiplicative noise. IEEE Transactions on pattern analysis and machine intelligence, (2), 157-166.

[11] Kuan, D.T., Sawchuk, A. A., Strand, T. C., &Chavel, P. (1987). Adaptive restoration of images with speckle. IEEE Transactions on Acoustics, Speech, and Signal Processing, 35(3), 373-383.

[12] Lopes, A., Touzi, R., &Nezry, E. (1990). Adaptive speckle filters and scene heterogeneity. IEEE transactions on Geoscience and Remote Sensing, 28(6), 992-1000.

[13] Perona, P., & Malik, J. (1990). Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on pattern analysis and machine intelligence, 12(1), 629-639.

[14] Yu, ¥., & Acton, S. T. (2002). Speckle reducing anisotropic diffusion. IEEE Transactions on image processing, 77(11), 1260-1270.

[15] Yongjian Yu and S. T. Acton, "Edge detection in ultrasound Imagery using the instantaneous coefficient of variation", IEEE Transaction on image processing. Vol 13, No. 12, pp. 1640 - 1655, 2004.

[16] Aja-Femández, S., &Alberola-López, C. (2006). On the estimation of the coefficient of variation for anisotropic diffusion speckle filtering. IEEE Transactions on Image Processing, 15(9), 26942701.

[17] Puvanathasan, P., &Bizheva, K. (2009). Interval type-II fuzzy anisotropic diffusion algorithm for speckle noise reduction in optical coherence tomography images. Optics express, 17(2), 733746.

[18] J. Wu and c. Tang, "PDA based random valued impulse noise removal based on new class of controlling functions", Vol. 20, No. 9, pp. 2428 - 2438, 2011.

[19] Fabbrini, L., Greco, M., Messina, M., &Pinelli, G. (2014). Improved edge enhancing diffusion filter for speckle-corrupted images. IEEE Geoscience and Remote Sensing Letters, 77(1), 99-103.

[20] Ramos-Llordén, G., Vegas-Sánchez-Ferrero, G., Martin-Fernandez, M., Alberola-López, C., &AjaFemández, S. (2014). Anisotropic diffusion filter with memory based on speckle statistics for ultrasound images. IEEE transactions on Image Processing, 24(1), 345-358.

[21] Zhou, Z., Guo, Z., Dong, G., Sun, J., Zhang, D., & Wu, B. (2015b). A doubly degenerate diffusion model based on the gray level indicator for multiplicative noise removal. IEEE Transactions on Image Processing, 24(1), 249-260.

[22] Zhou, Z., Guo, Z., Dong, G., J., Zhang, D., & Wu, B. (2018c). A nonlinear diffusion equation based model for ultrasound speckle noise removal. Journal of Nonlinear Science, 28(2), 443470.

[23] Hu, Z., & Tang, J. (2016, September). Cluster driven anisotropic diffusion for speckle reduction in ultrasound images. In 2016 IEEE International Conference on Image Processing (ICIP) (pp. 23252329). IEEE.

[24] Mishra, D., Chaudhury, S., Sarkar, M., Soin, A. S., & Sharma, V. (2018). Edge probability and pixel relativity-based speckle reducing anisotropic diffusion. IEEE Transactions on Image Processing, 27(2), 649-664.

[25] Gao, M., Kang, B., Feng, X., Zhang, W., & Zhang, W. (2019). Anisotropic Diffusion Based Multiplicative Speckle Noise Removal. Sensors, 79(14), 3164.

[26] Xu, H. H., Gong, Y. C., Xia, X. Y., Li, D., Yan, Z. Z., Shi, J., & Zhang, Q. (2019). Gabor-based anisotropic diffusion with lattice Boltzmann method for medical ultrasound de-speckling. Mathematical biosciences and engineering: MBE, 16(6), 75467561.

[27] Krissian, K., Westin, C. F., Kikinis, R., &Vosburgh, K. G. (2007). Oriented speckle reducing anisotropic diffusion. IEEE Transactions on Image Processing, 16(5), 1412-1424.

[28] Buades, B. Coll and J.-M. Morel,"A review on image de-noising algorithms, with a new one", Mathematical Modeling and Simulation, Vol 4, No. 2, pp. 490 - 530, 2005

[29] Coupé, P., Hellier, P., Kervrann, C., &Barillot, C. (2009). Nonlocal means-based speckle filtering for ultrasound images. IEEE transactions on image processing, 78(10), 2221-2229.

[30] C. A. Deledalle, V. Duval and J. Salmon,"Non-local Methods with Shape Adaptive Patches (NLM - SAP)", Journal of Mathematical Imaging and Vision, Vol. 43, pp. 103 - 120, 2011.

[31] Baselice, F. (2017). Ultrasound image de-speckling based on statistical similarity. Ultrasound in Medicine & Biology, 43(9), 2065-2078.

[32] Singh, K., Kanade, S. K., & Singh, C. (2017). A hybrid algorithm for speckle noise reduction of ultrasound images. Computer methods and programs in biomedicine, 148, 55-69.

[33] Shahdoosti, H. R., &Rahemi, Z. (2018). A maximum likelihood filter using non-local information for de-speckling of ultrasound images. Machine Vision and Applications, 29(4), 689-702.

[34] Nadeem, M., Hussain, A., &Munir, A. (2019). Fuzzy logic based computational model for speckle noise removal in ultrasound images. Multimedia Tools and Applications, 78(13), 18531-18548.

[35] Eskicioglu, A. M., & Fisher, P. S. (1995). Image quality measures and their performance. IEEE Transactions on Communications, 73(12). 29592965.

[36] F. Mei, D. Zhang, Y. Yang. (2020). Improved nonlocal self-similarity measures for effective speckle noise reduction in ultrasound images. Computer Methods and Programs in Biomedicine, Vol. 196, 105670.

[37] H.Yan, P.Zhao, Z.Du, Y.Xu, P. Liu (2021). Frequency division de-noising algorithm based on VIF adaptive 2D-VMD ultrasound image. Pios one. Vol. 16, No. 3, 60248146, pp. 1-22.

[38] Niveditta Thakur, Nafis Uddin Khan and Sunildatt Sharma (2022). An efficient fuzzy inference system based approximated anisotropic diffusion for image de-noising. Cluster Computing, 25(43), 4303 - 4323.

[39] Niveditta Thakur, Nafis Uddin Khan and Sunildatt Sharma (2021). A Review on Performance Analysis of PDE Based Anisotropic Diffusion Approaches for Image Enhancement. Informatica, 45, 89 - 102.

[40] N. Singh and A. K. Bhandari (2022). Noise Aware L2-LP Decomposition-Based Enhancement in Extremely Low Light Conditions With Web Application. IEEE Transactions on Consumer Electronics, Vol. 68, No. 2, pp. 161 - 169.

[41] C. Yu and L. Z. Hou (2022). Realization of a RealTime Image De-noising System for Dashboard Camera Applications. IEEE Transactions on Consumer Electronics, Vol. 68, No. 2, pp. 181 - 190.

[42] P. A. A. Penna and N. D. A. Mascarenhas (2019). SAR Speckle Nonlocal Filtering With Statistical Modeling of Haar Wavelet Coefficients and Stochastic Distances. IEEE Transactions on Geoscience and Remote Sensing, Vol. 57, No. 9, pp. 7194-7208.

[43] S. Tan, X. Zhang, H. Wang, L. Yu, Y. Du, J. Yin and B. Wu (2022). A CNN-Based Self-Supervised Synthetic Aperture Radar Image De-noising Approach. IEEE Transactions on Geosciene and Remote Sensing, Vol. 60, pp. 1-15, Art no. 5213615.

[44] A. G. Mullissa, D. Marcos, D. Tuia, M. Herold and J. Reiche (2022). deSpeckNet: Generalizing Deep Learning-Based SAR Image De-speckling. IEEE Transactions on Geosciene and Remote Sensing, Vol. 60, pp. 1-15, Art no. 5200315.

[45] S. S. Rajput, A. Singh, K. V. Arya and J. Jiang (2018). Noise robust face hallucination algorithm using local content prior based error shrunk nearest neighbors representation. Signal Processing, Vol. 147, pp. 233 -246.

[46] S. S. Rajput and K.V. Arya (2019). Noise robust face hallucination via outlier regularized least square and neighbor representation. IEEE Transactions on Biometrics, Behavior, and Identity Science, Vol. 1, Issue 4, pp. 252 - 263.

[47] S. S. Rajput, K.V. Arya and Vinay Singh (2018). Robust face super-resolution via iterative sparsity and locality-constrained representation. Information Sciences, Vol. 463, pp. 227 - 244.

[48] Nafis uddin Khan and K. V. Arya (2018). Two Stage Image De-noising and Edge Enhancement by SVD on Large Scale Heterogeneous Anisotropic Diffused Image Data. Multimedia Tools and Applications, Volume 77, pp. 22543 - 22566.

[49] Nafis uddin Khan, K. V. Arya and Manisha Pattanaik (2014). Edge Preservation of Impulse Noise Filtered Images by Improved Anisotropic Diffusion. Multimedia Tools and Applications, Volume 73, pp. 573-597.

[50] Nafis uddin Khan, K. V. Arya and Manisha Pattanaik (2013). Histogram Statistics based Variance Controlled Adaptive Threshold in Anisotropic Diffusion for Low Contrast Image Enhancement. Signal Processing, Volume 93, Issue 6,pp. 1684-1693.

[51] Nafis uddin Khan, K. V. Arya and Manisha Pattanaik (2012). A Robust PDE based Image Denoising Method. International Journal of Computer and Electrical Engineering, Volume 4, No. 3, pp. 303-306.

[52] Nafis uddin Khan, K. V. Arya and Manisha Pattanaik (2011). Image Enhancement and Denoising by Diffusion based Singular Value Decomposition. International Journal of Computer Applications, Volume 32, No. 8, pp. 14-22.

[53] Nafis uddin Khan, K. V. Arya and Manisha Pattanaik (2010). An Efficient Image Noise Removal and Enhancement Method. Proceedings of 2010 IEEE International Conference on Systems, Man & Cybernetics (SMC 2010), October 10-13, 2010, Istanbul, Turkey, pp. 3735-3740.