Content area
Block discrete cosine transform coding has been widely used in image and video compression standards. However, at low bit rate coding, the compressed image produces obvious block effects at the block boundaries, which seriously affect the image visualization. This paper combines Gaussian curvature regularization and structural sparse representation to remove the block artifacts appearing in the compressed images, while preserving sharp edges. More precisely, we use the internal structural sparse prior to remove the image noise, and apply the external structural sparse prior to prevent image overfitting. Meanwhile, we perform Gaussian curvature regularization constraint that blends image gradient information, in order to remove the detrimental structure of the compressed image. Concretely, we incorporate filtering technique into the alternating iteration method for handling the nonconvexity problem of the proposed model. Experimental results demonstrate that our algorithm achieves several state-of-the-art deblocking algorithms in terms of both objective and visual perception.
Introduction
In recent years, with the rapid development of the Internet, images occupy an increasingly important position in the field of modern communication technology. Usually, due to the large amount of data, image data is compressed by using the block-based transform coding, such as JPEG, H.264/A VC, H.265/HEVC, in order to effectively transmit and store images. As a core algorithm about image compression, BDCT is widely used in current coding standards, which can lead to the appearance of visual blocking artifacts in the compressed images. Therefore, it is important to research the issue about removing the block effect in the compressed images.
In order to suppress the bad visual effects produced by the blocking artifacts, scholars have been conducted many studies and proposed different approaches. On the whole, the deblocking methods can be divided into two types, namely, pre-processing methods and post-processing methods. The pre-processing approach eliminates the block effect by changing the compression framework. Unlike the pre-processing approach, the post-processing approach [1] requires no changes about existing coding scheme, replaced by performing the deblocking process at the coding end. The post-processing approach based on traditional methods mainly consists of image enhancement deblocking methods and image restoration deblocking methods.
The image enhancement deblocking methods utilize the special structure of the image block effect and human visual perception for the improvement of image quality, which mainly include spatial domain filtering methods and transformation domain filtering methods. Foi et al. [2] exploited Wiener filtering to process the blocking artifacts of the compressed images. Zhai et al. [3] presented an effective deblocking scheme with local coefficient regularization, block-wise shape adaptive filtering and quantization constraint. Kim et al. [4] introduced an adaptive image blocking artifact reduction method by the directional activity of wavelet-based block analysis. Nath et al. [5] proposed a non-iterative image deblocking algorithm in the wavelet domain for reducing blocking artifacts. Mahalakshmi et al. [6] presented an efficient algorithm for automatic detection of visually impaired regions in the compressed images. Nath et al. [7] designed a non-iterative image deblocking algorithm by performing the hard thresholding operation in non-subsampled shearlet transform domain.
Image restoration deblocking methods regard the block removal problem as a reconstruction problem by treating a coded image as a degraded image of the original image. Image restoration deblocking methods utilize the image prior knowledge in an attempt to recover the original images, which mainly include convex set projection methods, total variational methods, low-rank prior methods and sparsity-based prior methods.
Convex set projection methods define each prior of the original image as a closed convex set by projecting the coded image onto the intersection of these closed convex sets. Chu et al. [8] developed an iterative recovery algorithm by the projection onto convex sets to reduce motion-related artifacts in the magnetic resonance imaging. Lee et al. [9] described an algorithm based on the projection onto convex sets for reducing the N/2 ghost in echo-planar imaging. Jing et al. [10] proposed a non-local algorithm by the iterative-correction projection onto convex sets. Generally speaking, convex set projection methods require a large number of iterations to achieve the removal of the block effect, so these methods have a high computational complexity.
Total variational methods recovery the original images by computing energy minimization. Alter et al. [11] devised a total variation minimization deblocking method that overcomes the limitation of unquantized cosine coefficients. Wei et al. [12] proposed a weighted adaptive total variation method to remove the blocking artifacts. Zhu et al. [13] imposed the continuity constraint on the block boundaries to deal with the blocking artifacts. Suo et al. [14] combined an adaptive high-dimensional nonlocal total variation and narrow quantization constraints to reduce blocking artifacts. By interpreting images as graph signals, Zhou et al. [15] proposed an artifact-free image deblocking method based on the graph signal total variational framework. Total variational methods assume that the image is binned and continuous, so the corresponding reconstructed image will lose some texture information of the images with rich texture features.
The low-rank prior methods and sparsity-based prior methods formulate the image deblocking problem as an optimization problem on the basis of the maximum a posteriori (MAP) framework. Yeh et al. [16] formulated the deblocking problem as an image decomposition problem based on morphological component analysis through sparse representation. Amiri et al. [17] proposed a sparse representation method by learning a dictionary via the single input blocky image. Li et al. [18] utilized a shape-adaptive low-rank prior to preserve edges well, and used an extra prior to restore the lost high-frequency components. Zhao et al. [19] adopted the structural sparse representation prior and quantization constraint prior to conduct image deblocking operation. Zhang et al. [20] jointly used the weighted kernel norm minimization and deep prior model to deal with image artifacts. Zha et al. [21, 22, 23–24] designs different image restoration algorithms from these three types of prior information, nonlocal self-similarity priors, group sparsity and low-rank priors, and illustrated that applicability of different types of priors. Hu et al. [25] made use of the shape-adaptive low-rank prior, the quantization constraint prior and sparsity-based detail enhancement to obtain initial deblocked images. Liu et al. [26] introduced an image deblocking model by combining the sparse representation of fractional paradigms and low-rank regularization. Yuan et al. [27] provided a non-local self-similar deblocking method by adaptively solving the mixed-paradigm minimization models. To improve the applicability of the deblocking model, Arya et al. [28] used the wavelet transform to extract the local sparse structure in image frequency domain and the group sparse representation to extract the non-local self-similar structure in image spatial domain. More precisely, Zha et al. [29] proposed a novel hybrid structural sparsification error (HSSE) model for image restoration, which jointly exploited non-local self-similar prior using both the internal and external image data on account of highly image restoration tasks. The low-rank prior deblocking methods involve the low-rank minimization, resulting in higher computational complexity. Sparsity prior-based prior deblocking methods can cover more pixel information, thus favoring structure enhancement and noise suppression. Particularly, sparsity-based methods not only have a wide range of applications in image restoration problems, but still play an important role in image super-resolution reconstruction [30, 31].
In the past few years, as a powerful regularization technique, the curvature of image surface has been applied in image denoising [32, 33, 34–35], image restoration [36, 37–38] and image fusion [39]. Aiming at blind image deblurring problem, Ge et al. [38] combined Gaussian curvature regularization with sparse prior of the gradient image to increase small and large gradient information of the latent clean image. Jiang et al. [39] fused the medical images by using a new entropy measure that combines intuitionistic fuzzy sets with Gaussian curvature filtering. Wang et al. [40] presents a scalar auxiliary variable method for solving the mean curvature and Gaussian curvature minimization problems with applications to image noise, image deblurring and image super resolution. Liu et al. [41] proposed an operator splitting method to solve Gaussian curvature model, the efficiency of this method was proved by experiments of surface smoothing and image denoising. In this paper, we focus on how to take advantage of the Gaussian curvature regularization with its good geometric interpretability and strong continuity, so that our model can better constrain the solution space of image deblocking method based on the MAP framework.
In this paper, we propose an image deblocking algorithm based on Gaussian curvature and structural sparse representation, which can preserve the edge information well. Our contributions can be summarized as follows:
We introduce a novel Gaussian curvature regularization term that contains the information about Gaussian curvature and image gradient, which is added to the HSSE model for removing harmful structures of the compressed image.
We embed Gaussian curvature filtering technique into half quadratic splitting algorithm in order to effectively solve our proposed model.
The experimental results show that our proposed model is competitive with several state-of-the-art models in terms of both objective and visual perception.
The rest of this paper is organized as follows. Section 2 introduces the constraint terms of the original image, including the internal prior, the external prior and Gaussian curvature regularization. Section 3 gives our proposed model, as well as its solving approach. Experiments and conclusions are provided in Sect. 4 and Sect. 5, respectively.
Preliminary
In this section, we firstly review the quantization noise model and HSSE model for image deblocking, then introduce the proposed Gaussian curvature regularization.
The quantization noise model
As we all know, the blocking image is usually denoted as
1
where is the JPEG-coded image with blocking artifacts, is the original image, is the compression operator, is the quantization noise produced in the image compression process. Let be the 8-by-8 quantization matrix determined by the quality factor , and be the mean value of the nine upper-left entries of defined byZhang et al. [29] indicated that the variance of the quantization noise , denote by , can be computed by
2
Under the MAP framework, the estimation model of the original image is
3
where the first term represents the data-fidelity that can be formulated as4
and the second term corresponds to the image prior.The HSSE model
The HSSE model takes into account both the internal and external priors to deal with the sparse representation of images. To be precise, an image is divided into the overlapped patches. The block matching method regroups these patches into a series of non-local self-similar blocks, denoted by . Notice that we use to represent the operator for searching non-local self-similar blocks of an image. Without loss of generality, we use the symbol to express the number of non-local self-similar blocks. For each group of image blocks , let and respectively denote the corresponding internal and external dictionaries. Moreover, let and respectively denote the internal and external group sparse coefficients of . The HSSE model works on computing the coefficients and ,, by solving the following model
5
where and respectively express the Frobenius norm and -norm, is the balancing factor, , and are all regularization parameters.Zha et al. [29] constructed many groups from external image corpus, and then developed a group-based Gaussian mixture model (GMM) method to learn the external dictionaries , which was used to compute the external group sparse coefficients . The coefficients are computed by solving the following minimization problem
6
For each , let be the sparse coefficients of about the external dictionary , namely . Further, let and denote the vectorization of the matrices and , then the model (6) can be simplified as
Obviously, the closed solution of the above model can be obtained by the soft-thresholding, namely,
Next, the HSSE model updates the dictionary about each group by means of principal component analysis (PCA). The external group sparse coefficients are computed by the same way , namely,
7
To simplify the model (7), the HSSE model introduces the joint image blocks defined by.
Let be the coefficient representation relying on the dictionary . Let and respectively denote the vectorization of the matrices and , let , then the model (7) is reduced to the following modelthe corresponding closed solution is given by.
Gaussian curvature
Gaussian curvature filtering can preserve image edges and remove detrimental structures while performing image denoising task. Let denote the image surface corresponding to the image , where is the pixel coordinate and is the corresponding pixel value. Let and denote the gradient of in the and directions, respectively. Let , and denote the corresponding second-order partial derivatives. Then Gaussian curvature of the image can be represented by
In order to prevent the correlation between neighborhood pixels, Gaussian curvature filtering proposed the image domain decomposition which divides the pixels of two-dimensional image into four categories, namely, black rectangles, white rectangles, black circles and white circles, as shown in Fig. 1. Obviously, there are 8 kinds of projections from a pixel to its tangent plane which corresponds to 8 kinds of distance. The minimum value of 8 kinds of projection distances is taken as the projection operator to calculate the distance from the tangent plane formed by neighborhood pixels in the window. Algorithm 1 illustrates the specific process of Gaussian curvature filtering.
Fig. 1 [Images not available. See PDF.]
Schematic illustration of image domain decomposition
In this paper, we introduce a regularization term that contains the Gaussian curvature and image gradient, which is defined by
8
where is the -norm, and is the positive parameter balancing the effectiveness between Gaussian curvature regularization and -norm of image gradients.The proposed algorithm
Deblocking model
Inspired by the HSSE model and Gaussian curvature regularization, we propose an image deblocking algorithm based on Gaussian curvature and structural sparse representation. On the basis of using the symbols in Eq. (1) and Eq. (5), our proposed model is defined by
9
where is the variance of the quantization noise determined by Eq. (2), is a balancing factor, is Gaussian curvature regularizing operator defined by Eq. (8).The minimization problem (9) is difficult due to the simultaneous existence of both sparse prior term and Gaussian curvature regularization term. To overcome this problem, we adopt the half quadratic splitting algorithm by introducing auxiliary variable and to respectively substitute appearing in the second term and last term of Eq. (9). Accordingly, the minimization problem (9) is rewritten as.
10
where and are balancing factors.
Alternating minimization to solve the deblocking model
In this subsection, we adopt the alternating minimization by incorporating the Gaussian curvature filtering technique to effectively solve the proposed deblocking model. Evidently, the minimization problem (10) involves solving a sequence of variables, namely, , , , and .
Firstly, our model determines the group sparse coefficients and when the variables , , are fixed. In this case, the corresponding sub-model is the same as (5). Therefore, we adopt the same method as the HSSE model to compute and .
Next, when and are fixed, the model (10) is simplified as
11
Apparently, the sub-model (11) can be divided into the following sub-problems to solve, namely,
12
13
14
For the fixed , we use Algorithm 2 to solve the problem (12) by using curvature filtering technology and enumerating the developable surfaces in the neighborhood.
When and are fixed, the closed solution of the sub-problem (13) is
15
Similarly, for the fixed , the closed solution of the problem (14) is
16
Adaptive parameter adjustment
We use the same method as the HSSE model to adaptively update the parameters , , , . For the sake of completeness, we present the calculation of the parameters. Let be the noise variance at the -th iteration computed by
17
where is a scaling factor, then the parameters and at the -th iteration are directly determined by18
where and are scaling factors. The parameters and are updated by
19
where and are respectively the estimated standard deviation of and that are respectively the sparse coefficients satisfying and , and is a small constant to avoid dividing by zero. Additionally, we directly update and by.
where and are amplification factors. The complete description of the proposed algorithm for image deblocking is exhibited in Algorithm 3.
Experimental results
All experiments are implemented by MATLAB R2022a. We compare our proposed method with SSR-QC [19], JPG-SR [22] and HSSE [29] methods. Due to limited space, we only present the deblocking results at the quality factor . The experimental results are divided into experiments about the CBSD68 dataset and Set5 dataset. Due to the fact that the human system is more sensitive to illumination, this paper conducts the quantitative results of color images only in the illumination channel, which operation is the same as the SSR-QC, JPG-SR and HSSE methods.
We choose the indexes of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) to quantitatively evaluate the deblocking results. The PSNR index can measure the intensity difference between two images. The larger the PSNR, the better the performance of the deblocking algorithm. The SSIM index can reflect the structure similarity between two images. The larger the SSIM, the smaller the difference between the deblocking image and the original image.
Parameters setting
In this paper, we cite an adaptive parameter setting scheme for noise estimation, namely, is determined by Eq. (2). We empirically use the size of each patch as 7 × 7 and nonlocal similar patch number as , which are the same as the HSSE model. The parameters and related to Gaussian curvature regularization term are set to be 45 and 0.01, respectively. The initial values of the balance factors and are respectively set to be 2.5 and 1, which are respectively updated by amplification factors and . Since the parameters , , belong to the HSSE model, we pick the values of these parameters to be consistent with the HSSE model. To be specific, the parameter is chosen as 0.6 for the case , and 0.5 for the case .The parameter is set to be for , respectively. The parameter is set to be 1.2.
Experiments results on the CBSD68 dataset
We evaluate the qualitative and quantitative performance of our proposed algorithm on the CBSD68 dataset, which contains 68 images. The CBSD68 dataset [42] is commonly used to measure the performance of image denoising algorithms. The average PSNR and SSIM values of different methods on the CBSD68 dataset are provided in Table 1. The average PSNR improvements of our proposed algorithm for the restored image at over SSR-QC, JPG-SR, HSSE are respectively 0.2376 dB, 0.2304 dB, 0.0240 dB, the average SSIM improvements of our algorithm for the restore image over contrast algorithms are respectively 0.0146, 0.0049, 0.0020. Similarly, when , our proposed model achieves {0.2370 dB, 0.2365 dB, 0.0145 dB} gains in the average PSNR and {0.0117, 0.0048, 0.0020} gains in the average SSIM values. Unsatisfactorily, when and , our algorithm respectively achieves 0.0050 dB and 0.0045 dB over the HSSE model in the average PSNR values, and the average SSIM values of our algorithm cannot outperform the HSSE algorithm, which is understandable since hybrid prior models incorporating Gaussian curvature regularization are more effective for heavily degraded images.
Table 1. Average PSNR (dB)/SSIM comparisons of different methods on the CBSD68 dataset
SSR-QC | JPG-SR | HSSE | OURS | |||||
|---|---|---|---|---|---|---|---|---|
PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
q = 1 | 23.8502 | 0.5992 | 23.8574 | 0.6088 | 24.0638 | 0.6118 | 24.0878 | 0.6138 |
q = 5 | 26.1736 | 0.7091 | 26.1741 | 0.7160 | 26.3961 | 0.7188 | 26.4106 | 0.7208 |
q = 10 | 28.6202 | 0.7998 | 28.6521 | 0.8087 | 28.8699 | 0.8107 | 28.8749 | 0.8106 |
q = 20 | 30.9674 | 0.8710 | 31.0612 | 0.8787 | 31.2199 | 0.8801 | 31.2244 | 0.8801 |
Due to space constraints, we choose four images from CBSD68 dataset to demonstrate the deblocking effect of different algorithms, as shown in Figs. 2, 3, 4, 5. In Fig. 2, our proposed model can recover more details of the penguin's beak and better depict the boundary between the beak and the background without over-smoothing the penguin's head, which shows that our model has an advantage in dealing with complex shapes and textures, while being able to maintain the naturalness and details of the image. As can be seen in Fig. 3, our model recovers more details of the tail letter “A” and performs better in reducing artifacts. In Fig. 4, the performance of SSR-QC, JPG-SR, HSSE and our model is shown to be very similar by comparing the seaweed details, however, the SSR-QC, JPG-SR and HSSE methods show artifacts in the background blades of grass due to the block effect, which suggests that our model has an advantage in processing local details in the image. From Fig. 5, we can see that our model performs better on the details of the tiger's front paw, which satisfactorily recovers the hair texture details without artifacts. This shows that our model has an advantage in handling complex textures and shapes.
Fig. 2 [Images not available. See PDF.]
Deblocking results of different methods on image 106024 from CBSD68 at q = 1. a Original image. b JPEG compressed image (PSNR = 22.5486, SSIM = 0.6394). c SSR-QC (PSNR = 27.6864, SSIM = 0.7773). d JPG-SR (PSNR = 27.4500, SSIM = 0.7690) e HSSE (PSNR = 27.6818, SSIM = 0.7664). f OURS (PSNR = 27.7761, SSIM = 0.7747)
Fig. 3 [Images not available. See PDF.]
Deblocking results of different methods on image 3096 from CBSD68 at q = 5. a Original image. b JPEG compressed image (PSNR = 28.3034, SSIM = 0.8596). c SSR-QC (PSNR = 32.8735, SSIM = 0.9428). d JPG-SR (PSNR = 32.8930, SSIM = 0.9429) e HSSE (PSNR = 32.9810, SSIM = 0.9398). f OURS (PSNR = 33.0342, SSIM = 0.9415)
Fig. 4 [Images not available. See PDF.]
Deblocking results of different methods on image 210088 from CBSD68 at q = 10. a Original image. b JPEG compressed image (PSNR = 25.4765, SSIM = 0.7460). c SSR-QC (PSNR = 33.2449, SSIM = 0.9196). d JPG-SR (PSNR = 33.0636, SSIM = 0.9195) e HSSE (PSNR = 33.2324, SSIM = 0.9211). f OURS (PSNR = 33.2451, SSIM = 0.9214)
Fig. 5 [Images not available. See PDF.]
Deblocking results of different methods on image 108082 from CBSD68 at q = 20. a Original image. b JPEG compressed image (PSNR = 23.0406, SSIM = 0.7063). c SSR-QC (PSNR = 30.9639, SSIM = 0.8843). d JPG-SR (PSNR = 31.1390, SSIM = 0.8942) e HSSE (PSNR = 31.3070, SSIM = 0.8945). f OURS (PSNR = 31.3151, SSIM = 0.8951)
Experiments results on Set5 dataset
The Set5 dataset [43] is a dataset commonly used for testing performance of image super-resolution models. The Set5 dataset consists five images, namely, “baby”, “bird”, “butterfly”, “head” and “woman”. Table 2 lists the computation results of different algorithms, where the highest value in each case is shown in bold. From Table 2, we can observe that on average, our method achieves higher performance than other competing methods except for the case . When , the average SSIM result of our algorithm is the same as the HSSE algorithm. Figure 6 shows the deblocking results of “baby” image under different block effects. SSR-QC, JPG-SR, HSSE and our model can preserve the eyebrows on the front of the eye socket, as well as the skin around the eyes. However, in terms of color saturation, our method tends more to the original image.
Table 2. PSNR (dB)/SSIM comparisons of different methods on the Set5 dataset
q = 1 | SSR-QC | JPR-SR | HSSE | OUR | ||||
|---|---|---|---|---|---|---|---|---|
PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
Baby | 26.3425 | 0.7057 | 26.4440 | 0.7194 | 26.6173 | 0.7162 | 26.6310 | 0.7217 |
Bird | 25.4915 | 0.6376 | 25.5393 | 0.6858 | 25.8564 | 0.6842 | 25.8726 | 0.6855 |
Butterfly | 22.8619 | 0.7965 | 21.8581 | 0.7712 | 22.8236 | 0.7849 | 22.8712 | 0.7875 |
Head | 26.8488 | 0.4881 | 27.0452 | 0.5479 | 27.1597 | 0.5597 | 27.2360 | 0.5666 |
Woman | 24.5199 | 0.6843 | 24.4837 | 0.7185 | 24.8536 | 0.7160 | 25.0533 | 0.7279 |
Average | 25.2129 | 0.6624 | 25.0741 | 0.6885 | 25.4621 | 0.6922 | 25.5328 | 0.6978 |
q = 5 | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM |
Baby | 29.1918 | 0.7989 | 29.3820 | 0.8046 | 29.5063 | 0.8070 | 29.5195 | 0.8075 |
Bird | 28.8313 | 0.8295 | 28.6545 | 0.8267 | 29.2225 | 0.8424 | 29.2335 | 0.8426 |
Butterfly | 25.2936 | 0.8641 | 24.3895 | 0.8414 | 25.0455 | 0.8523 | 25.0650 | 0.8532 |
Head | 28.9854 | 0.6631 | 29.0850 | 0.6683 | 29.2288 | 0.6717 | 29.2356 | 0.6720 |
Woman | 27.6039 | 0.8468 | 27.2010 | 0.8423 | 27.6320 | 0.8473 | 27.9796 | 0.8553 |
Average | 27.9812 | 0.8005 | 27.7424 | 0.7966 | 28.1270 | 0.8041 | 28.2066 | 0.8061 |
q = 10 | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM |
Baby | 31.7779 | 0.86250 | 32.0246 | 0.8694 | 32.1628 | 0.8711 | 32.20725 | 0.87213 |
Bird | 32.2444 | 0.90375 | 32.3040 | 0.9061 | 32.6131 | 0.9113 | 32.68168 | 0.91288 |
Butterfly | 27.8882 | 0.91490 | 26.8418 | 0.8961 | 27.4706 | 0.9050 | 27.43039 | 0.90495 |
Head | 30.6592 | 0.71923 | 30.7360 | 0.7267 | 30.8650 | 0.7286 | 30.89971 | 0.73044 |
Woman | 30.6881 | 0.90934 | 30.1857 | 0.9033 | 30.6295 | 0.9098 | 30.63960 | 0.91019 |
Average | 30.6516 | 0.8619 | 30.4184 | 0.8603 | 30.7482 | 0.8651 | 30.7717 | 0.8661 |
q = 20 | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM |
Baby | 34.0656 | 0.9070 | 34.3083 | 0.9137 | 34.4507 | 0.9146 | 34.4721 | 0.9148 |
Bird | 35.3899 | 0.9465 | 35.7447 | 0.9515 | 35.7426 | 0.9509 | 35.7411 | 0.9508 |
Butterfly | 29.8201 | 0.9410 | 29.7878 | 0.9376 | 29.5892 | 0.9351 | 29.5867 | 0.9351 |
Head | 32.1588 | 0.7788 | 32.2634 | 0.7871 | 32.3923 | 0.7893 | 32.3938 | 0.7893 |
Woman | 33.2169 | 0.9426 | 33.2686 | 0.9445 | 33.3048 | 0.9443 | 33.3063 | 0.9443 |
Average | 32.9303 | 0.9032 | 33.0746 | 0.9069 | 33.0959 | 0.9068 | 33.1000 | 0.9068 |
Fig. 6 [Images not available. See PDF.]
Deblocking results of different methods on image “baby” from Set5 at q = 20, q = 10, q = 5 and q = 1. a Original image. b JPEG compressed image at q = 20 (PSNR = 27.7181, SSIM = 0.8622). c SSR-QC at q = 20. d JPG-SR at q = 20. e HSSE at q = 20. f OURS at q = 20. g JPEG compressed image at q = 10 (PSNR = 26.9038, SSIM = 0.8147). h SSR-QC at q = 10. i JPG-SR at q = 10. j HSSE at q = 10. k OURS at q = 10. l JPEG compressed image at q = 5 (PSNR = 25.3739, SSIM = 0.7392). m SSR-QC at q = 5. n JPG-SR at q = 5. o HSSE at q = 5. p OURS at q = 5. q JPEG compressed image at q = 1 (PSNR = 23.6984, SSIM = 0.6631). r SSR-QC at q = 1. s JPG-SR at q = 1. t HSSE at q = 1. u OURS at q = 1
Convergence analysis
From Algorithm 3, we know that our method adopts posterior error estimation as the iteration termination condition. Consequently, our algorithm terminates with a different number of iterations for different images. The convergence is verified by analyze the change of the PSNR and SSIM values as the number of iterations increases. In Figs. 7, 8, we list the corresponding results on Set5 dataset. Concretely the PSNR and SSIM values of the computed image significantly increase for the first 10 iterations, and the PSNR and SSIM values gradually stabilize after the 10th iteration, which proves the convergence of our proposed method.
Fig. 7 [Images not available. See PDF.]
PSNR values of the computed image for different iteration number on the Set5 dataset. aq = 1. bq = 5. cq = 10. dq = 20
Fig. 8 [Images not available. See PDF.]
SSIM values of the computed image for different iteration number on the Set5 dataset. aq = 1. bq = 5. cq = 10. dq = 20
Parameter analysis
Since the parameters , , , are same as the HSSE model, we only explore the influence of the parameters , , , , , on the proposed algorithm by only changing one parameter at a time. We compute the average PSNR values of our algorithm on the Set5 dataset to compare the deblocking effect when one parameter is changed. As seen in Fig. 9, our algorithm is insensitive to parameter variations within a certain range.
Fig. 9 [Images not available. See PDF.]
Sensitivity analysis of the parameters in the proposed algorithm. a Effect of the parameter on PSNR values. b Effect of the parameter on PSNR values. c Effect of the parameter on PSNR values. d Effect of the parameter on PSNR values. e Effect of the parameter on PSNR values. d Effect of the parameter on PSNR values
Conclusion
This paper proposes an image deblocking algorithm based on Gaussian curvature and structural sparse representation. The core of our algorithm is to combine internal and external structural sparse prior with Gaussian curvature for image unlocking. We first use the GMM approach to learn an external dictionary for improved generation of structural sparse representation models. After that, we use the internal prior information to characterize the non-local self-similarity of the original image. Considering the preservation of image edge information, we jointly employ Gaussian curvature and image gradient constraints in the sparse representation model. Finally, we integrate the curvature filtering technique into the semi-quadratic splitting method to solve our proposed model. Experimental results on the BSD68 dataset and the Set5 dataset show that our proposed algorithm outperforms other competing methods in terms of the average PSNR values. Visual comparisons illustrate that for severely degraded images, our proposed method not only eliminates artifacts to some extent, but also achieves better visual perception in terms of details and textures. In the future, we will work on designing Gaussian curvature regularization term that fuse features from the frequency domain of the original image to preserve richer edge and texture information of the image.
Acknowledgements
The authors would like to acknowledge the contributions of all the reviewers and thank them for their insightful comments on the early drafts of this paper.
Funding
This paper is supported by the National Nature Science Foundation of China under Grant (No.12171054) and the fund of the Department of Education of Jilin Province (JJKH20230788KJ).
Data availability
The data that support the findings of this study are available from corresponding author upon reasonable request.
Declarations
Conflict of interest
No conflict of interest exists in the submission of this manuscript, and the manuscript is approved by all listed authors for publication. The authors would like to declare that the work was the original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1. Shen, M; Kuo, CJ. Review of postprocessing techniques for compression artifact removal. J. Vis. Commun. Image Represent.; 1998; 9,
2. Foi, A; Katkovnik, V; Egiazarian, K. Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images. IEEE Trans. Image Process.; 2007; 16,
3. Zhai, GT; Zhang, WJ; Yang, XK; Lin, WS; Xu, Y. Efficient deblocking with coefficient regularization, shape-adaptive filtering, and quantization constraint. IEEE Trans. Multi.; 2008; 10,
4. Kim, J. Adaptive blocking artifact reduction using wavelet-based block analysis. IEEE Trans. Consum. Electron.; 2009; 55,
5. Nath, V. K., Hazarika, D.: Blocking artifacts suppression in Wavelet transform domain using local Wiener filtering. In: the 3rd National Conference on Emerging Trends and Applications in Computer Science. pp. 93–97 (2012)
6. Mahalakshmi, V; Srinivas, R. An optimized algorithm for ringing region detection in compressed images. Int. J. Mod. Eng. Res.; 2011; 1,
7. Nath, V. K., Baruah, H.G., Hazarika, D.: An image deblocking approach based on non-subsampled Shearlet transform. In: the 2nd International Conference on Data Engineering and Communication Technology. Advances in Intelligent Systems and Computing, vol. 828. Springer, Singapore, pp. 231–238 (2019)
8. Chu, ML; Chang, HC; Chung, HW. POCS-based reconstruction of multiplexed sensitivity encoded MRI (POCSMUSE): a general algorithm for reducing motion-related artifacts. Magn. Reson. Med.; 2015; 74,
9. Lee, KJ; Barber, DC; Paley, MN. Image-based EPI ghost correction using an algorithm based on projection onto convex sets (POCS). Magn. Reson. Med.; 2010; 47,
10. Jing, H; Ma, J; Nan, L. Sparse angular CT reconstruction using non-local means based iterative-correction POCS. Comput. Biol. Med.; 2011; 41,
11. Alter, F. Adapted total variation for artifact free decompression of JPEG images. J. Math. Imaging Vis.; 2005; 23, pp. 199-211.
12. Wei, D., Au, O.C., Zhu, W.: BDCT compressed image deblocking using weighted adaptive total variation. In: IEEE International Conference on Multimedia and Expo, San Jose, CA, USA, pp. 1–6 (2013)
13. Zhu, W., Au, O.C., Wei, D.: Image deblocking using convex optimization with modified total variation method. In: 2013 IEEE Global Conference on Signal and Information Processing, Austin, TX, USA , pp. 1069–1072 (2013)
14. Suo, SY; Xiao, HE; Xiong, SH. Image Deblocking algorithm using adaptive high-dimensional nonlocal total variational for compressed image. Sci. Technol. Eng.; 2018; 18,
15. Zhou, F; Li, H; Yang, L. A novel image deblocking approach within a graph framework. Digit. Signal Process.; 2022; 120, [DOI: https://dx.doi.org/10.1016/j.dsp.2021.103270]
16. Yeh, CH; Kang, LW; Chiou, YW; Lin, CW; Jiang, SJF. Self-learning-based post-processing for image/video deblocking via sparse representation. J. Visual Commun. Image Represent.; 2014; 25,
17. Amiri, SA; Hassanpour, H. Deblocking joint photographic experts group compressed images via self-learning sparse representation. Mater. Energy Res. Center.; 2016; 29,
18. Li, T; He, X; Qing, L. An iterative framework of cascaded deblocking and super-resolution for compressed images. IEEE Trans. Multi.; 2017; 20,
19. Zhao, C; Zhang, J; Ma, S; Fan, X; Zhang, Y; Gao, W. Reducing image compression artifacts by structural sparse representation and quantization constraint prior. IEEE Trans. Ci.r Sys. Video Technol.; 2017; 27,
20. Zhang, J; Xiong, R; Zhao, C. CONCOLOR: Constrained non-convex low-rank model for image deblocking. Trans. Img. Proc.; 2016; 25,
21. Zha, Z; Yuan, X; Zhou, J. Image restoration via simultaneous nonlocal self-similarity priors. IEEE Trans. Image Process.; 2020; 29, pp. 8561-8576. [DOI: https://dx.doi.org/10.1109/TIP.2020.3015545]
22. Zha, Z; Yuan, X; Wen, B; Zhang, J; Zhou, J; Zhu, C. Image restoration using joint patch-group-based sparse representation. IEEE Trans. Image Process.; 2020; 29, pp. 7735-7750. [DOI: https://dx.doi.org/10.1109/TIP.2020.3005515]
23. Zha, Z; Yuan, X; Wen, B. Nonconvex structural sparsity residual constraint for image restoration. IEEE Trans. Cybern.; 2022; 52,
24. Zha, Z; Wen, B; Yuan, X. Low-rankness guided group sparse representation for image restoration. IEEE Trans. Neural Netw. Learn Syst.; 2022; [DOI: https://dx.doi.org/10.1109/TNNLS.2022.3144630]
25. Hu, J; Zhou, X; Ren, C. Image deblocking via shape-adaptive low-rank prior and sparsity-based detail enhancement. Signal Process. Image Commun.; 2020; 86, [DOI: https://dx.doi.org/10.1016/j.image.2020.115874] 115874.
26. Liu, S; Tian, S; Cao, J. A joint sparse representation and low rank prior regularization model for image deblocking. Optik; 2023; 286, 171013. [DOI: https://dx.doi.org/10.1016/j.ijleo.2023.171013]
27. Yuan, W; Liu, H; Liang, L. Rank minimization via adaptive hybrid norm for image restoration. Signal Process.; 2023; 206, 108926. [DOI: https://dx.doi.org/10.1016/j.sigpro.2022.108926]
28. Arya, AS; Mukhopadhyay, S. Adaptive sparse modeling in spectral and spatial domain for compressed image restoration. Signal Process.; 2023; 213, [DOI: https://dx.doi.org/10.1016/j.sigpro.2023.109191]
29. Zha, Z; Wen, B; Yuan, X. A hybrid structural sparsification error model for image restoration. IEEE Trans. Neural Netw. Learn Syst.; 2021; 33,
30. Altantawy, DA; Saleh, AI; Kishk, SS. Texture-guided depth upsampling using Bregman split: a clustering graph-based approach. Vis. Comput.; 2020; 36, pp. 333-359. [DOI: https://dx.doi.org/10.1007/s00371-018-1611-x]
31. Altantawy, DA; Saleh, AI; Kishk, SS. Bi-perspective fisher discrimination for single depth map upsampling: a self-learning classification-based approach. Neurocomputing; 2020; 380, pp. 321-340. [DOI: https://dx.doi.org/10.1016/j.neucom.2019.08.074]
32. Lee, SH; Seo, JH. Noise removal with Gauss curvature-driven diffusion. IEEE Trans. Image Process.; 2005; 14,
33. Zhu, W; Chan, T. Image denoising using mean curvature of image surface. SIAM J. Imag. Sci.; 2012; 5,
34. Brito-Loeza, C; Chen, K; Uc-Cetina, V. Image denoising using the Gaussian curvature of the image surface. Numer. Methods Partial Differential Eq.; 2016; 32,
35. Gong, Y., Sbalzarini, I.F.: Local weighted Gaussian curvature for image processing. In: IEEE International Conference on Image Processing. pp. 534–538 (2013)
36. Zhu, H; Shu, H; Zhou, J; Bao, X; Luo, L. BayesIan algorithms for PET image reconstruction with mean curvature and Gauss curvature diffusion regularizations. Comput. Biol. Med.; 2007; 37,
37. Zhong, Q; Yin, K; Duan, Y. Image reconstruction by minimizing curvatures on image surface. J. Math. Imaging Vision; 2021; 63,
38. Ge, X; Tan, J; Liu, J. Blind image deblurring with Gaussian curvature of the image surface. Signal Process Image Commun.; 2022; 100, 116531. [DOI: https://dx.doi.org/10.1016/j.image.2021.116531]
39. Jiang, Q; Huang, J; Jin, X. Medical image fusion using a new entropy measure between intuitionistic fuzzy sets joint Gaussian curvature filter. IEEE Trans. Radiat. Plasma Med. Sci.; 2023; 7,
40. Wang, C; Zhang, Z; Guo, Z. Efficient SAV algorithms for curvature minimization problems. IEEE Trans. Circuits Syst. Video Technol.; 2022; 33,
41. Liu, H; Tai, XC; Glowinski, R. An operator-splitting method for the Gaussian curvature regularization model with applications to surface smoothing and imaging. SIAM J. Sci. Comput.; 2022; 44,
42. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Eighth IEEE International Conference on Computer Vision. Vancouver, BC, Canada, pp. 416–423 (2002)
43. Kim, J., Lee, J.K. and Lee, K. M.: Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp.1646–1654 (2016)
Copyright Springer Nature B.V. Jan 2025