An Entropy-Histogram Approach for Image Similarity and Face Recognition

Abstract

Image similarity and image recognition are modern and rapidly growing technologies because of their wide use in the field of digital image processing. It is possible to recognize the face image of a specific person by finding the similarity between the images of the same person face and this is what we will address in detail in this paper. In this paper, we designed two new measures for image similarity and image recognition simultaneously. The proposed measures are based mainly on a combination of information theory and joint histogram. Information theory has a high capability to predict the relationship between image intensity values. The joint histogram is based mainly on selecting a set of local pixel features to construct a multidimensional histogram. The proposed approach incorporates the concepts of entropy and a modified 1D version of the 2D joint histogram of the two images under test. Two entropy measures were considered, Shannon and Renyi, giving a rise to two joint histogram-based, information-theoretic similarity measures: SHS and RSM. The proposed methods have been tested against powerful Zernike-moments approach with Euclidean and Minkowski distance metrics for image recognition and well-known statistical approaches for image similarity such as structural similarity index measure (SSIM), feature similarity index measure (FSIM) and feature-based structural measure (FSM). A comparison with a recent information-theoretic measure (ISSIM) has also been considered. A measure of recognition confidence is introduced in this work based on similarity distance between the best match and the second-best match in the face database during the face recognition process. Simulation results using AT&T and FEI face databases show that the proposed approaches outperform existing image recognition methods in terms of recognition confidence. TID2008 and IVC image databases show that SHS and RSM outperform existing similarity methods in terms of similarity confidence.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Facial recognition technology will change society in many ways. Face recognition system is available nowadays. Face recognition has been used in many commercial and law enforcement applications [1] such as some of the airport’s systems, and ATM and electronic payment started to test facial recognition in real events. This availability of efficient face recognition algorithms leads to the fact that it can be used in real-time security issues where there is nowhere to hide.

People can recognize each other by the spectacular diversity of facial features and this is essential to the formation of complex societies. The face has the capability to send emotional signals, either voluntarily or involuntarily. The current biometric system technology reads faces as efficiently as humans do. Holy places use face recognition to track the presence of worshipers; retailers also use facial recognition technology to monitor thieves or to arrest suspects. Face recognition technology helps to verify the ID of the ride-hailing driver, verify the permits of tourists to enter tourist places, and let people buy things with a smile [2].

Face recognition is falling within the similarity of images where it is possible to recognize the face by finding the similarity and dissimilarity between the image stored in the database and the current image of the same person [3, 4]. There are different approaches for face recognition, especially statistical and information-theoretic. In this paper, we focus on an information-theory approach to design a similarity measure capable of testing similarity for the purpose of face recognition.

The measurement of image similarity is a significant point in the applications of the real world and several fields like optical character recognition (OCR), identity authentication, human-computer interfacing, surveillance, and other pattern recognition tasks [5].

To measure the similarity between two digital images, there is a simple method to calculate the similarity which is mean squared error. The advantage of mean squared error is easy to calculate, but, at the same time, it is not accurate for pattern recognition.

It is possible to use information-theory approach in image processing for analysis if we consider the image is a two-dimensional random variable, giving rise to the use of information-theoretic measures (such as joint histogram) to define similarity and recognition measures between images [6]. There are two useful measures of information which are Shannon entropy and Renyi entropy measures. In this paper, we present an information-theoretic image similarity-recognition measures and show its superior performance versus the similarity measures SSIM, FSIM, and FSM and recognition measures ZMSIM and ZESIM.

This paper handles the information-theoretic approach and includes the following sections: Section 2 describes the related works that give high-performance measures; Section 3 presents the design of novel information-theoretic measures based on entropy, combined with the joint histogram; Section 4 shows simulation results and performance analysis; and Section 5 presents conclusions and future work.

2. Related Works

There are several works that addressed the face recognition approach and images similarity measure by employing the information theory and entropy concepts. All previous work has solved a high level of challenges of the face recognition and image similarity to support this system to work in real time. The authors developed SSIM and explained the performance of SSIM by using some examples [7].

Zhang et al. (2011) presented a feature similarity index measure (FSIM) for image-quality assessment. Phase congruency was used as a primary feature in the feature similarity index measure, whereas the gradient magnitude was used as a secondary feature in the feature similarity index measure to compute the feature similarity index. Experiment results were on the six-benchmark image-quality assessment database. Later we will demonstrate that entropy metrics performance is much higher than FSIM [8].

The feature extraction using the discrete cosine transform (DCT) with the approach of illumination normalization in the domain of the logarithm is proposed by Arindam Kar et al. in 2013 [9]; then, in the second step, they applied the entropy measures on the discrete cosine transform coefficients. Finally, they applied the kernel entropy component analysis with an extension of arc cosine kernels on the extracted DCT coefficients; their system was tested on the four databases such as FERET, AR, FRAV2D, and AT & T.

In 2013 Darshana Mistry et al. used the concepts of entropy measure, joint entropy, and joint histogram to find the similarity between two digital images and test these measures on the brain images as a database [10]

In 2014, Lee et al. suggested a method for face recognition using Shannon entropy and fuzzy logic [11]. This method is based on combination of Shannon and fuzzy logic. In this work, the use of entropy is to calculate the element ratio between two faces images and the use of fuzzy logic is to calculate the entropy membership with one.

In 2015, Yulong Wang et al. introduced a MEEAR (Minimum Error Entropy-based Atomic Representation) framework for facial recognition system. MEEAR is based on the minimum error entropy (MEE) model to be more robust under noise condition [12]. MEEAR can be used for developing new classifiers. MEEAR can provide distinctive representation vector by reducing the atomic norm regularized Renyi’s entropy of the reconstruction error.

Images similarity index based on entropy function and group theory is proposed by Y. G. Suarez et al. in 2015 [13]. An algebraic group theory of images is considered in this image similarity index. Images subtraction is provided in this similarity index by inner law.

In 2016 Q. R. Zhang et al. proposed the Improved Relative Entropy (IRE) method for face recognition approach. The IRE method is based on Shannon entropy and it is more accurate than Linear Discriminant Analysis and Locality Preserving Projections methods. The experimental results of IRE using CMU PIE and YALE B databases showed the high performance of the IRE versus LDA and LPP [14].

The system of emotion recognition based on facial expression is proposed by Y. D. Zhang et al. in 2016 [15]. Seven different facial expressions are considered in this approach such as sad, happy, angry, surprised, disgust, neutral, and fear. To extract the features the biorthogonal wavelet entropy has been used and utilizes the fuzzy multiclass support vector machine as a classifier.

To improve the kernel entropy component analysis (KECA), X. Ruan et al. in 2017 [16] did this improvement in three stages. Extract the features of faces by using Gabor wavelets in the first stage. In the second stage use the algorithms of nonlinear dimension reduction. In the third and last stage, use the k-nearest neighbor to the final classification on the fusion of different weighted multiresolution image of a human face.

FRIQA (Full-Reference Image-Quality Assessment) is an algorithm proposed by Y. Ren et al. in 2017 [17]. In FRIQA algorithm, the local entropy of images is analyzed in the first step, and then it calculates the similarity of local entropy between two images (reference image and the distorted version of it). Finally, the quality is computed for distorted version of the reference image from local energy similarity.

In a recent development, the authors in [18] introduced state-of-the-art FSM which combines the SSIM and FSIM methods. Canny edge detector has been used in FSM. The performance of FSM is tested under Gaussian noise condition and a wide range of PSNR, using FEI and AT & T databases. Experimental results show that the proposed FSM is better than the SSIM and FSIM approaches in similarity and recognition of human faces.

3. A Brief on Efficient Similarity and Recognition Measures

The distance between two sets of various data points based on a given norm is called a “similarity measure.” If we have a dataset and a function that gives a large distance between this set and members of a database, except probably one member, then we have a similarity algorithm that can detect similarity between given data and members of a database. In this paper, two information-theory measures are designed based on entropies combined with a joint histogram of two images. Performance comparison is considered with well-known similarity and recognition measures. All the methods of recognition of the face image depend on the extraction of certain features of the images; the similarity shows the features of the statistical correlation or informatics correlation. To find the similarity between two images, several approaches are utilized; some are used for face and facial expression recognition. Here we present a brief description of well-known similarity and recognition measures for the sake of performance comparison, which is overviewed as follows.

3.1. Structural Similarity Index Measure (SSIM)

The structural similarity index measure (SSIM) is one of the most popular metrics used to find the similarity between two images. Zhou Wang et al. proposed this measure in 2004 [7]. SSIM has been widely utilized for many algorithms of digital image-processing systems and image-quality assessment. The technique used in structural similarity is based on using statistical measurements, and it has an ability to extract the statistical image features for image recognition purpose such as standard deviation $(\sigma )$ and mean $\left(\mu \right)$, to get a definition for a distance function that can measure the SSIM between a training image and a test image. The measure is given by this formula:$$\begin{array}{}\text{(1)}& \rho \left(x,y\right)=\frac{\left(\mathrm{2}{\mu }_x{\mu }_y+c_{\mathrm{1}}\right)\left(\mathrm{2}{\sigma }_{xy}+c_{\mathrm{2}}\right)}{\left({\mu }_x^{\mathrm{2}}+{\mu }_y^{\mathrm{2}}+c_{\mathrm{1}}\right)\left({\sigma }_x^{\mathrm{2}}+{\sigma }_y^{\mathrm{2}}+c\mathrm{2}\right)},\end{array}$$where $\rho \left(x,y\right)$ is a structural similarity measure of a statistical similarity between the test image $(x)$ and training image $(y)$. The quantity ${\mu }_x$ is the statistical mean of pixels in image $x$, ${\sigma }_x^2$ is the statistical variance of pixels in the image $x$, ${\mu }_y$ is the statistical mean of pixels in image $y$, and ${\sigma }_y^2$ is the statistical variance of pixels in image $y$. The quantities $c_1$ and $c_2$ are constants: $c_1={{(k}_{1}L)}^2$ where $k$ is a small constant and $L$ is a maximum value of pixels; $c_2={(k_{2}L)}^2$ where $L=255$.

3.2. Feature Similarity Index Measure (FSIM)

In 2011, Zhang et al. [8] presented a feature-based similarity index for image-quality assessment (FSIM), which has become a very common measure to find the similarity in images. The phase congruency $(PC)$ was used as a primary feature and the gradient magnitude $(GM)$ was used as a secondary feature in feature similarity index measure to compute feature the similarity index. To calculate the similarity between images the FSIM definition is used: $$\begin{array}{}\text{(2)}& FSIM=\frac{{\sum }_{xϵ\Omega }{S}_L\left(x\right)·{PC}_m\left(x\right)}{{\sum }_{xϵ\Omega }{PC}_m\left(x\right)},\end{array}$$where $\Omega$ means the whole image spatial domain, $PC$ is a phase congruency, and $S_L$ is a similarity resulting from the combined similarity measure for phase congruency $S_{PC}(x)$ and similarity measure for gradient $S_G\left(x\right)$, as given by the formulas$$\begin{array}{}\text{(3)}& S_L\left(x\right)={\left[S_{PC}\left(x\right)\right]}^{\alpha }·{\left[S_G\left(\mathrm{x}\right)\right]}^{\beta },\end{array}$$where $\alpha$ and $\beta$ are parameters used to adjust the relative importance of phase congruency $(PC)$ and gradient magnitude $(\mathrm{G}\mathrm{M})$ features. The phase congruency $PC$ is given by the equation$$\begin{array}{}\text{(4)}& \mathrm{P}\mathrm{C}\left(\mathbf{x}\right)=\frac{\mathrm{E}\left(\mathbf{x}\right)}{ϵ+{\sum }_{\mathrm{n}}{\mathrm{A}}_{\mathrm{n}}\left(\mathbf{x}\right)},\end{array}$$where $ϵ$ is a small positive constant and$$\begin{array}{}\text{(5)}& E\left(\mathbf{x}\right)=\sqrt{K^{\mathrm{2}}\left(\mathbf{x}\right)+H^{\mathrm{2}}\left(\mathbf{x}\right)},\end{array}$$where $H\left(\mathbf{x}\right)={\sum }_n^{}{o}_n\left(\mathbf{x}\right)$ and $K\left(\mathbf{x}\right)={\sum }_n^{}{e}_n\left(\mathbf{x}\right)$, $o_n\left(\mathbf{x}\right)=\xi \left(\mathbf{x}\right)\ast M_n^e$, and $e_n(\mathbf{x})=\xi \left(\mathbf{x}\right)\ast M_n^o$, noting that $M_n^e$ and $M_n^o$ are even and odd symmetric filters on scale $n$ and “$\ast$” denotes convolution. The function $\xi \left(\mathbf{x}\right)$ is a 1D signal obtained after arranging pixels in different orientations. The local amplitudes $A_n(\mathbf{x})$ are defined as$$\begin{array}{}\text{(6)}& {\mathrm{A}}_{\mathrm{n}}\left(\mathbf{x}\right)=\sqrt{{\mathrm{e}}_{\mathrm{n}}^{\mathrm{2}}\left(\mathbf{x}\right)+{\mathrm{o}}_{\mathrm{n}}^{\mathrm{2}}\left(\mathbf{x}\right)},\end{array}$$where $\mathbf{x}$ is the position on scale $n$.

3.3. FSM: A Feature-Based Rational Measure

In 2017 NA Shnain et al. [18] have proposed a new structure for image similarity measure. The new structure is a rational function of measure with different statistical properties. FSM combines the best features of the well-known SSIM and FSIM approaches, trading off between their performances for similar and dissimilar images. Canny’s edge detection in FSM is used as a distinctive structural feature, where (after processing by Canny’s edge filter) two binary images, $g_x$ and $g_y$, are obtained from the original two images $x$ and $y$. FSM can be given by [18]$$\begin{array}{}\text{(7)}& \mathcal{F}\left(x,y\right)=\frac{\left(a+c\right)·R\left(g_x,g_y\right)·\Phi \left(g_x,g_y\right)+b·R\left(g_x,g_y\right)+e}{\left(a+b\right)·\rho \left(g_x,g_y\right)+c·R\left(g_x,g_y\right)·\Phi \left(g_x,g_y\right)+e},\end{array}$$where $\Phi$ stands for the feature similarity index measure (FSIM) and $\rho$ stands for the structural similarity measure (SSIM). The constants are given the values $a$ = 5, $b$ = 3, c = 7, and $e$ = 0.01, while $R\left(x,y\right)$ is a correlative function given by$$\begin{array}{}\text{(8)}& R\left(x,y\right)=\frac{{\sum }_i{\sum }_j\left(x_{ij}-x_{\mathrm{o}}\right)\left(y_{ij}-y_{\mathrm{o}}\right)}{\sqrt{\left[{\sum }_i{\sum }_j{\left(x_{ij-}{x}_{\mathrm{o}}\right)}^{\mathrm{2}}\right]\left[{\sum }_i{\sum }_j{\left(y_{ij-}{y}_{\mathrm{o}}\right)}^{\mathrm{2}}\right]}},\end{array}$$where $x_{\mathrm{o}}$ and $y_{\mathrm{o}}$ represent the image means. This function is not applied here to the original images themselves but to their edge-detected versions using $R\left(g_x,g_y\right)$.

3.4. Zernike-Moments Approach for Image Recognition

Zernike moments provide an efficient, rotation-invariant, and noise-resistant approach for image and face recognition, including the complicated effect of face expressions [19]. Zernike moments are rotation-independent as they are defined in polar coordinates $\left(r,\theta \right)\in R^{\mathrm{2}}$, with the help of Zernike radial functions $R_{pq}\left(r\right)$ that are defined as follows [20]:$$\begin{array}{}\text{(9)}& R_{pq}\left(r\right)={\displaystyle \sum }_{k=\mathrm{0}}^{\left(p-\left|q\right|\right)/\mathrm{2}}\frac{{\left(-\mathrm{1}\right)}^k\left(p-k\right)!}{k!\left(\left(p+\left|q\right|\right)/\mathrm{2}-k\right)!\left(\left(p-\left|q\right|\right)/\mathrm{2}-k\right)!}{r}^{p-\mathrm{2}k}\end{array}$$where $p$, $q$ are integers that satisfy the conditions: $\left|p-q\right|$ is even and $\left|q\right|\le p.$ In the 2-dimensional radial domain, Zernike moments are defined as follows:$$\begin{array}{}\text{(10)}& Z_{pq}=\frac{p+\mathrm{1}}{\pi }{{\displaystyle \int }}_{\mathrm{0}}^{\mathrm{2}\pi }{{\displaystyle \int }}_{\mathrm{0}}^{\mathrm{1}}{R^{\ast }}_{pq}\left(r\right)e^{-jq\theta }f\left(r,\theta \right)·r·dr·d\theta ,\hspace{1em}\left|r\right|\le \mathrm{1},\end{array}$$where $\ast$ indicates complex conjugation. In order to use these moments for image recognition, we should approximate them in the discrete Cartesian coordinate system. Therefore, we perform a linear transformation of the image Cartesian coordinates $(i,j)$ from the inside of the unit circle $\{(r,\theta ):\left|r\right|\le 1\}$ to the inside of the square $\{(i,j):i,j=\mathrm{0,1},\dots ,N-1\}$ as follows:$$\begin{array}{}\text{(11)}& Z_{pq}=\frac{p+\mathrm{1}}{N-\mathrm{1}}{\displaystyle \sum }_{j=\mathrm{0}}^{N-\mathrm{1}}{\displaystyle \sum }_{i=\mathrm{0}}^{N-\mathrm{1}}{R}_{pq}\left(r_{ij}\right)e^{-jq\theta }f\left(i,j\right)\end{array}$$with$$\begin{array}{c}\text{(12)}& r_{ij}=\sqrt[\mathrm{2}]{{x_i}^{\mathrm{2}}+{y_i}^{\mathrm{2}}};x_i=\frac{2i}{N-1}-1,\\ y_i=\frac{2j}{N-1}-1,\\ {\theta }_{ij}={\tan }^{-1}\left(\frac{y_j}{x_i}\right)\end{array}$$We extract face features as various Zernike moments (which we call here Zernike domain) and then define a similarity measure after imposing a distance measure in this domain. In this work we will consider Euclidean and Minkowski distance metrics. Features of an image $\mathbf{x}$ can be represented by a vector of selected Zernike moments, ${\mathbf{Z}}_{\mathbf{x}}$. These distance measures are applied to feature vectors ${\mathbf{Z}}_{\mathbf{x}}$, ${\mathbf{Z}}_{\mathbf{y}}$ of two images in the Zernike domain. They are defined as follows.

3.4.1. Euclidean Metric

$$\begin{array}{}\text{(13)}& d_E\left(X,Y\right)=\sqrt[\mathrm{2}]{{\displaystyle \sum }_{i=\mathrm{1}}^n{\left|x_i-y_i\right|}^{\mathrm{2}}}\end{array}$$

3.4.2. Minkowski Metric

$$\begin{array}{}\text{(14)}& d_M\left(X,Y\right)=\sqrt[t]{{\displaystyle \sum }_{i=\mathrm{1}}^n{\left|x_i-y_i\right|}^t};\hspace{1em}t\ge \mathrm{1}\end{array}$$

In this work we will use $t=3$ for Minkowski metric. When these two metrics are applied to test similarity between the Zernike-domain image features ${\mathbf{Z}}_{\mathbf{x}}$, ${\mathbf{Z}}_{\mathbf{y}}$ (for two images x and y), we call the two Zernike-based similarity measures Zernike-Euclidean Similarity (ZESIM) and Zernike-Minkowski Similarity (ZMSIM). From another viewpoint, we establish a comparison with an efficient, rotation-invariant method for face recognition based on Euclidean and Minkowski distance in the Zernike domain. We selected Zernike feature vector as$$\begin{array}{}\text{(15)}& \mathbf{Z}=\left[\left|Z_{\mathrm{0,0}}\right|,\left|Z_{\mathrm{1,1}}\right|,\left|Z_{\mathrm{2,0}}\right|,\left|Z_{\mathrm{2,2}}\right|,\left|Z_{\mathrm{3,1}}\right|,\left|Z_{\mathrm{3,3}}\right|,\left|Z_{\mathrm{4,0}}\right|,\left|Z_{\mathrm{4,2}}\right|,\left|Z_{\mathrm{4,4}}\right|,\left|Z_{\mathrm{5,1}}\right|,\left|Z_{\mathrm{5,3}}\right|,\left|Z_{\mathrm{5,5}}\right|,\cdots \left|Z_{\mathrm{6,0}}\right|,\left|Z_{\mathrm{6,2}}\right|,\left|Z_{\mathrm{6,4}}\right|,\left|Z_{\mathrm{6,6}}\right|,\left|Z_{\mathrm{7,1}}\right|,\left|Z_{\mathrm{7,3}}\right|,\left|Z_{\mathrm{7,5}}\right|,\left|Z_{\mathrm{7,7}}\right|,\left|Z_{\mathrm{8,0}}\right|,\left|Z_{\mathrm{8,2}}\right|,\left|Z_{\mathrm{8,4}}\right|,\left|Z_{\mathrm{8,6}}\right|,\left|Z_{\mathrm{8,8}}\right|,\cdots \left|Z_{\mathrm{9,1}}\right|,\left|Z_{\mathrm{9,3}}\right|,\left|Z_{\mathrm{9,5}}\right|,\left|Z_{\mathrm{9,7}}\right|,\left|Z_{\mathrm{9,9}}\right|,\left|Z_{\mathrm{10,0}}\right|,\left|Z_{\mathrm{10,2}}\right|,\left|Z_{\mathrm{10,4}}\right|,\left|Z_{\mathrm{10,6}}\right|,\left|Z_{\mathrm{10,8}}\right|,\left|Z_{\mathrm{10,10}}\right|\right].\end{array}$$Also, we used $t=3$ for Minkowski metric. Comparison showed that the proposed measures outperform Zernike-Euclidean (ZESIM) and Zernike-Minkowski (ZMSIM) recognition approaches. This is so despite the fact that Zernike measures are so powerful that they apply to face expression recognition as well as face recognition.

3.5. ISSIM: A Functionally Relative 2D Histogram-Based Similarity Measure

In [21] an efficient information-theoretic measure (called ISSIM) has been proposed. This measure used a functionally normalized error function based on 2D joint histogram between the two images $x$, $y$ under testing as follows:$$\begin{array}{}\text{(16)}& E\left(x,y\right)=\sqrt{\frac{{\sum }_i^{}{\sum }_j^{}{\left[\left(H_{ij}-H_{ji}\right)\mathrm{1}/\left(h_i+c\right)\right]}^{\mathrm{2}}}{\mathrm{2}{L}^{\mathrm{2}}}}\end{array}$$where $H_{ij}$ and $H_{ji}$ represent elements of the joint histogram between two images, $c$ is a small positive number to avoid division by zero. Normalization has been done relative to the function $h_i$ (the histogram of the reference image) and the maximum pixel value $L=255$. Other (scalar) normalization steps have been added to ensure that the proposed measure stays inside $[\mathrm{0,1}]$.

3.6. The Proposed Measures

Researchers have proposed several similarity and recognition metrics used in image-processing field; each has its weaknesses and strengths. The most disturbing problem in image similarity for face recognition is the confusing high similarity given by a specific measure between the reference image and other images in the database.

In this paper, we propose novel information-theoretic similarity-recognition measures for image similarity and face recognition. The proposed measures reduce confusion when used in face recognition by giving a very small similarity between unrelated images. Information theory has already been applied to pattern recognition [22]; here we apply it to design two similarity-recognition measures that are useful for face recognition and image similarity. The two measures apply the concept of entropy (Shannon & Renyi) to image a joint histogram as a probabilistic distribution. The names Renyi Similarity Measure (RSM) and Shannon Similarity Measure (SHS) are given to the new measures, according to the use of Renyi and Shannon entropies. Performance tests have been applied against the popular metrics SSIM, FSIM, ZMSIM, ZESIM, and FSM. Additional tests also include comparisons with the information-theoretic ISSIM.

3.6.1. Shannon and Renyi Entropies

Entropy is the expected value of the information. Entropy has several applications in statistical mechanics, coding theory, statistics, and related areas. Emerging fields have also used entropy, such as image similarity [23]. The most significant entropy in applications is Shannon entropy, whose mathematical formula is given by$$\begin{array}{}\text{(17)}& S\left(x\right)=-{\displaystyle \sum }_{i=\mathrm{1}}^{n}p\left(x_i\right){\log }_{\mathrm{2}}\left[p\left(x_i\right)\right],\end{array}$$where $S$ represents the entropy, $x$ is discrete random variable $x=\{x_i,x_2,\dots ,x_n\}$, and $p\left(x_i\right)$ is probability of event $x_i$, $p\in \left[\mathrm{0,1}\right].$ Here the probabilistic events are the elements of the 2D joint histogram between two images (test and reference images).

Renyi entropy is another significant measure of information, given by$$\begin{array}{}\text{(18)}& R_{\alpha }\left(x\right)=\frac{\mathrm{1}}{\mathrm{1}-\alpha }{\log }_{\mathrm{2}}\left({{\displaystyle \sum }}_{i=\mathrm{1}}^n{p}_i^{\alpha }\right),\end{array}$$where $\alpha \ge 0$, $\alpha \ne 1$, $x$ is a discrete random variable, and $p$ is corresponding probabilities for $i=1,\dots ,n$. This entropy is a mathematical generalization of Shannon entropy.

The main difference between Shannon entropy and Renyi entropy is the placement of the logarithm in the entropic equations, giving a flexible measure of the entropy as a result of the parameter $\alpha$, enabling several measurements of dissimilarity [24]. This entropy, if applied to a joint histogram, gives high performance for face recognition.

3.6.2. A Joint Entropic-Histogram Similarity and Recognition Measure

In a huge database for digital images like a face database, there might be identical histograms for very different images. This fact will be a problem when researchers want to compare images using a histogram as a distinctive feature. To solve this problem, Pass et al. [25] proposed an alternative to the classical histogram, called a joint histogram, which includes additional information without losing the powerful feature of the histogram. The joint histogram is based mainly on selecting a set of local pixel features to construct a multidimensional histogram.

A 2D joint histogram entry $H_{ij}(x,y)$ for two images $x$ and $y$ represents the probability that a pixel intensity value $i$ from image $x$ cooccurs with pixel intensity value $j$ from image $y$. The normalized joint histogram for two images $\mathbf{x}$ and $\mathbf{y}$ of size $M\times N$ is defined here as follows:$$\begin{array}{}\text{(19)}& H\left(\mathbf{x},\mathbf{y}\right)=\left[H_{ij}\right],\end{array}$$where$$\begin{array}{}\text{(20)}& H_{ij}=\frac{\text{Number\hspace{0.17em}\hspace{0.17em}of\hspace{0.17em}\hspace{0.17em}joint\hspace{0.17em}\hspace{0.17em}events\hspace{0.17em}\hspace{0.17em}}\left(\mathbf{x}=i,\mathbf{y}=j\right)}{M\times N},\end{array}$$or$$\begin{array}{}\text{(21)}& H_{ij}=\frac{\left|\left\{x=i\right\}\cap \left\{y=j\right\}\right|}{M\times N}.\end{array}$$

Now we apply the entropy to measure the information held in the joint histogram that represents the joint probability of pixel cooccurrence. Note that both $i$ and $j$ range from $0$ to $L=255$. First, the Shannon entropy measure is applied to get Shannon-Histogram Similarity Measure (SHS) as follows:$$\begin{array}{}\text{(22)}& {\rho }_{SHS}\left(x,y\right)=-{\displaystyle \sum }_{k=\mathrm{1}}^{M\times N}T\left(k\right)·{\log }_{\mathrm{2}}\left[T\left(k\right)\right],\end{array}$$where $T=H(:)$ reshapes the 2D joint histogram $H$ into a one-dimensional column vector $T$ via the colon operator, as defined in MATLAB, with a new dimension $1\times (M·N)$. Applying the Renyi entropy in this approach gives the Renyi-Histogram Similarity Measure (RSM) as follows:$$\begin{array}{}\text{(23)}& {\rho }_{RSM}\left(x,y\right)=\frac{\mathrm{1}}{\mathrm{1}-\alpha }{\log }_{\mathrm{2}}\left({\displaystyle \sum }_{k=\mathrm{1}}^{M\times N}{\left[T\left(k\right)\right]}^{\alpha }\right),\end{array}$$where $\alpha \ge 0$; $\alpha \ne 1$. Using other entropies could be more helpful. However, this is beyond the scope of this paper at the moment and will be investigated in future works.

3.6.3. Motivation

One of the most difficult challenges for researchers in measuring image similarity for face recognition is that there is a high level of scepticism about the similarity between the reference image and test image in the same database, particularly when the image has low resolution or distortion in terms of illumination or background changes.

The differences in facial expressions and head poses for human faces often give rise to scepticism. Official government security systems do not rely entirely on face recognition systems, because the latter still suffer from challenges such as different facial expressions, illumination, and changing shape with age. However, a face recognition system can be very supportive of current routine security systems.

In this work, we have contributed to reducing these challenges regarding similarity of images, especially for the purpose of face recognition. We proposed new image similarity measures that can be utilized in face recognition. These measures are built using an information-theory approach; they proved to be very accurate in finding similarity between face images with more confidence than existing images similarity and image recognition measures. Our method is motivated by the problem of finding image similarity in large databases, where reduced confidence may open the door for big confusion.

The aim of this work is to provide metrics to find similarity between images for the purpose of face recognition; also, this can be used in case of nonface images. High performance and accuracy are the main features of proposed measures as compared to existing measures. Although other measures may have the ability to find the similarity between images (even for face recognition), the proposed measures have high confidence by giving almost a near-zero value in case of different images, while other measures give a nontrivial amount of similarity when comparing different images.

4. Experimental Results and Performance

We have implemented the proposed measures on MATLAB and tested their performance against other measures as follows.

4.1. Test Environment: Image Databases

In this work, we used well-known face databases, AT & T and FEI [26, 27], as a test environment. AT & T database as shown in Figure 1 has 40 persons each, with 10 different poses (including facial expressions); hence the total number of AT & T face images used in this test is 400 images. FEI database, as shown in Figure 2, has 200 persons, each with 14 different poses (including facial expressions), and the total number of FEI face images used in this test is 700 images as part of it.

[figure omitted; refer to PDF] [figure omitted; refer to PDF]

We divided the AT & T and FEI databases into two subgroups: testing group and the training group. In training group, we choose a random face image from the database to be a reference image, and then we select a different facial expression and pose from the testing group, for the same person as a challenging image to test the performance of measures in recognition and similarity.

On the other hand, there are several publicly available image databases in the image similarity community, including TID2008 and image and video-communication (IVC). Both are used here for algorithm validation and comparison. TID2008, as shown in Figure 3, contains 25 reference images and 1,700 distorted images (25 reference images × 17 types of distortions x 4 levels of distortions) [28]. The IVC database as shown in Figure 4 has 10 original images and 235 distorted images generated from four different processes: JPEG, JPEG2000, LAR coding, and blurring [29].

[figure omitted; refer to PDF] [figure omitted; refer to PDF]

For each reference image in the TID2008 and IVC databases, we use six complex distorted versions as image poses to test, compare, and prove that the proposed SHS & RSM outperforms the existing measures in terms of a recognition and similarity tests.

Note that although we obtained good results using this standard database, better results could be obtained using the Viola-Jones face detection algorithm [30], local analysis [31, 32], or hybrid analysis [33]. Note using face detection algorithm with giving emphasis to the relevant face features while ignoring artefacts.

4.2. Performance Criterion

Performance of the proposed measures has been tested against other efficient similarity and recognition metrics: SSIM, FSIM, FSM, ZESIM, and ZMSIM. The criterion for good performance is the amount of confusion in deciding whether an image belongs to a database or not. This confusion is measured by the difference in similarity produced (by a specific measure) between the reference image and the database images, with a focus on the best match and the second-best match. If a measure gives little difference in similarity between the same persons, then the confusion is high and the performance is low.

4.3. Results and Discussion

To evaluate the performance of the proposed SHS and RSM against SSIM, FSIM, ZSIM, ZMSIM, and the state-of-the-art FSM, we have to describe the experimental procedure in detail.

In this paper, we use four challenging datasets which are AT & T, FEI, IVC, and TID2008. AT & T and FEI are used to test the performance of all the measures in terms of face recognition, and TID2008 and IVC are used to test the performance of all the measures in terms of image similarity in the figures listed below.

Figure 5 refers to the test image, person number 17 in AT & T database with all poses, and note that we chose pose 10 as a reference image. Figure 6 shows the result of applying the proposed measures (SHS & RSM) and existing measures for the sake of recognizing a specific person recognized as an indicated in Figure 5, and here the similarity differences between best and second-best match are $d_{RSM}=0.9579$, $d_{SHS}=0.9374$, $d_{SSIM}=0.6595$, $d_{FSIM}=0.2869$, $d_{ZESIM}=0.5568$, $d_{ZMSIM}=0.4942,\hspace{0.17em}\hspace{0.17em}{d}_{FSM}=0.7662$.

[figure omitted; refer to PDF] [figure omitted; refer to PDF]

Figure 7 refers to the test image, person number 17 in AT & T database with all poses, and note that we chose pose 3 as a reference image. Figure 8 shows the result of applying the proposed measures and existing measures for the sake of recognizing a specific person that we chose in Figure 7, and here the similarity differences between best and second-best match are $d_{RSM}=0.9554$, $d_{SHS}=0.9355$, $d_{SSIM}=0.5860$, $d_{FSIM}=0.2549$, $d_{ZESIM}=0.4124$, $d_{ZMSIM}=0.3260$, and $d_{FSM}=0.7513$.

[figure omitted; refer to PDF] [figure omitted; refer to PDF]

Note that the measures have also been tested using different facial expressions with different illumination and different head pose in the same databases. Such cases represent the current challenges of any face recognition or image similarity measure. Results show more confidence in our proposed measurement.

In Figure 9 we chose person number 20 in AT & T database as a test image and we chose pose number 10 as a reference image. Figure 10 shows the result of applying SHS and RSM measures against SSIM, FSIM, ZSIM, and ZMSIM and the recent FSM measures for the specific person recognition as indicated in Figure 9, and here the similarity differences between best and second-best match are $d_{RSM}=0.957$, $d_{SHS}=0.9265$; and $d_{SSIM}=0.7056$, $d_{FSIM}=0.0580$, $d_{ZESIM}=0.4328$, $d_{ZMSIM}=0.3339$, and $d_{FSM}=0.7513$.

[figure omitted; refer to PDF] [figure omitted; refer to PDF]

FEI database is used which represents the most face recognition challenges because it is containing a different facial expression with different illumination (white homogenous background) and different head pose (about 180 degrees).

Figure 11 refers to the test image, person number 17 in FEI database with all poses; note that we chose pose number 8 as a reference image. Figure 12 shows the result of applying the proposed measures and existing measures for the sake of recognizing a specific person that we chose in Figure 11, and here the similarity differences between best and second-best match are $d_{RSM}=0.9138$, $d_{SHS}=0.8036$, $d_{SSIM}=0.1975$, $d_{FSIM}=0.1865$, $d_{ZESIM}=0.2976$, $d_{ZMSIM}=0.2899,\hspace{0.17em}\hspace{0.17em}{d}_{FSM}=0.6280$.

[figure omitted; refer to PDF] [figure omitted; refer to PDF]

Figure 13 refers to person number 28 with all poses in FEI database as a test image; note that we chose pose number 6 as a reference image while Figure 14 shows the result of applying the proposed SHS and RSM against existing measures to recognizing a specific person that we chose in Figure 13, and here the similarity differences between best and second-best match are $d_{RSM}=0.9437$, $d_{SHS}=0.8686$; and $d_{SSIM}=0.3085$, $d_{FSIM}=0.2508$, $d_{ZESIM}=0.6784$, $d_{ZMSIM}=0.6503,\hspace{0.17em}\hspace{0.17em}{d}_{FSM}=0.7334$.

[figure omitted; refer to PDF] [figure omitted; refer to PDF]

As face recognition can be pose-dependent, we did averaging of similarity confidence measure for every pose in the AT & T dataset. The global average can be obtained as the mean of all these subaverages. Let $s_{ik}$ denote the similarity confidence when the image of person $i$ with pose $k$ is the reference image while recognizing person $i$ between 40 people under pose $k$. Then the global confidence average is taken as $s_{av}=\left(1/400\right){\sum }_{k=1}^{10}{\sum }_{i=1}^{40}{s}_{ik}$. Table 1 shows the performance of the proposed SHS & RSM versus other methods. Note that the average similarity difference of other databases gave nearly similar results. It is clear that the only near match to the proposed measures is the recently proposed FSM.

Table 1

The global average similarity difference of best match and second-best match within all persons.

Measures	RSM	SHS	FSM	SSIM	ZESIM	ZMSIM	FSIM
Average Confidence, $s_{av}$	0.0239	0.0234	0.0192	0.0150	0.0119	0.0100	0.0066

The preparation of the database that is more suitable for this approach (e.g., in security applications) should take into consideration some important factors like lighting, expression, and viewpoint, while the reference image should consider the same factors.

It is clear that the proposed joint entropic-histogram measures give more confident decisions in face recognition and image similarity, whereas other measures, although they decide the proper person correctly, give low confidence in their decision.

Using a database with distorted images in the test of image similarity and image recognition measures is a real challenge to the proposed and existing measures. In this work, we tested the SHS & RSM on distorted images for the sake of image similarity and image recognition. The figures listed below show that the proposed methods are still superior versus others.

Figures 15, 16, 17, and 18 have three images: (a) is the original reference image from databases, (b) is the distorted version of the reference image, and (c) represents the performance of our proposed image similarity-recognition measures compared with the existing measures. The proposed RSM & SHS demonstrates better performance in terms of recognition and similarity confidence. Although the other measures correctly decide the proper image with maximum similarity, they give low confidence in their decision because there are many cases of distrust (big similarities with wrong images) in their decisions (similarities). This is a big challenge when we employ these measures in security recognition tasks. SHS and RSM give more confidence to decide the proper image from a database.

[figures omitted; refer to PDF]

[figures omitted; refer to PDF]

[figures omitted; refer to PDF]

[figures omitted; refer to PDF]

The difference in the values of the peaks of each measure is a new feature showing the high performance of the proposed measures (SHS & RSM). If the distance between the highest match and the second-best match is higher, that means the measure has better performance and vice versa; i.e., if the distance is less, that means the measure has been confused in deciding the best match by giving a nontrivial similarity between the different images. The new feature of recognition confidence can be very useful in security systems of big databases.

Receiver Operating Characteristics (ROC). ROC graph is used for performance evaluation of classifiers; it is a two-dimensional graph in which true positive rate (tpr, also called hit rate and recall) is plotted versus false positive rate (fpr, also called false alarm rate), defined as follows [34]: $$\begin{array}{}\text{(24)}& tpr=\frac{\text{Positives\hspace{0.17em}\hspace{0.17em}correctly\hspace{0.17em}\hspace{0.17em}classified}}{\text{Total\hspace{0.17em}\hspace{0.17em}positives}}\text{(25)}& fpr=\frac{\text{Negatives\hspace{0.17em}\hspace{0.17em}incorrectly\hspace{0.17em}\hspace{0.17em}classified}}{\text{Total\hspace{0.17em}\hspace{0.17em}negatives}}\end{array}$$

An ROC graph essentially shows the relationship between advantages (true positives) and disadvantages of the classifier (false positives). Tables 2 and 3 show FPR and TPR using AT & T database (40 persons), while Figure 19 shows an ROC graph with 8 classifiers (similarity measures), including the recent information-theoretic ISSIM [33]. The difference $d$ between the best match and the second-best match is used as a confidence measure in this experiment used to confirm that a face image belongs to the database. Thresholds of confidence are used as given by in the following vector: $T=\left[0.1\hspace{0.5em}.3\hspace{0.5em}.5\hspace{0.5em}.7\hspace{0.5em}.8\hspace{0.5em}.9\hspace{0.5em}.95\right].$ To confirm that a face image does not belong to the database, the measure $1-d$ is used, with the same thresholds as above. Note that when all images are not related to a specific test image (i.e., the image does not belong to the database), we expect low values for $d$ since only similarity features (including correlative, structural, and information-theoretic features) can push $d$ up.

Table 2

True positive rate (tpr) according to the threshold vector of confidence using AT & T database, with pose 10 (of each of the forty persons) as a reference image (included in the database).

Thr	SSIM	FSIM	ZESIM	ZMSIM	ISSIM	FSM	SHS	RSM
0.1	1	1	1	1	1	1	1	1
0.3	1	0.275	1	0.875	0.975	1	1	1
0.5	1	0	0.275	0.075	0.05	1	1	1
0.7	0.15	0	0	0	0	1	1	1
0.8	0	0	0	0	0	1	1	1
0.9	0	0	0	0	0	0.075	1	1
0.95	0	0	0	0	0	0	0.275	0.85

Table 3

False positive rate (fpr) according to the threshold vector of confidence using AT & T database, with pose 10 (of each of the forty persons) as a reference image (excluded from the database).

Thr	SSIM	FSIM	ZESIM	ZMSIM	ISSIM	FSM	SHS	RSM
0.1	0	0	0	0	0	0	0	0
0.3	0	0	0	0	0	0	0	0
0.5	0	0	0	0	0	0	0	0
0.7	0	0	0	0	0	0.05	0	0
0.8	0	0	0	0	0	0.15	0	0
0.9	0.3	0	0.875	0	0	0.3	0	0
0.95	0.5	0.025	1	1	0.1	0.6	1	1

[figure omitted; refer to PDF]

5. Conclusions

This paper presented an efficient approach for face recognition and image similarity. The approach is based on an information-theoretic similarity measure derived using the entropy of a 1D version of the 2D joint histogram between two images. Two entropies have been used, Shannon and Renyi, giving rise to two measures: Shannon-Histogram Similarity (SHS) and Renyi Similarity Measure (RSM). The performance of RSM and SHS was tested against efficient existing similarity metrics feature-based similarity (FSIM), structural similarity (SSIM), and also Zernike-moments recognition approaches, specifically Zernike-Euclidean Similarity (ZESIM) and Zernike-Minkowski Similarity (ZMSIM) and the state-of-the-art FSM. A comparison with a recent information-theoretic ISSIM has also been considered. Experimental results showed superior performance for the proposed measures in terms of correct decisions with minimal confusion in face recognition and image similarity, using the AT & T and FEI face databases and TID2008, and IVC image databases. Confusion in recognition is introduced as a performance factor, measured as the difference between the similarity produced by the best match and that produced by the second-best match.

In this work, global face analysis has been applied, where the whole image is treated at once. Although good results were obtained using a standard database, difficulties may arise in practice. The Viola-Jones face detection algorithm and local analysis of face images played a significant role in improving face recognition. The authors intend to pursue this point in future works and extend their previous studies on local analysis to improve the performance of the measures defined above.

Ethical Approval

The Authors declare that there are no ethical issues regarding this work.

Conflicts of Interest

The authors declare no conflicts of interest regarding this work.

Authors’ Contributions

All authors extensively discussed the contents of this paper and contributed to its preparation. Mohammed Abdulameer Aljanabi and Zahir M. Hussain have proposed and developed the model, performed experiments, and drafted the manuscript. Results analysis, mathematics check-up, and simulation revision of this manuscript were done by Songfeng Lu. All of the authors have contributed to the literature overview and modelling discussions.

Acknowledgments

This work is supported by the Natural Science Foundation of Hubei Province of China under Grant no. 2016CFB541 and the Applied Basic Research Program of Wuhan Science and Technology Bureau of China under Grant no. 2016010101010003 and the Science and Technology Program of Shenzhen of China under Grant nos. JCYJ20170307160458368 and JCYJCYJ20170818160208570.

References

[1] L. Best-Rowden, A. K. Jain, "Longitudinal Study of Automatic Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40 no. 1, pp. 148-162, DOI: 10.1109/TPAMI.2017.2652466, 2018.

[2] The Economist, Ever better and cheaper, face-recognition technology is spreading, 2017.

[3] M. A. Aljanabi, N. A. Shnain, S. F. Lu, "An image similarity measure based on joint histogram — Entropy for face recognition," Proceedings of the 3rd IEEE International Conference on Computer and Communications (ICCC '17), pp. 1626-1631, DOI: 10.1109/CompComm.2017.8322815, .

[4] N. A. Shnain, S. F. Lu, Z. M. Hussain, "HOS image similarity measure for human face recognition," Proceedings of the 3rd IEEE International Conference on Computer and Communications (ICCC '17), pp. 1621-1625, DOI: 10.1109/CompComm.2017.8322814, .

[5] E. Chalom, E. Asa, E. Biton, "Measuring image similarity: An overview of some useful applications," IEEE Instrumentation & Measurement Magazine, vol. 16 no. 1, pp. 24-28, 2013.

[6] A. F. Hassan, D. Cai-lin, Z. M. Hussain, "An information-theoretic image quality measure: Comparison with statistical similarity," Journal of Computer Science, vol. 10 no. 11, pp. 2269-2283, DOI: 10.3844/jcssp.2014.2269.2283, 2014.

[7] Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13 no. 4, pp. 600-612, DOI: 10.1109/TIP.2003.819861, 2004.

[8] L. Zhang, L. Zhang, X. Mou, D. Zhang, "F{SIM}: a feature similarity index for image quality assessment," IEEE Transactions on Image Processing, vol. 20 no. 8, pp. 2378-2386, DOI: 10.1109/TIP.2011.2109730, 2011.

[9] A. Kar, D. Bhattacharjee, D. K. Basu, M. Nasipuri, M. Kundu, "A Face Recognition Approach Based on Entropy Estimate of the Nonlinear DCT Features in the Logarithm Domain Together with Kernel Entropy Component Analysis," International Journal of Information Technology and Computer Science, vol. 5 no. 9, pp. 31-42, DOI: 10.5815/ijitcs.2013.09.03, 2013.

[10] M. Darshana, A. Banerjee, A. Tatu, "Image similarity based on joint entropy (joint histogram)," Proceedings of the Int. Conf. on Advances in Engineering and Technology, .

[11] J. Lee, T. Yun, Fuzzy and entropy facial recognition, 2014.

[12] Y. Wang, Y. Y. Tang, L. Li, "Robust Face Recognition via Minimum Error Entropy-Based Atomic Representation," IEEE Transactions on Image Processing, vol. 24 no. 12, pp. 5868-5878, DOI: 10.1109/TIP.2015.2492819, 2015.

[13] Y. Garcés, E. Torres, O. Pereira, R. Rodríguez, "New similarity index for images based on entropy and group theory," IEEE Latin America Transactions, vol. 13 no. 5, pp. 1584-1592, DOI: 10.1109/TLA.2015.7112019, 2015.

[14] Q. R. Zhang, J. Y. Zhang, Y. Wang, J. Kao, W. Sung, "The improved relative entropy for face recognition," Matec Web of Conferences, vol. 63,DOI: 10.1051/matecconf/20166304006, 2016.

[15] Y.-D. Zhang, Z.-J. Yang, H.-M. Lu, X.-X. Zhou, P. Phillips, Q.-M. Liu, S.-H. Wang, "Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation," IEEE Access, vol. 4, pp. 8375-8385, DOI: 10.1109/ACCESS.2016.2628407, 2016.

[16] X. Ruan, S. Wang, "Face recognition based on weighted multi-resolution kernel entropy component analysis," Proceedings of the 29th Chinese Control and Decision Conference, CCDC '17, pp. 6118-6123, .

[17] D. Sandić-Stanković, D. Kukolj, P. Le Callet, "DIBR synthesized image quality assessment based on morphological wavelets," Proceedings of the 7th International Workshop on Quality of Multimedia Experience, QoMEX '15, .

[18] N. A. Shnain, Z. M. Hussain, S. F. Lu, "A feature-based structural measure: an image similarity measure for face recognition," Applied Sciences (Switzerland), vol. 7 no. 8, 2017.

[19] S. M. Lajevardi, Z. M. Hussain, "Zernike moments for facial expression recognition," Proceedings of the IEEE International Conference on Communication, Computer and Power (ICCCP'09), .

[20] S.-K. Hwang, W.-Y. Kim, "A novel approach to the fast computation of Zernike moments," Pattern Recognition, vol. 39 no. 11, pp. 2065-2076, DOI: 10.1016/j.patcog.2006.03.004, 2006.

[21] F. Hassan Asmhan, M. Hussain Zahir, Dong. Cailin, "An Information-Theoretic Measure for Face Recognition: Comparison with Structural Similarity," (IJARAI) International Journal of Advanced Research in Artificial Intelligence, vol. 3 no. 11, 2014.

[22] Y. S. Al-Halabi, "Information Theory and its Application to Pattern Recognition in Image Processing," International Journal of Engineering Inventions, 2012.

[23] L. Golshani, E. Pasha, G. Yari, "Some properties of Rényi entropy and Rényi entropy rate," Information Sciences, vol. 179 no. 14, pp. 2426-2433, DOI: 10.1016/j.ins.2009.03.002, 2009.

[24] D. Xu, D. Erdogmuns, "Renyis entropy, divergence and their nonparametric estimators," Information Theoretic, 2010.

[25] G. Pass, R. Zabih, "Comparing images using joint histograms," Multimedia Systems, vol. 7 no. 3, pp. 234-240, DOI: 10.1007/s005300050125, 1999.

[26] "Laboratories, A.T. The Database of Faces," .

[27] "FEI Face Database," .

[28] A. Ninassi, P. Le Callet, F. Autrusseau, .

[29] N. Ponomarenko, V. Lukin, A. Zelensky, K. Egiazarian, M. Carli, F. Battisti, "TID2008A database for evaluation of full-reference visual quality assessment metrics," Adv. Modern Radioelectron, vol. 10, pp. 30-45, 2009.

[30] P. Viola, M. Jones, "Rapid object detection using a boosted cascade of simple features," Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. I511-I518, .

[31] S. M. Lajevardi, Z. M. Hussain, "Novel higher-order local autocorrelation-like feature extraction methodology for facial expression recognition," IET Image Processing, vol. 4 no. 2, pp. 114-119, DOI: 10.1049/iet-ipr.2009.0100, 2010.

[32] S. M. Lajevardi, Z. M. Hussain, "Facial expression recognition: Gabor filters versus higher-order correlators," Proceedings of the IEEE International Conference on Communication, Computer and Power (ICCCP'09), .

[33] S. M. Lajevardi, Z. M. Hussain, "Feature extraction for facial expression recognition based on hybrid face regions," Advances in Electrical and Computer Engineering, vol. 9 no. 3, pp. 63-67, DOI: 10.4316/aece.2009.03012, 2009.

[34] T. Fawcett, "An introduction to ROC analysis," Pattern Recognition Letters, vol. 27 no. 8, pp. 861-874, DOI: 10.1016/j.patrec.2005.10.010, 2006.