Content Based Image Retrieval by Multi Features using Image Blocks

Abstract

Content based image retrieval (CBIR) is an effective method of retrieving images from large image resources. CBIR is a technique in which images are indexed by extracting their low level features like, color, texture, shape, and spatial location, etc. Effective and efficient feature extraction mechanisms are required, to improve existing CBIR performance. This article presents a novel approach of CBIR system in which higher retrieval efficiency is achieved by combining the information of image features color, shape and texture. The color feature is extracted, using color histogram for image blocks, for shape feature Canny edge detection algorithm is used, and the HSB extraction in blocks is used for texture feature extraction. The experiments show that, the fusion of multiple features retrieval gives better retrieval results than another approach used by Rao et al. This article presents comparative study of performance of the two different approaches of CBIR system, in which the image features color, shape and texture are used.

Headnote

Abstract

Content based image retrieval (CBIR) is an effective method of retrieving images from large image resources. CBIR is a technique in which images are indexed by extracting their low level features like, color, texture, shape, and spatial location, etc. Effective and efficient feature extraction mechanisms are required to improve existing CBIR performance. This paper presents a novel approach of CBIR system in which higher retrieval efficiency is achieved by combining the information of image features color, shape and texture. The color feature is extracted using color histogram for image blocks, for shape feature Canny edge detection algorithm is used and the HSB extraction in blocks is used for texture feature extraction. The feature set of the query image are compared with the feature set of each image in the database. The experiments show that the fusion of multiple features retrieval gives better retrieval results than another approach used by Rao et al. This paper presents comparative study of performance of the two different approaches of CBIR system in which the image features color, shape and texture are used.

Keywords

Content Based Image Retrieval, color histogram, Canny edge detection, Euclidian distance, HSV, HSB, texture, shape.

(ProQuest: ... denotes formulae omitted.)

1. Introduction

Advances in image acquisition and data storage technologies have facilitated the creation of large image data sets. Content based image retrieval is an appropriate information system to efficiently manage these image collections. In CBIR images low level features of images like color, shape, texture etc are used to index images [1, 2, 3].

A typical content-based image retrieval system can be described by the figure 1 here the visual contents of the images in the database are described and extracted by multi-dimensional feature vectors. This builds a feature database from the feature vectors. To retrieve images, users provide the retrieval system with query image. Feature vectors are then built. The distances between the feature vectors of the query image and images of database are computed for similarity. In this paper we have used the features color, shape and texture and the relevant retrieval experiments show that the multi features retrieval brings better visual feeling than the single feature retrieval, which means better retrieval results. The results of our experiments are the compared with that of another approach. This Comparison clearly shows that our approach gives better performance over the approach used by Rao et al in their research.

2. Typical CBIR System

CBIR is a good alternative to traditional text-based image searching or an image retrieval system and can greatly improve the accuracy of the information being returned. CBIR aims to develop an efficient content-based technique to search and retrieve relevant images from heavy digital image collections. Most proposed CBIR techniques are based on keywords, text-phrase, image, graphics, and even combination of all. Our CBIR system is based on multiple features i.e. color, shape and texture. The CBIR system is divided into two stages. First, pre-processing in which the image is first processed in order to extract the features. Second is feature extraction where features such as shape, texture and color are used to describe the content of the image. The CBIR technique automatically extracts low-level features to measure the similarities between database images and the query image by comparing the feature differences.

The overall structure of a typical image retrieval system is shown in figure 1. It consists of images, feature extraction engine, feature database and retrieval process. One of the images from the image collection is considered as query image. The feature extraction engine extracts the features of images in the database and the query image and obtains the feature vector. The search and retrieval process search the query image matched with images in the database and finally arrange the received images according to their matching score.

3. Proposed System

The main objective of the proposed system to provide an efficient tool for efficient image retrieval from a huge content of image database using features based on color, shape and texture. Then retrieve the images to identify the most similar images to the query image. The terms and methods used in our approach are explained below [4].

1. Color

Most widely used visual feature in image retrieval is color of the image. Surface texture, shading and viewing conditions alters the facade of the color of real world objects. There are many advantages of characterizing images using color feature. This makes the CBIR system robust, effective, implementation is not complicated and it requires low storage space [5, 15, 16]. In our system we are using histograms in blocks and respective block histograms are matched.

2. Shape

Shape is also an important and very powerful feature when used for indexing and retrieving images from image database. Histogram of edge detection is used for extracting the shape feature. In the proposed method we have used the Canny edge detection to get edge information of the image [6, 7, 16, 17, 19].

3. Texture

Texture features are intended to capture the granularity and repetitive patterns of surfaces within a picture. For example, cloth, grass, and flower petals differ in texture, by smoothness as well as patterns. The study of texture features have been done in image processing, computer vision, and computer graphics. In image processing, a popular way to form texture features is by using the coefficients of a certain transform on the original pixel values, or, more sophisticatedly, by statistics computed from these coefficients. Examples of texture features using the wavelet transform and the discrete cosine transform can be found in Do and Vetterli and Li et al. We have used HSB based extraction in blocks for texture feature extraction.

4. Similarity feature extraction

We have explained the different feature extraction methods. The similarity feature which is used for comparing the various features is the Euclidean Distance. To retrieve the similarity images from the large image dataset, three types of Distance Metric Measures like Euclidean Distance, Chi-Square Distance and Weighted Euclidean but in the proposed method Euclidean distance is used.

The formula of Euclidean distance is

...

The images are sorted according to the value of distance and the image with least distance matches most with the query image. Other metrics can also give good results in some cases. Before computing the distance between query image and the images in database each feature component should be normalized so as to neutralize the effect of squaring the distances.

4. Algorithm for CBIR using Multi Features

Step 1: Load the image database in the Mat lab workspace.

Step 2: Get the block size from user.

Step 3: For each image convert image from RGB to HSV.

Step 4: Generate the histogram of hue, saturation and value per block.

Step 5: Store it into the mat file.

Step 6: Load the Query image.

Step 7: Apply the procedure 3-5 to find histogram per block of Query image.

Step 8: Determine the distance of signature of Query image with stored signature of database for each block.

Step 9: To perform indexing sort the distance values.

Step 10: Apply Canny edge detection algorithm for extracting the shape feature of image database.

Step 11 : Apply Canny edge detection algorithm for extracting the shape feature of Query image.

Step 12: Compare the Query image with image database.

Step 13: Compute the HSB for image database per block for texture feature

Step 14: Compute the HSB for Query image per block

Step 15: Compare HSB per block of Query image and images in database if they are same images match in texture.

Step 16: Get the composite result of the features

Step 17: Create a folder in the name of corresponding variants.

Step 18: Create a file in the name of corresponding variants & save

5. Performance Measures

To establish the performance of a CBIR system level of retrieval accuracy achieved by a system is important. The performance of retrieval system can be measured in terms of its recall and precision. We have found some recent literature [9-12] use this pair to measure the retrieval performance.

Recall (R) = Number of Relevant Images retrieved / Total Number of Relevant Images=A/A+C

Precision (P) = Number of Relevant Images Retrieved / Total Number of Retrieved Images = A/A+B

Recall is defined by R and is defined as the ratio of the number of retrieved relevant images A to the total number A+C of relevant images in the whole database [8]. Where C is number of relevant items those were not retrieved. Precision P is defined as the ratio of the number of retrieved relevant images A to the total number of retrieved images A+B [8], B is the number of irrelevant items. The number of relevant items retrieved is the number of the returned images that are similar to the query image in this case. The number of relevant items in collection is the number of images that are in the same particular category with the query image. The total number of items retrieved is the number of images that are returned by the search engine.

Recall measure the ability of the system to retrieve all the models that are relevant, while precision measures the ability of the system to retrieve only models that are relevant. Recall measures the robustness of the retrieval and precision measures the accuracy of the retrieval. The above approach results in a high recall and precision of retrieval, and is effectively used in content-based image retrieval systems.

6. Results

The data set used is the Wang's [13] dataset comprising of 1000 Corel images with ground truth. There are 10 categories and there are 100 images per category in the dataset. The images are of the size 256 x 384 or 384X256. But the images with 384X256 are resized to 256X384. This is the same data set used by Rao et al.

Results of the proposed system are compared with that of one of the papers of Rao et al [14].They have also fused dominant color, shape and texture features for CBIR and the data set used is also the same Wang data set. They have obtained texture of an image by using Gray Level Co-occurrence Matrix (GLCM) and shape information is captured in terms of edge images computed using Gradient Vector Flow fields. Table 1 shows the comparision of proposed method and Rao et al in terms of CBIR by color, CBIR by color and texture and finally CBIR by fusing color, texture and shape.

7. Conclusion

The proposed system uses three image features namely color, shape and texture. Since these features can describe different properties of an image, the proposed method integrates these three features to retrieve the images. The experiments were carried out on the Wang's dataset comprising of 1000 Corel images. The image set comprises of 10 categories with 100 images per category. We evaluated the experimental results based on the performance measures: precision and recall. The experimental results demonstrated that the proposed approach yielded better retrieval effectiveness than the approach used by Rao et al. Comparision of precision of both methods showed that proposed system outperforms Rao et al methods.

8. Future Scope

It would be beneficial to use the three features color, shape and texture in many applications like searching a certain disease from a database of medical images. Therefore, an efficient combination of these features into multimodal descriptors of the audio-visual content should be considered in more detail in future work. Another issue is the development of a flexible and dynamically adjustable similarity measure based on relevance feedback obtained from the end-user should be taken into account. The issue of distance measures between feature vectors also need to be considered. Euclidean distance was used in this method because of its simplicity and interpretability, but it would be valuable to evaluate other distance measures and their effect on retrieval performance.

References

References

[1] G.T. Shrivakshan, Dr.C. Chandrasekar, "A Comparision of various Edge Detection Techniques used in Image Processing", IJCSI International Journal of Computer Science Issues, Vol. 9, Issue 5, No 1, September 2012.

[2] Ryszard.S.Chora"Image Feature Extraction Techniques and Their Applications for CBIR and Biometrics Systems" International Journal of Biology And Biomedical Engineering ,vol.l, no.7, pp. 6-16,2008

[3] Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z. Wang "Image Retrieval: Ideas, Influences, and Trends of the New Age" ACM Computing Surveys, vol. 40, No. 2, Article 5,2008

[4] P. V. N. Reddy, P. V. N. Reddy, ''Color and Texture Features for Content Based Image Retrieval", International Journal of Computer Technology Application, Volume 2 (4), 10161020, ISSN:2229-6093, July-August 2011

[5] Suhasini, P. S.; Krishna, K. Sri Rama; Krishna, I. V. Murali, "CBIR Using Color Histogram Processing", Journal of Theoretical & Applied Information Technology, 2009. Vol. 6 Issue 1, pi 16-122.

[6] George Gagaudakis, Paul L. Rosin, " Incorporating shape into histograms for CBIR", Elsevier, Pattem Recognition, Volume 35, Issue 1, January 2002, Pages 81-91

[7] G.T. Shrivakshan, Dr.C. Chandrasekar, "A Comparision of various Edge Detection Techniques used in Image Processing", IJCSI International Journal of Computer Science Issues, Vol. 9, Issue 5, No 1, September 2012.

[8] F. Long, H. J. Zhang, and D. D. Feng, "Fundamentals of Content-based Image Retrieval," in Multimedia Information Retrieval and Management, D. Feng Eds, Springer, 2003.

[9] L. Chen, G. Lu, and D. S. Zhang, "Effects of Different Gabor Filter Parameters on Image Retrieval by Texture," in Proc. of IEEE 10th International Conference on Multi-Media Modelling, Australia, 2004, pp. 273-278.

[10] J. R. Smith and S.-F. Chang, "VisualSEEk: A fully automated content-based image query system," in Proc. of ACM Multimedia, 1996.

[11] Y. Song, W. Wang, and A. Zhang, "Automatic Annotation and Retrieval of Images." vol. 6, no.2: World Wide Web: Internet and Web Information Systems, 2003, pp. 209-231.

[12] Z. Lu, S. Li, and H. Burkhardt, "A Content-Based Image Retrieval Scheme in JPEG Compressed Domain," International Journal of Innovative Computing, Information and Control, vol. 2(4), pp. 831-839, 2006.

[13] http://wang.ist.psu.edu/

[14] M.Babu Rao, Dr.B.Prabhakara Rao, Dr.A.Govardhan, " Content Based Image Retrieval Using Dominant Color, Texture and Shape", International Journal of Engineering Science and Technology (IJEST), Vol. 3 No. 4 Apr 2011, ISSN: 0975-5462.

[15] L. Chen, G. Lu, and D. S. Zhang, "Effects of Different Gabor Filter Parameters on Image Retrieval by Texture," in Proc. of IEEE 10th International Conference on Multi-Media Modelling, Australia, 2004, pp. 273-278.

[16] Y. Song, W. Wang, and A. Zhang, "Automatic Annotation and Retrieval of Images." vol. 6, no.2: World Wide Web: Internet and Web Information Systems, 2003, pp. 209-231.

[17] Z. Lu, S. Li, and H. Burkhardt, "A Content-Based Image Retrieval Scheme in JPEG Compressed Domain," International Journal of Innovative Computing, Information and Control, vol. 2(4), pp. 831-839, 2006.

[18] Ying Liu, Dengsheng Zhang, Guojun Lu, WeiYing Ma, "A survey of content based image retrieval with high level semantics",The Journal of the pattem recognition society, Pattem Recognition ,vol. 40, no. 9, pp. 262-282,2007.

[19] Raman Maini, Dr. Himanshu Aggarwal," Study and Comparision of Various Image Edge Detection Techniques", International Journal of Image Processing (IJIP), Volume (3) : Issue (1),2009.