Vision-Based Bicycle Detection Using Multiscale Block Local Binary Pattern

Abstract

Bicycle traffic has heavy proportion among all travel modes in some developing countries, which is crucial for urban traffic control and management as well as facility design. This paper proposes a real-time multiple bicycle detection algorithm based on video. At first, an effective feature called multiscale block local binary pattern (MBLBP) is extracted for representing the moving object, which is a well-classified feature to distinguish between bicycles and nonbicycles; then, a cascaded bicycle classifier trained by AdaBoost algorithm is proposed, which has a good computation efficiency. Finally, the method is tested with video sequence captured from the real-world traffic scenario. The bicycles in the test scenario are successfully detected.

(ProQuest: ... denotes non-US-ASCII text omitted.)

Hongyu Hu 1 and Pengfei Tao 2 and Zhenhai Gao 1 and Qingnian Wang 1 and Zhihui Li 2 and Zhaowei Qu 2

Academic Editor:Wuhong Wang

1, State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China
2, College of Transportation, Jilin University, Changchun 130022, China

Received 24 June 2014; Accepted 21 September 2014; 1 October 2014

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

In the past two decades, China's motorization was developed rapidly with the economic growth. The motor vehicle has become one of the most important travel tools. However, national economy and per capita GDP are still lower than most developed countries. In some underdeveloped cities in China, many people yet select bicycles as travel tools [1]. Motor vehicles and bicycles are sharing the same roadway. Cycling has heavy proportion among all travel modes in these cities. In the developed countries, bicycle travel is recognized as "green traffic," which has low energy consumption, is healthy to the users, and does not damage the health of others. It is relatively fast over short distances and provides a reliable and affordable form of transport for most sectors of the population [2]. Therefore, cycling is still one of the most sustainable travel modes [3] around the world.

In relation to the bicycle traffic, it is crucial for control and management as well as facility design of mixed traffic. Scholars have done much research work on it. Unfortunately, deficiencies and limitations in the existing sources of data often hamper these efforts. And several current data collection techniques for bicycle study still depend on observer-based manual operations, which remain time-consuming and resource intensive. With the emergence of a wide variety of automated detection technologies, a few applications have been developed for bicycle detection in recent years, including inductive loops, microwave, infrared, and vision-based. In previous works, most scholars [4-6] utilized inductive loops or their improved forms to design detection systems for acquiring bicycle parameters. But limited by the detection range, it is difficult to solve problems of multiple bicycles passing together or vehicles and bicycles passing together. In addition, although microwave detector can detect the occupying area of an object, the affordable automotive detector can only detect reflectors. As a result, it has no ability to recognize object's category reliably which is the same to the infrared detector.

The vision-based method has the advantages of large detection range and high scalability compared to other methods and therefore is one of the most reliable techniques used for bicycle detection. However, in existing literatures of vision-based detection methods in ITS, vehicles and pedestrians remain the primary objects focused on [7, 8]. Bicycles have so far been limited. Messelodi et al. [9] presented a feature-based bicycle recognition algorithm. The algorithm extracts some visual projective features focusing on the wheel regions of the targets. And then support vector machine (SVM) is applied to distinguish bicycles from motorcycles in real-world traffic scenes. Rogers and Papanikolopoulos [10] detected moving objects through the scene by means of a background differencing technique. They localized the wheels by searching for ellipses using the generalized Hough-Transform in the edge map to recognize bicycles. In a similar way, in Dukesherer and Smith's method [11], Hough-Transform is utilized to locate wheel regions of bicycles, and then the Hausdorff distance is used for matching the candidates with simple bicycle templates. A bicycle is recognized as two arcs of a circle separated by an approximately known distance. David et al. [12] developed and tested a bicycle detection and classification algorithm by active-infrared overhead vehicle imaging sensor technology. In the method, several message concepts are defined derived from four stages of the movement of a target underneath the sensor. A bicycle could be accurately detected and classified using the sequence of messages.

As can be seen from the literature review, some achievements of vision-based bicycle detection have already been made. Nevertheless, an individual or a small number of bicycles are considered as the research object to be detected in most methods, which do not adapt to the case of bicycles in large numbers. Since cycling has heavy proportion among all travel modes in China, there is always a great volume of bicycles in rush hour. In addition, due to the low speed, low occupancy in space, and the flexibility of bicycle travelling, cyclists often move together in groups. Therefore, it is very necessary to design a multiple bicycle detection method, which could provide real-time bicycle's traffic information (the volume, the velocity, etc.) for traffic control and management.

This paper aims to propose a real-time multiple bicycle detection algorithm based on video. The remainder of this paper is arranged as follows: in Section 2, an effective feature called multiscale block local binary pattern is provided for bicycle feature representation; followed by the recognition task of bicycle, a cascaded classifier trained by AdaBoost algorithm is proposed. Lastly, the validity of the proposed approach with video sequence captured from realistic traffic scenario is tested and conclusions are drawn.

2. MBLBP Feature Representation

The LBP operator was first introduced as a complementary measure for local image contrast [13]. It is a gray-scale invariant texture primitive statistic, which has shown excellent performance in the classification of various kinds of textures [14]. A texture T in a local neighborhood of a monochrome texture image is defined as the joint distribution of the gray levels of P ( P > 1 ) image pixels: [figure omitted; refer to PDF] where _{g c} corresponds to the gray value of the center pixel of the local neighborhood and _{g p} ( p = 0,1 , ... , P - 1 ) correspond to the gray values of P equally spaced pixels on a circle of radius R ( R > 0 ) that form a circularly symmetric neighbor set. If the coordinates of _{g c} are ( 0,0 ) , then the coordinates of _{g p} are given by ( - R sin ( 2 π p / P ) , R cos ... ( 2 π p / P ) ) . Much information of the joint gray level distribution about the textural characteristics can be conveyed by the joint difference distribution [15]: [figure omitted; refer to PDF] For each _{g p} , a binary code can be produced by thresholding its neighborhood with the value of _{g c} : [figure omitted; refer to PDF] where [figure omitted; refer to PDF] By assigning a binomial factor ^{2 p} for each s ( _{g p} - _{g c} ) , a unique LB _{P P , R} can be constructed that characterizes the spatial structure of the local texture: [figure omitted; refer to PDF] When P = 8 and R = 1 , LB _{P 8,1} can be obtained, which is the basic LBP descriptor. An illustration of the basic LBP operator is shown in Figure 1. In this way, a 256-bin histogram can be created to collect up the occurrences of different binary patterns over an image.

Figure 1: The basic LBP operator.

[figure omitted; refer to PDF]

The basic LBP is defined for each pixel by thresholding the 3 * 3 neighborhood pixel value with the center pixel value. MBLBP is the extendable descriptor of the basic LBP, with respect to neighborhoods of different sizes. In MBLBP, the comparison operator between single pixels in LBP is replaced with the comparison between average intensities of subregions. Each subregion is a block containing neighboring pixels. A MBLBP descriptor is composed of 9 blocks, which is shown in Figure 2. In this way, an output value of the MBLBP can be obtained: [figure omitted; refer to PDF] where _{b c} is the average gray values of the center block (size w × h , w is the width of the block; h is the height of the block); _{b p} ( p = 0,1 , ... , 7 ) are those of its neighborhood blocks. Particularly, when w = 1 , h = 1 , MBLBP is in fact the basic LBP. Compared with the basic LBP, MBLBP can capture large-scale structures that may be the dominant features of images. In addition, MBLBP could be calculated fast using integral image method [15], which incurs a little more cost than the basic 3 * 3 LBP operator.

Figure 2: 9 * 9 MBLBP operator (including 9 blocks, 3 * 3 pixels in each block).

[figure omitted; refer to PDF]

Figure 3 gives some examples of MBLBP with different sizes for bicycle and nonbicycle images. From this figure we can see that, for a small scale, local, micropatterns of a bicycle structure are well represented, which may be beneficial for discriminating local details. But using average values over the blocks could reduce noise and make the representation more robust.

MBLBP calculation with different sizes for bicycle and nonbicycle images.

(a) Bicycle's MBLBP calculation

[figure omitted; refer to PDF]

(b) Nonbicycle's MBLBP calculation

[figure omitted; refer to PDF]

3. AdaBoost Learning

A cascaded classifier is constructed for obtaining possible bicycle candidates. It can effectively remove most nonbicycle subimages and accelerate the detection algorithm. The MBLBP features are adopted as the basic elements to construct the cascaded classifiers, and each layer is trained by AdaBoost algorithm [16].

The basic idea of AdaBoost algorithm is to use large capacity of general classification of the weak classifier by a certain method of cascade to form a strong classifier. The cascade structure containing N stages is illustrated in Figure 4, where _{A i} is referred to as an AdaBoost classifier in the i th stage. As can be seen from the structure, the cascade classifier is a degenerated decision tree. At each stage, a classifier is trained to detect almost all bicycle candidates while rejecting a certain fraction of nonbicycle objects. Therefore, negative subimages that do not contain bicycles can be abandoned in some early stages of the cascade [17, 18]. Only the subimage passing all stages can be identified to be the bicycle. Detailed procedure of the cascaded AdaBoosting classifier is described as follows.

(a) Give training samples S = { ( _{x 1} , _{y 1} ) , ( _{x 2} , _{y 2} ) , ... , ( _{x M} , _{y M} ) } , where _{y i} = { 0,1 } corresponds to the types (nonbicycle and bicycle, resp.). The training set consists of k nonbicycle samples and l bicycle samples.

(b) Initialize the i th sample's weights _{w 1 , i} = D ( i ) . If the sample is a nonbicycle object, the weight is represented as D ( i ) = 1 / 2 k ; if the sample is a bicycle, the weight is represented as D ( i ) = 1 / 2 l .

(c) For each training stage (where N is the number of training stages):

: ... normalize the weights [figure omitted; refer to PDF]

: ... For each feature, the corresponding weak classifier is trained as [figure omitted; refer to PDF]

: where _{p j} denotes the direction of the inequality sign, its value only refers to 1 or -1, _{f j} ( x ) denotes the feature value, and _{h j} is the threshold.

: ... Choose the simple classifier _{h n} ( x ) with the lowest error _{[straight epsilon] n} : [figure omitted; refer to PDF]

: ... Update the weights according to the best simple classifier _{h n} ( x ) : [figure omitted; refer to PDF]

: where _{e i} = 0 if sample _{x i} is classified correctly, _{e i} = 1 ; otherwise, _{β n} = _{[straight epsilon] n} / 1 - _{[straight epsilon] n} .

(d) At last, a strong classifier is formed as [figure omitted; refer to PDF]

As can be seen from the process proposed above, a strong classifier based on a set of weak classifiers by reweighting the training samples can be constructed using the boosting idea. At each stage of boosting, the feature-based classifier that best classifies the weighted training samples is used. Because the classifier should achieve the desired false alarm rate at a given hit rate, the number of weak classifiers should be increased. Lastly, all the weak classifiers are combined to form a strong classifier by different weights [16].

Figure 4: The cascade structure of the AdaBoost classifier.

[figure omitted; refer to PDF]

4. Experimental Results

For bicycle recognition in urban road environment, a number of bicycle images are selected manually and a bicycle sample dataset is constructed. In the sample dataset, the positive samples are typical bicycles with different size, pose, and cycler's clothing. Some preexperimental studies have shown that the selection of negative samples is particularly important for reduction of false alarms. Thus, boles, trash cans, telegraph poles, and bushes, which are likely to be mistaken for bicycles, as well as some normal objects such as roads, vehicles, and other infrastructures are selected to form negative samples. A total of 4650 hand labeled samples were adopted to train the cascaded classifiers. They include 1650 positive samples and 3000 negative samples. In the bicycle dataset, each sample image is normalized to 16 × 64 pixels for training. Figure 5 shows some samples of the bicycle dataset.

Samples of the bicycle dataset: (a) positive samples; (b) negative samples.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

Based on the cascaded classifier with 50 stages, 600 test samples (the first 300 samples are positives; the rest are negatives) are recognized. From Figure 6 we can see that feature outputs of bicycles and nonbicycles are obviously different. That means the MBLBP is an effective feature for bicycle detection.

Figure 6: Output values of test samples (the first 300 samples are positives; the rest are negatives).

[figure omitted; refer to PDF]

A 9 * 9 mode MBLBP is used for feature representation of the training data. And then, multiple bicycles of the test set are recognized by an AdaBoost classifier trained. The results (2 examples) show a relative high false positive rate in Figure 7. In order to achieve a better detection performance, a two-layer detection strategy is operated in this paper. In the first layer, 9 * 9 mode MBLBP features of samples are calculated. Some not well-recognized samples are selected (false positives and false negatives) to compose a new training set. Multiple modes of MBLBP with 3 * 3 and 9 * 9 modes are calculated for feature representation in this layer. And then, an additional AdaBoost classifier is constructed for the second round recognition. In this way, false positive rate could be improved (Figure 8). In addition, the two-layer detection has better timewise performance than that of one-layer detection with 3 * 3 mode MBLBP calculation. Through our test, the average detection time is around 0.1 s with 10 detection scales, which could be used for the real-time application of intelligent urban traffic management and control.

Figure 7: Detection results with one-layer classifier with 10 detection scales and without overlapping processing.

[figure omitted; refer to PDF]

Figure 8: Detection results with two-layer classifier and overlapping processing.

[figure omitted; refer to PDF]

5. Conclusions

According to this paper, a multiple bicycle detection algorithm for intelligent urban traffic management and control has been developed. The research conclusions are as follows: an extended LBP descriptor called MBLBP is proposed for feature representation, which is a well-classified feature to distinguish between bicycles and nonbicycles; then a cascaded bicycle classifier is constructed based on AdaBoost algorithm, followed by testing from real-world traffic scenarios. Reliable and timewise performances are shown on vision-based bicycle recognition. The processing speed could reach 10 frames/s which could satisfy the real-time requirement. In future work, we will further study the key technologies for the analysis of cyclist motion characteristics and behaviors.

Acknowledgments

This work is partly supported by National Science Foundation of China (nos. 51108208 and 51278220), Postdoctoral Science Foundation funded project of China (no. 2013T60330), Science and Technology Development Project of Jilin Province (no. 20130522121JH), and Fundamental Research Funds for the Central Universities of China (no. 201103146).

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

[1] D. H. Wang, T. J. Feng, C. Y. Liang, "Research on bicycle conversion factors," Transportation Research Part A: Policy and Practice , vol. 42, no. 8, pp. 1129-1139, 2008.

[2] K. J. Krizec, "Estimating the economic benefits of bicycling and bicycle facilities: an interpretive review and proposed methods," Essays on Transport Economics , pp. 219-248, 2007.

[3] B. Gatersleben, K. M. Appleton, "Contemplating cycling to work: attitudes and perceptions in different stages of change," Transportation Research Part A: Policy and Practice , vol. 41, no. 4, pp. 302-312, 2007.

[4] R. Leschinski, "Evaluation of inductive loops for bicycle detection," in Proceedings of the 17th ARRB Conference, vol. 17, part 5, pp. 15-19, Gold Cosat, Queensland, Australia, 1994.

[5] R. Kidarsa, T. Pande, S. V. Vanjari, J. V. Krogmeier, D. M. Bullock, "Design considerations for detecting bicycles with inductive loop detectors," Transportation Research Record , vol. 1978, pp. 1-7, 2006.

[6] G. He, G. Wang, S. Ma, "Research on the classifier for bicycle flow detecting device," in Proceedings of the 3rd International Conference on Transportation and Traffic Studies, pp. 1272-1277, Guangxi, China, July 2002.

[7] H. Hu, Z. Qu, Z. Li, J. Hu, F. Wei, "Fast pedestrian recognition based on multisensor fusion," Discrete Dynamics in Nature and Society , vol. 2012, 2012.

[8] H.-Y. Hu, Z.-W. Qu, Z.-H. Li, Q.-N. Wang, "Robust detection and tracking algorithm of multiple objects in complex scenes," Applied Mathematics and Information Sciences , vol. 8, no. 5, pp. 2485-2490, 2014.

[9] S. Messelodi, C. M. Modena, G. Cattoni, "Vision-based bicycle/motorcycle classification," Pattern Recognition Letters , vol. 28, no. 13, pp. 1719-1726, 2007.

[10] S. Rogers, N. Papanikolopoulos, "A robust video-based bicycle counting system," in Proceedings of the ITS America 9th Annual Meeting, Washington, DC, USA, 1999.

[11] J. H. Dukesherer, C. E. Smith, "A hybrid Hough-Hausdorff method for recognizing bicycles in natural scenes," in Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, pp. 2493-2498, October 2001.

[12] N. A. David, A. Gajendran, R. Dharmaraju, "Development of bicycle and bicycle detection and classification algorithm for active-infrared overhead vehicle imaging sensors," Transportation Research Record , vol. 1982, pp. 202-209, 2006.

[13] T. Ojala, M. Pietikäinen, D. Harwood, "A comparative study of texture measures with classification based on feature distributions," Pattern Recognition , vol. 29, no. 1, pp. 51-59, 1996.

[14] T. Ojala, M. Pietikäinen, T. Mäenpää, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 24, no. 7, pp. 971-987, 2002.

[15] P. Viola, M. Jones, "Robust real time object detection," in Proceedings of the IEEE ICCV Workshop on Statistical and Computational Theories of Vision, Vancouver, Canada, July 2001.

[16] L. Guo, P.-S. Ge, M.-H. Zhang, L.-H. Li, Y.-B. Zhao, "Pedestrian detection for intelligent transportation systems combining AdaBoost algorithm and support vector machine," Expert Systems with Applications , vol. 39, no. 4, pp. 4274-4286, 2012.

[17] Y. Freund, R. E. Schapire, "Experiments with a new boosting algorithm," in Proceedings of the 13th International Conference on Machine Learning, pp. 148-156, San Francisco, Calif, USA, 1996.

[18] W. Wang, X. Jiang, S. Xia, Q. Cao, "Incident tree model and incident tree analysis method for quantified risk assessment: an in-depth accident study in traffic operation," Safety Science , vol. 48, no. 10, pp. 1248-1262, 2010.