Face Detection under Complex Background and Illumination

Shao-Dong Lv, Yong-Duan Song, Mei Xu, and Cong-Ying Huang

1. Introduction

Face detection is to find out whether the given images or a video sequence contains human faces, if so, then it gives the specific location of the face, size, quantity, etc.Face detection is a complex pattern recognition problem,which is mainly affected by the complex background and illumination outside. In recent years, with the actual demand for identification and safety certification, the face detection is expected to be adapted to the environment,which forces the researchers to develop a robust, efficient,and real-time face detection algorithm.

So far, researchers have done lots of research in face detection and have proposed some classical algorithms,for improving the detection rate, but the algorithm training time was very long, and the false detection rate also increased rapidly while the detection rate is improved.

For face detection under the complex background and illumination, at present there is no single algorithm can achieve a good detection results. We propose an effective and robustness method in this paper. First, we cluster the skin color in the color space by using the skin color to extract the skin likelihood area in images, excluding a large number of irrelevant backgrounds; then for remedying the deficiencies of the Adaboost algorithm, we introduce the cost-sensitive function into the Adaboost algorithm; finally,combine the skin color and cost-sensitive Adaboost algorithm for the face detection. Experimental results show that the proposed detect method has high robustness and a high detection rate, and detection speed.

2. Skin Color Segmentation

Skin color is one of the most significant features of the surface, which relies on facial expressions and rotation, but has a certainly robustness for different light. Experimental research shows that the differences in skin color between different ages and races are mainly the differences of brightness, which have nothing to do with the chrominance.After removing the brightness, the color of skin takes on the characteristic of clustering in some color space, so the color is one kind of the most commonly used features in face detection[9]. The commonly used color space are RGB(red, green, blue—three primary colors), YCbCr(encoding the chrominance model CCIR601), HSV (hue, saturation,brightness), etc. In order to reduce the influence of brightness difference as much as possible, this paper selected the YCbCrcolor space for color segmentation.

2.1 YCbCr Skin Space

In the Joint Photographic Experts Group (JPEG)Standard, the RGB image is converted to a luminance-chrominance space, usually called YCbCrcolor space, where Y represents the luminosity, and Cband Crrepresent chroma. The YCbCrhas the advantages that the luminance component is isolated from the chrominance,and the skin color can be well clustered in the Cb-Cr-dimensional space[10]. So it is widely used in the face skin color detection. YCbCrcan be obtained by linear transformation of RGB, shown as

As shown in Fig. 1, we manually select 30 images, and then select 30 skin pixels and 30 non-skin pixels in each image for a statistical skin distribution in the Cb-Crspace,as shown in Fig. 1.

Fig. 1. YCbCr skin statistics.

2.2 Skin Color Segmentation

This paper used the YCbCrcolor space for the face detection, and after testing on a great number of face images, we choose the skin segmentation parameters as shown in (2) and some skin segmentation examples are shown in Fig. 2.

3. Face Detection

Face detection is essentially a two-classification problem, which distinguishes the face and non-face areas generally by the pre-established rules. Viola proposed the Adaboost face detection algorithm based on Haar-Like in 2001, which is a milestone in the face detection history. But the traditional Adaboost algorithm is not perfect. This section starts from the analysis of Adaboost algorithm, then combines with the cost-sensitive learning function[11]-[13]to propose a new method that uses the cost-sensitive Adaboost for the face detection.

Fig. 2. Skin segmentation in YCbCr color space: (a) original picture and (b) skin segmentation result.

3.1 Adaboost Algorithm

The Adaboost algorithm is an iterative algorithm, and the core idea is training different classifiers by the same training set, and then combining these weak classifiers to form a final stronger classifier. The algorithm is implemented by changing the data distribution. In each round of training, the algorithm changes the weight of each sample according to whether this sample has been classified correctly and the overall classification accuracy. Then it sends the modified weight of new data to the lower classifier for training, finally puts each trained classifier fusion together as the final decision classifier. The specific derivation could refer to [8].

In the original form, in the process of initializing the samples and updating the sample weights, the Adaboost algorithm treats the negative samples the same as the positive samples. Besides, it selects the best classifier according to the minimum classification error. This is unrealistic in the actual face detection. First, face detection is the small sample detection, we should give enough attention to the positive samples. Second, the structure of Adaboost face detection is the cascade classifier structure,only the area through all of the cascade classifier windows can be tested as the face. The classifiers in the higher lever of the cascade structure are required to detect almost all of the face samples and exclude non-face samples, at this moment, if we just consider the minimum classification error, it will affect the detection rate for face samples.

In recent years, the cost-sensitive learning problem[14]-[16]has been widely concerned. People not only concern the minimum classification error, but also begin to focus on the minimum of the classification costs. Inspired by them, we introduce the cost-sensitive function into both the samples weights initialization and samples weights updated process, and finally combine it with the color segmentation for the actual face detection.

3.2 Proposed Cost-Sensitive Adaboost Algorithm

1) For a given set ofn samples:L ={(xi, yi)},i = 1 ,2,… ,n, where xi∈Xrepresents a set of samples,andyi∈Y={-1,1}represents a sample class, set the number of positive samples forpand that of the negative samples forq .

2) Define the cost-sensitive function as

where c is the cost-sensitive factor.

3) Initialize the sample weights:

4) For the cycling index t=1, 2, ⋅⋅, T, loop Step 4) for T times to find T weak classifiers ht.

a) The classifier error can be expressed as

where ωt(i)is the ith sample weight in the tth cycling.Select the weak classifierhtin each cycle, which has the smallest errorεt.

b) Setat= ( 1 2)ln((1 - εt)εt)to update sample weights:

c) Normalize the sample weights separately:

Then return to a) until T weak classifiers hthave been obtained.

5) Finally, we get the strong classifier:

where sgn(⋅) is the sign function.

From the derivation of the above algorithm, the main differences between the algorithm proposed in this paper and the Adaboost algorithm are: 1) First, this paper introduces the cost-sensitive function Ciand applies it into the samples initialization and the process of updating the sample weights. 2) In the sample normalization process,the positive and negative samples are separately normalized.The aim are: 1) After the introduction of cost-sensitive functionCi, as long as c>1 then the initially positive samples weights will be larger than the negative samples,and the classifier will pay more attention to the positive samples. 2) We are able to make the positive sample weights larger than negative sample weights by setting the cost-sensitive functionCi. But if the positive samples and negative samples are not normalized separately, then after a few iterations, the negative sample weights are likely to exceed the weight of the positive samples. Therefore, we normalized the positive and negative samples separately to ensure the main position of the positive samples in the whole process. 3) According to the analysis of the above weight update rule, when the samples are correctly classified, namelyyiht(xi) = 1, the sample weight will fall.But at this time the extent of the positive samples will decrease more slowly than that of the negative samples.Similarly, when the samples are misclassified, namely yiht(xi) = -1, the sample weights will increase, but at this time the degree of positive samples will increase more greatly than that of the negative samples. All of these actions ensure that the modified algorithm will always give enough attention to the positive samples, thus the classifier will have good classification capabilities for the positive samples.

4. Experimental Results and Discussion

In order to prove the effectiveness of the proposed cost-sensitive Adaboost, we verify it on the Massachusetts Institute of Technology (MIT) face database. Fig. 3 (a)shows a strong classifier face detection rate under different cost-sensitive functions, (Note that when c=1, the cost-sensitive Adaboost algorithm is equivalent with the Adaboost algorithm). As shown in Fig. 3, under the same false alarm the cost-sensitive Adaboost algorithm has a higher detection rate, so the performance is superior to the traditional Adaboost algorithm. Fig. 3 (b) shows the relationship between the detection rate and weak classifier number. It can be seen that the detection rate of traditional Adaboost algorithm does not increase with the increase of the number of weak classifiers, when the number of weak classifiers is more than 160. This is because the traditional Adaboost algorithm will always increase the weights of the misclassification samples. When the negative samples exist,which is difficult to classify, the classifier will increase the negative samples weights all the time. As a result, the classifier performance degrades. The proposed cost-sensitive Adaboost algorithm can well avoid this phenomenon, because the positive samples always occupy the main position.

Fig. 3. Performance comparison between cost-sensitive Adaboost and Adaboost algorithm: (a) face detection rate vs. false alarms with different cost-sensitive functions and (b) face detection rate vs. number of weak classifier.

In this paper, we combine the skin color segmentation and cost-sensitive Adaboost together for the face detection.In order to verify the effectiveness of the proposed method,we conduct experiments from following two aspects: 1)First, we use the proposed algorithm and the traditional Adaboost to test 80 copies of still images, respectively.These pictures are from the Internet or the camera.Statistical experimental data are shown in Table 1 and Table 2, some of the detection results of the images are shown in Fig. 4. It can be seen from Table 1 and Table 2 that the proposed algorithm has better performance. The detection rate, false detection rate, and detection time are superior to those of the Adaboost. 2) In addition, we conduct the experiment by using our proposed detection method and other methods on the MIT standard face database, the experiments results are shown in Table 3. As shown in Table 3, on the same false alarm rate, the proposed method has a higher detection rate. So, the effectiveness of the proposed algorithm is proved.

Table 1: Comparison of the face detection performance of our detection method and Adaboost on still images

Table 2: Average performance of our detection method and Adaboost

Fig. 4. Some test images and results.

Table 3: Comparison of the face detection performance of our detection method and other methods on MIT standard face database

5. Conclusions

Aiming at reducing the influence of the complex backgrounds and illumination conditions, this paper proposes a face detection method. First, we extract the skin area in the YCbCrcolor space, thereby reducing the calculation complexity for the subsequent face detection.Then based on the Adaboost algorithm, we introduce the cost-sensitive function into the Adaboost algorithm. The experimental results show that the algorithm based on cost-sensitive Adaboost face detection has better performance. Finally, the skin color segmentation and cost-sensitive Adaboost algorithm are combined for the face detection. We test the proposed algorithm on both the real live pictures and MIT standard face database. The experimental results show that the proposed face detection algorithm has a better detection rate, detection speed, and robustness compared with the other methods.

[1] S. L. Phung, A. Bouzerdoum, and D. Chai, “Skin segmentation using color pixel classification: Analysis and comparison,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 148-154, 2005.

[2] H. Wu, Q. Chen, and M. Yachida, “Face detection from color images using a fuzzy pattern matching method,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 21,no. 6, pp. 557-563, 1999.

[3] R. Brunelli and T. Poggio, “Face recognition: Features versus templates,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1042-1052, 1993.

[4] J. Haddadnia, F. Karim, and M. Ahmadi, “Design of RBF neural network using an efficient hybrid learning algorithm with application in human face recognition with pseudo Zernike moment,” IEICE Trans. on Information and Systems,vol. 86, no. 2, pp. 316-325, 2003.

[5] P. Juell and R. Marsh, “A hierarchical neural network for human face detection,” Pattern Recognition, vol. 29, no. 5,pp. 781-787, 1996.

[6] R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, “Face detection in color images,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, 2002.

[7] P. Kakumanu and N. Bourbakis. “A local-global graph approach for facial expression recognition,” in Proc. of the 18th IEEE Int. Conf. on Tools with Artificial Intelligence,2006, pp. 685-692.

[8] H. A. Rowley, S. Baluja, and T. Kanade, “Neural network-based face detection,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 23-38,1998.

[9] P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proc. of IEEE Conf.on Computer Vision and Pattern Recognition, 2001, pp.511-518.

[10] E. Hjelmås and B. K. Low, “Face detection: A survey,”Computer Vision and Image Understanding, vol. 83, no. 3,pp. 236-274, 2001.

[11] Y. Ban, S. K. Kim, and S. Kim, “Face detection based on skin color likelihood,” Pattern Recognition, vol. 47, no. 4,pp. 1573-1585, 2014.

[12] Z.-H. Zhou, Cost-Sensitive Learning, Heidelberg: Springer,2011, pp. 17-18.

[13] P. Viola and M. J. Jones, “Robust real-time face detection,”Int. Journal of Computer Vision, vol. 57, no. 2, pp. 137-154,2004.

[14] Y. Sun, M. S. Kamel, and A. K. C. Wong, “Cost-sensitive boosting for classification of imbalanced data,” Pattern Recognition, vol. 40, no.12, pp. 3358-3378, 2007.

[15] K. M. Ting, “A comparative study of cost-sensitive boosting algorithms,” in Proc. of the 17th Int. Conf. on Machine Learning, 2000, pp. 983-990.

[16] W. Fan, S. J. Stolfo, and J. Zhang, “AdaCost:misclassification cost-sensitive boosting,” in Proc. of the 16th Int. Conf. on Machine Learning, 1999, pp. 97-105.

[17] Y. Ma and X. Ding, “Real-time multi-view face detection and pose estimation based on cost-sensitive adaboost,”Tsinghua Science & Technology, vol. 10, no. 2, pp. 152-157,2005.

[18] Lin C, “Face detection in complicated backgrounds and different illumination conditions by using YCbCrcolor space and neural network,” Pattern Recognition Letters, vol. 28,no. 16, pp. 2190-2200, 2007.