An improved Vibe algorithm for illumination mutations

LIANG Jincheng, WANG Xiaopeng, WANG Qingsheng

(School of Electronic and Information Engineering， Lanzhou Jiaotong University, Lanzhou 730070, China)

Abstract： The visual background extractor(Vibe) algorithm can lead to a large area of false detection in the extracted foreground target when the illumination is mutated. An improved Vibe method based on the YCbCr color space and improved three-frame difference is proposed in this paper. The algorithm detects the illumination mutation frames accurately based on the difference between the luminance components of two frames adjacent to a video frame. If there exists a foreground moving target in the previous frame of the mutated frame, three-frame difference method is utilized; otherwise, Vibe method using current frame is used to initialize background. Improved three-frame differential method based on the difference in brightness between two frames of the video changes the size of the threshold adaptively to reduce the interference of noise on the foreground extraction. Experiment results show that the improved Vibe algorithm can not only suppress the “ghost” phenomenon effectively but also improve the accuracy and completeness of target detection, as well as reduce error rate of detection when the illumination is mutated.

Key words： moving target detection； visual background extractor (Vibe) algorithm； YCbCr color space； three-frame difference method

0 Introduction

Moving target detection[1]is a process that extracts the part of the interested image sequence. The performance of the detection has an impact on target recognition and target tracking accuracy. The current moving target detection methods contain frame difference method[2], optical flow method[3]and background difference method[4]. The frame difference method detects a moving target by calculating the difference between two consecutive frames and using a threshold determination, which is simple. But the contour of the detection target is incomplete, and contains “holes” inside. The optical flow method detects moving targets through the motion state vector of each pixel. However, it is difficult to apply widely in practice because of the high computational complexity, high hardware requirements and poor real-time performance. The background difference method constructs a background model to differ the image sequence from the current background model. The foreground and background of a moving target are distinguished by a threshold. Visual background extractor (Vibe) algorithm[5], which is a pixel-level background modeling method with the advantages of fast initialization of the background model, optimal sample decay and high computational efficiency, is an effective background differential method. However, in complex background conditions such as background perturbations (water surface ripples, leaf shaking, etc.) and noise interference (wind, rain, etc.), the Vibe algorithm has a high false detection rate.

In recent years,many improved algorithms have been proposed. In Ref.[6], small target discarding and void filling strategies are applied to remove noise. This algorithm improves the accuracy and recognition rate. In Ref.[7], an improved Vibe algorithm is proposed to address the difficulty of adapting to complex background environments with fixed thresholds. This algorithm introduces the maximum inter-class variance method to improve the accuracy and integrity of target detection by changing the fixed pixel thresholds to dynamic thresholds. Ref.[8] addresses the problem of inaccurate detection of foreground moving targets due to noise and interference in the video. The proposed algorithm dynamically adjusts the update period and update method of the model according to the complexity of the scene, so that the model can effectively eliminate the effects of vignettes and background noise.

Although those improved Vibe algorithms show some adaptability in complex background environments, it is difficult to detect moving targets completely and accurately under mutated illumination, and the detected foreground often contains large areas of false detection. In this paper, an improved Vibe algorithm based on YCbCr color[9]space is proposed with precise positioning of illumination mutation frames in YCbCr color space by judging the difference between the luminance components of two frames adjacent to a video frame. Also the luminance components is used to change the threshold adaptively in improved three-frame difference method. If a foreground moving target exists in the previous frame of the mutated frame, three-frame difference method is utilized; otherwise, Vibe[10-14]method using current frame is used to initialize background. As a result, “ghost” phenomenon, which is false detection during frame initialization, can be suppressed.

1 Vibe algorithm

Vibe algorithm does not require any prior knowledge of the scene during background initialization. The initialization of background where each pixel is used to create its own set of samples (samples in each set are collected probabilistically from pixels in pixel vicinity), can be done by using the first frame. Here,M(x) is defined as the background sample of pixelxwhere each pixel hasNsamples, andviis the 8-neighborhood random sample for pixelx. The set is defined as

M(x)={v1,v2,…,vN-1,vN}i∈[1,N].

(1)

When foreground segmentation is applied, the Vibe algorithm distinguishes between foreground and background pixels by comparing the similarity between the current pixel and the corresponding pixel sample set in the background initialization. Firstly, set current pixelxasv(x); secondly, make a circleSR(v(x)) centered atxwith radiusR; finally, set the threshold asUmin. Fig.1 shows a schematic representation of the foreground extraction with Vibe algorithm on the 2D color space (C1,C2) component.

In Fig.1, pixelx is judged to be the background pixel if the number of samples of pixelxinSR(v(x)) is larger than the thresholdUmin; otherwise it is judged to be the foreground.

(2)

wherebgis the background image, andfgis the foreground image.

Fig.1 Two-dimensional color space diagram with Vibe algorithm

During background update, Vibe algorithm uses random subsampling. Only if a pixel is determined as background in the foreground detection, then each sample in the alternative background is updated with a probability of 1/φ(φis the time sampling factor). Spatial information neighborhood spreading mechanism is used to spread background information to neighboring pixels and the background model of the pixelxis updated randomly with the same probability while updating the background model of its neighboring pixels.

2 Improved Vibe algorithm

2.1 Precise detection of illumination mutation frames

If there exists an illumination mutation in the video, thenYcomponent corresponded to the two frames before and after the mutation will also change. When dealing with the illumination mutation, the change ofYcomponent is used to describe the illumination mutation, and thus accurate judgment of illumination mutation of the video frame can be achieved. However, images are generally obtained in RGB color space, so that the conversion between RGB and YCbCr are required, which is

(3)

Suppose that there aresframes (t=1,2,3,…,s) in a video andKpixels (i=1,2,3,…,K) in each frame. The brightness of each pixel isY(i) and the average of all pixels in an image isYt_mean. It can be expressed as

(4)

This paper considers average brightness values of two adjacent video frames. The absolute difference between the average brightness values of frametand framet-1 isY(t), which represents the image brightness variation scale of two adjacent frames. At this point, thresholdYthis introduced. IfY(t)≥Yth, then illumination mutation is occurred, and vice versa.

|Yt-1_mean-Yt_mean|=Yt.

(5)

2.2 Improved three-frame difference method

Three-frame difference[15-19]is an improved frame difference algorithm which allows complete extraction of moving targets when the target moves too fast. Assume that there are three video framest-1,tandt+1 areFt-1,FtandFt+1, and grayscales of pixels in the three frames areFt-1(x,y),Ft(x,y) andFt+1(x,y). As shown in Fig.2,RtandRt+1are the difference results ofFt-1andFtas well asFtandFt+1, respectively.

Rt(x,y)=|Ft(x,y)-Ft-1(x,y)|.

(6)

Binary images of the moving targets is extracted according to thresholdT.PtandPt+1are obtained by

(7)

(8)

The conventional three-frame difference method applies fixed thresholdT. IfTis too large, some information of the target may be blocked. IfTis too small, noise may not be suppressed. Also, the fixedTis not able to accommodate the illumination mutation.

Improved three-frame difference method is introduced based onYtand weightsτ, which can change the threshold adaptively. When the light change is small,τYtis close to 0. When the change is too large,τYtis significantly large. In this algorithm,τ=0.25.τis derived from the experiment. The selection criterion is top reserving the characteristics of the foreground targetand eliminating the influence of noise on the extraction of foreground pixels. The formula is

(9)

The thresholdT1andT2of the differences between framest-1,tand framest,t+1 shown in Eq.(9) is controlled byY(t) andY(t+1). Binary differential imagesPtandPt+1are obtained byT1andT2.

(10)

(11)

In Fig.2, Fig.2(a) is the 60th frame of the Canoe video, and Fig.2(b) is the binary image acquired by the conventional three-frame difference method. In Fig.2(b), false detection appears widely in the foreground due to the changes of the water surface ripples. Fig.2(c) shows the binary image acquired by improved three-frame difference method with adaptive threshold according to luminance. The algorithm is performed well and adapts to the changes in the water surface area, and the false detection rate of foreground is reduced.

Fig.2 Improved three-frame differential detection results

The flowchart of improved Vibe algorithm is shown in Fig.3.

The improved Vibe algorithm is achieved by the specific steps.

Step 1: Calculate the difference of YCbCr spatial luminance component for the current frame of the input video according to Eqs.(3)-(5), and compare it with the threshold, if an illumination mutation occurs, perform the step 3, otherwise perform step 2.

Step 2: Foreground extraction is performed according to the conventional Vibe algorithm when illumination mutation is not detected in the current frame.

Step 3: When an illumination mutation is detected in the current frame, and if the foreground pixel exists in the previous frame, step 4 is performed, otherwise perform step 5.

Step 4: Use improved three-frame difference method to extract the foreground target according to Eqs.(9)-(11).

Step 5: The Vibe algorithm is used for background initialization and foreground extraction of the current frame, as a result, accurate detection and extraction of the foreground under illumination mutations can be achieved.

Fig.3 Flowchart of improved Vibe algorithm

3 Results and discussion

To verify the accuracy and validity of the algorithm for foreground detection under illumination mutations and the ability to suppress “ghost” phenomenon, three videos are used to validate the algorithm in this paper. The parameters set for the experiments are number of samples collectedN=20, minimum number of matches #min=2, distance thresholdR=20, frame sampling frequencyφ=16, thresholdYth=20, The Intel©CorTMi5-4200U CPU@1.60 GHz 2.300 GHz/memory 8.00 GB hardware platform was used in the experiment, and the simulation experiment was carried out under the MTALAB R2012b environment. Lightingswitch and two self-captured videos are tested.

The first experiment is validated by using lightingswitch. As shown in Fig.4(a) and (b), illumination mutation caused by target rapid approach to the camera in a dark environment is located in the 327th and 328th frames. The second experiment is conducted with a video captured indoors by using the camera. Illumination mutation caused by the light switch is occurred in 107th and 108th frames. The mutation frames are shown in Fig.4(c) and (d). Presence of foreground pixels are detected.

Fig.4 Illuminated mutation frames with foreground targets

Fig.4(a) and (b) are two frames before and after the light mutation of lightingswitch. Fig.4(c) and (d) are two frames before and after the illuminated mutation from the second set of video images.

Experiments are conducted by conventional Vibe algorithm, three-frame difference method and improved Vibe algorithm. Visual results are shown in Fig.6. In the results of the Vibe algorithm, the foreground target can no longer be distinguished due to the effect of illumination. Foreground targets can be identified in the results of the three-frame difference method with large areas of noise and false detection. The improved Vibe algorithm can clearly identify the foreground target and suppress false detection, which is better than the conventional Vibe algorithm and the three-frame difference method.

Fig.5(a) and (e) are the original frame images with illumination mutation. Fig.5(b) and (f) are the results of Vibe algorithm. Fig.5(c) and (g) are the results of three-frame difference method. Fig.5(d) and (h) are the results of improved Vibe algorithm.

Fig.5 Foreground detection renderings containing foreground images

The third experiment is conducted with another self-captured video.There is no foreground targets exists in the image when the illumination mutation occurs. Fig.6(a) and (b) are the two frames detected by this algorithm with illumination mutation which are frames 20 and 21.

Fig.6 Illuminated mutation frame detection images without foreground targets

Comparison experiments are conducted by using conventional Vibe algorithm, three-frame difference method and the algorithm in this paper. Results are shown in Fig.7. After illumination mutation, the three-frame difference method only detects an incomplete moving target contour of frames 23 and 24. False detection appears in the result of conventional Vibe algorithm widely. Improved Vibe algorithm can detect the moving targets precisely and completely. When the moving target appears completely in the image, comparison results of the three algorithms in frame 32 proves that the improved Vibe algorithm performs better than conventional Vibe algorithm and three-frame difference method.

Fig.7 Foreground detection map for illumination mutation frames without foreground images

Fig.7(a), (e) and (i) are the frames 23, 24 and 32 of the third set of videos. Fig.7(b), (f) and (j) are the processing results of three-frame difference. Fig.7(c), (g) and (k) are the processing results of Vibe algorithm. Fig.7(d), (h) and (l) are the processing results of improved Vibe.

When the current frame has lighting mutation, if foreground pixel exists in the previous frame, improved three-frame difference method is used for foreground extraction. If the Vibe algorithm is used for background initialization and foreground extraction of the current frame, “ghost” phenomenon is appeared in the foreground detection.

Fig.8(a), (b), (c) and (d) are the original images in frames 328-331 of the video. Fig.8(e), (f), (g) and (h) are the results of frames 328-331 of the video by the improved three-frame difference method. Fig.8(i), (j), (k) and (l) are the results of frames 328-331 of the video by initializing the background of the current frame by using conventional Vibe algorithm.

Fig.8 shows that when an illumination mutation phenomenon is detected and foreground pixel is existed in the frame just before the mutated frame, the foreground target can be detected in real time and extracted completely by using improved three-frame difference method. However, if a foreground target is detected by using the Vibe algorithm for initial model processing, “ghost” phenomenon appears in the foreground target. Fig.8(e) shows that when a illumination mutation is detected in frame 328, the foreground can be extracted accurately by using the improved three-frame difference method. However, one frame is required for background initialization to initialize the current frame by using Vibe. Phenomenon of “ghost” occurs with Vibe initialization modeling in the followed three frames of frame 328, and it is difficult to eliminate “ghost”. Foreground target can be accurately extracted by using improved three-frame difference in this paper.

Fig.8 Experimental results for ghost elimination

Relevant performance metrics are used to quantify the performance of the algorithms.

(12)

(13)

(14)

whereDTPis the number of correctly detected foreground pixels;DFPis the number of incorrectly detected pixels;DTNis the number of pixels in the foreground that are not identified as background;DPCCis the overall percentage of correct classification, expressed as the percentage of all pixels that were correctly detected as foreground and background pixels;DFPRdenotes the false alarm rate, which represents the percentage of false foreground pixels detected as a percentage of the overall pixel count;DFNRis the false-negative rate, indicating the proportion of foreground pixels that are missed as background pixels.

Three videos are tested and calculated by using three different methods for moving target detection under illumination mutation conditions and the results are averaged and the data are shown in Table 1.DFPRandDFNRof the Vibe algorithm and the three-frame difference method are high due to illumination mutation, resulting in lower foreground detection accuracy. For overall evaluation criterionDPCC, it can be shown that the results of the algorithm detection in this paper are excellent among all indicators. As can be seen from the data in Table 2, although the algorithm in this paper adds steps compared with conventional Vibe algorithm, the execution time of the proposed algorithm is only a little bit longer than the conventional Vibe algorithm with a few seconds. There is little difference in real-time performance

Table 1 Comparison of indicators of three algorithms

Table 2 Processing time of three algorithms for video

4 Conclusions

An improved Vibe algorithm based on YCbCr color space and improved three-frame difference method is proposed to solve the problem of foreground target false detection by using conventional Vibe algorithm in the presence of illumination mutation. It can realize precise positioning of illumination mutation frames in YCbCr color space by comparing the difference between the luminance components of two frames adjacent with a video frame with threshold. Different processing methods for subsequent frames depend on whether there exists foreground pixels in the previous frame of the mutated frame; “ghost” phenomenon can be suppressed. Experimental results show that the proposed algorithm is able to detect moving targets under illumination mutation accurately. The improved Vibe algorithm reduces the false detection rate effectively and improves the adaptability of the algorithm under illumination conditions.