REAL-TIME EYE DETECTION AND TRACKING UNDER VARIOUS LIGHT CONDITIONS

This paper describes a real-time online prototype automobile and truck driver-fatigue monitor. It uses remotely located charge-coupled-device cameras equipped with active infrared illuminators to acquire video images of the driver. Various visual cues that typically characterize the level of alertness of a person are extracted in real time and systematically combined to infer the fatigue level of the driver. The visual cues employed characterize eyelid movement, gaze movement, head movement, and facial expression. A probabilistic model is developed to model human fatigue and to predict fatigue based on the visual cues obtained. The simultaneous use of multiple visual cues and their systematic combination yields a much more robust and accurate fatigue characterization than using a single visual cue. This system was validated under real-life fatigue conditions with human subjects of different ethnic backgrounds, genders, and ages; with/without glasses; and under different illumination conditions. It was found to be reasonably robust, reliable, and accurate in fatigue characterization.


INTRODUCTION
Automobile and truck drivers have various emotions, such as fatigue, stress, and distraction.Among these emotions, fatigue is one of the most significant ones that lead to an accident or incident on the road.Some studies have demonstrated that drowsiness accounts for 16% of all crashes and over 20% of motorway crashes ( (Fasel & Luettin, 2003).The key to resolve the problem is to developing a system in which can recognize driver fatigue.Around this idea, many studies have been done in both the visual field and non-visual field.
Driver fatigue is closely related to brain activity, eye movement, degree of mouth openness, heart rate, body posture, skin conductance, and so on.Although we can detect driver fatigue by any of them, it is very important not to disturb the driver when getting the relevant data.Therefore, we use a camera to get the driver's image, which will not disturb the driver.Froma visual viewpoint, drivers in fatigue exhibit certain visual cues such as eye movement, head movement, mouth opening and closing, and facial expressions that are easily observed from an image.In this direction, much progress has been achieved.Images taken by a normal camera vary according to light conditions.Zhu, et al. (2002) give an algorithm for eye tracking by combining the right-pupilbased Kalman filter eye tracker with the mean shift eye tracker.This combines filtered accounts of the dynamics of eye movement with mean shift tracking of the eyes based on the eyes' appearance.Ji et al. (2004Ji et al. ( & 2002) ) developed this algorithm for when the pupils are not bright due to oblique face orientations, eye closures, external illumination interferences, or sudden head movements.Wang et al. (2005) presented a method of detecting mouth motion to determine driver drowsiness, and Furugori et al. (2003) measured driver fatigue based on postural change.Despite compelling reasons to use these different methods, we believe that the best information can be obtained by tracking the state of the eyes.With the help of an IR camera, we can separate the rectangle of the eyes from the image more easily.
The remainder of this paper is organized as follows: section 2 introduces some related works.Our algorithm is presented in section 3, and section 4 gives out some of our experiment results.Section 5 is the conclusion.

RELATED WORKS
In the last two decades, many countries have begun to pay attention to driver safety problems and to investigate the mental states of the driver as related to driving safety.Also, warning systems for drivers are also being developed (Onken, 1994;Yammamoto & Higuchi, 1992;Ericksson & Papanikolopoulos, 1999;Singh & Papanikolopoulos, 1999;Betke & Mullally, 2000).In these papers, driver fatigue was detected based on image processing techniques for driving safety.The authors mainly used the symmetric property of faces to detect facial areas on an image.Then, they used pixel difference to find the edges on the facial region to locate the vertical position of the eyes.When the edge detection method could not easily locate ocular locations accurately, they used a threshold to improve it.After finding the approximate eyes positions, a concentric circle template was designed to locate the exact locations, and the template was used for tracking.Face symmetry is an obvious feature for an upright face.
However, facial symmetry usually fails to locate the correct face position when the face tilts, rotates, or is shadowed.Instead of using symmetric central line method, Singh and Papanikolopoulos (1999) used the Gaussian distribution of the skin colors to distinguish skin and non-skin pixels for face detection.They also built a database of eye images as templates for eye detection and location.Although the Gaussian distribution of skin colors based on the RGB color model was used to predict skin quite well, the method does not work well under dark circumstances.In addition, the eye images in the database are impossible to match with all drivers' eyes, thus reducing the detection accuracy.
In order to make the system more robust under variable lighting conditions and facial orientations, Ji et al (2004Ji et al ( & 2002) ) proposed a real-time method for eye tracking, based on combining the appearance-based methods and the active infrared (IR) illumination approach.However, their system prototype of image quality requirements is excessively difficult and is not suitable for practical application.Because of reductions in the picture quality in the situation, traditional eye localization algorithms, such as the histogram (Sobottka & Pitas, 1997) and hough transformations (Chow & Li, 1993) method cannot achieve good results.Traditional eye localization methods such as the histogram match algorithm also do not obtain good results because the gray level of the eye part in the picture as determined by an IR camera and compared with ones taken by a normal camera, simultaneously, fail because the two eyes are not always in the same level line.Moreover, because the bright-pupil usually has a circular characteristic, some authors use the edge detection first and then use the hough transformation to find the human eye position.However, this method is time consuming and performs worse when the bright-pupil's circular characteristic is not obvious; it is also unsuitable in practice.
Therefore, we propose a robust locating algorithm based on our Round Template Two Values Matching (RTTVM) algorithm, which our experimentation proves to have fast and good results.

OUR SYSTEM MODEL
Suppose the size of an image taken by an IR camera is N M × , and the rectangular area ( f R ) of the driver's face is L K × .In the usual situation, our camera is fixed to be motionless; therefore, it may obtain the driver's face rectangular region through difference operations on the video frequency sequence.At the same time, in most situations, the part gray level of the background is low, whereas the gray level of the prospect (the driver's face) is high.This may help us in locating the driver's facial region (k, l expresses the region's central coordinates) accurately.Also, because the eyes are located in the upper half of the human face, we may search the eyes' pixels in the region e R with the central coordinates (k, l/2) and size The round template is located at center of the rectangular template.First we count the pixel number w P in the rectangular template region, in which the gray level is greater than τ .Then, according to the round area formula, we calculate roughly the bright-pupil's possible size, defining its bright-pupil's radius as: The radius we get the first time is usually somewhat larger than the actual value because w P may contain some pixels with a gray level higher than τ in the rectangle fringe field.Therefore, in the circle region, we take c R as the radius (position of the circle center is unchangeable), and we count the pixel number ' w P , in which the gray level is greater than τ and calculate the bright-pupil's radius value ' c R once more. where where V (i, j) means the gray value of the image at pixel coordinates (i, j).

EXPERIMENTAL RESULTS
The three images are three parts of a driver's face ( f R ) from full surveillance images captured by our IR camera in a car.The red rectangles show the possible region e R for the eyes.The middle cross divides e R into el R and er R , and the other two crosses show the locations of the bright-pupils.We did our analysis using programs written in VC++6.0 and ran on a PⅢ-800 MHz computer.
As a result of the influence of light on the bright-pupils brightness, the sizes of the three images are all different.
In Figure 1 (a), the driver is looking up to the right; in Figure 1 (c), the glasses produce a reflection that is similar to that of bright-pupils.However, as a result of our consideration of the bright-pupils' size, region, symmetry, and so on, our algorithm is able to obtain a good location.This also confirms that the RTTVM algorithm has a good robustness.At the same time, we also compared our results with those from the histogram algorithm and the hough transformation algorithm.

CONCLUSIONS
Our RTTVM algorithm is suitable for use in locating the bright-pupil in a driver surveillance system, as it takes a short amount of time and is quite accurate.However, this algorithm lacks handling ability when the brightpupils are too small or oversized.We will research this problem in future research.
to the bright-pupil's radius ' c R , we divide the rectangular template into two sub-regions: the round template region c σ and the non-round template region bσ .We define a match function as: We first use the distribution density of the edge to separate e R into two areas el R and er R , which represent the areas the left eye and right eye are located in.Then, we use the formulas in equations (3) to (7) to find the two positions that most closely match the match function in region el R and er R .The positions having the maximal value max M are the eyes' locations.

Figure 1 .
Figure 1.The red rectangles show the possible two eye regions e R .The middle cross divides e R into el R and

Table 1 .
Table 1 lists the contrasting experimental results, which indicate that the RTTVM algorithm performs slightly better in speed and accuracy.RTTVM algorithm results compared with the histogram algorithm and the hough transformation algorithm