Event-Based Kilohertz Eye Tracking using Coded Differential Lighting

PDF  BibTex

Abstract. Pixels in an event camera operate asynchronously and independently, reporting changes in intensity as events - tuples of (x, y) position, polarity p and timestamp t at microsecond resolution. Event cameras operate at low power 5mW and respond to changes in the scene with a latency on the order of microseconds. These properties make event cameras an exciting candidate for eye tracking sensors on mobile platforms such as VR headsets, since these systems have hard real-time and power constraints. One proven method for eye tracking and gaze estimation is corneal glint detection. We exploit the fact that corneal glint tracking only requires a sparse set of pixels in the image, by making use of the natural sparsity of event cameras, which only detect changes in the scene. To enhance this effect, we design an illumination scheme, Coded Differential Lighting, which enhances specular reflections, suppresses all other events, and solves the light-to-glint correspondence. This is the first purely event-based corneal glint detection and tracking algorithm, which operates on standard hardware at kHz sampling rate.

Reference: T. Stoffregen, H. Daraei, C. Robinson, A. Fix, “Event-Based Kilohertz Eye Tracking using Coded Differential Lighting”, Winter Conference on Applications of Computer Vision (WACV), 2022.