Eye tracking has spent thirty years inside expensive specialist hardware. It now works on an ordinary webcam, and there is a page online where anyone can test it in about ninety seconds.
Watch anyone use a computer and the same delay repeats, hundreds of times an hour. They look at the button. Then they move the mouse to the button. The eye arrives first, every time, and then waits for the hand to catch up. Bigger targets, keyboard shortcuts, autocomplete, gestures: much of interface design exists to compensate for a pointer slower than the person holding it.
For most of computing history there was no alternative. Knowing where someone was looking required a dedicated device with infrared illuminators, a price in the thousands, and a controlled environment to stay accurate. Tobii, the Swedish company that dominates the field, built a serious business on it. For those with ALS or a spinal injury who speak by looking at letters on a screen, that hardware is life changing. It is also gated behind a purchase order.
The gate is coming off from two directions. From above, the platforms: Apple ships Eye Tracking as a standard accessibility feature on iPhone and iPad using nothing but the front camera, and Vision Pro made gaze the pointer and the pinch the click. From below, the open-source stack: face-tracking models that were research projects a few years ago now run in a browser tab, at camera framerate, for nothing.
The demo
Which is where Lugano comes in. Mont3, a software studio in the city, has put a working example online. The page asks for the camera, walks the visitor through about a minute of calibration following a dot around the screen, and then a marker starts tracking their gaze. Two small games are attached: in one you blink to zap a bug you are looking at, in the other, a version of Snake, you steer by looking where you want to go. There is no install, no account and no upload: the camera feed stays on the visitor’s machine, the model is trained and stored in the browser, and closing the tab ends it.
Mid-calibration. The green ring is the target the user is asked to look at, the blue marker is where the page thinks they are looking, and the strip on the right is the iris tracking it is reading. Credit: Mont3
How it works
The obvious way to build this is to feed camera frames to a model and ask where the person is looking. It works impressively until the head tilts, at which point it collapses, because what the model learned was a posture.
So the picture is discarded almost immediately. Google’s MediaPipe FaceLandmarker, an open-source face mesh that runs on the device, returns 478 points per frame, including a ring around each iris, and that iris tracking is its own piece of published research. About a dozen numbers survive: where each iris sits inside its own eye socket, the yaw, pitch and roll of the head, and the distance between the eyes as a proxy for viewing distance.
That is the trick that survives contact with a real person: iris position measured inside the eye cancels out where the face happens to be in the frame, so leaning back in a chair no longer destroys the estimate.
From face landmarks to a point on screen. Credit: Mont3
One HTML file
It is worth being precise about what this costs to make, because that is the part that changed. The page is a single HTML file: around 1,100 lines, 51 KB, one script tag, no framework, no build step, no server, no npm install. Plain JavaScript, running in whatever browser the reader already has open. The one external dependency is Google’s face mesh, pulled from a public CDN in a single line. The regression at the end is 65 terms of hand-written linear algebra, solved in closed form, with no machine learning library involved.
A competent developer could reproduce it over a weekend. Fifteen years ago the same result was a funded research project with a hardware budget.
How accurate it really is
Webcam gaze lands at roughly 2 to 4 degrees of error after calibration. At laptop distance that is somewhere between a large coin and a credit card: dependable at the level of a 4×4 grid of zones. It is not a cursor, and anyone claiming otherwise on consumer hardware is selling something.
That limit is more interesting than it sounds. Plenty of interface needs no pixel precision: which of six tiles has your attention, whether you have finished a paragraph, whether you are still in front of the screen. Dwell selection, the technique assistive technology has used for decades, works fine at zone accuracy.
Where this goes
This is what it looks like when a technology leaves its enclosure. A capability sits for years inside expensive hardware, serving the people who cannot manage without it, until the sensing gets good enough for commodity parts and the models go open. The first beneficiaries are the ones who always needed it, for whom the gap between a device costing thousands and a page costing nothing is the gap between speaking and not. What follows is more ordinary: interfaces that notice where attention went, and stop asking the hand to explain it.
Ninety seconds and a browser are enough to see how far along we already are, at mont3.ch/control_with_eyes. Chrome, even light on the face, head an arm’s length from the screen.
Notes for the editor
- Word count: 838 words (body only, excluding title, standfirst, captions and this block).
- Framing: a piece about where eye control is going, with the Lugano demo as the thing a reader can go and verify. The company is named once in the body and once in the closing URL. No links to its commercial pages, no quotes from it, and the only first-person plural is the editorial one.
- Byline: left blank on purpose.
- Images: two, both supplied by the studio, no third-party rights involved. Image 1 is a screenshot of the page in use, taken by its author. Image 2 is an original diagram made for this article. Credit line “Credit: Mont3”, which can be dropped or moved to a single line at the foot if house style prefers.
- External links used: Tobii, Apple Newsroom, Google MediaPipe documentation, arXiv. All checked and live at the time of writing.
- The demo needs a camera and works best in Chrome. It does nothing without an explicit permission grant and uploads nothing.
Be the first to comment