So when do we start doing eyeball tracking?

[jducoeur]

And I don't mean "how many people have read this page?" -- I'm musing about, literally, when we're going to have our computers start really paying attention to the user's eyes.

I was thinking about that as I was proofing my last entry. When I wanted to change a word, I exited proofing mode, used the mouse to go to the right position, backspaced over the word, and used the keyboard to re-enter it. But what I really wanted was to simply say, "strike 'fill' and replace it with 'feel'", and have the computer know which word I was talking about.

I'm sure it's *possible* -- it just requires combining good speech-sensitive editing capabilities with a camera sensitive enough to track the user's eye movements, and well-enough aligned with the screen to serve as a pointer. (I envision an alignment UI sort of like the one you get when you boot a Palm PDA -- "Look at this dot. Now look at this dot.") Not easy, but I suspect it's feasible with current high-end technology, and ought to become more straightforward as camera tech improves.

The mouse is a nice tool, and good for precision work, but a hassle to use most of the time -- you have to stop whatever you're doing with your hands to use it. Geeks use keyboard shortcuts instead, but I suspect that we're going to see the emergence of alternatives better-suited to mass use. Minority Report style virtual displays are one approach (especially if you think the keyboard itself will go away), but eye tracking seems to have a lot of potential for very intuitively doing what you want...

Comments

[metahacker.livejournal.com]

The tech for eyetracking is currently good enough to do this. However, that's not consumer tech, that's experimental tech. Word-level precision is probably doable assuming you don't use an exceptionally small font.

One of the real issues, however, is that "pointing" with your eyes turns out to be very non-intuitive and not at all how the eyes work. You don't continue to look at something after you've mentally marked it; your eyes flicker across the page, distracted by a hundred things a second. It may be possible to fix this with training, but past studies (can't find links right now) haven't been too favorable. So basically it's an area waiting for a good design solution; how to figure out what the user is *thinking* of, based on where their eyes went...

[bkdelong.livejournal.com]

There's been a lot of research in eye-tracking - I remember seeing an excellent poster presentation at WWW6 back in 1997. Gesture Recognition ala Minority Report has come a long way though I'm not sure if we'll have the augmented reality displays anytime soon.

One should be able to use any PDA/phone camera that records video. i.e. during that preview mode you'd see it attempt to zero in on your retina and hopefully do a bit of recognition as well.

I honestly wish there was an Open source voice-recognition platform where I could continuously hone the voiceprint and take it around or import it into other applications. IMO it's still not really there yet and I want us to hurry up and get to subvocal recognition - though some people see subvocalization as a problem rather than an enhancement.

[talvinamarich.livejournal.com]

Already, if you wind up in court on any of an hundred or more offenses, what you Googled for is going to come out. "He was researching how to make a bomb." "He scanned Google Images for blondes. The victim was an attractive blonde."

Next, they will be ransacking your computer for *what your eyes lingered on most*, and using it against you.

I'm gonna head down to Chinatown and buy a used abacus.

No, new. The last person might have had something on his fingers when he used it, and they'll try to pin it on me!

--Talvin