Nintendo Files Patent for "Eye Tracking" Device to Enable 3D Experiences on 2D Displays
Posted by Thomas Whitehead
Also proposes enhanced experiences through smell, temperature and light
Nintendo may have submitted some rather peculiar patent applications in the past, but the most recent may have some foundation in reality, indicating a potential move into Virtual Reality for the company. It proposes a wearable device — potentially glasses — that is detected by a camera, and adjusts your viewpoint to give an illusion of 3D on a 2D display, which would then also be used in various ways to impact interactive game experiences.
You can see a typically crude illustration above, though the explanation of how the system will work places this technology somewhere between Microsoft's Kinect and VR headsets such as Oculus Rift and Sony's Morpheus. The technology to give an impression of 3D without an autostereoscopic screen (like that on the 3DS) or 3D glasses is not new, as there are already some smart devices that can track your eyes to achieve similar results. Nintendo's proposal appears to aim to take this technology much further in terms of interaction — examples given include dodging objects by moving your head, adjusting your view to see something on screen that others can't (an idea suggested as potentially suited to multiplayer experiences) and also the ability for in-game characters to react to where you're looking on screen.
There are more ambitious ideas for immersion, too. Additional devices could expand the experience further, while emotional state could also be measured.
 Enhancing the VR experience Additional output devices that enhance the experience can be provided. For example, we can put light out that is correlated to the image to provide "ultra wide field of view correlated lighting." Given that your eye does not see clearly in the periphery, this could still be useful and interesting.
 Additionally, smell is a very strong sense. There may be some ways to produce aromas for a very strong experience.
 Virtual wind could enhance the experience.
 Temperature: blowing cool air on your face.
 Physiologically comfortable stereo viewing is a way to prevent headaches. If you find a little spec on your windshield, focus on that and then far field and then back again. Eye strain happens quite quickly. Lots of folks in the past require the users to focus far field and close up, but this can cause headaches. We can stay on one side of the focal point cone, to provide higher level of comfort.
 Detect Emotions via monitoring mental state. Brain wave detection, detect eye movement, heart rate monitor or the like can be used. If we provide goggles, we can also provide detectors (electrodes) fairly easily.
It's certainly intriguing, and if it becomes a reality — which is far from certain, as is always the case with patent applications — would continue Nintendo's policy of adopting existing technology for new, potentially innovative experiences. It also fits into the rhetoric of Nintendo's senior figures that it's currently unsure of the reclusive nature of headsets such as Oculus Rift — this patent's setup could be more social in the living room — while maintaining recently that it'll continue to seek innovation as an entertainment business.
What do you think of this concept? Does it look exciting to you, or do you feel it won't work well or ever see the light of day? Let us know in the comments below.
Thanks to Otto Petersen for the heads up.