Clearing the VR hurdles
Originally published 2002 in Atomic: Maximum Power Computing Last modified 03-Dec-2011.
OK, forget the flying cars. I just want proper VR.
I never particularly dug Diablo and Diablo II, because I've played Nethack and ADOM and Angband, baby. I want one of those games in full VR (as opposed to the usual presentation), with user-assignable gestures to cast common spells. Including, if you must, an I Dream Of Jeannie crossed-arms-nod to activate the Scroll Of Return.
I want a god game in which I can point to shoot lightning, pinch to pick up a peasant, grab to reshape the landscape.
I want an RTS that lets me just look around the battlefield, not scroll or click on a mini-map, and which automatically highlights the battle group I'm looking at and applies my commands to them.
Unfortunately, virtual reality isn't a very popular concept at the moment.
Oh, sure, there are plenty of researchers plugging away at it, and Ray Kurzweil has lots to say about it (and about the "nanobots" that he expects to create it...). But I can't help but notice that I still can't "jack in".
That's a concept popularised by William Gibson. He, famously, envisaged people connecting to cyberspace via a direct cable link to the brain.
For the benefit of that portion of this page's readership that wasn't born when Neuromancer was first published: Like in "The Matrix", kids. Like that.
The problem with the jacking-in idea is that, barring some very surprising breakthroughs in, um, a whole lot of fields, it isn't going to be happening soon.
Happily, though, that doesn't mean that we're not going to be able to play with - um, I mean greatly increase worker productivity and shareholder value by proactively utilising - some pretty darn compelling virtual reality in the quite near future.
And we need exactly one piece of hardware to make it happen.
Virtual reality was a bit of a fad, ten years ago. I played Dactyl Nightmare on a Virtuality (or "W Industries", as they might then still have been) 1000CS system. Even with only about 30,000 untextured polygons and a shuddery 20 frames per second, it was pretty fun. A similar game on current more-than-a-thousand-times-faster hardware would be a knockout.
But you still couldn't afford it, because head-position-tracking displays (you need head tracking, if you want to be able to look around by moving your head) with reasonable resolution are still quite heftily expensive.
You wouldn't have to pay the $US60,000 that a 1000CS originally cost to get such a Head Mounted Display (HMD) rig happening today. But you're still talking a fair few thousand. And, at that price, uptake of a mass market version of the technology would be slow enough that anybody who decided to make the things would really be wanting a blank cheque from Microsoft, to keep them solvent while they produced their first three money-losing generations of hardware.
This, however, is gonna change.
Never mind fantasies about bodysuits that let you directly control your in-game avatar, and magic Holodeck-ish rooms that somehow stop you headbutting the walls as you move around. Consider, instead, just a good high resolution wide angle head-tracking HMD. No bulkier than one of Elton John's middleweight pairs of glasses. No more expensive than a top-flight video card. For use with a sit-down control rig, not much different from the controllers we use today.
This display is the key technology. Everything else is easy.
Graphics cards with dual monitor outputs are already quite mainstream items. Hardware that can drive two-eye displays, and which can come built into game consoles as well as on PC video cards, won't be much of a leap.
CPU power's forging ahead, too. Budget processors today already have several hundred times as much grunt as the 1000CS could manage. Far more, for some tasks. VR applications aren't, from the CPU's point of view, qualitatively different from normal 3D apps; you just have to render two views of everything, one for each eye.
So none of that stuff is a huge hurdle. It's the display that we've got to get working.
Thus far, not many HMDs have motion tracking, because you don't want motion tracking, for many applications. If you're using an HMD for "virtual screen" applications - heads-up info, watching movies, espionage gadgets like Arnold and Ahnold used in "True Lies" - then you probably want the screen to stay in front of your face at all times, and head tracking is useless.
HMDs also don't have a really wide view, stretching out into the peripheral vision area. Again, this is because it's a pointless feature if you want to display a normal screen image. People can't read with their peripheral vision, and stretching movies out that much makes the whole image very fuzzy, because you don't have enough pixels in the first place. A 720 pixel wide DVD image, stretched out to a mere 90 degree horizontal field of view, will have an eighth of a degree of arc per pixel; for comparison, that means the full moon...
...displayed to cover the same amount of your field of view as it does in real life...
...would only be four by four pixels.
Fuzziness like this doesn't matter in peripheral vision. It does matter when it's in the little sharp macular vision spot in the middle of your view. Which presents a problem, because convincing VR needs wide view as well as head tracking, and doing a whole wide view area at a high enough resolution that the centre of the image looks good would seem to require, at first glance, outrageous image sizes.
Head tracking, by itself, is a bit tricky but not horribly expensive to implement, especially if the user has the decency to sit in a chair and not prance around the room.
And wide vision can be done without monster resolution, if you use clever optics fed by an appropriately pre-distorted rendered image. The original image, viewed without the optics, will seem to bulge in the middle. That's where the high-res section's rendered. The optics flatten out the bulge, and match the image resolution in each area of the picture to the visual resolution of the part of the retina that's seeing it.
A few consumer video cameras have had eye tracking for years now; it allows the hardware to know where the user's eye is pointing, and could be used in VR to put the high-res portion of the display in the right place, provided it can respond fast enough. Presto - you're not wasting power rendering zillions of pixels that the user can't clearly see.
The cheaper solution here, of course, is to just put the high res part in the middle of the image, and let the user learn to turn their head, not move their eyes.
Brute-forcing wide vision by creating a 10,000 by 10,000 pixel image that's sharp all over would do, too, but there'll have to be a revolution in tiny video display technology for that to be an affordable solution.
Wearable display technology, based on little panels and on more exotic tech like laser retinal scanning, is forging ahead. And none too soon, as anybody who's tried to do actual work on a mobile phone screen will tell you.
So I'm hopeful that, some time soon, I'll be able to pop on a jet-black set of Buddy Holly shades, look down, and see Conan's pectorals.
Or Red Sonja's.
I'm a broadminded guy.
