In Praise of the Fisheye

Publication date: 14 December 2011
Originally published 2011, in Atomic: Maximum Power Computing
Last modified 14-Dec-2011.

If you're fortunate enough to have normal human eyesight, you seem to have a view of the world that's both quite wide, and quite sharp. But actually, there's only a little high-res spot of "macular" vision in the middle, surrounded by much lower resolution. That spot in the middle of your field of view isn't just where you're directing your attention at the moment; it's the only sharp vision you have.

Your whole horizontal field of view, from the super-blurry peripheral left edge to the similarly blurry right edge, approaches 180 degrees. (If you happen to be an animal with eyes on the sides of your head, you may manage damn near 360 degrees, at the cost of little to no binocular 3D vision at all.)

The portion of your total field of view that you perceive as sharp is a lot smaller than the full field, though. In photography, a "normal lens" is one that approximates the perceived view of a human standing in the camera's location. A normal lens has a left-to-right field of view of only about 45 degrees.

All of this is complicated enough already, but then people have to go and start playing computer games.

3D games are famous for giving you an odd view of the world. If the field of view is wide enough to give you decent situational awareness in an action game, then objects at the edges of the screen will appear oddly stretched.

(A very narrow, 45-degree field of view in Skyrim. This game defaults to a wider, but still unusually narrow, 75° horizontal field of view; that may work better for people playing on consoles, but PC gamers should probably set it wider, by bringing up the console [no relation to the aforementioned consoles] and typing "fov [number]".)

(A 90° FoV.)

Anybody who's fiddled with field-of-view settings in games will be familiar with this; extreme settings give you a sort of land-speed-record view of the world...

(120°)

...with massive radial stretching and things straight in front of you seeming weirdly far away.

(160°, as wide as Skyrim will go.)

Actually, though, there's no distortion at all.

When I'm sitting here with my eyes about 90 centimetres away from my beloved 30-inch monitor, the monitor subtends roughly a normal-lens 45-degree portion of my field of view. If I then set my field of view in a game to 45 degrees, there's no stretch effect at all.

But a 45-degree FoV is not very useful; it feels like trying to view the world through a toilet-paper tube. (Through binoculars made out of about half of a toilet-paper tube on each side, to be fair. Standard military night-vision goggles still commonly manage only about a 45-degree field of view.)

About a 90-degree horizontal FoV is the standard for action games. But then, there's that stretchy effect again.

In photography, this effect is called "perspective distortion"; it happens when you view any image that shows a field of view different from the amount of your own field of view that the image takes up.

Pretend your monitor is a literal, physical window into the game-world. If the field of view in the game is set the same as the field of view you'd get through a physical window the size of your monitor and the same distance away from your eyes, then everything looks fine. If you widen the field of view - let's say, by a factor of two - all you have to do to make everything look fine again is halve your distance from the monitor.

Try it. Run some game with FoV adjustment, wind the field of view up to 120 degrees or something, and move much closer to the screen than your mother thinks is healthy. When the amount of your actual biological field of view that the monitor covers is the same as your FoV setting in the game, there won't be any distortion.

(This even works for multi-monitor gaming, which has a long-standing problem with field-of-view settings. If a standard three-screen multi-monitor PC is displaying a game on, as far as the computer knows, one very wide yet strangely short screen, there will be severe stretching effects on the side screens. This problem can be solved by rendering three separate camera views, effectively giving three separate magic windows onto the game-world.)

It is, of course, completely impractical to play a game with your head very close to the monitor. Nobody wants to stare at a screen that's ten centimetres away from their eyes in the middle and thirty at the corners. We're going to have to wait for affordable wraparound curved screens, or head-mounted displays that create the same effect, before super-wide view angles will be useful.

In the meantime, a better option exists.

All normal 3D games use similar simple "rectilinear projection", where straight lines stay straight, but get longer and longer as the view gets wider. There are many other projections, though, which trade distortion - straight lines aren't straight any more - for a better view. Standard fisheye, for instance, gives a distinctive circular distortion effect that lets things in front of you seem realistically close and not very distorted, while the outer portion of the image is a less faithful, but very wide, view. Which is actually pretty much what your eyes already do.

(Door peepholes use low-quality fisheye optics.)

But everything, except for the occasional Xenomorph's-Eye View or CPU-intensive add-on for Warsow, is still rectilinear. Until mainstream FPS players find themselves getting pwned by people who've clicked the "fisheye" button, there'll be no demand for said button to exist.

It's not easy to do, either. If you want to implement fisheye rendering in a quick demo program where the camera just flies around in a checkerboarded room, modern CPU power makes it quite easy. If you want to do the same thing in an actual game, though, you will immediately run into the fact that current 3D game rendering, from the raw hardware to the most abstracted development-kit tools, is a vast teetering pile of hacks and kludges shot though with numerous assumptions, one of which is rectilinear rendering.

Even something as simple as widening the rectilinear field of view can create an instant problem: If the game renders the player characters hands and/or weapons on the screen, then widening the view will make your arms stretch bizarrely, along with the rest of the image. To avoid that, many games add yet another cheat to the collection, and render hands and guns and swords and such as a fixed extra layer on top of everything else, which ignores field-of-view entirely. Try changing the FoV in Half-Life 2, for instance; you can stretch the world into unrecognisability, but your crowbar will look exactly the same.

Despite all of these obstacles, I hope someone creates game projections more interesting than rectilinear. If only because fisheye-versus-stereographic-projection flame wars can only be more interesting than Nvidia versus ATI.