The Future of 3D Technology - Holography & Holodeck....

The Unlimited Potential of 3D

A Different Perspective for Everyone - Less Technical

Modern 3D technology has largely focused on giving everyone an improved sense of 3D. There are a number of different ways to do this, but almost all of them involve sending two different images to every person. One image shows a scene and the other image shows the same scene but from a slightly different angle. Since we are accustomed to viewing real life from two perspectives at once, this dual-image technique gives us a natural sense of 3D and a more intimate connection to whatever we are viewing. This is the basis of the technology which makes 3D TVs work.

Still, doesn’t it seem like there should be more to 3D? Do you really feel like you’re part of the environment just because things pop out at you more? The answer is both yes and no for most people. You do indeed feel like you are part of the environment because you have a renewed sense of depth and it feels closer to what you’re used to seeing in real life. On the other hand, you can’t move around the environment and focus on things properly, and if you look at it from a steep angle, the 3D looks weirdly skewed because it’s actually still on a flat surface. These gripes aren’t just conditional, they are part a fundamental gap between 3D technology and real life. This gap can be stated in a surprisingly simple way.

How many images are needed to show a sports stadium worth of people what they would see at a real stadium? The answer is two times the number of people (assuming each person has two working eyes, which is actually somewhat unlikely). To truly recreate the visual environment of a stadium means to show every person what they should see out of each of their two eyeballs. It may seem like a lot more images than you need considering that most of those images will be almost identical; but that’s not really true when you consider how the minor changes between two people slowly add up to completely different perspectives on the other side of the stadium. Also consider how looking around is completely different for each person because they each see different people nearby (surely two people looking at each other won’t see the same thing… unless they’re identical twins, I suppose).

Every person who looks at a dual-image 3D TV sees the same two images. This presents a nice means of fully conveying a sense of 3D without an extremely high data requirement. Sending out two images only takes twice the data of one (or less if each one is reduced in resolution or frame rate). Still, we know from the stadium example that each person really should not see the same set of two images in any situation, because that’s not what it’s like to be in the real world. To truly call something 3D, we must convey the sense of individual perspective.

Let’s go back to the sports stadium. How many different views can we have of the sport being played there? 1, 2, 3? A trillion? In all honesty, we could have infinite different perspectives on any situation because we could always find a new perspective between two others no matter how close they are. The idea of infinite perspectives may not have a lot of meaning for our eyes because we can’t distinguish infinitely small distances, but that’s actually a good thing because it means we won’t have to make infinite images. Instead, we can just make as many as it takes for a moving person to observe a continuous change in imagery with both eyes.

Making enough images to give a person a sense of continuity is not as easy as it sounds, because a viewer’s distance from a scene can change the requirement. So let’s restate our attempt to make a full 3D environment as “making as many images as possible and spreading them out as evenly as possible.” There are at least two major goals that branch from this endeavor. The first is to make an image for every potential viewer spread out horizontally. The second is to make an image for every potential viewer in any direction. There are any number of steps required to reach these goals (e.g. 5 horizontal perspectives, 100 total perspectives from all directions, etc…), but the end results should strive to reach one or the other.

Let’s start with the first type of 3D environment, one that is spread out horizontally. Think about being at an outdoor wedding. Every person at such a wedding will probably be seated at the same level. This means that each of the people watching can only get a different view by changing seats at the same level. This is a pretty common occurrence for many environments because we don’t move up and down nearly as often as we walk around. Having many horizontal perspectives works well with our eyes because they too are arranged horizontally. The biggest drawback to a system that shows lots of horizontal perspectives is that we need to show that many more images at the exact same time. That’s a lot more information; but the increased 3D effect will be well worth it.

The next type of display is one made for every perspective. This type is suited to viewers who can see a scene from many different levels or can otherwise move up and down (even standing up can cause a noticeable difference in perspective). This is the case in the sports stadium because the people watching the game need to see over each other to look down at the action. To recreate this kind of environment accurately, we need to make as many images as we can from all different directions. If someone at a sports game wanted to see it from an overhead view by flying in a blimp, they should be able to see the same view on a 3D display that recreates it. The drawback to this type of 3D display is that it requires a vast number of images, but the additional weight of the data is nothing compared to the 3D freedom it burgeons.

Holograms are one type of display that can create a different perspective for every viewer. They are essentially laser recordings of light from many different angles (although they aren’t actually stored as images, so they can only be recalled with the same type of laser used to create them). Because they have been around for so long, research and development in this area is continuously advancing. Even changing holograms have started to appear. Unfortunately, the advancements are still a matter of ingenuity, so there is no way to estimate when holographic televisions and computer displays will be commercially available.

The only other option currently available to create fully 3D environments on a flat surface is to use the techniques described in “How to Make a Holodeck.” The basic concept is to create the many perspectives we need from as many miniature curved displays. The jump from many perspectives to mini-pictures on the screen may seem somewhat mysterious, but that’s because the approach of many perspectives is slightly flawed. Consider how I stated that there are infinite places from which to view a scene. This is not just true in an area around a scene, but in the entire volume around it. Imagine you’re at the outdoor wedding looking at the bride and groom. You take a step back. You’re still looking at the environment from the same angle, but the perspective images you perceive have definitely changed. There is at least one way to account for the infinite volume of perspectives, and that is to know the relative placement of the flat screen display that will be used to recreate the environment.

Try to imagine how you would put millions of images on top of each other on a flat screen (one for each perspective). You can’t, it just doesn’t make sense. So the trick is to only recreate the perspectives that occur at the screen. This reduces our task from creating a ‘volume’ of images to creating a tiny perspective image for each point on the screen. This can be done by putting each perspective image on a curved surface, then shrinking it around the viewing point. “How to Make a Holodeck” describes several means to do this, along with some unexpected results of the approach (such as the ability to cel-shade 3D objects from every angle simultaneously without the shading covering the characters).

The goal of 3D technology is not just “to create two images at once”, but to recreate all of the visual and sensory aspects of the real world. In terms of 3D displays, this means successively integrating each and every aspect of a real 3D environment. This task will continue until no one perspective can distinguish a 3D display from the reality it strives to recreate.

4/22/11

Change is Silver

Author of “How to Make a Holodeck” (5Deck.com)
~The book that describes exclusive new techniques for arbitrarily generating “infinite images.”
Creator of Unili arT (UniliarT.com)
~Random designs on random products speak volumes in the right places.

The Unlimited Potential of 3D

Creating Infinite Images on a Single Surface - More Technical

Note: this article can be read alone or act as bridge for understanding the difference between 3D TV and the “4V” technology presented in the book “How to Make a Holodeck” from 5Deck.com.

Modern 3D technology is primarily focused on creating two images from two different perspective points, then splitting them at some point so they each reach a different eye of each viewer. The method of splitting can occur at any point between the screen and the viewer’s eyes, but no matter what technique is used there are still only two images worth of information at any given time. It seems pointless to ponder the possibility of more than two images because we don’t have three or more eyes, but improving the quality of a 3D experience is more than just improving the 3D perspective of a single scene. It is also a matter of improving the range and accuracy of that scene.

What if there were a thousand viewers watching a single 3D screen? In theory you could make a thousand different images, one for each of their eyes. Does that make any sense? Actually, it does, because each person viewing something in real life cannot see exactly the same thing as another person at the exact same time. To do so would mean that two people’s eyes would have to overlap. In fact, even one person cannot see the same thing with each of their two eyes (for the same reason). So the question for 3D technology is, “when is it important for two people looking at the same thing see it differently?” The answer is “in real life.”

Each eye of each viewer watching a dual-image 3D TV will see the same two images as every other person, even if they are skewed from the angle of view-but if the objects on the screen were truly in 3D, no viewer’s eye would see the exact same image or even skewed versions of the same images. This is where the future of 3D lies: in sending out a different image to every viewer’s individual eyes. The task is easily stated, but when approached literally it can seem like a monumental, if not completely impossible task.

Consider members of an audience of several levels looking at an opera. At any given point in time, there are as many perspectives in the room as there are audience member eyeballs. Some of these eyes can even see behind things that others cannot (like stage crew if you’re far enough to the side). So the question becomes: how many different perspectives can we fit in an opera house? You might imagine a room stuffed with people from wall to wall and floor to ceiling. That seems like a lot of people already, but it’s actually only the limit of the building, not the limit of perspectives.

What if we have the opera outside (and apologize in advance for the reduced acoustics)? Now how many perspectives can you have? If you stacked a sphere of people lying on top of each other around the opera (in little tube rooms maybe, haha), you could probably get millions of perspectives. And what if that sphere was further away? Billions? Further? Infinity is the final answer (or at least a ridiculously high number if you consider that any given environment only releases a finite number of photons). A single 3D scene has as many perspectives as there are locations in which light can be received. This means to reproduce a 3D scene completely, we need to record and then reproduce infinite images.

Making “infinite images” is impossible (as you may well have guessed), so instead we should try to make “as many images as possible.” The first way to do this is to consider the fact that people looking at a scene are usually all at the same level. In the case of the opera this was not true, but it is true at a single-level play without stadium seating. Here everyone will be content to see different perspectives from different seats at the same level. No one will be climbing up and down ladders or swinging from chandeliers (I suppose it depends on the play…). This means that our first approximation to “infinite images” should be “as many images as possible for viewers who only move horizontally.”

If we could create a multi-image display that showed a different image to every eye of every viewer that walked past it, then we have fully recreated all of the 3D perspectives available in a single-level play. This means that anything from the play can now be seen accurately from any horizontal position. Displays that can do this are typically called “single parallax” or “horizontal parallax” displays (sometimes they are called “volumetric”, but this is actually a misnomer as the objects only appear to have volume along one axis, but still skew as flat images along the other axis). Such devices require quite a lot more information than dual-image displays (an entire extra dimension worth of data), but that’s the price of catering to more people’s unique perspectives.

Is this good enough? Kind of… for most purposes people only view a screen from approximately one height and multiple horizontal positions (like the two eyes of a single person or multiple people sitting on a couch). But what about viewers who like stadium seating in a play or multi-level balconies in an opera? Most of the viewers in those cases will see the equivalent 3D display as if they were looking at skewed flat images-a confusing effect when both eyes already see realistic 3D objects along one axis. Even viewers at the proper level will notice the lack of vertical differentiation if they move up or down just a little.

So for at least some situations we definitely want more vertical perspectives. Since we already know that all perspectives require infinite images, let’s just jump to the new goal “to create as many images as possible for each horizontal position and for each vertical position from there.” Woah! That’s a big jump. The data we need to go from dual-image to a row of images is already a lot, so you can imagine how much more we need to create a entire new column of images at each point along that row. That being said, there is one way to do it that has been around for decades. Holograms.

Holograms are like “3D film” in the way they record information. They use a material and light specific technique to uniquely capture every angle of light incident on a surface. The basic concept is to use a laser where all the photons are lined up and then record how each of those photons changes when it hits an object in a 3D scene (the original beam is split so that the changes can be recorded against the original). In this sense, a Hologram naturally records each angle of light independently and can thus reproduce each angle of light independently. Even though a hologram is not normally thought of as “infinite images,” the end result is exactly the same.

Holograms have a great deal of potential, but many of their limitations lie in the fact that they are not recorded ‘as images’ but rather as a material/method-specific pattern (you can’t record a hologram without a laser hitting the recorded objects and you can’t see a hologram without the original laser configuration). Commercial 3D displays that use holograms may be a ways off, but the technique is still the only developed means of creating the “infinite images” we need for a complete 3D display.

The book I wrote, “How to Make a Holodeck,” is a conceptual starting point for an alternate approach to creating infinite images. The basic concept is just to create a dense array of as many images as possible on a single surface (the images are not actually from a viewer’s perspective as I’ve described here, but from the individual perspectives of the points on the intended screen). In other words it has the same end result as holograms, but uses a much more literal approach. The technique is hampered by extremely high data requirements; but those data requirements also allow for complete, arbitrary generation of 3D scenes. This means that recorded environments can be sent digitally and simulated environments can change instantly. 3D technology in this form would not just give a better sense of 3D. It would recreate the opera for every member of its audience.

Note: The term “3D” used in this article refers to three “spatial dimensions.” In “How to Make a Holodeck,” I use the term 3D with time as a possible dimension. Because almost all 3D situations involve time, I prefer the term 4D for most instances where people use 3D, but I still use 3D for ease of understanding.

4/22/11

Change is Silver

Author of “How to Make a Holodeck” (5Deck.com)
~The book that describes exclusive new techniques for arbitrarily generating “infinite images.”
Creator of Unili arT (UniliarT.com)
~Random designs on random products speak volumes in the right places.