How can you get 3D from a photo? Or better yet, 3DTV from TV?

How can you get 3D from a single photo? Or even better yet, get 3DTV from plain old TV?

Well, the clever scientists at MIT have a way — using a “coded aperture” single-lens camera:

Image and Depth from a Conventional Camera with a Coded Aperture

This is a normal, “iris” camera aperture, and next to it, a “coded” aperture:

Normal aperture

Normal aperture

Coded aperture

Coded aperture

This sort of thing really isn’t new — here’s a picture of a “production coded aperture” used on ESA’s INTEGRAL (The International Gamma-Ray Astrophysics Laboratory):


IBIS Coded Mask

Typical lenses don’t work real well with gamma rays and high energy x-rays, so engineers have to get creative — hence the big, thick coded aperture mask above. This acts as a “lens” for high-frequency radiation. Resolution is pretty limited in this case, even going through all of this considerable trouble. But if you use a coded aperture mask like this with visible light, you get an added benefit.

In almost every image taken with a camera with a wide aperture, one gets a blurry image. In essence, every point source of light outside of the field of focus gets blurred by something akin to having a pinhole camera, with too-wide of a hole. This “blur” takes the shape of the aperture itself. From MIT’s site, here are images of a single point of light, taken through a camera using both the normal and coded apertures:

Normal aperture

Normal aperture

Coded aperture

Coded aperture

The real problem is, images are not single points of light — an image taken with a wide aperture is a mishmosh of blurred point sources, all merged together. If one uses a circular aperture, it’s really hard to figure out the amount of blur for each pixel. But it turns out that if you “stack the deck” with a coded aperture, figuring out the amount of blur for each pixel is not impossible — it’s still hard — but much more tractable. This is essentially a specific case of “blind deconvolution” — you don’t know what the blur IS, but you know (or assume) something about it generally that lets you reduce the dimensionality of the equations needed to solve the problem.

The linear equations required to solve this kind of problem make my brain hurt, and I won’t repeat them here. If you’re really interested in Blind Deconvolution, I really suggest looking at Eurasip — which has the benefit of offering plenty of papers on the subject that DON’T require a several-hundred dollar-per-annum subscription NOR a $30-per-paper download charge. You can get at Eurasip for free (as in beer) here:


Okay, big deal — let’s say we now know how much each pixel is blurred. What does knowing the amount of blur tell you?

The amount of blur, or “how much out-of-focus” that particular pixel is, tells you how far away from the focal plane of the imaging system it is. In short, it tells you how far away it is.

MIT’s results are extremely impressive, considering that the aperture itself and the camera lens housing was cobbled together by very crude methods (hacksaw, razorblade, cardboard, oh my):

Coded aperture camera image

Coded aperture image

Depth Image

Depth map from image

Ultimately, it’s the post-processing math, in concert with the selection of coded aperture, that makes the magic happen.

Some time later, we’ll talk about what sort of information video can add, on top of depth-from-blur (depth-from-focus is the more “proper” term).


~ by opticalflow on February 18, 2009.

2 Responses to “How can you get 3D from a photo? Or better yet, 3DTV from TV?”

  1. very interesting info, i will be “monitoring” your blog 🙂
    best, and keep it coming 🙂

  2. Thanks! Many people tell me that I’m a card-carrying member of the “Keepers of Odd Knowledge Society”. In any case, there will definitely be more on this subject in the future…


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: