Skip to main content
Part of complete coverage on

A camera that can see around corners

By Ramesh Raskar, Special to CNN
updated 9:05 AM EDT, Sun August 19, 2012
STORY HIGHLIGHTS
  • Ramesh Raskar's team at the MIT Media Lab is re-imagining what photography can do
  • By creating movies of light in motion at a trillion frames per second, camera can see around corner
  • Raskar: Technology is years away from applications in the real world
  • He says it could be helpful in preventing car crashes and aiding rescues

Editor's note: Ramesh Raskar is Associate Professor at MIT Media Lab and head of the Camera Culture research group. He is an Alfred P. Sloan Research Fellow (2009), a recipient of the DARPA Young Faculty award, the co-author of "Spatial Augmented Reality", and the holder of more than 50 US patents. He spoke at the TEDGlobal conference in June. TED is a nonprofit dedicated to "Ideas worth spreading" which it makes available through talks posted on its website.

(CNN) -- A camera that can see around a corner?

I know this sounds like something in a sci-fi movie or a superhero comic, but this is a real-world technology we've made possible with a camera that is aware of the travel time of light, an imaging technique that can create movies of light in motion with an effective rate approaching a trillion frames per second: the speed of light.

Before I joined the MIT faculty in 2008, I had done deep research in "computational photography," a field of new imaging techniques dramatically improving the capture and synthesis of photos. But, I knew there was more to photography than just depicting what the eye can see. I wanted to create a camera that could see beyond the line of sight. The speed of light isn't infinite: light travels about a foot per billionth of a second.

Photography could make X-rays obsolete

If I could build a camera fast enough to analyze light at high speeds in room-sized environments, I knew we could then create cameras to solve major problems in scientific and consumer imaging, and enable completely new functionality.

I spoke to top researchers in ultrafast lasers and photonics to understand what was currently possible. When I did, most of them asked some version of: "Why? Why spend years building a camera to look around corners when no commercial application is screaming for it and no funding agency has a call for it?" In addition, it's rare to shoot light pulses and analyze at such high speeds in large environments. Ultra-fast imaging experiments are usually limited to centimeter- or smaller-size samples.

I continued the work and in the spring of 2008, with James Davis from UC Santa Cruz, wrote a proposal that laid out the mathematical foundation and various experimental solutions for exploration.

TED.com: Boaz Almog 'levitates' a superconductor

I also began working with Media Lab graduate student Matthew Hirsch to build a prototype, hoping that we would have something to demonstrate within a few months. The grant proposal was rejected for administrative reasons (I made a paperwork error!), which meant we had to wait nearly a year to apply again. But those two years didn't yield any meaningful results, as our lab components weren't designed to be used the way we wanted.

After nearly three years of experimental work, the team -- especially postdoctoral associate Andreas Velten and MIT professor Moungi Bawendi, many students, and several collaborators -- cobbled together pieces of the puzzle and built a software program to create a first demonstration of looking around corners. Very soon afterward, we could also start creating surreal movies of light in motion.

We call this new imaging technology femto photography because we capture a segment of the image with a flashlight (in this case, a laser pulse) on for a few millionths of a billionth of a second (or a few femtoseconds) and an exposure time approaching a trillionth of a second.

Just how fast is femto photography? Think of it this way: if we took one-thousandth of a second of footage from the femto camera video output and slowed it down to the speed of 30 seconds per frame -- the approximate speed of a standard TV broadcast -- it would take us a lifetime to watch.

Photographers know that at very short exposures and even at the most sensitive setting for dark scenes, we will record barely any light. So what about in a trillionth of a second? We actually record and average millions of photos to get enough light, each photo made to look the same via carefully timed synchronization with the light pulse. So even if our exposure time is indeed nearly a trillionth of a second, to get sufficient light we must take an average. Thus, as of now, we can only record repeatable events, but this is not a fundamental limitation.

TED.com: A robot that flies like a bird

Unlike conventional cameras, our femto camera captures an image as one thin slice at a time of one-dimensional space using a "streak tube," a laboratory instrument that is commonly used by chemists to study light passing through chemical samples. We then take hundreds of these narrow videos (each shot at a slightly different angle) and create a carefully synchronized, slow-motion composite of the light pulses. It takes about an hour to collect and aggregate the data (view a demonstration of a light pulse as it travels through an ordinary Coca-Cola bottle).

To see around the corner, we use femto photography to analyze scattered light. We bounce light off of visible parts into hidden parts and then measure the time and direction of returned light.

Usually the scattering of energy is considered a nuisance -- whether driving in fog or poor reception from a cell phone tower -- and most techniques either try to avoid it (by turning on fog lights) or reduce the impact of scattering (by selecting energy for the phone only from direct paths).

In contrast, we exploit the scattering. For the camera, a laser pulse is fired at a wall, and the impact of hitting the wall causes the particles of light to scatter. Some of the scattering particles return to the camera at different times. This is repeated about 60 times per image as the camera measures how long it takes for the light to travel back and where the particles land. An algorithm then crunches the data to reconstruct the hidden image. This technique even allows us to see a three-dimensional object such as a mock-up of running person.

TED.com: Test for Parkinson's with a phone call

As exciting as this work is, don't look for this technology to be in practice tomorrow -- we're still years away from bringing this to market. But, we can already imagine multiple ways that it could have a significant, positive impact on our everyday lives.

By allowing us to "see" around a corner, for example, this technology added to our cars could let us know if there's another vehicle approaching around a blind curve. It also could give us a new way to look deep inside our bodies without X-rays, or to look through a window into a burning building from a distance to see if anyone is left inside --without risking a firefighter's life.

When I gave a TEDGlobal talk on femto photography in June, I began with a reference to Doc Edgerton, a very popular MIT professor of electrical engineering who, in 1964, wowed the world with an image of a bullet in midair, having just passed through an apple. He accomplished this by using a strobe light to freeze the action of the bullet at a millionth of a second.

What we're talking about here -- the speed of light --is a million times faster, and is opening the door to a complete rethinking of what we mean by, and can do with, photography. It is a first step toward a new world of imaging that far exceeds human ability, resynthesizing data and depicting it in ways that are within human comprehensibility.

Follow @CNNOpinion on Twitter

Join us at Facebook/CNNOpinion

The opinions expressed in this commentary are solely those of the author, Ramesh Raskar, who would like to thank his collaborators: Moungi G. Bawendi, Professor, MIT Department of Chemistry; Andreas Velten and Christopher Barsi, postdoctoral associates, MIT Media Lab; Everett Lawson, Otkrist Gupta, Nikhil Naik, Amy Fritz, Di Wu, MIT Media Lab; Matt O'toole, Kyros Kutulakos, University of Toronto; Diego Gutierrez, Belen Masia, and Elisa Amoros, Universidad de Zaragoza; Kavita Bala, Shuang Zhao, Cornell University; Ashok Veeraraghavan, Rice University.; Thomas Willwacher, Harvard University.

ADVERTISEMENT
Part of complete coverage on
updated 11:38 AM EDT, Wed July 23, 2014
The idea is difficult to stomach, but John Sutter writes that eating dog is morally equivalent to eating pig, another intelligent animal. If Americans oppose it, they should question their own eating habits as well.
updated 12:30 PM EDT, Wed July 23, 2014
Bill van Esveld says under the laws of war, civilians who do not join in the fight are always to be protected. An International Criminal Court could rule on whether Israeli airstrikes and Hamas rocketing are war crimes.
updated 8:05 AM EDT, Wed July 23, 2014
LZ Granderson says Ronald Reagan went horseback riding and took a vacation after the Korean Air Crash of 1983. So why does the GOP keep airbrushing history to bash Obama?
updated 9:38 AM EDT, Wed July 23, 2014
Aaron Miller says Kerry needs the cooperation of Hamas, Israel, Egypt and others if he is to succeed in his peacemaking efforts
updated 8:51 AM EDT, Wed July 23, 2014
Errol Louis says the tragic death of Eric Garner at the hands of the NYPD has its roots in the "broken windows" police strategy from the crime-ridden '80s.
updated 10:08 AM EDT, Wed July 23, 2014
Gordon Brown says the kidnapped Nigerian girls have been in captivity for 100 days, but the world has not forgotten them.
updated 7:27 AM EDT, Wed July 23, 2014
Ruben Navarrette says Texas Gov. Rick Perry is right to immediately send 1,000 National Guard troops to the U.S.-Mexico border in response to the border children crisis.
updated 9:56 AM EDT, Tue July 22, 2014
Ukraine's president says the downing of MH17 was a terrorist act, but Richard Barrett says it would be considered terrorism only if it was intentional
updated 4:15 PM EDT, Tue July 22, 2014
Robert McIntyre says the loophole that lets firms avoid taxes should be closed
updated 11:35 AM EDT, Tue July 22, 2014
Jeronimo Saldana and Malik Burnett say Gov. Perry's plan to send National Guard to the border won't solve the escalating immigration problem.
updated 1:42 PM EDT, Tue July 22, 2014
Sally Kohn: The world's fish and waters are polluted and under threat. Be very careful what fish you eat
updated 8:42 AM EDT, Tue July 22, 2014
Les Abend says threat information that pilots respond to is only as good as the intelligence from air traffic controllers. And none of it is a match for a radar-guided missile
updated 8:35 AM EDT, Mon July 21, 2014
Frida Ghitis: Anger over MH17 is growing against pro-Russia separatists. It's time for the Dutch government to lead, she writes
updated 8:27 AM EDT, Mon July 21, 2014
Julian Zelizer says President Obama called inequality the "defining challenge" of our time but hasn't followed through.
updated 7:57 AM EDT, Mon July 21, 2014
Gene Seymour says the 'Rockford Files' actor worked the persona of the principled coward, charming audiences on big and small screen for generations
updated 10:17 AM EDT, Mon July 21, 2014
Daniel Treisman says that when the Russian leader tied his fate to the Ukraine separatists, he set the stage for his current risky predicament
updated 12:42 PM EDT, Fri July 18, 2014
Andrew Kuchins says urgent diplomacy -- not sanctions -- is needed to de-escalate the conflict in Ukraine that helped lead to the downing of an airliner there.
updated 9:50 PM EDT, Fri July 18, 2014
Jim Hall and Peter Goelz say there should be an immediate and thorough investigation into what happened to MH17.
updated 11:07 AM EDT, Fri July 18, 2014
Pilot Bill Palmer says main defense commercial jets have against missiles is to avoid flying over conflict areas.
updated 1:55 PM EDT, Sun July 20, 2014
Valerie Jarrett says that working women should not be discriminated against because they are pregnant.
updated 3:53 PM EDT, Mon July 21, 2014
David Wheeler says the next time you get a difficult customer representative, think about recording the call.
updated 3:33 PM EDT, Fri July 18, 2014
Newt Gingrich says the more dangerous the world becomes the more Obama hides in a fantasy world.
updated 6:11 AM EDT, Fri July 18, 2014
Michael Desch: It's hard to see why anyone, including Russia and its local allies, would have intentionally targeted the Malaysian Airlines flight
updated 3:14 PM EDT, Thu July 17, 2014
LZ Granderson says we must remember our visceral horror at the news of children killed in an airstrike on a Gaza beach next time our politicians talk of war
updated 8:06 AM EDT, Thu July 17, 2014
Sally Kohn says now the House GOP wants to sue Obama for not implementing a law fast enough, a law they voted down 50 times, all reason has left the room.
updated 8:14 AM EDT, Thu July 17, 2014
A street sign for Wall Street
Sens. Elizabeth Warren, John McCain and others want to scale back the "too big to fail" banks that put us at risk of another financial collapse.
updated 4:16 PM EDT, Thu July 17, 2014
Newt Gingrich writes an open letter to Robert McDonald, the nominee to head the Veterans Administration.
updated 12:01 PM EDT, Fri July 18, 2014
Paul Begala says Dick Cheney has caused an inordinate amount of damage yet continues in a relentless effort to revise the history of his failures.
updated 10:04 AM EDT, Fri July 18, 2014
Kids who takes cell phones to bed are not sleeping, says Mel Robbins. Make them park their phones with the parents at night.
updated 1:29 PM EDT, Thu July 17, 2014
Buzz Aldrin looked at planet Earth as he stood on talcum-like lunar dust 45 years ago. He thinks the next frontier should be Mars.
updated 2:04 PM EDT, Wed July 16, 2014
Mark Zeller never thought my Afghan translator would save his life by killing two Taliban fighters who were about to kill him. The Taliban retaliated by placing him on the top of its kill list.
updated 11:18 AM EDT, Thu July 17, 2014
Jeff Yang says an all-white cast of Asian characters in cartoonish costumes is racially offensive.
updated 9:24 PM EDT, Wed July 16, 2014
Gary Ginsberg says the late John F. Kennedy Jr.'s reaction to an event in 1995 summed up his character
updated 12:41 PM EDT, Wed July 16, 2014
Meg Urry says most falling space debris lands on the planet harmlessly and with no witnesses.
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT