Previous news story    Next news story

Lytro's Ren Ng sheds some light on the company's ambitions

By Richard Butler on Aug 18, 2011 at 22:42 GMT

Lytro's announcement that it will be launching a plenoptic 'light field' camera that allows images to be re-focused after they've been taken, was met with equal amounts of interest and skepticism. Interested to find out more, we spoke to the company's founder and CEO, Ren Ng, to hear just what he has planned and how far towards a product the company has got.

The first thing to understand, he stressed, is how the system works: by placing an array of microlenses some distance in front of an imaging sensor, points of light arriving through the lens are scattered across multiple photo sites, depending on the angle they've arrived from. This information, captured in a single exposure, provides the ability to render images as if they'd been focused at different distances. The company says it will begin selling such a device before the end of the year (2011). Not only would such a device be able to produce re-focusable images, but it also wouldn't need focusing at the point of shooting.

"Our vision is a product that allows people to
shoot and share very simply"

The first device will be aimed at the consumer end of the market, says Ng, explaining that the company is targeting: 'people who really like to have fun with pictures and share them with friends and family. Our vision is a product that allows people to shoot and share very simply.' And this product is not far from becoming a reality, he says: 'The product will be out in 2011 and priced competitively for a consumer product. It's already in the hands of photographers.' (Of the people shooting the samples on the company's website, only Eric Cheng, its director of photography, is an employee).

Despite the consumer focus for the first product, Ng believes the nature of the technology means this won't just entail people pointing and shooting: 'We're looking at someone really interested in what photography means, who wants to experiment with the capabilities of this new approach, and wants to explore and enjoy the artistic possibilities of working with a new medium.'

Sharing the experience

'Light field photography creates a fundamentally different type of data. When we moved from film to digital it made all sorts of changes to what we could do with photographs, but we were still collecting essentially the same 2D data that we always had been, right back to the days of the daguerreotype. There are opportunities as an artistic process for people to experiment and be creative. The type of data is very resonant with that - you can create an image and invite the viewers to explore the picture. There are opportunities in terms of crafting and posing pictures in a way that gives a sense of discovery to the viewer. A sense of discovering a story for themselves.'

The first product's focus will be on making this capability accessible and easy to share, he says: 'Five years ago, this would have been impossible. It's only the development of web infrastructure, technologies such as Flash and HTML5 that allow us to program the interaction through an internet browser without having to download or install additional software. That's what powers the experience of our product, just as much as the instant shutter, instant focus or any of the other benefits.'

"The end user gets the full 'living picture'
experience without onerous downloads"

'The software to convert the captured 'light field' into an image, which we're calling the Light Field Engine, is in the camera. It is installed on your PC as well and, when you share your images through social networks, mobile devices or all the other places people share images, the Light Field Engine goes with the picture, so that the end user gets the full 'living picture' experience without onerous downloads.'

What about those samples?

He notes the concerns expressed about the samples that have already been shown on the web, explaining that, while they show how shareable the images can be, they are not representative of the camera's full capabilities: 'The ability to focus after-the-fact is fully continuous - you can focus at any depth. There are two factors that make this less apparent in the samples. The first is the tendency in photography for depth to appear compressed, so objects of similar distances appear together [as they do when you shoot a portrait with a long focal length, as an extreme example]. Depending on composition and arrangement of subjects, there may only be two or three significant depths within an image. Also the way we've packaged the data for easy viewing on the internet has an effect. It's not the full light field you're seeing - it's a subset to make it more portable. It's analogous to comparing the Raw data that an enthusiast photographer might take, with the small, compressed JPEG that Facebook might serve up if you view it on your smart phone.'

Also, while he explains that the sample images come from devices taken from the production line, they are not yet final: 'The devices themselves look very close to final on the outside, but the hardware internals, software and image quality are not production standard yet,' he says.

"The 0.1MP resolution we were producing then is not
consumer-ready, so we've come a long way from
there to make a commercializable product"

This is a long way beyond the point dpreview last spoke to him (in 2005), when Ng has adapted a 16MP medium format camera to produce 900,000 pixel images: 'An important thing to note is that at that stage of development, the focus was on: "how do we take a multi-camera array and miniaturize it to a single device?" The results at that time were not anywhere near commercializable. It was a scientific breakthrough we were working towards. The next step we've been working on has been making a commercial breakthrough. The 0.1MP resolution we were producing then is not consumer-ready, so we've come a long way from there to make a commercializable product, that can sell in the highest volumes. And doing that has required making a product that makes it easy to share the results on the internet. If you look at the way people use pictures, the vast majority of pictures are on the web.'

More creativity to come

The initial software won't allow a great deal of post-shot editing, he explains: 'At first we'll be making those decisions for the user - so that we can make the process as simple as possible but, further down the line, we'll provide tools to give more control over the final output. It's important to understand that Lytro's camera will record full light fields at day one, and folks will be able to do more and more with those same files as the software grows into the future. It's a bit like DSLR shooters working with the initial Raw formats: the new editing features you could achieve with those Raw file increased over time as software support matured.'

"We're very keen to see light field images develop
through an ecosystem of software"

'We're very keen to see light field images develop through an ecosystem of software, to allow people to share images and edit images, as with normal, 2D images. We're producing a format with an API to provide developer access to the format's capabilities. Kurt Akeley, our Chief Technology Officer used to work for SGI, where he invented the OpenGL API, so we've got some truly world class experience in this sort of thing.'

'It's not going to just drop into existing software, it's going to require a bit of work - it's a richer data with greater possibilities. The light field, when turned into pictures, is 2D but there are opportunities to work on light fields directly to access their full possibility. Tapping the full potential is a huge opportunity, and paves the road for a great deal of exciting R&D.'

"In the same way that Polaroid changed the market -
it brought an immediacy and shareability to photography"

But, even at his most positive, Ng says he doesn't expect light field photography to replace conventional, 2D photography: 'The folks at dpreview are not going to replace all their cameras with our first product. But, once it comes and opens up all these other capabilities, I think they're going to be enchanted by what it represents for photography. It provides new opportunities - allowing you to create compositions that tell a story in a way you never could before. They're going to keep their existing tools but add this as well. In the same way that Polaroid changed the market - it brought an immediacy and shareability to photography, but that wasn't at the expense of conventional film photography, it was in addition.'

Over time, he believes, people will find additional creative options in the images. For example, some enthusiast photographers discuss the quality of the out-of-focus regions of their photographs, which is influenced by the complexity of the design of the lens used to shot the image. This could be an area people want to experiment in, Ng proposes: 'At the beginning, the out-of-focus region will look as it would through an optical viewfinder. For people who want to shape their bokeh, this could be the thing that really interests them. The ability to control your bokeh after the effect, could be another example of creative control on the editing side. The photographic possibilities will explode as people experiment with this sort of thing.'

Pushing sensor technology

And, if it achieves the level of success he and the company are hoping for, he says he can envisage light field cameras influencing sensor technology: 'As well as a scientific and commercial breakthrough, this could cause a technological breakthrough. We've got to the stage where we're seeing 14-16MP sensors for compacts and 20-24MP in larger sensors. It's not technological limitations that are defining that figure, it's a marketing-driven progression. When we went from VGA to 1MP to 4MP sensors, that was technology growth.'

"You could in theory make a sensor with
hundreds of millions of pixels"

'Growth in that underlying industry capacity hasn't stopped, there's just no demand for it. With 14MP, for print or web use, those are enormous images, so there's no great pressure to move on from there. But if you applied the technology being developed for mobile phone cameras and applied it to an APS-C sensor, you could in theory make a sensor with hundreds of millions of pixels - an order of magnitude beyond what we're currently seeing. With such a sensor in a light field camera, we'd be able to measure hundreds of millions of rays of light. Light field technology can utilize and re-invigorate amazing growth in density of sensors.

And the lower output resolution of light field cameras, compared to conventional ones, could be a real benefit: 'Light field technology is inherently more capable in low light - we can shoot wide-open with apertures larger than make any sense for conventional photography. And we're not just trying to make enormous pictures. One dead or noisy pixel in conventional photography is expected to result in one output pixel in the final image. In light field photography it translates to a dead 'ray' which won't have as much impact on the final output - the sensitivity to defects from the sensor will go down.'

Going it alone

Trying to go to market with a product based on a fundamentally different technology may sound ambitious for a camera company nobody had heard of two months ago, but Ng is unfazed by the challenge: 'We feel we're better placed to bring the full benefits of this technology. It's a transformational technology, it needs a transformational product. If you look at most digital cameras, they're very good but they've come about as a result of a series of incremental changes to the previous technology. Trying to do this as an incremental change to an existing technology would rob the consumer of many of the most disruptive benefits.'

"We feel we're better placed to bring the full
benefits of this technology"

There's another reason for producing the camera themseleves, Ng says: 'because we can build this kind of company today. Ten years ago [doing all this themseleves] would have been impossible but, as with the advances in web infrastructure that make the pictures sharable, there have been great advances in manufacturing and distribution that make it possible for a new company to do this. In the past, to get the message out, you'd have needed to buy an ad during the Superbowl - which is a very expensive thing to do and doesn't get your message to the right people. The web has made it so much easier in terms of localizing the message. Just the pictures we've posted, spreading out across the web has generated so much interest.'

Comments

Total comments: 116
12
fotografer
By fotografer (Aug 19, 2011)

I can see one very important application. It's excellent for security camera that takes pictures every, say, half a millisecond. If we need to determine potential witness or culprits of thefts/burglary, then all one needs is to click on the area of interest and 'hey presto' it's focused. Perhaps one day CCTV will be able to incorporate this technology that perhaps even features/details like scars and tattoos could be post-focused after the images are captured...

0 upvotes
Joseph S Wisniewski
By Joseph S Wisniewski (Aug 19, 2011)

It's not useful, at all, for security. It actually limits resolution, severely. In fact, none of the new camera technologies that have actually been promoted for security (other variations on light field cameras, "stutter shutter" motion estimation, or single shot HDR techniques) have actually turned out to be useful for security.

Multispectral imaging has forensics use, but forensics is the diametric opposite of security, being done by people who have some idea what they're doing.

0 upvotes
ponyman
By ponyman (Aug 19, 2011)

I wondered about security cameras as well - the combination of a large aperture for low light and the ability to focus on multiple points which would normally not be possible, due to the narrow dof, could be of benefit in this area.

0 upvotes
Cy Cheze
By Cy Cheze (Aug 19, 2011)

Security cameras must be wide angle to cover a property or place of business. Most of them have very low resolution, barely VGA. Like "beware of dog" signs, they may be more bark than bite. Plenoptics would not make a culprits face clear if there are only a hundred pixels for that face in the first place. Some security cameras can employ medium-format resolutions, but that would be well beyond the ken of plenoptics, except at a cost of millions per unit. However, were that possible, would it be any better than a deep DOF still? And a clear picture of a face behind a stocking or mask would not be very conclusive anyway. Only a dope would consider a big heist without scouting the site and spotting where the cameras were or not.

0 upvotes
Tim Ashton
By Tim Ashton (Aug 19, 2011)

There are a couple of "photographers" I know of who would benefit. Came through from P&S cameras and have totally failed to master the focussing systems of todays DSLRs :-)
Other wise, Jogger'ds comment nails it
A solution looking for a problem

Tim

0 upvotes
RedFox88
By RedFox88 (Aug 18, 2011)

Further development of this would be great for the average camera user (digicams not SLRs). Where as you download your photos you can have the option to change focus or not, touching the part of the image you want in focus and then move onto the next image.

There are a lot of focus issues on P&S cameras that even their built-in large DOF are noticeable and robs detail and makes photos look blah.

0 upvotes
Jogger
By Jogger (Aug 18, 2011)

solution looking for a problem.. do people really have a hard time focusing.. also, i dont think this tech will ever work on an FF camera with am 85/1.4 (for example).. its for dinky pinhead sized sensors that have infinite dof to begin with.. the oof area are digitally simulated (dont expect crasy dof separation from a point and shoot sensor)

0 upvotes
RedFox88
By RedFox88 (Aug 18, 2011)

So you've never had a photo mis-focused that you would have loved to be in focus?

And when you see a bunch of photos taken by average camera (P&S) users, you'll see hat there are focus errors often and they cause lack of detail and blurriness. Yes it sounds like a great idea to have. You'd have to be crazy to not want a new tool like this to be developed or you're a purist of photography and thinks nothing should be done to an image after pressing the button.

0 upvotes
Michael Williams
By Michael Williams (Aug 19, 2011)

I disagree. This technology not just controls the focal distance, but can control the DoF (far and near focal limits)... If I'm indoors with a simple lamp trying to take pictures of the boys running around, even at I'm having to open up to f/1.8 or 2.0 at ISO 800 just to get a decent non-flash shot. With this, you can open up and take the picture, then adjust DoF later, so not ALL shots have that f/1.8 feel. I can't imagine why you'd think this wouldn't apply to FF sensors, if anything a FF sensor would make this technology even better!

I think the folks that are protesting this technology are doing so because it's going to give the average person control over focus, DoF, and bokeh after the fact, thereby nullifying the need for legitimate skill and understanding to take better quality photos... and that scares people who rely on that skill for a profession.

1 upvote
mikiev
By mikiev (Aug 19, 2011)

While in the 3rd paragraph of the 'Pushing sensor technology' section, he states:

"Light field technology is inherently more capable in low light - we can shoot wide-open with apertures larger than make any sense for conventional photography."

They don't require 'pinhead-sized sensors with infinite DOF' for this to work. They -want- to move to bigger sensors, so they can have more pixels in the output.

1 upvote
Joseph S Wisniewski
By Joseph S Wisniewski (Aug 19, 2011)

If he states that it's "inherently more capable in low light", he's lying.

First, there's a limit to the aperture you can shoot at with a light field camera. If you take a 10mp camera, with an f1.4 lens, and set it up with a 10:1 decimation microlens array, you now have selectable DOF between different planes of about f2 to f14. You can't get at either the f1.4 low light ability of your lens, or the shallow DOF. And that involves reducing a 10mp camera to 0.1mp. ;)

And you can't cheat information theory. When you trade off the math to combine the lightfield, you add noise. And you can't get around "information theory", on a depth equivalent to f8, there's no more than f8 worth of light "information". Toss in the stop you lose from the microlens overlap (they're not round, you know), and a stop or two from the math, and you've got a camera that's performing 3 stops worse than a simple camera stopped down to the same DOF as the final print.

0 upvotes
Nazgman
By Nazgman (Aug 19, 2011)

And now consider a 100mp camera. Imagine the 3rd generation light field engine software which gets rid of most of the math stops. And suddenly we have something which looks quite interesting.

The first digital cameras produced horrible pictures, and many people claimed digital photography could never surpass analogue photography.

I think we're seeing the same here.

1 upvote
djezraj
By djezraj (Aug 19, 2011)

It is so easy to cast judgement from a perspective other than that of the intended audience. I see this technology as catchy for the consumer who does not want to understand depth of field or focus accuracy. Casual Gadget gurus or enthusiast tourist photogs may appreciate this indeed!
At least to me I do not have an issue knowing how to get proper depth of feild or focus so I am not sure I "need" the failsafe so maybe it is not me that they are after?
As for the high end "Pro" film industry this is a highly standardized and much smaller industry than the consumer market. A consumer release has the potential for higher volume and better profit margins. this is not to say that they should not enter the pro market but they should be careful and may be better to have a strong entry into the market first. The consumer market will be more able to live with shortcomings or flaws with the first release of the technology.

Lets all give it at least 5 years and then revisit this.

0 upvotes
Erik Magnuson
By Erik Magnuson (Aug 20, 2011)

I'm not sure who the intended audience for this camera really is. Some of the suggestions here don't seem to apply. The example shots show careful composition to have a useful photographic effect. Like when shooting stereo, you really need to have a distinct foreground, middle ground, and background which most casual shooters do not bother with. A large number of blurry shots have nothing to do with focus and are camera shake or subject motion. One thing consumers *will* appreciate is lack of AF lag, but I have a 10yo 1.3 MP digital camera with zero focus lag already. I suspect like Nimslo in the 80's it will be a fad and a few photographers will exploit it, but unless they pay to get on the major photo sharing sites like Facebook or Flckr, the cameras will quickly end up being dumped.

0 upvotes
Graystar
By Graystar (Aug 18, 2011)

It's interesting. Dunno how useful it will be. But the market will decide.

0 upvotes
MonkRX
By MonkRX (Aug 18, 2011)

+1
I support him. It sounds like a great direction to explore, while it certainly doesn't meet any needs I require, this new camera might create a new style of art & photography.

0 upvotes
Total comments: 116
12