Lytro's announcement that it will be launching a plenoptic 'light field' camera that allows images to be re-focused after they've been taken, was met with equal amounts of interest and skepticism. Interested to find out more, we spoke to the company's founder and CEO, Ren Ng, to hear just what he has planned and how far towards a product the company has got.

The first thing to understand, he stressed, is how the system works: by placing an array of microlenses some distance in front of an imaging sensor, points of light arriving through the lens are scattered across multiple photo sites, depending on the angle they've arrived from. This information, captured in a single exposure, provides the ability to render images as if they'd been focused at different distances. The company says it will begin selling such a device before the end of the year (2011). Not only would such a device be able to produce re-focusable images, but it also wouldn't need focusing at the point of shooting.

"Our vision is a product that allows people to
shoot and share very simply"

The first device will be aimed at the consumer end of the market, says Ng, explaining that the company is targeting: 'people who really like to have fun with pictures and share them with friends and family. Our vision is a product that allows people to shoot and share very simply.' And this product is not far from becoming a reality, he says: 'The product will be out in 2011 and priced competitively for a consumer product. It's already in the hands of photographers.' (Of the people shooting the samples on the company's website, only Eric Cheng, its director of photography, is an employee).

Despite the consumer focus for the first product, Ng believes the nature of the technology means this won't just entail people pointing and shooting: 'We're looking at someone really interested in what photography means, who wants to experiment with the capabilities of this new approach, and wants to explore and enjoy the artistic possibilities of working with a new medium.'

Sharing the experience

'Light field photography creates a fundamentally different type of data. When we moved from film to digital it made all sorts of changes to what we could do with photographs, but we were still collecting essentially the same 2D data that we always had been, right back to the days of the daguerreotype. There are opportunities as an artistic process for people to experiment and be creative. The type of data is very resonant with that - you can create an image and invite the viewers to explore the picture. There are opportunities in terms of crafting and posing pictures in a way that gives a sense of discovery to the viewer. A sense of discovering a story for themselves.'

The first product's focus will be on making this capability accessible and easy to share, he says: 'Five years ago, this would have been impossible. It's only the development of web infrastructure, technologies such as Flash and HTML5 that allow us to program the interaction through an internet browser without having to download or install additional software. That's what powers the experience of our product, just as much as the instant shutter, instant focus or any of the other benefits.'

"The end user gets the full 'living picture'
experience without onerous downloads"

'The software to convert the captured 'light field' into an image, which we're calling the Light Field Engine, is in the camera. It is installed on your PC as well and, when you share your images through social networks, mobile devices or all the other places people share images, the Light Field Engine goes with the picture, so that the end user gets the full 'living picture' experience without onerous downloads.'

What about those samples?

He notes the concerns expressed about the samples that have already been shown on the web, explaining that, while they show how shareable the images can be, they are not representative of the camera's full capabilities: 'The ability to focus after-the-fact is fully continuous - you can focus at any depth. There are two factors that make this less apparent in the samples. The first is the tendency in photography for depth to appear compressed, so objects of similar distances appear together [as they do when you shoot a portrait with a long focal length, as an extreme example]. Depending on composition and arrangement of subjects, there may only be two or three significant depths within an image. Also the way we've packaged the data for easy viewing on the internet has an effect. It's not the full light field you're seeing - it's a subset to make it more portable. It's analogous to comparing the Raw data that an enthusiast photographer might take, with the small, compressed JPEG that Facebook might serve up if you view it on your smart phone.'

Also, while he explains that the sample images come from devices taken from the production line, they are not yet final: 'The devices themselves look very close to final on the outside, but the hardware internals, software and image quality are not production standard yet,' he says.

"The 0.1MP resolution we were producing then is not
consumer-ready, so we've come a long way from
there to make a commercializable product"

This is a long way beyond the point dpreview last spoke to him (in 2005), when Ng has adapted a 16MP medium format camera to produce 900,000 pixel images: 'An important thing to note is that at that stage of development, the focus was on: "how do we take a multi-camera array and miniaturize it to a single device?" The results at that time were not anywhere near commercializable. It was a scientific breakthrough we were working towards. The next step we've been working on has been making a commercial breakthrough. The 0.1MP resolution we were producing then is not consumer-ready, so we've come a long way from there to make a commercializable product, that can sell in the highest volumes. And doing that has required making a product that makes it easy to share the results on the internet. If you look at the way people use pictures, the vast majority of pictures are on the web.'

More creativity to come

The initial software won't allow a great deal of post-shot editing, he explains: 'At first we'll be making those decisions for the user - so that we can make the process as simple as possible but, further down the line, we'll provide tools to give more control over the final output. It's important to understand that Lytro's camera will record full light fields at day one, and folks will be able to do more and more with those same files as the software grows into the future. It's a bit like DSLR shooters working with the initial Raw formats: the new editing features you could achieve with those Raw file increased over time as software support matured.'

"We're very keen to see light field images develop
through an ecosystem of software"

'We're very keen to see light field images develop through an ecosystem of software, to allow people to share images and edit images, as with normal, 2D images. We're producing a format with an API to provide developer access to the format's capabilities. Kurt Akeley, our Chief Technology Officer used to work for SGI, where he invented the OpenGL API, so we've got some truly world class experience in this sort of thing.'

'It's not going to just drop into existing software, it's going to require a bit of work - it's a richer data with greater possibilities. The light field, when turned into pictures, is 2D but there are opportunities to work on light fields directly to access their full possibility. Tapping the full potential is a huge opportunity, and paves the road for a great deal of exciting R&D.'

"In the same way that Polaroid changed the market -
it brought an immediacy and shareability to photography"

But, even at his most positive, Ng says he doesn't expect light field photography to replace conventional, 2D photography: 'The folks at dpreview are not going to replace all their cameras with our first product. But, once it comes and opens up all these other capabilities, I think they're going to be enchanted by what it represents for photography. It provides new opportunities - allowing you to create compositions that tell a story in a way you never could before. They're going to keep their existing tools but add this as well. In the same way that Polaroid changed the market - it brought an immediacy and shareability to photography, but that wasn't at the expense of conventional film photography, it was in addition.'

Over time, he believes, people will find additional creative options in the images. For example, some enthusiast photographers discuss the quality of the out-of-focus regions of their photographs, which is influenced by the complexity of the design of the lens used to shot the image. This could be an area people want to experiment in, Ng proposes: 'At the beginning, the out-of-focus region will look as it would through an optical viewfinder. For people who want to shape their bokeh, this could be the thing that really interests them. The ability to control your bokeh after the effect, could be another example of creative control on the editing side. The photographic possibilities will explode as people experiment with this sort of thing.'

Pushing sensor technology

And, if it achieves the level of success he and the company are hoping for, he says he can envisage light field cameras influencing sensor technology: 'As well as a scientific and commercial breakthrough, this could cause a technological breakthrough. We've got to the stage where we're seeing 14-16MP sensors for compacts and 20-24MP in larger sensors. It's not technological limitations that are defining that figure, it's a marketing-driven progression. When we went from VGA to 1MP to 4MP sensors, that was technology growth.'

"You could in theory make a sensor with
hundreds of millions of pixels"

'Growth in that underlying industry capacity hasn't stopped, there's just no demand for it. With 14MP, for print or web use, those are enormous images, so there's no great pressure to move on from there. But if you applied the technology being developed for mobile phone cameras and applied it to an APS-C sensor, you could in theory make a sensor with hundreds of millions of pixels - an order of magnitude beyond what we're currently seeing. With such a sensor in a light field camera, we'd be able to measure hundreds of millions of rays of light. Light field technology can utilize and re-invigorate amazing growth in density of sensors.

And the lower output resolution of light field cameras, compared to conventional ones, could be a real benefit: 'Light field technology is inherently more capable in low light - we can shoot wide-open with apertures larger than make any sense for conventional photography. And we're not just trying to make enormous pictures. One dead or noisy pixel in conventional photography is expected to result in one output pixel in the final image. In light field photography it translates to a dead 'ray' which won't have as much impact on the final output - the sensitivity to defects from the sensor will go down.'

Going it alone

Trying to go to market with a product based on a fundamentally different technology may sound ambitious for a camera company nobody had heard of two months ago, but Ng is unfazed by the challenge: 'We feel we're better placed to bring the full benefits of this technology. It's a transformational technology, it needs a transformational product. If you look at most digital cameras, they're very good but they've come about as a result of a series of incremental changes to the previous technology. Trying to do this as an incremental change to an existing technology would rob the consumer of many of the most disruptive benefits.'

"We feel we're better placed to bring the full
benefits of this technology"

There's another reason for producing the camera themseleves, Ng says: 'because we can build this kind of company today. Ten years ago [doing all this themseleves] would have been impossible but, as with the advances in web infrastructure that make the pictures sharable, there have been great advances in manufacturing and distribution that make it possible for a new company to do this. In the past, to get the message out, you'd have needed to buy an ad during the Superbowl - which is a very expensive thing to do and doesn't get your message to the right people. The web has made it so much easier in terms of localizing the message. Just the pictures we've posted, spreading out across the web has generated so much interest.'