Previous news story Next news story

Bell Labs creates lensless single-pixel camera

Jun 4, 2013 at 20:06:11 GMT

Print view

Scientists at Bell Labs have built a prototype camera that uses no lens and a single-pixel sensor. This rather counter-intuitive idea is based around a grid of small apertures that each direct light rays from different parts of the scene to the sensor, and can be opened and closed independently. Using a technique called 'compressive sensing', the sensor makes a series of measurements with different combinations of open apertures, and uses this data to reconstruct the scene in front of the camera. Because there's no lens to focus the resultant image has infinite depth of field, rather like a pinhole camera.


This is the principle of the lensless camera. An aperture array directs light rays from different parts of the subject to a single sensor.	This is Bell Labs' prototype camera. The white assembly is the aperture array, and the black box (appropriately) contains the sensor and electronics.

Here's the image of the stack of books shown above, a 302 x 217 pixel file created using 16384 individual measurements.	Luckily the lensless single-pixel camera can still be used to take pictures of cats. As long as they're asleep, anyway.

Don't necessarily expect to find this technology coming to a camera near you soon, though. The single pixel design means that a series of measurements has to be made to generate the final image, and the higher the number of measurements, the higher the image quality.

The prototype camera used a 302 x 217 aperture array to produce a 65534 pixel file, which required 8192 or 16384 measurements to make a reasonable quality image (which even then wouldn't stand up to serious pixel-peeping). Scale this up to 12MP - towards the low end of the scale by modern camera standards - and you'd need to make several million measurements to construct an image. As you can imagine, this doesn't make for a quick-shooting action camera - it only works for still life subjects. Meanwhile, zooming would require moving the entire aperture assembly back and forwards relative to the sensor.

Click here for a link to download the original paper (from Cornell University Library)

(via engadget)

OTHER NEWS TECHNOLOGY

Comments

Total comments: 56

Thanks for sharing this interesting article. The science involved is a bit over my head, as usual, but staying abreast of the bleeding edge technology in our digital world here is very important to me. Such interesting times ...

0 upvotes

I can see this in fixed focal length applications such as flatbed scanners
or webcams

but for a camera.. how will it zoom? how will we control bokeh etc..

0 upvotes

You can zoom over an effectively infinite range by moving the sensor back and forth (it's a point sized sensor, it's easy) behind the modulator.

You can control bokeh by using a small array of sensors. They're single pixel, so relatively cheap.

Still not a particularly useful technique.

4 upvotes

Actually, I think you'd probably control bokeh in post. Just a thought, but an interesting way you'd accomplish this is by combining the infinite depth of field camera with some sort of low power laser to do a depth map of the scene. if you recorded the two set of data together (could actually do a 2 pixel camera, 1 sensitive to the visual light, one which would record the laser reflections), you'd have the ability to really control just about everything about how the image ends up appearing. Could be interesting.

0 upvotes

Could be interesting, but too complicated. No real advantage over current photographic equipment.

0 upvotes

How will it zoom? You walk closer to what your taking a picture of, that is what i call "manual zooming" :p

0 upvotes

This is also known as coded aperture imaging which has been around for awhile. Luckily for us, somebody invented the focusing lens to sample the light field simultaneously, transform the field to an angular field map, and somebody else invented a sensor array to also sample the angular field map simultaneously! Very clever. Of course you still have to sample this field at multiple depth points to get the full image information. Stereo photography is another creative combination of these approaches that provides some depth information (no 3d wavefront information) but still does most of the sampling simultaneously.

2 upvotes

That's what I was thinking too, it seems to be the same idea used in X-ray and gamma-ray astronomy.

0 upvotes

This is an example of "compressed sensing". This technology was invented to reconstruct data from incomplete measurements (which is the title of the original paper too). Not to replace complete measurements by incomplete measurements.

For fundamental physical reasons such as the Heisenberg principle of Uncertainty, the data reconstructed from incomplete measurements must always remain inferior to complete measurements (e.g., because the patterned aperture creates more diffraction, i.e., a blurred signal at the central pixel). This technology only narrows the gap.

So, while this has ist applications, photography is none of them.

Comment edited 2 minutes after posting

2 upvotes

Way too noisy pictures. That's where we land thanks to the MP-race - one pixel is too much!

3 upvotes

Hehehe love the irony

0 upvotes

This makes the most sense in applications where the sensor is incredibly expensive and the aperture array can be made more cheaply.

I could imagine some applications in doing high speed optical data transmissions where there are multiple end-points.

1 upvote

It's commonly done in X-ray and gamma-ray astronomy, where it's extremely difficult to make a mirror or lens.

0 upvotes

I wish people would stop looking at every new technology as the shape of things to come. Of all things being invented nowadays, only a few become feasible and even fewer are truly useful. It's all nice and well being all for innovation and progress and call «narrow-minded» to the ones who show their reservations to new technologies, but the truth is that not all inventions are that useful. The technology described in this article could point towards the future, but then again it may prove completely useless. Most of us are not scientists, so most of us have no clue of whatever technology will replace conventional cameras and lenses. (I certainly don't.)
Judging this technology by the quality of the sample images shown here is a pointless because it is so recent and there's certainly room for improvement, so the odds are this can evolve into something much better. Let's just wait and see.

1 upvote

Either way the fact that they are experimenting to see if there can be some new innovation for photography to improve it is always a positive thing, now if only canon could do the same (cough same 18mp sensor in how many models now? To bad im so used to canon or i would go nikon in a heartbeat)

0 upvotes

See the Nuit-blanche blog http://nuit-blanche.blogspot.com/ about compressed sensing.

0 upvotes

Now That's One Mega Pixel!

5 upvotes

Damn megapixel race! Thats one megapixel to many ;3

0 upvotes

Oh well, I guess in 5+ years hipsters will be all over this ;)

1 upvote

Don't look down on the idea. Probably there is no application value at this point of time. But some genius may be able to utilize this idea to re-invent the photography industries.
LED was invented ages ago. But were no value at that time. But now, what you are reading now is lid by the LED, if you say otherwise, you should start get yourself updated to appreciate the invention ppl make.

0 upvotes

Weird. I remember growing up watching the LED become the number one display for clocks, and a common display for lots of other instrumentation from calculators to thermometers to the exposure display in my Nikon FM2 (a camera that launched about 1/3 century ago). Those "no value at that time" displays were pretty ubiquitous. How long ago did they start appearing in keychain flashlights? 25 years? 30? I remember the high brightness LEDs causing a stir in the mid 80s, and appearing on the first LED car brake lights around 1990, better than 20 years ago.

No one "looked down on" the LED. It had obvious applications from day 1, and it just kept evolving, improving in power, cost, color range, and efficiency, and each new improvement brought new uses. This aperture coding invention has no obvious uses that will cause it to grow and thrive.

As far as "what you are reading now is lid by the LED", I'm using a 30 inch Dell with a fluorescent backlight. ;)

Comment edited 54 seconds after posting

2 upvotes

To bad they refer to led as "flash" in cellphones and to bad they went the led route instead of a real flash like nokia has often done.

0 upvotes

Sounds similar to the early mechanical Television systems that used a single neon bulb and a rotating perforated disc for the display, no?

http://en.wikipedia.org/wiki/Nipkow_disk
http://en.wikipedia.org/wiki/Mechanical_television

Comment edited 2 times, last edit 10 minutes after posting

1 upvote

At first glance, I thought it was a senseless camera
after reading the article, indeed, it is senseless.

0 upvotes

New digital Lomo
Kind of cool.
How small can they make the camera?

1 upvote

Not of much interest for standard photography, but theoretically impressive. Imagine some kind of 'image' made up of difficult to detect energy. Maybe each sensor needs to be large and expensive, so building an array of adequately small sensors isn't possible. This shows one way of constructing an image using just one sensor.

3 upvotes

Really impressive comment! I wonder if this could cause a revolution in space photography ? Movement of the earth could be compensated, the rest of the Universe it not going anywhere. Wavelengths could be utilized to build an image which previously were not accessible. Maybe now we can see farther or see things previously hidden ?

0 upvotes

Hubert, we do all those things now. Terrestrial telescopes make use of the parallax from the rotation of the earth and the movement of the earth about the sun (stellar parallax). Orbital eccentricities provide vertical parallax. Coordination of telescopes in different locations provides synthetic apertures.

And a lot of important astronomical imaging is done with single pixel scanning systems, especially longer wavelengths. Once you get to the far IR, single pixel devices, either coded aperture or scanning, become common. In x-ray astronomy, you see both array detectors like in the Chandra on, and single pixel detectors. Some are scanners, others use coded optics.

Comment edited 2 times, last edit 12 minutes after posting

1 upvote

"Because there's no lens to focus the resultant image has infinite depth of field...."

/sarcasm "Digital 135 (i.e., FF) fanatics need not apply!!!" /end sarcasm

0 upvotes

A programmable pin-hole camera in effect. Might be useful in computational photography going forward.

0 upvotes

A pinhole camera principle gets revisited every now and again, for more unexplored possibilities. It shows that the story of light-bending is not all told, and probably never will be.
I wish someone seriously reinvestigated the light-bending by EM field.
It would surely make use of the various-density materials (and also mirrors) for that purpose obsolete.
However, I wouldn't count very much on it as long as there still is more of the expensive glass to sell, not unlike the obvious reasons why the oil people won't allow the mass production of clean electric cars, regardless of the fact that technology now has all the means to efficiently replace the internal combustion engines.
As the technology advancement accelerates, some kind of greed always manages to slow it down...

Comment edited 4 minutes after posting

0 upvotes

16384 individual measurements to produce a noisy and blurry image. Let's be charitative and say it has lots of room for improvement...

0 upvotes

The first computer was as large as a factory and only calculates simple maths. And the first train was even slower than riding a horse. Having seen so many great products that start from stupid invention, it is better to keep hopes and wait.

9 upvotes

Actually, I have higher hopes on single-lens, multi-sensor cameras. They're even bulkier than this but somehow their concept seems more promising. This looks to me as a bold step in the wrong direction, but maybe it isn't. Or maybe it is. Let's wait and see.

0 upvotes

The first memory IC I bought was 2 Bit. Not 2 Bytes, Not 2 kB, Not 2 MB, Not 2 GB, Not 2 TB. The 2 Bit were very useful to me, and more memory was always put to more use too. I think you get the idea!

0 upvotes

I remember floppy disks, too...

0 upvotes

Its basicly like 99% of my fathers 35mm film photos then (not kidding the photos of us as kids are almost all blurry, a problem i have not had when using manual focus slrs)

0 upvotes

Yeah..but does lightroom support it yet?

1 upvote

No. You'll have to subscribe Creative Cloud.

Comment edited 15 seconds after posting

4 upvotes

ManuelVilardeMacedo - Lightroom is not covered by the Creative Cloud (thank god).

0 upvotes

LR is part of the CC, it just available separately as well.

0 upvotes

@ Wilfred Wong

God knows for how long this will last ! Let us hope indefinitely rather than one-two years.
That said, given the frequent updates needed to include new cameras, I find the monthly fee more justifiable for LR than for PS.

It is like Norton antivirus and firewall : you pay a yearly fee to get the regular updates, but you know that you need these updates. You don't expect new features with each updates, but recognize that to keep the product uptodate (aka able to handle the last viruses or cameras&lenses) there is a cost which the company has to recoup.

Comment edited 4 minutes after posting

0 upvotes

People, both Mssimo and I were just joking!

0 upvotes

@ Manuel

A good example how cynicism is not well understood by everybody. Especially consider non native speaker. But I have to say Adobe really stepped on my toes with Photoshop CC and their fuzzy answers on LR and updates in CC or not just made me more upset. So your comments did made me smile. For that I thank you.

Comment edited 33 seconds after posting

1 upvote

@ ManuelVilardeMacedo

Of course you both were joking, I had a good laugh too, but some jokes are an indication that the pill was hard to swallow, aka not all jokes are innocent. So it got me started.. I should have included a LOL at the beginning of my first answer.

0 upvotes

Another invention waiting to have a usefulness...

Keep 'em coming lab gurus...

1 upvote

Bill Atkinson talked about this years ago but it looks like a lot more computer power is needed.

0 upvotes

Interesting concept but I cannot see the application of this. You can do anyway with lens and get infinite DOF using pin hole camera as well.

With a single sensor, the camera needs to poll the sensor each time it modify the aperture array, which I believe is much slower than polling sensor.

0 upvotes

Military applications, perhaps? According to their abstract, "the architecture can be used for capturing images of visible and other spectra such as infrared, or millimeter waves, in surveillance applications for detecting anomalies or extracting features such as speed of moving objects." Millimeter waves lie somewhere between far IR and the highest frequency microwaves.

0 upvotes

They could use conventional sensor or any modified sensor. With such a small aperture, the image is going to be noisy as hell even with good light.

0 upvotes

This offer one way for lens-less image capturing. So minus away the distortion, light loss, defraction, color shift etc.

0 upvotes

I wonder if this could cause a revolution in space photography ? Movement of the earth could be compensated, the rest of the Universe it not going anywhere ( for the duration of days at least ). Wavelengths could be utilized to build an image which previously were not accessible. Maybe now we can see farther or see things previously hidden ?

0 upvotes

A very similar method is already used by astronomers to image in X-ray and gamma ray energies:

http://en.wikipedia.org/wiki/Coded_aperture

0 upvotes

Wonderful.. they've probably spent millions and invented..

.. the pinhole camera.

0 upvotes

I suppose it's a "beginning", of something... not sure what this would be useful for.

0 upvotes

Yea...

0 upvotes

Total comments: 56

Editorial content

Product database

Community


Roger Cicala investigates accuracy of lens adapters Sep 30, 2013	Enfojer project turns your smartphone into an enlarger for B&W prints Sep 8, 2013	Billboards offer tourists a glimpse of smog-free Hong Kong Sep 1, 2013	Artist Rolf Sachs' photo series inspired by train travel, embraces blur Aug 21, 2013

	Nikon 1 AW1 Sep 19, 2013
	Fujifilm X-A1 Sep 17, 2013
	Olympus OM-D E-M1 Sep 10, 2013
	Leica C (Typ112) Sep 8, 2013
	Nikon Coolpix P7800 Sep 5, 2013
	Nikon Coolpix S02 Sep 5, 2013
	Sony Cyber-shot DSC-QX10 Sep 4, 2013
	Sony Cyber-shot DSC-QX100 Sep 4, 2013
	Sony Alpha A3000 Aug 27, 2013
	Sony Alpha NEX-5T Aug 27, 2013

Here is the K3	137
Okay Fuji, so now that the experiment is over....	107
D610 to be announced within 48 hours (as of 10/5)	101
How many shots do you take to get a shot?	86
More leaks about K3 specifications	84
While everyone ooohs and aaaahs over the ...	79
After 3 weeks & thousands of shots: NEX-6 + ...	77
Anyone for Sunday Scapes (2013-10-06) ?	60
E-M1 Trouble focusing on moving objects	56
The Sunday BIRD Volume 9, Issue 15 October 6, 2013	53
E-M1 as E-5 successor - first week experience	51
Counterfeit lens caps from Amazon	49
GX7: Extended ISO125 Image Degradation?	48
Worth selling my Canon DSLR to move to Olympus ...	44
Shutter shock verdict is not present here. ...	40
Camera For Landscape Photography....NEED ...	34
I'm going broke buying E-M5 Eyecups!	32
Sensor Technology (The Software Engineer Way)	31
Why you shouldn't become a professional	30
Five Worst Clichés About Photography	29

Your email address:
Your name: