Previous news story    Next news story

Bell Labs creates lensless single-pixel camera

By dpreview staff on Jun 4, 2013 at 20:06 GMT

Scientists at Bell Labs have built a prototype camera that uses no lens and a single-pixel sensor. This rather counter-intuitive idea is based around a grid of small apertures that each direct light rays from different parts of the scene to the sensor, and can be opened and closed independently. Using a technique called 'compressive sensing', the sensor makes a series of measurements with different combinations of open apertures, and uses this data to reconstruct the scene in front of the camera. Because there's no lens to focus the resultant image has infinite depth of field, rather like a pinhole camera.

This is the principle of the lensless camera. An aperture array directs light rays from different parts of the subject to a single sensor. This is Bell Labs' prototype camera. The white assembly is the aperture array, and the black box (appropriately) contains the sensor and electronics.
Here's the image of the stack of books shown above, a 302 x 217 pixel file created using 16384 individual measurements. Luckily the lensless single-pixel camera can still be used to take pictures of cats. As long as they're asleep, anyway.

Don't necessarily expect to find this technology coming to a camera near you soon, though. The single pixel design means that a series of measurements has to be made to generate the final image, and the higher the number of measurements, the higher the image quality.

The prototype camera used a 302 x 217 aperture array to produce a 65534 pixel file, which required 8192 or 16384 measurements to make a reasonable quality image (which even then wouldn't stand up to serious pixel-peeping). Scale this up to 12MP - towards the low end of the scale by modern camera standards - and you'd need to make several million measurements to construct an image. As you can imagine, this doesn't make for a quick-shooting action camera - it only works for still life subjects. Meanwhile, zooming would require moving the entire aperture assembly back and forwards relative to the sensor.

(via engadget

Comments

Total comments: 56
Camediadude
By Camediadude (10 months ago)

Thanks for sharing this interesting article. The science involved is a bit over my head, as usual, but staying abreast of the bleeding edge technology in our digital world here is very important to me. Such interesting times ...

0 upvotes
Cameras do 1080p do we still need 1080p camcorders

I can see this in fixed focal length applications such as flatbed scanners
or webcams

but for a camera.. how will it zoom? how will we control bokeh etc..

0 upvotes
Joseph S Wisniewski
By Joseph S Wisniewski (10 months ago)

You can zoom over an effectively infinite range by moving the sensor back and forth (it's a point sized sensor, it's easy) behind the modulator.

You can control bokeh by using a small array of sensors. They're single pixel, so relatively cheap.

Still not a particularly useful technique.

4 upvotes
eyrieowl
By eyrieowl (10 months ago)

Actually, I think you'd probably control bokeh in post. Just a thought, but an interesting way you'd accomplish this is by combining the infinite depth of field camera with some sort of low power laser to do a depth map of the scene. if you recorded the two set of data together (could actually do a 2 pixel camera, 1 sensitive to the visual light, one which would record the laser reflections), you'd have the ability to really control just about everything about how the image ends up appearing. Could be interesting.

0 upvotes
ManuelVilardeMacedo
By ManuelVilardeMacedo (10 months ago)

Could be interesting, but too complicated. No real advantage over current photographic equipment.

0 upvotes
Anepo
By Anepo (10 months ago)

How will it zoom? You walk closer to what your taking a picture of, that is what i call "manual zooming" :p

0 upvotes
fieldray
By fieldray (10 months ago)

This is also known as coded aperture imaging which has been around for awhile. Luckily for us, somebody invented the focusing lens to sample the light field simultaneously, transform the field to an angular field map, and somebody else invented a sensor array to also sample the angular field map simultaneously! Very clever. Of course you still have to sample this field at multiple depth points to get the full image information. Stereo photography is another creative combination of these approaches that provides some depth information (no 3d wavefront information) but still does most of the sampling simultaneously.

2 upvotes
kkoba
By kkoba (10 months ago)

That's what I was thinking too, it seems to be the same idea used in X-ray and gamma-ray astronomy.

0 upvotes
falconeyes
By falconeyes (10 months ago)

This is an example of "compressed sensing". This technology was invented to reconstruct data from incomplete measurements (which is the title of the original paper too). Not to replace complete measurements by incomplete measurements.

For fundamental physical reasons such as the Heisenberg principle of Uncertainty, the data reconstructed from incomplete measurements must always remain inferior to complete measurements (e.g., because the patterned aperture creates more diffraction, i.e., a blurred signal at the central pixel). This technology only narrows the gap.

So, while this has ist applications, photography is none of them.

Comment edited 2 minutes after posting
2 upvotes
Managarm
By Managarm (10 months ago)

Way too noisy pictures. That's where we land thanks to the MP-race - one pixel is too much!

3 upvotes
Anepo
By Anepo (10 months ago)

Hehehe love the irony

0 upvotes
Mark Schormann
By Mark Schormann (10 months ago)

This makes the most sense in applications where the sensor is incredibly expensive and the aperture array can be made more cheaply.

I could imagine some applications in doing high speed optical data transmissions where there are multiple end-points.

1 upvote
kkoba
By kkoba (10 months ago)

It's commonly done in X-ray and gamma-ray astronomy, where it's extremely difficult to make a mirror or lens.

0 upvotes
ManuelVilardeMacedo
By ManuelVilardeMacedo (10 months ago)

I wish people would stop looking at every new technology as the shape of things to come. Of all things being invented nowadays, only a few become feasible and even fewer are truly useful. It's all nice and well being all for innovation and progress and call «narrow-minded» to the ones who show their reservations to new technologies, but the truth is that not all inventions are that useful. The technology described in this article could point towards the future, but then again it may prove completely useless. Most of us are not scientists, so most of us have no clue of whatever technology will replace conventional cameras and lenses. (I certainly don't.)
Judging this technology by the quality of the sample images shown here is a pointless because it is so recent and there's certainly room for improvement, so the odds are this can evolve into something much better. Let's just wait and see.

1 upvote
Anepo
By Anepo (10 months ago)

Either way the fact that they are experimenting to see if there can be some new innovation for photography to improve it is always a positive thing, now if only canon could do the same (cough same 18mp sensor in how many models now? To bad im so used to canon or i would go nikon in a heartbeat)

0 upvotes
bizi clop
By bizi clop (10 months ago)

See the Nuit-blanche blog http://nuit-blanche.blogspot.com/ about compressed sensing.

0 upvotes
RXVGS
By RXVGS (10 months ago)

Now That's One Mega Pixel!

5 upvotes
Anepo
By Anepo (10 months ago)

Damn megapixel race! Thats one megapixel to many ;3

0 upvotes
electrophoto
By electrophoto (10 months ago)

Oh well, I guess in 5+ years hipsters will be all over this ;)

1 upvote
spidercrown
By spidercrown (10 months ago)

Don't look down on the idea. Probably there is no application value at this point of time. But some genius may be able to utilize this idea to re-invent the photography industries.
LED was invented ages ago. But were no value at that time. But now, what you are reading now is lid by the LED, if you say otherwise, you should start get yourself updated to appreciate the invention ppl make.

0 upvotes
Joseph S Wisniewski
By Joseph S Wisniewski (10 months ago)

Weird. I remember growing up watching the LED become the number one display for clocks, and a common display for lots of other instrumentation from calculators to thermometers to the exposure display in my Nikon FM2 (a camera that launched about 1/3 century ago). Those "no value at that time" displays were pretty ubiquitous. How long ago did they start appearing in keychain flashlights? 25 years? 30? I remember the high brightness LEDs causing a stir in the mid 80s, and appearing on the first LED car brake lights around 1990, better than 20 years ago.

No one "looked down on" the LED. It had obvious applications from day 1, and it just kept evolving, improving in power, cost, color range, and efficiency, and each new improvement brought new uses. This aperture coding invention has no obvious uses that will cause it to grow and thrive.

As far as "what you are reading now is lid by the LED", I'm using a 30 inch Dell with a fluorescent backlight. ;)

Comment edited 54 seconds after posting
2 upvotes
Anepo
By Anepo (10 months ago)

To bad they refer to led as "flash" in cellphones and to bad they went the led route instead of a real flash like nokia has often done.

0 upvotes
Turbguy1
By Turbguy1 (10 months ago)

Sounds similar to the early mechanical Television systems that used a single neon bulb and a rotating perforated disc for the display, no?

http://en.wikipedia.org/wiki/Nipkow_disk
http://en.wikipedia.org/wiki/Mechanical_television

Comment edited 2 times, last edit 10 minutes after posting
1 upvote
qwertyasdf
By qwertyasdf (10 months ago)

At first glance, I thought it was a senseless camera
after reading the article, indeed, it is senseless.

0 upvotes
Trollshavethebestcandy
By Trollshavethebestcandy (10 months ago)

New digital Lomo
Kind of cool.
How small can they make the camera?

1 upvote
MarkInSF
By MarkInSF (10 months ago)

Not of much interest for standard photography, but theoretically impressive. Imagine some kind of 'image' made up of difficult to detect energy. Maybe each sensor needs to be large and expensive, so building an array of adequately small sensors isn't possible. This shows one way of constructing an image using just one sensor.

3 upvotes
HubertChen
By HubertChen (10 months ago)

Really impressive comment! I wonder if this could cause a revolution in space photography ? Movement of the earth could be compensated, the rest of the Universe it not going anywhere. Wavelengths could be utilized to build an image which previously were not accessible. Maybe now we can see farther or see things previously hidden ?

0 upvotes
Joseph S Wisniewski
By Joseph S Wisniewski (10 months ago)

Hubert, we do all those things now. Terrestrial telescopes make use of the parallax from the rotation of the earth and the movement of the earth about the sun (stellar parallax). Orbital eccentricities provide vertical parallax. Coordination of telescopes in different locations provides synthetic apertures.

And a lot of important astronomical imaging is done with single pixel scanning systems, especially longer wavelengths. Once you get to the far IR, single pixel devices, either coded aperture or scanning, become common. In x-ray astronomy, you see both array detectors like in the Chandra on, and single pixel detectors. Some are scanners, others use coded optics.

Comment edited 2 times, last edit 12 minutes after posting
1 upvote
FooFighter007
By FooFighter007 (10 months ago)

"Because there's no lens to focus the resultant image has infinite depth of field...."

/sarcasm "Digital 135 (i.e., FF) fanatics need not apply!!!" /end sarcasm

0 upvotes
thx1138
By thx1138 (10 months ago)

A programmable pin-hole camera in effect. Might be useful in computational photography going forward.

0 upvotes
Deleted pending purge
By Deleted pending purge (10 months ago)

A pinhole camera principle gets revisited every now and again, for more unexplored possibilities. It shows that the story of light-bending is not all told, and probably never will be.
I wish someone seriously reinvestigated the light-bending by EM field.
It would surely make use of the various-density materials (and also mirrors) for that purpose obsolete.
However, I wouldn't count very much on it as long as there still is more of the expensive glass to sell, not unlike the obvious reasons why the oil people won't allow the mass production of clean electric cars, regardless of the fact that technology now has all the means to efficiently replace the internal combustion engines.
As the technology advancement accelerates, some kind of greed always manages to slow it down...

Comment edited 4 minutes after posting
0 upvotes
ManuelVilardeMacedo
By ManuelVilardeMacedo (10 months ago)

16384 individual measurements to produce a noisy and blurry image. Let's be charitative and say it has lots of room for improvement...

0 upvotes
Anfernee Cheang
By Anfernee Cheang (10 months ago)

The first computer was as large as a factory and only calculates simple maths. And the first train was even slower than riding a horse. Having seen so many great products that start from stupid invention, it is better to keep hopes and wait.

9 upvotes
ManuelVilardeMacedo
By ManuelVilardeMacedo (10 months ago)

Actually, I have higher hopes on single-lens, multi-sensor cameras. They're even bulkier than this but somehow their concept seems more promising. This looks to me as a bold step in the wrong direction, but maybe it isn't. Or maybe it is. Let's wait and see.

0 upvotes
HubertChen
By HubertChen (10 months ago)

The first memory IC I bought was 2 Bit. Not 2 Bytes, Not 2 kB, Not 2 MB, Not 2 GB, Not 2 TB. The 2 Bit were very useful to me, and more memory was always put to more use too. I think you get the idea!

0 upvotes
ManuelVilardeMacedo
By ManuelVilardeMacedo (10 months ago)

I remember floppy disks, too...

0 upvotes
Anepo
By Anepo (10 months ago)

Its basicly like 99% of my fathers 35mm film photos then (not kidding the photos of us as kids are almost all blurry, a problem i have not had when using manual focus slrs)

0 upvotes
Mssimo
By Mssimo (10 months ago)

Yeah..but does lightroom support it yet?

1 upvote
ManuelVilardeMacedo
By ManuelVilardeMacedo (10 months ago)

No. You'll have to subscribe Creative Cloud.

Comment edited 15 seconds after posting
4 upvotes
Rickard Hansson
By Rickard Hansson (10 months ago)

ManuelVilardeMacedo - Lightroom is not covered by the Creative Cloud (thank god).

0 upvotes
Wilfred Wong
By Wilfred Wong (10 months ago)

LR is part of the CC, it just available separately as well.

0 upvotes
rrr_hhh
By rrr_hhh (10 months ago)

@ Wilfred Wong

God knows for how long this will last ! Let us hope indefinitely rather than one-two years.
That said, given the frequent updates needed to include new cameras, I find the monthly fee more justifiable for LR than for PS.

It is like Norton antivirus and firewall : you pay a yearly fee to get the regular updates, but you know that you need these updates. You don't expect new features with each updates, but recognize that to keep the product uptodate (aka able to handle the last viruses or cameras&lenses) there is a cost which the company has to recoup.

Comment edited 4 minutes after posting
0 upvotes
ManuelVilardeMacedo
By ManuelVilardeMacedo (10 months ago)

People, both Mssimo and I were just joking!

0 upvotes
HubertChen
By HubertChen (10 months ago)

@ Manuel

A good example how cynicism is not well understood by everybody. Especially consider non native speaker. But I have to say Adobe really stepped on my toes with Photoshop CC and their fuzzy answers on LR and updates in CC or not just made me more upset. So your comments did made me smile. For that I thank you.

Comment edited 33 seconds after posting
1 upvote
rrr_hhh
By rrr_hhh (10 months ago)

@ ManuelVilardeMacedo

Of course you both were joking, I had a good laugh too, but some jokes are an indication that the pill was hard to swallow, aka not all jokes are innocent. So it got me started.. I should have included a LOL at the beginning of my first answer.

0 upvotes
CameraLabTester
By CameraLabTester (10 months ago)

Another invention waiting to have a usefulness...

Keep 'em coming lab gurus...

.

1 upvote
AbrasiveReducer
By AbrasiveReducer (10 months ago)

Bill Atkinson talked about this years ago but it looks like a lot more computer power is needed.

0 upvotes
joe6pack
By joe6pack (10 months ago)

Interesting concept but I cannot see the application of this. You can do anyway with lens and get infinite DOF using pin hole camera as well.

With a single sensor, the camera needs to poll the sensor each time it modify the aperture array, which I believe is much slower than polling sensor.

0 upvotes
edu T
By edu T (10 months ago)

Military applications, perhaps? According to their abstract, "the architecture can be used for capturing images of visible and other spectra such as infrared, or millimeter waves, in surveillance applications for detecting anomalies or extracting features such as speed of moving objects." Millimeter waves lie somewhere between far IR and the highest frequency microwaves.

0 upvotes
joe6pack
By joe6pack (10 months ago)

They could use conventional sensor or any modified sensor. With such a small aperture, the image is going to be noisy as hell even with good light.

0 upvotes
forsakenbliss
By forsakenbliss (10 months ago)

This offer one way for lens-less image capturing. So minus away the distortion, light loss, defraction, color shift etc.

0 upvotes
HubertChen
By HubertChen (10 months ago)

I wonder if this could cause a revolution in space photography ? Movement of the earth could be compensated, the rest of the Universe it not going anywhere ( for the duration of days at least ). Wavelengths could be utilized to build an image which previously were not accessible. Maybe now we can see farther or see things previously hidden ?

0 upvotes
kkoba
By kkoba (10 months ago)

A very similar method is already used by astronomers to image in X-ray and gamma ray energies:

http://en.wikipedia.org/wiki/Coded_aperture

0 upvotes
JamesMortimer
By JamesMortimer (10 months ago)

Wonderful.. they've probably spent millions and invented..

.. the pinhole camera.

0 upvotes
Timmbits
By Timmbits (10 months ago)

I suppose it's a "beginning", of something... not sure what this would be useful for.

0 upvotes
ybizzle
By ybizzle (10 months ago)

Yea...

0 upvotes
Total comments: 56