78

LG patent describes smartphone camera with 16 sensors and lenses

With a total of five, LG's current flagship smartphone V40 ThinQ already offers more camera modules than most of its competitors. However, it appears the current dual and triple-camera setups are only the beginning of a multi-camera arms race in the mobile industry.

According to a recent patent filing discovered by Dutch technology publication LetsGoDigital, the Korean manufacturer could be working on a smartphone with 16 sensors and lenses in the main camera array. The individual modules are arranged in a 4 x 4 matrix and designed to record the same scene from slightly offset angles, enabling the capture of 3D video or interesting image manipulation effects.

In an example it is shown how it would be possible to rotate the head of a portrait subject (or a teddy bear in the illustration below) to the angle you like best after capture.

Another example shows how the multi-cam technology could be combined with artificial intelligence features. Users could select a subject by drawing around it. The system then searches for other photos of the same person and then offers other, potentially more flattering, "heads" to replace the original with. This could be a useful function when one person in a group shot has their eyes closed, for example.

The patent also describes how the main camera array could be used to take selfies using a simple mirror, a concept that users of older flip-phones or compact cameras might still be familiar with.

As usual there is no way of knowing if or when the technology will make it into a final product but it's great to see manufacturers are working on new ways of making the cameras in our pockets even more powerful. For more detail, you can have a look at the patent documents on the USPTO website.

View Comments (78)

Comments

All (78)
Most popular (5)
Editors' picks (0)
DPR staff (0)
Oldest first
Jesse_Just_Him
Jesse_Just_Him

When in doubt, add more camera~!!!

Dec 4, 2018
peterwr
peterwr

*Cough* Light L16 *cough*...

Dec 3, 2018
Mike FL

By using Winnie the Pooh, LG is asking Chinese communist leader XI to ban the LG. Take a look:

https://www.theguardian.com/world/2018/aug/07/china-bans-winnie-the-pooh-film-to-stop-comparisons-to-president-xi

Nov 29, 2018
Jake2046
Jake2046

well...at least LG has a reason to jack up their price.

Nov 27, 2018
Eskilsson
Eskilsson

Someone should invent a monocular that I could use as an evf för my phone. No matter how good your screen is, you always will have problem with extrem sunlight.

Nov 27, 2018
SandySandy

I would have preferred a camera with 4 sensors, 2 with 48MP 1/1.7" and 2 with 12MP and 1/1.55". And some superb computational photography for 5x Zoom at 12MP and usable ISO to ISO6400 equivalent. And 14 stops in HDR photo in HEIF format.

Also if it could have a 5k video mode, for high speed photography, that would be the cherry on the cake.

Nov 27, 2018
Franz Weber
Franz Weber

The main benefit of this 4x4 array is that you can have lenses combined from different manufacturers and all these flavors are blendet to one super-jpg photo. Zeiss, Nikon, Fuji... you name it. Think of the possibilties: A Zuiko Batis L lense.. with anti vibration, stadyshot AND image stabilizer.

It will give you more flexibility with assesories as well: Attach a lens hood on some lenses for image quality and leave it off on other lenses for compactness and convenience. And for savety reasons allways apply a lens cap on one lense which stays there while shooting

Nov 27, 2018*
ozturert

With a tiny sensor, I wonder why you'd want to pay for a Zuiko or Zeiss lens.
Plus, these phones keep getting more and more expensive every day.

Nov 27, 2018
Mike FL

"wonder why you'd want to pay for a Zuiko or Zeiss lens." while Zeiss can not wait to sell you some phone lenses. Take a look:

https://exolens.com/products.html

Nov 28, 2018
barrym1966

be cool if they 1st increased the battery life to more than 1 day and also got them to run android properly instead of being lagtastic

Nov 27, 2018
Crypt Keeper
Crypt Keeper

LG has always been one of the most underrated companies, despite being very innovative.
As I recall, they created the first 3d smartphone (LG Optimus 3D), they were the first to produce higher screen to body ratio smartphones, optical image stabilization in an smartphone camera, high resolution, high pixel density smarphone screens, modular smartphones, etc, etc, etc....

Nov 27, 2018*
SandySandy

That's true. LG innovates extremely well. Their only downside has been implementation which is not refined or perfected and quality control which could improve. If they perfected those two, they could easily be the No1 or No2 smartphone company.

Nov 27, 2018
Critical Thinker

And here comes the barrage of comments from old dinosaurs complaining about having this many lenses because they think its literally 16 different focal lengths and didnt read the article.

Nov 27, 2018
madeinlisboa
madeinlisboa

No. It's just ridiculous...

Nov 27, 2018
ozturert

I have read the article and I think it's stupid if the price is over 1000+. I want one big and good sensor in my phone. I don't need to change heads of my friends in photos.

Nov 27, 2018
beavertown
beavertown

Only Apple knows how to do photography with smartphones.

Nov 27, 2018
HowaboutRAW

Panasonic is still leading the smartphone camera category, with the CM1--discontinued 2015 (or perhaps early 2016).

Nov 27, 2018
Bobthearch
Bobthearch

Really? According to these sample comparison shots the Apple phone camera is even worse than the Google and Huawei models.

https://www.dxomark.com/huawei-p20-pro-camera-review-innovative-technologies-outstanding-results/

Nov 27, 2018
HowaboutRAW

Bobthearch:

I assume beavertown is joking.

No, in all seriousness, DXO sensor scores are next to useless.

Separately, yes, the raws I've extracted (not shot myself) with the Huawei are impressive for a current (2018 available) smartphone.

Nov 27, 2018
Bobthearch
Bobthearch

I give no credence to their sensor scores, as the scores are only relative to other camera phones; I was looking only at the side-by-side photo samples.

The Huawei photos are pretty bad, but the Apple phone's are even worse.

Nov 27, 2018
HowaboutRAW

Bobthearch:

Right, Apple has some catching up to do even in the limited market that is smartphone cameras.

Samsung could introduce a smartphone based on the already developed NX-Mini ILC system and have the best smartphone camera out. Or Panasonic could introduce the CM2 with the same lens/lens and a bigger screen, more RAM and a faster CPU and best every other smartphone camera for a while.

Nov 27, 2018*
Charlie Jin

Apple knows how to hypnotize people.

Nov 27, 2018
ozturert

Only Apple knows how to turn your face to plastic and sell their "effect" for USD1000+.

Nov 27, 2018
Mariano Pacifico

I agree ... Apple knows how to do photography with $50.00 lens ... I ABSOLUTELY AGREE !!! And they have beauty effect that makes ugly faces beautiful and appealing ...

Nov 27, 2018*
EXkurogane
EXkurogane

Meanwhile, Google pixel 3 with only 1 camera: Am I a joke to you?

Nov 26, 2018
Mariano Pacifico

1 camera in smartphone is sooooo 6 months ago.

Nov 27, 2018
falconeyes
falconeyes

I browsed through the patent application which dates back 5 years.

It seems it is looking desparately for applications of a 4x4 camera array. And it fails at that, it’d say.

The most important and innovative use case of a 4x4 array would be to multiply an otherwise huge crop factor by 0.25, bringing smartphones pretty close to system cameras in terms of image quality. And that seems to not be even mentioned in the patent.

Nov 26, 2018*
Mariano Pacifico

Ohhh Mmmm Geeee !!! How am I going to hold the smartphone without smudging 16 lenses?

Will there be a Nikon & Canon smartcamera? Right at this moment these soon-to-be-irrelevant behemoths do not have smartcameras while cellphones have already evolved into smartphones.

Nov 26, 2018
ikan154
ikan154

use 16-legged pod for best stability

Nov 26, 2018
HowaboutRAW

Sorry, which smartphone competes with say a half decent Canon lens mounted on a Canon 6D Mark 1 for plain image quality?

Which 2018/17 smartphone can you easily shoot at ISO 2500 again?

Nov 26, 2018*
mxx
mxx

Or at 600mm?

Nov 27, 2018
brn

With a 16 way zoom lens attachment, of course.

Nov 27, 2018
Mariano Pacifico

16-lens smartphones will have ultra-wide, normal wide, 35mm, normal lense 50mm, 80 & 105 portrait lenses ... how many is that now ... 200mm ... 300mm ... 600mm ... that is 9 lens total 7 more lenses to go ... wheeew !!! That is more than a bagful of lenses ...

Of course it cannot compete with 6D Mark 1 & ISO 2500 but the convenience when we need it is there at an instant in our pocket. If we have to fiddle and tweedledee we lost that "moment".

Nov 27, 2018
HowaboutRAW

MP:

But then a Sony RX100 IV or a Panasonic ZS200 would be better pocket cameras.
And both of those one can readily shoot at ISO 2500.

No bag full of lenses needed with the Panasonic and Sony.

Nov 27, 2018
RUcrAZ
RUcrAZ

Great news! It means that there will be a war among cellphone makers, to improve their camera function. Cellphone-cams are still at their infancy, give them time. (I remember the early digicams were like the Sony that took 1 single digital photo on a whole floppy disk; and look how far we've come!) If it wasn't for WWI and WWII, airplanes may still be biplane stringbags! The (new to me) software-massaging photo approach by Google Pixel, coupled with multiple lenses & better sensors is obviously the start (just the start) of something significant.

Nov 26, 2018
photophile
photophile

My Gillette razor now has 17 blades - but I think I'd get a closer shave if I had 18.

Nov 26, 2018
webber15

No...fewer but larger blades are the way to go πŸ˜‚

Nov 26, 2018
Lars V

Yeah we did this (and patented) at Pelican Imaging several years ago. This is nothing new. I see lawsuits coming up.

Nov 26, 2018
Jon Stern

This patent seems to be for an array with each camera having a different FOV.

Quote translated from a German site: "The lenses are placed in a certain curvature within the matrix arrangement, so that the different lenses can make photographs from multiple perspectives."

Nov 27, 2018
Lars V

Thanks, Jon. Presumably a convex curvature. Or possibly multiple perspectives refers to parallax resulting from offset, which is then used to estimate depth. I hate reading patents.

Nov 27, 2018
Jon Stern

It seems to be curvature of the mounting plane, not just parallax.

Nov 27, 2018
UllerellU

I could never take photos with a cell phone. Not even if in the future they were able to offer good image quality and low light capacity. For me the hergonomy and the pleasure of use is everything. I have replaced my compact RX100 for an LX100 for the same reason, I have made good shots with the RX100 in these years, but every time I use it I realize that I do not enjoy everything I would like, regardless of the result. And I do not like to shot without viewfinder, nor in my third camera body. And with the RX100 I have a physical button for the exposure lock, apertura ring, focus ring for fine MF, exposure compensation in two touches, shooting mode in two touches and functions such as measurement mode, white balance etc. in a quick submenu. Despite not being a camera that is too pleasant to use, it is a million times more pleasant to use than a mobile phone.

Nov 26, 2018
Paul_B Midlands UK
Paul_B Midlands UK

Wait till you try a Leica Q ...such a lovely thing to make photos. I used to like Sony till I got this.

Nov 27, 2018
mxx
mxx

The cell phone may not be great on ergonomics, but if it's the only camera you have on you, it sure is very handy.

Nov 27, 2018
UllerellU

I always carry my compact, even on business trips, it is easier for me to forget the phone at home, in the hotel or in the car before I forget the camera...

Nov 27, 2018
madeinlisboa
madeinlisboa

LOL! Another proof that cellphone companies are going completely nuts

Nov 26, 2018
JochenIs

Nices sales argument. My LG G3 is still working fine btw. :-)

Nov 26, 2018
QuangFoto
QuangFoto

No kidding. I replaced my G4 with the V30+, and this thing is amazing.
https://youtu.be/nAth8g05tfs

Nov 26, 2018
sfft
sfft

What about a patent that cools off motherboards in LG mobile phones?

Nov 26, 2018*
duartix

I'd buy that!

Nov 26, 2018
Marty4650
Marty4650

One good sensor will outperform 16 mediocre ones. No matter how many computational tricks you use.

Just remember.... those camera makers can use computational tricks too whenever they feel the urge to do so. And real cameras will ALWAYS have an ergonomic advantage. There is a fundamental conflict between "pocketable" and "can do everything you want done."

If you need images for web display, then a six year old smartphone will do the trick. If you need something significantly better, then you will need a real camera with a much larger sensor. So it really makes very little sense to make smartphones "more like a camera" or cameras "more like a smartphone." These are two different markets with two different type of user, ever since the bottom of the camera market collapsed.

Right now people are spending thousands of dollars for full frame cameras that are only marginally better than crop sensor cameras. And they do this because those marginal differences are very important for them.

Nov 26, 2018
duartix

Define mediocre.
Because if mediocre is 2x worse, with 16 cameras sensors you are still 8x ahead in light gathering capability.
Computational photography has come to stay. If camera manufacturers don't get it (some have already and have started producing SR from 4/8 exposures) , they will slowly continue to wither.

Nov 26, 2018
Michael Long

Reminds me of the early days when digital was just beginning to replace film and all of the film guys were talking about how digital would "never" outperform film.

It also strikes me that the OP simply doesn't realize just how many "computational tricks" are already being used to pull an image off a Bayer color filter array.

Next, "real" cameras have ergonomic advantages.... coupled with ergonomic disadvantages, namely size and weight.

Finally, marginal differences are in fact important... to an ever shrinking number of people. IQ is steadily increasing and each gain makes the final result "good enough" for an ever growing set of use cases.

IQ is important... but it's not the sole consideration, nor is it even the most important one. If it were, everyone would still be shooting with 8x10 view cameras...

Nov 26, 2018
HowaboutRAW

Michael L:

And yet as of late 2018, the IQ of not one current smartphone camera comes close to a decent current 4/3s sensored body and a good lens.

It's pushing things to use any smartphone available new above ISO 1000 in 2018.

And remember, digital took a good while to get up to producing an APSC sensor one could shoot at ISO 800 easily. But then things quickly took off, and raw extraction software got much better, and the flashcards got much faster, much cheaper, and more capacious.

Multi-lensed cameras show some promise, but it's years before they will replace conventionially lenses cameras of fixed focal length. Right, remember there's the non-tele/zoom problem with 4 or 16 sensors.

Nov 26, 2018
Arastoo Vaziri
Arastoo Vaziri

Duartix, the "light gathering" is a myth DPR staff invented in order to sell more smartphones to gullible gearheads who will swallow anything if it has a pseudo-scientific reasoning behind it, but there's no truth to it. Besides, the extra sensors aren't used in tandem, so the gathered light will never be the same as per a, say, APS-C sensor
Michael Long: digital photography hasn't caught up with medium and large format film yet. And a smartphone *is* an ergonomic nightmare. I'd go as far as to say it's a PITA to shoot with.

Nov 26, 2018*
panther fan

"And yet as of late 2018, the IQ of not one current smartphone camera comes close to a decent current 4/3s sensored body and a good lens."

https://www.youtube.com/watch?v=wrdKGvuw4pU&t=1s

Except in some use cases, where it already does. For example hand held low light photography... the last thing people expected from a smartphone... but it beats M43 handily. And if you need deep DOF it even beats FF

Computational photography is here to stay and will probably drastically improve in the next years. Something I am not so sure about is, that cameras will develop equally fast

Nov 26, 2018
HowaboutRAW

panther:

In 2018, there isn't really a smartphone one can shoot at ISO 3200.

While all new 2016-18 4/3 sensored cameras can easily shoot at ISO 3200.

There are of course many more fast lenses of m4/3s cameras.

So no, you are wrong.

"Computational photography is here to stay and will probably drastically improve in the next years."

It will stay around, it is unlikely to drastically improve in the next few years. 15-20 years out is a different subject.

"And if you need deep DOF it even beats FF."

Is this supposed to be a joke? Do you know what deep DOF means?

Please don't go on about stacked images from multisensored cameras being able to reduce final output noise. Someone here already embarrassed himself badly last week with claims about the Google Pixel 3 and moving images in the foreground.

Ironically, 2014 saw the release of a smartphone one could use at ISO 3200: The now discontinued Panasonic CM1.

Nov 26, 2018*
panther fan

@HowaboutRAW

The fact that you talk about ISO 3200 shows that you don't quite understand current tricks in computational photography.

We live in a time, where you need to pixel peep night shots to tell the difference. Just search the internet for google night sight vs DSLR and you will see tons of comparisons, showing that the pixel crushes night photography

PS: Go and search a comparison with moving objects. And you will see that the Pixel3 works phantastic:
https://www.youtube.com/watch?time_continue=553&v=csBDl85I69A
for example.
Or look at the bazillion low light portrait comparisons

Nov 26, 2018*
HowaboutRAW

panther fan:

You still have the image stacking problem; it's great for static images. Which are few and far between.

Nothing to do with pixel peeping. In 2018, there is no ISO (gain if ISO confounds you) 3200 capable smartphone.

"Just search the internet for google night sight vs DSLR and you will see tons of comparisons, showing that the pixel crushes night photography".

Again, not if something is moving the the foreground. Then then no, no smartphone in 2018 crushes any contemporary DSLR for low light. Now a DSLR from 2004 is close to that low bar the Pixel 3 pretends to leap.

Nov 26, 2018
panther fan

@HowaboutRAW

You didn't even bother to check it out yourself. Yes stacking has some problems. But movement surpression works really well in the pixel. And that includes complex moving subject like humans.

Yes you can push a modern FF a little further in some scenarios. And that is great. But does it matter for regular people? No it doesn't. And that is the real problem. Without a large camera market, we will simply see no fast development anymore. A real problem if you look at the grwothrate smartphone cameras have

Nov 26, 2018
HowaboutRAW

panther fan:

"You didn't even bother to check it out yourself. "

When you provide raws, I'll be happy to download them and extract them.

I'm not going by silly YouTube assertions.

"But movement surpression works really well in the pixel."

No it doesn't. This is well established, and that you don't need raws to check. You can find examples on YouTube of big failures of the Pixel 3. Now they're where you'd expect the failures to be--car moving across foreground.

"regular people" got along just fine with 35mm point and shoots in the 1980s and 1990s; then more serious types who cared about greater performance used SLRs. Today in 2018, DSLRs have the added advantage of much better sensors.

I'm guessing you don't remember when it was radical for a DSLR to be easy to use at ISO 3200. I'm guessing you have no idea how much AF has improved on mirrorless and DSLRs in the last say 8 years. Well I'm not actually guessing.

Nov 27, 2018
panther fan

" I'm guessing you have no idea how much AF has improved on mirrorless and DSLRs in the last say 8 years. "

I have. I did go mirrorless with the first FF wave. I owned an A7, A7 II and rented an A9 multiple times, because my A7 II just doesn't cut it for events.

"When you provide raws, I'll be happy to download them and extract them."

Give me a week. And I will provide you some RAWs. I will probably migrate from my Nexus 5X to the Pixel 3 in a few days. So I will happily upload some RAWs

Nov 27, 2018
HowaboutRAW

panther:

Great, I'll look at higher ISO raws.

Nov 27, 2018
Akpinxit

this is going to be some challenge for add on lens manufactures

Nov 26, 2018
ozturert

It's heaven for them. Just an excuse to raise prices.

Nov 27, 2018
zeratulmrye

Sony Semiconductor: atta boy, good job!

Nov 26, 2018
GodSpeaks
GodSpeaks

Seriously? I think I will pass.

Nov 26, 2018
Brian P Smith

This is getting to be like razor blades were years ago... just keeping adding more of the same.

Nov 26, 2018
duartix

Well, look at history. It worked for razor blades...

Nov 26, 2018
mxx
mxx

I still shave with a double blade, and it works fine, and is cheap.

Nov 27, 2018
Arastoo Vaziri
Arastoo Vaziri

This is getting ludicrous. It's not evolution, it's just gimmickry.

Nov 26, 2018
Suntan

Most likely the system is designed for better 3D mapping. IE, for augmented reality. Potentially a gimmick, but something that all the phone makers are working towards.

This isn't just "moar cameras" for the sake of more cameras.

Nov 26, 2018
mr_landscape
mr_landscape

Yeah, let them keep on stacking small sensor sized cr#p-over-the-top.
N808 is the best camera-phone till today.

Nov 26, 2018
princecody
princecody

Can the 1 inch compact pocket camera still exist with new smartphone tech?

Nov 26, 2018*
HowaboutRAW

Yes. And that will be true for years to come.

You "forgot" that those compacts have optical zooms.

Go find raws from the Light multi-sensored camera.

Nov 26, 2018*
Michael Long

You'll note, however, that the short zoom pocket camera is all but disappearing.

Nov 26, 2018
HowaboutRAW

ML:

Not familiar with the RX100 V and the Panasonic LX10 I see.

The LX100 II also doesn't have a particularly long zoom. But true that's more jacket pocketable.

And note how princecody specified 1 inch compacts, not 1/2.3 inch compacts.

Nov 26, 2018*
GeraldW

It really depends more on how many of us are willing to spend for that 1" sensor camera, and are serious enough about their photography to need one. I see the smart phone camera as the equivalent of the Instamatics and box Brownies of the past, and the low end digital P&S models. An easy to use camera for the snap shooter. Once the smart phone got decent cameras, those same buyers left the major camera brands. The result was that there were not enough buyers left to make them viable products.
What helps the 1" sensor models is the high prices. You need to be pretty sure about your photography needs to justify that kind of price. We can argue indefinitely that camera phones aren't a match to 1" sensors. For most folks, they couldn't care, and are happy with their phones.

Jerry

Nov 27, 2018