During its I/O 2017 conference yesterday, Google demonstrated a new algorithm-based technology that can remove unwanted objects from existing photographs. The demonstration showed the technology removing a chainlink fence from the foreground of an image, with the final result offering no discernible indications that the fence had ever existed (around 10:45 in the video below).
The technology was demonstrated on stage by Google's CEO Sundar Pichai during a conversation about the company's expanding visual technology. 'Coming very soon,' Pichai explained, 'if you take a picture of your daughter at a baseball game and there's something obstructing it, we can do the hard work and remove that structure and have the picture of what matters to you in front of you.'
It looks to be an evolution of the research Google and MIT have been collaborating on for some time – in fact, their demonstration from 2015 includes a very similar chain-link fence demo. This method takes advantage of the parallax effect to identify and remove obstructions from photos.
Unfortunately, Pichai didn't elaborate on when this technology will be made available aside from 'very soon,' nor did he specify where the technology will be available. Given the company's Google Photos announcements, however, it seems likely the technology will be implemented within that product.
Oops - I missed that there was a reference to 10:45. And - it is too late to edit my post. Anyhow - ignore my post. If someone from DPR sees this, please remove it (and this).
I'm Sorry, but that was not a demonstration of the technology. It was an illustration of the technology, or perhaps a description of it. Those chain-link fences were simple overlays, that they then removed, at the precise moment he says "in the future".
What? You are sure? In that case, I am quite stunned. Hmmm ... that the second image has a perfectly sharp fence hints at something. Enormous DOF for that camera.
I listened at 10:45, and yes, he said "coming very soon" we can do this. So, it was fake. And the audience cheered. Very effectful presentation. And also very manipulative. Lets see when and if this really is released.
yes. I saw that--it's from 18 months ago and involves removing the foreground from a still frame of video image...but the "demo" at 10:45 doesn't demonstrate to us that technology in any way. It's an illustration that what that technology "could" do, "soon". The "demo" doesn't appear to be showing an actual removal...the foreground fence doesn't look like it's part of the original image. I certainly have no doubt that this will be, or is, possible, but the "demo" in the video doesn't appear to be an actual demonstration of the technology (which the article reports, incorrectly)...which is all I said.
As for the technology itself, I'd love to see what it does if that little girl decided to "not be a statue" and actually swing the bat. The results might vary a little. :-)
Two more thoughts. I think the isolation of the reflected image off glass is actually more interesting...the fact that you could pull that much detail from the reflection has numerous creative and "forensic" uses. The second thought is why would anyone shoot with their smart phone 6-12 inches from glass or a chainlink fence? The lens on a smartphone is small enough that it would have little trouble peering through 1/2 inch wire fencing much less 2-3" chainlink. And pressing a phone camera lens against a window generally leads to largely reflection free photos...
This is not an idea. Talk about this has been going on for ages and there are some (more or less successful) tries. Now it is about implementation. The idea stadium was in another millenium.
Interesting - but there is a problem. In the second example the camera is focused on the chain link fence. When the fence is removed that does not change, so then the entire picture is out of focus!!! Loren
I find that if something is in the way from my viewpoint and I can't get round it then it makes me think harder, can I actually make it an active part of my composition ? Fences too.
They removed the chain link fence. Why stop? Remove the ugly construction materials and ugly buildings, replace the asphalt with grass and put leaves on the leafless trees.
Usually, I take the shot through a hole in the chainlink fence. It is even easier... ;-) Or take the shot with a lens allowing a DOF shallow enough to blurr the fence away.
All the years I sat at Little League games staring through chain-link fences, I kidded about inventing holographic sunglasses that would filter out chain-link fences. Maybe Google can help me...
It is but a matter of time. Narrow AIs can achieve above human level easily for particular tasks already. Some aspects of image recognition being one of those tasks... this is just a logical step in that direction.
Not long from now, they will probably do almost everything better and faster than us.
Not very sure, but they might have expanded the U-point technology that Nik first developed and which Google purchased a couple of years back? Nikon was using the U-point in their Capture NX software.
I can imagine turning up to a place to take a picture at a particular spot because I saw it on the internet and want to take one of my own, only to say, "Ugh, there is an ugly chain fence here!!". This tech will be a bit of both good and bad, depending on who you are I guess.
I can see this technology being useful, but then also leading to laziness too, for some. I think it may cause some photographers (particularly newbies) to become "lazy" and say "Oh, I'll just remove it in post processing" and not try to move around the obstruction (if it's possible) to get a better shot. I hear this statement a lot from beginners: "I'll just remove that later." Some things can be fixed in post, some things cannot.
The parallax method, requiring at least 2 photos from slightly different PoV can be done using Hugin. Done it several times to remove inconvenient telephone poles, tripod legs, or some random person.
The description says "existing photos" which in most case would not have been shot with obstruction removal in mind.
So which is it? A well understood compositing method which requires a pair of special shot photos, or a new method that can take an arbitrary single image, as shot, and perform these operations?
And if there isn't a hole where you want it to get the shot you want? If Google want to help photographers like with the free NIK filters,let them do it.
Today it is quite common to put fences at high buildings and bridges in order to hinder suicide and throwing large objects. Those fences are usually a bit away from you. No possibility to take the image through any hole. Actually, one of my all favorite photo spots is destroyed this way. A high bridge with a view over Stockholm. Now there is a fence. OK - it is possible to hold the camera out on the other side of the fence, but taking panoramas and also using tripod is no longer possible. This technique (if and when it works) sounds like a good idea for me.
Lots of things that make sense do not make sense. The bridge I am talking about is 26 meter high. The probability to survive is almost 100%. For some reason it is almost impossible to get statistics on suicides from the bridge. But, if I dig deeper, it looks like it almost never happens. It seems to be a myth. But, all "know" it is the most common way to commit suicide in Stockholm, something that definitely is untrue. Jumping from high houses, jumping in front of a train and medicine are the most common.
but if they stop putting fencing on every bridge and making kids wear helmets to chess matches , what , pray tell will our nanny state do?
I know of no cannabis fatalities here in America , but legal booze cigarettes and big -pharma heroin kill untold numbers but are just too cash imbued for the nannys to care ... instead they will continue to care where they are not wanted
Removing something with a consistent pattern is fine. What about removing more complex, discreet objects from the image? I occasionally make use of Photoshop context sensitive fill, and that sometimes takes quite a bit of work with layers and masks to i) make sure only what I want removing is actually removed and ii) what it gets replaced with looks credible.
This does look incredible. If it uses the parallax effect, that would imply the need for two slightly offset photos (like stereo photography), however, it states it can work on existing photos.
I would love to see this technology integrated into Google's Nik software in the future. In the meantime, it's made me think about taking two offset photos where there are obstructions close to the camera, overlay them in PS, then use the mask tools to eliminate the obstruction from the top layer. Hmmmm.
That's probably because it is. I'd like to see how it would deal with text or humans in the background, or a much more out of focus chain link fence. Reminds me very much of when Adobe acquired the rights to the Context Aware / Fill technology. Yes, it can be a useful starting point, but nowhere near as good as their examples suggested.
There are things I cannot do which others can do well. So I need other people. But what if I can create technology that begins where my capabilities end? I will no longer need others to do things I cannot do or those things I wish to avoid doing. I will be able to direct my technology to do everything I want to do but don't do well myself. I will be an emperor in an empire of my own choosing. I will only see a world of my choosing. I will only interact with others I choose. There will be no uncertainty. There will be no inefficiency. There will be no accidental natural wonder. And I will never have to face my limitations, other than the limits of my own imagination, because I will have no limitations to know. Or will there?
My thoughts. I'm pretty sure the world is nowhere near ready for everyone to be gliding around carefree in a seamless, hassle-free existence in which we are assisted and cared for in almost every manner by AI.
looks good unfortunately, the Pros amongst us wont use it and the $200 husband and wife teams on facebook are too lazy to edit their photos . So not sure who the market is
Photography is becoming more and more than just about the lenses and sensors. That's why Nikon is falling behind (I think). They are not really into software things. Nikon (and other manufacturers for that matter) must offer cool tricks like this to attract casual users.
It's not this (or any) technology that would cause someone to be uncreative, but the way one utilizes it. One can use a tool in an uncreative way and someone else may utilize it in a highly creative manner. The example that they show of removing fence seems pretty basic - obviously to show, very simply and in a way everyone enderstands, what this technology can do. I'm sure it can actually be used to do some very interesting things.
Right now I am using my creativity and my brains to either find a way to make a photograph where the fence doesn't show up on the image. Or try to incorporate the fence in an interesting photo.
Because of the google technology there is no need for that. It's that simple.
Did you watch the whole video? Google photos will also select your best photos for you and print it in a hardcover book. No need to spend precious time choosing the photos you want, let the algorithm decide!
Sadly no, the only possibly humorous part of this tech is that Google is taking it seriously enough that they are giving everyone in the (very huge) audience a free trial of the technology (i.e. a free photo book, with images automatically selected from all their Android photos).
Although I admit the facial recognition technology with auto-album update is kinda interesting. Now, if anyone takes a photo of your kids with an Android phone, you can automatically get a copy of the photo compliments of Google sharing technology :p
For most people, creativity never had anything to do with it in the first place. They're trying to preserve a memory or communicate, not make a work of art. This is just a tool to get a job done.
healing brush is completely software based and just guesses what's behind the fence
this is hardware based, it uses parallax error like is said in the article....same way humans with two eyes can see everything that is happening behind a fence
you're combining overlapping images..you're not guessing what's behind the fence like a healing brush, you can see what's behind it due to the overlapping image...the parallax error shifts the viewing frustrum, combine both correctly and the fence is gone
Might be difficult as most moire are very tiny patterns. If anything, the process will be ADDING the lines back that the moire ruined in the first place.
That's the future of photography. With increasing computational power more and more thing will become possible. And smartphones will use that power via cloud computing. Software will even overcome many limits of technology and will create images even DSLRs can't keep up with
with smartphone dual sensors they can now cleanly remove any element in the foreground without any loss in detail in the background
really, the next evolution will be cameras that can recognise objects with the help of dual cameras giving them depth..in the video the camera can regonize a flower..but this is just the beginning
with the help of depth and AI allowing them to recognize patters and colors and looking them up in a database...the cobots will truly have AI..see things..recognise it..learn and evolve..we'll have cobots doing household chores as if it's normal
All those are already possible with todays smartphones. You point camera around and in real-time it speaks what it see, like chair, apple, orange, teacup, chairs and table, newspaper on table etc...
Really? I don't get that attitude. LR automatically adds them when uploading to Flickr so why wouldn't you use it? Maybe it will stop someone from using your photo without telling you someday, who knows. And it's fun to put it there anyway, just more artistic expression.
@Docno I don't think that speaks to a sense of entitlement at all - it sounds like a different problem entirely. But I don't know the eastern half of the world well (except through the eastern immigrants who are definitely not entitled here) so I could not speak for the east.
In a broad and very general sweeping statement, I find that the people who are most possessive are the ones that have no reason to be, to put it politely. Of course that isn't the case for everything
You are affecting greater knowledge than you possess by making sociological assumptions about which you know very little. Now you can know what it means also (here's another example). ;)
I'll explain in simple terms shall I. I don't mean watermarks on agency images, I don't mean that I want to steal images from hard working individuals.
I rarely see good work posted in this forum that has a water mark attached to it or a 'photography by JT26' label somewhere on it for example. All the good images seem to be water mark or name free...
Secondly when I say 'posess' I meant photography ability. Shame I have to spell this out for you as it's kinda takes the edge off it all
The EF-M 32mm F1.4 is a welcome addition to Canon's APS-C mirrorless lens lineup. It's a good performer all-around and enjoyable to use on the EOS M50, and we hope to see more like it introduced to the EF-M range.
We don't often get excited about $900 cameras, but the Fujifilm X-T30 has really impressed us thus far. Find out what's new, what it's like to use and how it compares to its peers in our review in progress.
The S1 and S1R are Panasonic's first full-frame mirrorless cameras so there's a plenty to talk about. We've taken a look at the design and features of both cameras and have some initial impressions, as well.
The Olympus OM-D E-M1X is a more powerful dual-grip evolution of the E-M1 II. Aimed at sports shooters it promises improved AF, including advanced subject recognition, along with the highest-ever rated image stabilization system.
If you're looking for a high-quality camera, you don't need to spend a ton of cash, nor do you need to buy the latest and greatest new product on the market. In our latest buying guide we've selected some cameras that while they're a bit older, still offer a lot of bang for the buck.
A question frequently asked on the Internet is 'what's the best DSLR?' In this buying guide we've answered that question – but also whether it's the right question to be asking in the first place.
What's the best camera for under $500? These entry level cameras should be easy to use, offer good image quality and easily connect with a smartphone for sharing. In this buying guide we've rounded up all the current interchangeable lens cameras costing less than $500 and recommended the best.
Whether you've grown tired of what came with your DSLR, or want to start photographing different subjects, a new lens is probably in order. We've selected our favorite lenses for Sony mirrorlses cameras in several categories to make your decisions easier.
Whether you've grown tired of what came with your DSLR, or want to start photographing different subjects, a new lens is probably in order. We've selected our favorite lenses for Canon DSLRs in several categories to make your decisions easier.
We've had a little more time to shoot around with Panasonic's high-resolution full-framer and thus, top off our sample gallery. We've also had the chance to add a whole lot of Raw conversions.
Lens manufacturer Tokina has officially released details, price and on-sale dates for the Opera 16-28mm F2.8 lens it first showcased at Photokina in back in September. Expected to ship mid-March in Canon EF and Nikon F mounts, this wide-angle zoom will cost $699.
The Ricoh GR series has long been a favorite of street photographers, and the latest iteration - the GR III - brings a new sensor, redesigned lens, in-body stabilization and on-sensor phase detection. We spent some time with a pre-production model in London and have some initial impressions to share.
The Ricoh GR III made its official debut today, and DPR contributor Damien Demolder got his hands on the camera for a quick photo walk through London. Take a look at the results.
Ross Lowell was a man of many talents who had more than 25 patents to his name, created a lighting company and created gaffer tape, a staple in the camera bags of photographers and cinematographers the world over.
Light has announced it's teaming up with Sony to combined experience and technology in their respective fields to create the next-generation of multi-camera smartphones.
The Ricoh GR III will be going on sale this March for $899. It has a 24MP APS-C sensor, newly designed 28mm equiv. F2.8 lens, in-body image stabilization and on-sensor phase detection.
Ricoh's new WG-6 is the company's latest waterproof camera, with a 20MP sensor, 28-140mm equiv. lens and the ability to go 20m/65ft underwater. If you need something that's both crushproof and chemical-resistant, there's the G900, which is designed for industrial use.
Lomography has launched the Lomogon 32mm F2.5, a compact lens with full frame sensor coverage and a unique wheel of aperture stops that protrudes from the barrel.
At its Galaxy Unpacked event, Samsung has officially unveiled the Galaxy S10 and S10+ with a triple rear-camera array, as well as a more basic S10e model with a dual main camera unit. As expected, the S10 series' display is the center of attention with a hole-punch style front-facing camera embedded in the screen.
Samsung wasted no time unveiling the Galaxy Fold at its Unpacked event today – a foldable device with a 4.6" display when folded, and 7.3" display when unfolded. The device contains a total of six cameras – three on the back, two inside and one front-facing camera.
Photographers Ben Horne is asking for help to find the owners of a battered Fujifilm camera that fell from the top of Angels Landing in Zion National Park.
Taiwanese lens manufacturer William Optics is proposing to make a flatfield Petzval lens aimed at star gazers and photographers that it claims is the world’s sharpest 250mm.
After a rare Seattle snowstorm finally subsided, DPReview editor Jeff Keller was able to escape the snow and spend some time with the impressive Fujifilm X-T30, a camera that offers a lot of bang for the buck.
Given that it uses the same sensor and processor as the X-T3, it's no surprise that the Fujifilm X-T30 is capable of producing some excellent photos. We took a pre-production X-T30 all over the Seattle area and have plenty of photos for your viewing pleasure.
Tamron has announced three new full-frame lenses slated to launch in the middle of 2019: an SP 35mm F1.4 Di USD and 35-150mm F2.8-4 Di VC OSD for DSLRs, as well as an ultra-wide 17-28mm F2.8 Di III RXD for Sony E-mount cameras.
George Mendonsa, the gentleman kissing a woman believed to be Greta Zimmer Friedman in Alfred Eisenstaedt's iconic image titled 'V-J Day in Times Square,' has passed away at the age of 95.
Want to know more about the Canon EOS RP? We conducted a live Q&A that you can watch here. We'll be trying to address those comments we didn't get to in the comments.
The EF-M 32mm F1.4 is a welcome addition to Canon's APS-C mirrorless lens lineup. It's a good performer all-around and enjoyable to use on the EOS M50, and we hope to see more like it introduced to the EF-M range.
The data breach we reported on last week did not only affect 500px but a total of 16 websites, including mobile image sharing platform EyeEm, Animoto, Artsy and Fotolog.
Camera Rescue, a Finnish organization determined to rescue more than 100K analog, has already saved 46,000 cameras and plans to more than double that number by 2020.
Independent lens manufacturer Sigma has announced that its new 28mm T1.5 cine lens for full frame sensor cameras will be available from the middle of March.
Comments