During its I/O 2017 conference yesterday, Google demonstrated a new algorithm-based technology that can remove unwanted objects from existing photographs. The demonstration showed the technology removing a chainlink fence from the foreground of an image, with the final result offering no discernible indications that the fence had ever existed (around 10:45 in the video below).
The technology was demonstrated on stage by Google's CEO Sundar Pichai during a conversation about the company's expanding visual technology. 'Coming very soon,' Pichai explained, 'if you take a picture of your daughter at a baseball game and there's something obstructing it, we can do the hard work and remove that structure and have the picture of what matters to you in front of you.'
It looks to be an evolution of the research Google and MIT have been collaborating on for some time – in fact, their demonstration from 2015 includes a very similar chain-link fence demo. This method takes advantage of the parallax effect to identify and remove obstructions from photos.
Unfortunately, Pichai didn't elaborate on when this technology will be made available aside from 'very soon,' nor did he specify where the technology will be available. Given the company's Google Photos announcements, however, it seems likely the technology will be implemented within that product.
Oops - I missed that there was a reference to 10:45. And - it is too late to edit my post. Anyhow - ignore my post. If someone from DPR sees this, please remove it (and this).
I'm Sorry, but that was not a demonstration of the technology. It was an illustration of the technology, or perhaps a description of it. Those chain-link fences were simple overlays, that they then removed, at the precise moment he says "in the future".
What? You are sure? In that case, I am quite stunned. Hmmm ... that the second image has a perfectly sharp fence hints at something. Enormous DOF for that camera.
I listened at 10:45, and yes, he said "coming very soon" we can do this. So, it was fake. And the audience cheered. Very effectful presentation. And also very manipulative. Lets see when and if this really is released.
yes. I saw that--it's from 18 months ago and involves removing the foreground from a still frame of video image...but the "demo" at 10:45 doesn't demonstrate to us that technology in any way. It's an illustration that what that technology "could" do, "soon". The "demo" doesn't appear to be showing an actual removal...the foreground fence doesn't look like it's part of the original image. I certainly have no doubt that this will be, or is, possible, but the "demo" in the video doesn't appear to be an actual demonstration of the technology (which the article reports, incorrectly)...which is all I said.
As for the technology itself, I'd love to see what it does if that little girl decided to "not be a statue" and actually swing the bat. The results might vary a little. :-)
Two more thoughts. I think the isolation of the reflected image off glass is actually more interesting...the fact that you could pull that much detail from the reflection has numerous creative and "forensic" uses. The second thought is why would anyone shoot with their smart phone 6-12 inches from glass or a chainlink fence? The lens on a smartphone is small enough that it would have little trouble peering through 1/2 inch wire fencing much less 2-3" chainlink. And pressing a phone camera lens against a window generally leads to largely reflection free photos...
This is not an idea. Talk about this has been going on for ages and there are some (more or less successful) tries. Now it is about implementation. The idea stadium was in another millenium.
Interesting - but there is a problem. In the second example the camera is focused on the chain link fence. When the fence is removed that does not change, so then the entire picture is out of focus!!! Loren
I find that if something is in the way from my viewpoint and I can't get round it then it makes me think harder, can I actually make it an active part of my composition ? Fences too.
They removed the chain link fence. Why stop? Remove the ugly construction materials and ugly buildings, replace the asphalt with grass and put leaves on the leafless trees.
Usually, I take the shot through a hole in the chainlink fence. It is even easier... ;-) Or take the shot with a lens allowing a DOF shallow enough to blurr the fence away.
All the years I sat at Little League games staring through chain-link fences, I kidded about inventing holographic sunglasses that would filter out chain-link fences. Maybe Google can help me...
It is but a matter of time. Narrow AIs can achieve above human level easily for particular tasks already. Some aspects of image recognition being one of those tasks... this is just a logical step in that direction.
Not long from now, they will probably do almost everything better and faster than us.
Not very sure, but they might have expanded the U-point technology that Nik first developed and which Google purchased a couple of years back? Nikon was using the U-point in their Capture NX software.
I can imagine turning up to a place to take a picture at a particular spot because I saw it on the internet and want to take one of my own, only to say, "Ugh, there is an ugly chain fence here!!". This tech will be a bit of both good and bad, depending on who you are I guess.
I can see this technology being useful, but then also leading to laziness too, for some. I think it may cause some photographers (particularly newbies) to become "lazy" and say "Oh, I'll just remove it in post processing" and not try to move around the obstruction (if it's possible) to get a better shot. I hear this statement a lot from beginners: "I'll just remove that later." Some things can be fixed in post, some things cannot.
The parallax method, requiring at least 2 photos from slightly different PoV can be done using Hugin. Done it several times to remove inconvenient telephone poles, tripod legs, or some random person.
The description says "existing photos" which in most case would not have been shot with obstruction removal in mind.
So which is it? A well understood compositing method which requires a pair of special shot photos, or a new method that can take an arbitrary single image, as shot, and perform these operations?
And if there isn't a hole where you want it to get the shot you want? If Google want to help photographers like with the free NIK filters,let them do it.
Today it is quite common to put fences at high buildings and bridges in order to hinder suicide and throwing large objects. Those fences are usually a bit away from you. No possibility to take the image through any hole. Actually, one of my all favorite photo spots is destroyed this way. A high bridge with a view over Stockholm. Now there is a fence. OK - it is possible to hold the camera out on the other side of the fence, but taking panoramas and also using tripod is no longer possible. This technique (if and when it works) sounds like a good idea for me.
Lots of things that make sense do not make sense. The bridge I am talking about is 26 meter high. The probability to survive is almost 100%. For some reason it is almost impossible to get statistics on suicides from the bridge. But, if I dig deeper, it looks like it almost never happens. It seems to be a myth. But, all "know" it is the most common way to commit suicide in Stockholm, something that definitely is untrue. Jumping from high houses, jumping in front of a train and medicine are the most common.
but if they stop putting fencing on every bridge and making kids wear helmets to chess matches , what , pray tell will our nanny state do?
I know of no cannabis fatalities here in America , but legal booze cigarettes and big -pharma heroin kill untold numbers but are just too cash imbued for the nannys to care ... instead they will continue to care where they are not wanted
Removing something with a consistent pattern is fine. What about removing more complex, discreet objects from the image? I occasionally make use of Photoshop context sensitive fill, and that sometimes takes quite a bit of work with layers and masks to i) make sure only what I want removing is actually removed and ii) what it gets replaced with looks credible.
This does look incredible. If it uses the parallax effect, that would imply the need for two slightly offset photos (like stereo photography), however, it states it can work on existing photos.
I would love to see this technology integrated into Google's Nik software in the future. In the meantime, it's made me think about taking two offset photos where there are obstructions close to the camera, overlay them in PS, then use the mask tools to eliminate the obstruction from the top layer. Hmmmm.
That's probably because it is. I'd like to see how it would deal with text or humans in the background, or a much more out of focus chain link fence. Reminds me very much of when Adobe acquired the rights to the Context Aware / Fill technology. Yes, it can be a useful starting point, but nowhere near as good as their examples suggested.
There are things I cannot do which others can do well. So I need other people. But what if I can create technology that begins where my capabilities end? I will no longer need others to do things I cannot do or those things I wish to avoid doing. I will be able to direct my technology to do everything I want to do but don't do well myself. I will be an emperor in an empire of my own choosing. I will only see a world of my choosing. I will only interact with others I choose. There will be no uncertainty. There will be no inefficiency. There will be no accidental natural wonder. And I will never have to face my limitations, other than the limits of my own imagination, because I will have no limitations to know. Or will there?
My thoughts. I'm pretty sure the world is nowhere near ready for everyone to be gliding around carefree in a seamless, hassle-free existence in which we are assisted and cared for in almost every manner by AI.
looks good unfortunately, the Pros amongst us wont use it and the $200 husband and wife teams on facebook are too lazy to edit their photos . So not sure who the market is
Photography is becoming more and more than just about the lenses and sensors. That's why Nikon is falling behind (I think). They are not really into software things. Nikon (and other manufacturers for that matter) must offer cool tricks like this to attract casual users.
It's not this (or any) technology that would cause someone to be uncreative, but the way one utilizes it. One can use a tool in an uncreative way and someone else may utilize it in a highly creative manner. The example that they show of removing fence seems pretty basic - obviously to show, very simply and in a way everyone enderstands, what this technology can do. I'm sure it can actually be used to do some very interesting things.
Right now I am using my creativity and my brains to either find a way to make a photograph where the fence doesn't show up on the image. Or try to incorporate the fence in an interesting photo.
Because of the google technology there is no need for that. It's that simple.
Did you watch the whole video? Google photos will also select your best photos for you and print it in a hardcover book. No need to spend precious time choosing the photos you want, let the algorithm decide!
Sadly no, the only possibly humorous part of this tech is that Google is taking it seriously enough that they are giving everyone in the (very huge) audience a free trial of the technology (i.e. a free photo book, with images automatically selected from all their Android photos).
Although I admit the facial recognition technology with auto-album update is kinda interesting. Now, if anyone takes a photo of your kids with an Android phone, you can automatically get a copy of the photo compliments of Google sharing technology :p
For most people, creativity never had anything to do with it in the first place. They're trying to preserve a memory or communicate, not make a work of art. This is just a tool to get a job done.
healing brush is completely software based and just guesses what's behind the fence
this is hardware based, it uses parallax error like is said in the article....same way humans with two eyes can see everything that is happening behind a fence
you're combining overlapping images..you're not guessing what's behind the fence like a healing brush, you can see what's behind it due to the overlapping image...the parallax error shifts the viewing frustrum, combine both correctly and the fence is gone
Might be difficult as most moire are very tiny patterns. If anything, the process will be ADDING the lines back that the moire ruined in the first place.
That's the future of photography. With increasing computational power more and more thing will become possible. And smartphones will use that power via cloud computing. Software will even overcome many limits of technology and will create images even DSLRs can't keep up with
with smartphone dual sensors they can now cleanly remove any element in the foreground without any loss in detail in the background
really, the next evolution will be cameras that can recognise objects with the help of dual cameras giving them depth..in the video the camera can regonize a flower..but this is just the beginning
with the help of depth and AI allowing them to recognize patters and colors and looking them up in a database...the cobots will truly have AI..see things..recognise it..learn and evolve..we'll have cobots doing household chores as if it's normal
All those are already possible with todays smartphones. You point camera around and in real-time it speaks what it see, like chair, apple, orange, teacup, chairs and table, newspaper on table etc...
Really? I don't get that attitude. LR automatically adds them when uploading to Flickr so why wouldn't you use it? Maybe it will stop someone from using your photo without telling you someday, who knows. And it's fun to put it there anyway, just more artistic expression.
@Docno I don't think that speaks to a sense of entitlement at all - it sounds like a different problem entirely. But I don't know the eastern half of the world well (except through the eastern immigrants who are definitely not entitled here) so I could not speak for the east.
In a broad and very general sweeping statement, I find that the people who are most possessive are the ones that have no reason to be, to put it politely. Of course that isn't the case for everything
You are affecting greater knowledge than you possess by making sociological assumptions about which you know very little. Now you can know what it means also (here's another example). ;)
I'll explain in simple terms shall I. I don't mean watermarks on agency images, I don't mean that I want to steal images from hard working individuals.
I rarely see good work posted in this forum that has a water mark attached to it or a 'photography by JT26' label somewhere on it for example. All the good images seem to be water mark or name free...
Secondly when I say 'posess' I meant photography ability. Shame I have to spell this out for you as it's kinda takes the edge off it all
No Nikon camera we've tested to date balances stills and video capture as well as the Nikon Z7. Though autofocus is less reliable than the D850, Nikon's first full-frame mirrorless gets enough right to earn our recommendation.
Nikon's Coolpix P1000 has moved the zoom needle from 'absurd' to 'ludicrous,' with an equivalent focal length of 24-3000mm. While it's great for lunar and still wildlife photography, we found that it's not suited for much else.
As a stills camera the Fujifilm X-T3 is a pleasant update to one of our favorite APS-C cameras, significantly improving the autofocus. If you're interested in stills and video, though, it's knockout.
The Nikon Z7 is slated as a mirrorless equivalent to the D850, but it can't subject track with the same reliability as its DSLR counterpart. AF performance is otherwise good, except in low light where hunting can lead to missed shots.
The Canon EOS R is the first full frame mirrorless camera to use the new RF mount. We're well underway putting it through our range of standard tests – take a look at how it compares to the competition and our thoughts on using it so far.
What's the best camera for under $500? These entry level cameras should be easy to use, offer good image quality and easily connect with a smartphone for sharing. In this buying guide we've rounded up all the current interchangeable lens cameras costing less than $500 and recommended the best.
Whether you've grown tired of what came with your DSLR, or want to start photographing different subjects, a new lens is probably in order. We've selected our favorite lenses for Sony mirrorlses cameras in several categories to make your decisions easier.
Whether you've grown tired of what came with your DSLR, or want to start photographing different subjects, a new lens is probably in order. We've selected our favorite lenses for Canon DSLRs in several categories to make your decisions easier.
Whether you've grown tired of what came with your DSLR, or want to start photographing different subjects, a new lens is probably in order. We've selected our favorite lenses for Nikon DSLRs in several categories to make your decisions easier.
What’s the best camera for less than $1000? The best cameras for under $1000 should have good ergonomics and controls, great image quality and be capture high-quality video. In this buying guide we’ve rounded up all the current interchangeable lens cameras costing under $1000 and recommended the best.
The Z7 presented Nikon with a stiff challenge: how to build a mirrorless camera that measures up to its own DSLRs and can deliver a familiar experience to Nikon users. Chris and Jordan tell us whether they think Nikon succeeded.
Nikon's D3500 may be an inexpensive DSLR, but the company didn't cut corners when it comes to image quality. See how it handled fall colors and tropical seas in our sample gallery.
Nikon has released firmware version 1.02 that resolves a flickering issue when scrolling through images, an ISO limitation problem, and an occasional crash that could occur when displaying certain Raw files.
DxO announces the latest update to Nik Collection (version 1.1) that brings better compatibility, fewer bugs to the plugin suite it acquired from Google a year ago
Nikon USA has announced that its Z6 full-frame mirrorless camera will be shipping Friday, November 16th at a price of $1999 body-only and $2599 with the Nikkor Z 24-70 F4 S lens.
The Insta360 One X is the company's latest consumer 360-degree camera, supporting 5.7K video, including excellent image stabilization, as well as 18MP photos. And, in our experience, it's a really fun camera to use.
ON1 Photo RAW 2019 brings an updated interface, more powerful Lightroom migration, better camera/lens support, and more to ON1's flagship editing program.
Experiencing life through the lens of a camera might mean you miss out on special moments, warns Casey Cavanaugh as he shoots a short film through the viewfinder of his Hasselblad 500CM
The New York Times has teamed up with Google to start the process of digitizing more than five million photos stored in a vault nicknamed "the morgue."
Ambitious goals, new challenges and looking ahead to 100 years of the Z mount – we spoke with senior executives and engineers at Nikon about what lies ahead.
After years focused primarily on landscapes, Erez Marom leapt on an opportunity to return to his roots in wildlife photography. A trip to the mountains of Uganda photographing endangered mountain gorillas yielded some stunning photos – and an experience of a lifetime.
Digital cameras have made it incredibly easy to do time-lapse photography, thanks to the ability to take hundreds—or even thousands—of photos without interruption. This week, Chris and Jordan walk us through the process of planning and shooting compelling time-lapse videos.
Comments