During its I/O 2017 conference yesterday, Google demonstrated a new algorithm-based technology that can remove unwanted objects from existing photographs. The demonstration showed the technology removing a chainlink fence from the foreground of an image, with the final result offering no discernible indications that the fence had ever existed (around 10:45 in the video below).
The technology was demonstrated on stage by Google's CEO Sundar Pichai during a conversation about the company's expanding visual technology. 'Coming very soon,' Pichai explained, 'if you take a picture of your daughter at a baseball game and there's something obstructing it, we can do the hard work and remove that structure and have the picture of what matters to you in front of you.'
It looks to be an evolution of the research Google and MIT have been collaborating on for some time – in fact, their demonstration from 2015 includes a very similar chain-link fence demo. This method takes advantage of the parallax effect to identify and remove obstructions from photos.
Unfortunately, Pichai didn't elaborate on when this technology will be made available aside from 'very soon,' nor did he specify where the technology will be available. Given the company's Google Photos announcements, however, it seems likely the technology will be implemented within that product.
Oops - I missed that there was a reference to 10:45. And - it is too late to edit my post. Anyhow - ignore my post. If someone from DPR sees this, please remove it (and this).
I'm Sorry, but that was not a demonstration of the technology. It was an illustration of the technology, or perhaps a description of it. Those chain-link fences were simple overlays, that they then removed, at the precise moment he says "in the future".
What? You are sure? In that case, I am quite stunned. Hmmm ... that the second image has a perfectly sharp fence hints at something. Enormous DOF for that camera.
I listened at 10:45, and yes, he said "coming very soon" we can do this. So, it was fake. And the audience cheered. Very effectful presentation. And also very manipulative. Lets see when and if this really is released.
yes. I saw that--it's from 18 months ago and involves removing the foreground from a still frame of video image...but the "demo" at 10:45 doesn't demonstrate to us that technology in any way. It's an illustration that what that technology "could" do, "soon". The "demo" doesn't appear to be showing an actual removal...the foreground fence doesn't look like it's part of the original image. I certainly have no doubt that this will be, or is, possible, but the "demo" in the video doesn't appear to be an actual demonstration of the technology (which the article reports, incorrectly)...which is all I said.
As for the technology itself, I'd love to see what it does if that little girl decided to "not be a statue" and actually swing the bat. The results might vary a little. :-)
Two more thoughts. I think the isolation of the reflected image off glass is actually more interesting...the fact that you could pull that much detail from the reflection has numerous creative and "forensic" uses. The second thought is why would anyone shoot with their smart phone 6-12 inches from glass or a chainlink fence? The lens on a smartphone is small enough that it would have little trouble peering through 1/2 inch wire fencing much less 2-3" chainlink. And pressing a phone camera lens against a window generally leads to largely reflection free photos...
This is not an idea. Talk about this has been going on for ages and there are some (more or less successful) tries. Now it is about implementation. The idea stadium was in another millenium.
Interesting - but there is a problem. In the second example the camera is focused on the chain link fence. When the fence is removed that does not change, so then the entire picture is out of focus!!! Loren
I find that if something is in the way from my viewpoint and I can't get round it then it makes me think harder, can I actually make it an active part of my composition ? Fences too.
They removed the chain link fence. Why stop? Remove the ugly construction materials and ugly buildings, replace the asphalt with grass and put leaves on the leafless trees.
Usually, I take the shot through a hole in the chainlink fence. It is even easier... ;-) Or take the shot with a lens allowing a DOF shallow enough to blurr the fence away.
All the years I sat at Little League games staring through chain-link fences, I kidded about inventing holographic sunglasses that would filter out chain-link fences. Maybe Google can help me...
It is but a matter of time. Narrow AIs can achieve above human level easily for particular tasks already. Some aspects of image recognition being one of those tasks... this is just a logical step in that direction.
Not long from now, they will probably do almost everything better and faster than us.
Not very sure, but they might have expanded the U-point technology that Nik first developed and which Google purchased a couple of years back? Nikon was using the U-point in their Capture NX software.
I can imagine turning up to a place to take a picture at a particular spot because I saw it on the internet and want to take one of my own, only to say, "Ugh, there is an ugly chain fence here!!". This tech will be a bit of both good and bad, depending on who you are I guess.
I can see this technology being useful, but then also leading to laziness too, for some. I think it may cause some photographers (particularly newbies) to become "lazy" and say "Oh, I'll just remove it in post processing" and not try to move around the obstruction (if it's possible) to get a better shot. I hear this statement a lot from beginners: "I'll just remove that later." Some things can be fixed in post, some things cannot.
The parallax method, requiring at least 2 photos from slightly different PoV can be done using Hugin. Done it several times to remove inconvenient telephone poles, tripod legs, or some random person.
The description says "existing photos" which in most case would not have been shot with obstruction removal in mind.
So which is it? A well understood compositing method which requires a pair of special shot photos, or a new method that can take an arbitrary single image, as shot, and perform these operations?
And if there isn't a hole where you want it to get the shot you want? If Google want to help photographers like with the free NIK filters,let them do it.
Today it is quite common to put fences at high buildings and bridges in order to hinder suicide and throwing large objects. Those fences are usually a bit away from you. No possibility to take the image through any hole. Actually, one of my all favorite photo spots is destroyed this way. A high bridge with a view over Stockholm. Now there is a fence. OK - it is possible to hold the camera out on the other side of the fence, but taking panoramas and also using tripod is no longer possible. This technique (if and when it works) sounds like a good idea for me.
Lots of things that make sense do not make sense. The bridge I am talking about is 26 meter high. The probability to survive is almost 100%. For some reason it is almost impossible to get statistics on suicides from the bridge. But, if I dig deeper, it looks like it almost never happens. It seems to be a myth. But, all "know" it is the most common way to commit suicide in Stockholm, something that definitely is untrue. Jumping from high houses, jumping in front of a train and medicine are the most common.
but if they stop putting fencing on every bridge and making kids wear helmets to chess matches , what , pray tell will our nanny state do?
I know of no cannabis fatalities here in America , but legal booze cigarettes and big -pharma heroin kill untold numbers but are just too cash imbued for the nannys to care ... instead they will continue to care where they are not wanted
Removing something with a consistent pattern is fine. What about removing more complex, discreet objects from the image? I occasionally make use of Photoshop context sensitive fill, and that sometimes takes quite a bit of work with layers and masks to i) make sure only what I want removing is actually removed and ii) what it gets replaced with looks credible.
This does look incredible. If it uses the parallax effect, that would imply the need for two slightly offset photos (like stereo photography), however, it states it can work on existing photos.
I would love to see this technology integrated into Google's Nik software in the future. In the meantime, it's made me think about taking two offset photos where there are obstructions close to the camera, overlay them in PS, then use the mask tools to eliminate the obstruction from the top layer. Hmmmm.
That's probably because it is. I'd like to see how it would deal with text or humans in the background, or a much more out of focus chain link fence. Reminds me very much of when Adobe acquired the rights to the Context Aware / Fill technology. Yes, it can be a useful starting point, but nowhere near as good as their examples suggested.
There are things I cannot do which others can do well. So I need other people. But what if I can create technology that begins where my capabilities end? I will no longer need others to do things I cannot do or those things I wish to avoid doing. I will be able to direct my technology to do everything I want to do but don't do well myself. I will be an emperor in an empire of my own choosing. I will only see a world of my choosing. I will only interact with others I choose. There will be no uncertainty. There will be no inefficiency. There will be no accidental natural wonder. And I will never have to face my limitations, other than the limits of my own imagination, because I will have no limitations to know. Or will there?
My thoughts. I'm pretty sure the world is nowhere near ready for everyone to be gliding around carefree in a seamless, hassle-free existence in which we are assisted and cared for in almost every manner by AI.
looks good unfortunately, the Pros amongst us wont use it and the $200 husband and wife teams on facebook are too lazy to edit their photos . So not sure who the market is
Photography is becoming more and more than just about the lenses and sensors. That's why Nikon is falling behind (I think). They are not really into software things. Nikon (and other manufacturers for that matter) must offer cool tricks like this to attract casual users.
It's not this (or any) technology that would cause someone to be uncreative, but the way one utilizes it. One can use a tool in an uncreative way and someone else may utilize it in a highly creative manner. The example that they show of removing fence seems pretty basic - obviously to show, very simply and in a way everyone enderstands, what this technology can do. I'm sure it can actually be used to do some very interesting things.
Right now I am using my creativity and my brains to either find a way to make a photograph where the fence doesn't show up on the image. Or try to incorporate the fence in an interesting photo.
Because of the google technology there is no need for that. It's that simple.
Did you watch the whole video? Google photos will also select your best photos for you and print it in a hardcover book. No need to spend precious time choosing the photos you want, let the algorithm decide!
Sadly no, the only possibly humorous part of this tech is that Google is taking it seriously enough that they are giving everyone in the (very huge) audience a free trial of the technology (i.e. a free photo book, with images automatically selected from all their Android photos).
Although I admit the facial recognition technology with auto-album update is kinda interesting. Now, if anyone takes a photo of your kids with an Android phone, you can automatically get a copy of the photo compliments of Google sharing technology :p
For most people, creativity never had anything to do with it in the first place. They're trying to preserve a memory or communicate, not make a work of art. This is just a tool to get a job done.
healing brush is completely software based and just guesses what's behind the fence
this is hardware based, it uses parallax error like is said in the article....same way humans with two eyes can see everything that is happening behind a fence
you're combining overlapping images..you're not guessing what's behind the fence like a healing brush, you can see what's behind it due to the overlapping image...the parallax error shifts the viewing frustrum, combine both correctly and the fence is gone
Might be difficult as most moire are very tiny patterns. If anything, the process will be ADDING the lines back that the moire ruined in the first place.
That's the future of photography. With increasing computational power more and more thing will become possible. And smartphones will use that power via cloud computing. Software will even overcome many limits of technology and will create images even DSLRs can't keep up with
with smartphone dual sensors they can now cleanly remove any element in the foreground without any loss in detail in the background
really, the next evolution will be cameras that can recognise objects with the help of dual cameras giving them depth..in the video the camera can regonize a flower..but this is just the beginning
with the help of depth and AI allowing them to recognize patters and colors and looking them up in a database...the cobots will truly have AI..see things..recognise it..learn and evolve..we'll have cobots doing household chores as if it's normal
All those are already possible with todays smartphones. You point camera around and in real-time it speaks what it see, like chair, apple, orange, teacup, chairs and table, newspaper on table etc...
Really? I don't get that attitude. LR automatically adds them when uploading to Flickr so why wouldn't you use it? Maybe it will stop someone from using your photo without telling you someday, who knows. And it's fun to put it there anyway, just more artistic expression.
@Docno I don't think that speaks to a sense of entitlement at all - it sounds like a different problem entirely. But I don't know the eastern half of the world well (except through the eastern immigrants who are definitely not entitled here) so I could not speak for the east.
In a broad and very general sweeping statement, I find that the people who are most possessive are the ones that have no reason to be, to put it politely. Of course that isn't the case for everything
You are affecting greater knowledge than you possess by making sociological assumptions about which you know very little. Now you can know what it means also (here's another example). ;)
I'll explain in simple terms shall I. I don't mean watermarks on agency images, I don't mean that I want to steal images from hard working individuals.
I rarely see good work posted in this forum that has a water mark attached to it or a 'photography by JT26' label somewhere on it for example. All the good images seem to be water mark or name free...
Secondly when I say 'posess' I meant photography ability. Shame I have to spell this out for you as it's kinda takes the edge off it all
When the Fujifilm X-T2 arrived, it was more than just a modest upgrade to the already impressive X-T1. While the new X-T3 hasn't changed the overall design of the camera, this model is way more than an upgrade; rather, it's a quantum leap.
The Movie Maker is a compact, motorized slider designed for phones, action cams and small mirrorless cameras. We think it's a fun little kit and a good value proposition for the cost, provided you can work around a few of its weak points.
Nikon's Z7 is the first camera to use the all-new Z-mount, the company's first new full-frame mount since 1959. We've put together our first impressions based on quality shooting time with a pre-production camera - check out what we've found.
We got some hands-on time with a pre-production LX100 II to get a better feel for the improvements Panasonic's brought to the table. Here's a rundown of what's new – and what we think.
The instant camera market is heating up, and with four formats and 15+ cameras to choose from, we felt it was high time to examine them all and pick our favorite.
Whether you're hitting the beach in the Northern Hemisphere or the ski slopes in the Southern, a rugged compact camera makes a great companion. In this buying guide we've taken a look at seven current models and chosen our favorites.
What's the best camera for a parent? The best cameras for shooting kids and family must have fast autofocus, good low-light image quality and great video. In this buying guide we've rounded-up several great cameras for parents, and recommended the best.
What's the best camera for shooting landscapes? High resolution, weather-sealed bodies and wide dynamic range are all important. In this buying guide we've rounded-up several great cameras for shooting landscapes, and recommended the best.
What’s the best camera costing over $2000? The best high-end camera costing more than $2000 should have plenty of resolution, exceptional build quality, good 4K video capture and top-notch autofocus for advanced and professional users. In this buying guide we’ve rounded up all the current interchangeable lens cameras costing over $2000 and recommended the best.
Nikon's Sendai factory in the Tōhoku region North of Japan has been churning out cameras and lenses since 1971. We had the opportunity recently to visit Sendai during events to mark the launch of Nikon's new Z mount.
There's no mistaking the Nikon Coolpix P1000 – with a 24-3000mm equivalent zoom, it really is in a class of its own. It's a conspicuous-looking superzoom with one main job: getting you really close to far away subjects. We've put together a gallery showing the kind of results you can expect from it.
GoPro has announced its HERO7 camera lineup. The updated action cameras feature new HyperSmooth and TimeWarp modes, as well as improved video and photo specs.
The Sony 24mm F1.4 is the latest lens to join the company's premium G Master lineup. We've been shooting with one for a couple of days - here's what you need to know.
Camera bag manufacturer Lowepro has introduced mark II backpacks for its ProTactic AW range with models that are said to feature an improved handling experience as well as a collection of accessories that can be attached to the outside.
Canon has announced its latest superzoom camera, the PowerShot SX70 HS. Compared to the SX60 that came before it, the SX70 has the same lens but offers a higher resolution EVF, 4K video capture and support for Canon's new CR3 Raw format.
Sony has taken the wraps off of its new 24mm F1.4 GM full-frame lens, which the company claims is the lightest in its class. Despite its fast aperture, the 24mm F1.4 is remarkably light, weighing just 445 grams (15.7 ounces). The lens will set you back $1400 when it ships next month.
In this episode of DPReview TV we take a look at Sony's brand new 24mm F1.4 GM lens, a desirable focal length for many photographers. How does it perform? Chris and Jordan give us their first impressions.
We've had a little time to shoot with Sony's new wide/fast prime, both close to home and on the water in San Francisco. Check out our initial sample images.
Fujifilm released a firmware upgrade for its X-T3 mirrorless camera that addresses issues with distortion compensation and the mechanical lock on SD cards.
British photographic engineer MTF Services is claiming the world’s first third-party lens adapters for the new Nikon Z system with a collection of four units designed to allow cinema lenses to be mounted on the mirrorless full frame bodies.
The X-T3 is our first opportunity to analyze what's likely to be Fujifilm's next generation image sensor. Take a look at how it performs next to the competition in our studio test scene.
Nikon will cease offering Brazil-based customer service and technical support, though the company stresses that it will still offer technical assistance and warranty repairs for valid warranties.
Two years ago, CatLABS of JP announced a plan to save Packfilm from the dead. Now, it's announced it's giving up its efforts to better focus its resources elsewhere.
The GoPro Fusion is designed to make it easy to capture 360-degree video and stills. We took it out recently on a typically hot Seattle summer day to see what it can do.
Canon and Nikon finally entered the full-frame mirrorless market this summer with the brand-new RF and Z mounts. Now that we've had some time with the cameras, we wanted to revisit our earlier predictions and take stock.
Blackmagic Design has announced the public beta of its new Blackmagic RAW video codec. The company says the new format combines the benefits of shooting Raw video with the ease of use and smaller file sizes usually associated with non-Raw video files.
Comments