Algorithm that removes the water from underwater images

Came across this, maybe of interest to this forum. If you guys already know about this then my apologies.

It looks pretty awesome.

PHXAZCRAIG wrote:

You might be surprised how much water clarity you can enhance by playing with the black slider in Lightroom.

Yeah but this is different which is BTW mentioned in the video (@ 2:55)and quite obvious from the explanation of the method. Lightroom has no way of knowing what it should look like. Her algorithm does because of the reference card she uses. Simplified, the algorithm knows what red, green etc. is supposed to look like without the water. By looking at the reference card under water from it can calculate influence of underwater circumstances and apply the needed correction to the entire image. It's quite clever.

I only saw a bit of the video on my dentist's phone yesterday (he wanted to share it with me due to known interest).

But carrying a color card doesn't seem terribly different from use the sand as the WB point, so it still seems like maybe an exaggeration is being made, and something odd about calling it The Algorithm to give it extra importance.

I'll wait for the written paper.

kelpdiver wrote:

I only saw a bit of the video on my dentist's phone yesterday (he wanted to share it with me due to known interest).

But carrying a color card doesn't seem terribly different from use the sand as the WB point, so it still seems like maybe an exaggeration is being made, and something odd about calling it The Algorithm to give it extra importance.

I'll wait for the written paper.

Seems to me you're exaggerating the importance of the word algorithm. It's nothing more or less than some program code to solve a problem.

Anyway, I am not expert on under water photography, just thought it looked interesting.

Paper you don't have to wait for, here you go:

http://openaccess.thecvf.com/content_CVPR_2019/papers/Akkaynak_Sea-Thru_A_Method_for_Removing_Water_From_Underwater_Images_CVPR_2019_paper.pdf

The scientist from the video has been answering questions on Reddit. A few points I got from that:

- The card as seen in the video is not required, it was used to 'train' the software.

- She's also working on a stand-alone application

Reddit thread can be found here: https://www.reddit.com/r/videos/comments/dvts2j/this_researcher_created_an_algorithm_that_removes/

"Just a clarification: the method does NOT require the use of a color chart. That part was not clear in the video. All you need is multiple images of the scene under natural light, no color chart necessary."

And the second one means no help for our existing backlog of raw images.

The hardest test, even with that requirement, would be for true wide angle where nothing is within 20 m of you.

PHXAZCRAIG wrote:

I like the idea, which seems aimed for accuracy as much as anything. But also in shallow water perhaps, where there is still a range of colors to be had.

Yes. This is what it does. Goal is accuracy not perse a pleasing picture. Any photo-shopping comes down to what you think it would/should look like.

Quote from developer/scientist in yet another discussion:

But I believe you (and not you personally, but those who say "I can do this in Photoshop") are still missing the point about the method. I will try again to explain below, but please don't hesitate to ask if anything is still unclear. I know underwater photographers are very passionate, and I want to make sure we all understand each other.

1) You can of course do ANYthing in Photoshop - as one person commented, you can even paint the whole image pixel by pixel with the colors you want. But any correction you do in Photoshop is subjective. Give the same raw underwater image to 10 photographers, and you'll end up with 10 different corrections. Not to mention tedious. Can you imagine doing a survey of a reef, coming back with 1000s of photographs, and trying to correct them manually one by one in Photoshop ? (Actually that's how most marine scientists work at the moment.....)

Sea-thru operates on a physical equation describing of how light propagates in the water, it knows which parameters govern how light is absorbed and scattered, and it calculates those parameters for EACH PIXEL, and makes sure each pixel satisfies that equation. Because it always makes sure all pixels obey that equation, its corrections are objective, and repeatable. And it is automated.

2) Yes, in Photoshop, you may achieve a visually pleasing correction that will look great for many images. For the most part, these images will be the ones where the scene is close to the camera (low backscatter), and more or less all objects in the scene will have the same range from the camera (uniform attenuation). [Correcting scenes with varying in Photoshop ranges will get very difficult or very tedious. And yes, your comment about examples not showing that case is valid.]

Even so, the correction you end up with will likely not be physically accurate correction because (at this point), photoshop doesn't give you the tools to extract/estimate the parameters you need to do a proper correction. Until last year, what those relevant parameters are were not even fully understood, please have a look here:

http://openaccess.thecvf.co...

There is currently a "dehaze" filter in photoshop which stems from research, just like mine, from a field called atmospheric dehazing. But that filter won't work for underwater images because the physics of light propagation in the atmosphere and in the ocean are different.

So you can imaging Sea-thru will be a filter in Photoshop just like "dehaze", soon.

All of this might sound like overkill for non-scientific underwater photographers, amateur or professional. But this work comes from a scientific field called "underwater computer vision", and is intended to accelerate our pace of progress in marine science, by standardizing large sets of underwater images, so that we can use powerful AI methods on large datasets. As a bonus, it will be a cool tool for non-scientific underwater photographers too.

Hope this helps.
Derya

kelpdiver wrote:

And the second one means no help for our existing backlog of raw images.

Sure, if you want the glass is half empty.

Joep van Steen wrote:

kelpdiver wrote:

And the second one means no help for our existing backlog of raw images.

Sure, if you want the glass is half empty.

more like 90% empty.   Marine biology gets a lot of data from the legions of divers, and not all subjects are going to permit multiple shots.

To be blunt, the self promotion of the author sets off BS meters.   I'm not convinced that her's delivers the "accurate" view.

I don't want to see the before and after, which may be similar but different to what Craig might end up at.   I would want to see the befores, the processed image, and then an image after a ring of 10k lights is turned on.

I also don't subscribe to the general notion that accurate is better than pleasing.    I'm not shooting for science.

Sure. First you want the paper, I link to and it, and now you want more. Dr. Dino tactics. I'm done with this.

Joep van Steen wrote:

Sure. First you want the paper, I link to and it, and now you want more. Dr. Dino tactics. I'm done with this.

it's less clear what you want out of this thread.

Blind acceptance?   Praise for posting a link as a "discoverer" of something that's had remarkably good press?

Not sure how long you've been around the block, but a white paper != a viable tool to add to your workflow.   There's not even a non viable tool yet.   Just a maybe.

Jeep

thanksfor posting original and subsequent links. It’s an interesting new development made possible in part by development of AI. Future development might be more important to marine scientists, but even as a hobbyist I want to know what options for post processing might be available to me. Even With today’s WB tools I sometimes  color grade for “realism”, sometimes just for the most pleasing image, and sometimes for a more abstract effect. New developments will offer new options, but I won’t feel they will require me to change my approach unless I like the result.

Sure! Was just something I read about somewhere. Will be a while before it's a plugin or something.

Happy diving!

kelpdiver wrote:

Joep van Steen wrote:

Sure. First you want the paper, I link to and it, and now you want more. Dr. Dino tactics. I'm done with this.

it's less clear what you want out of this thread.

Blind acceptance? Praise for posting a link as a "discoverer" of something that's had remarkably good press?

Not sure how long you've been around the block, but a white paper != a viable tool to add to your workflow. There's not even a non viable tool yet. Just a maybe.

Ah, now you want my motives. First you already doubted the researcher's.

All it is, is I read an article, noticed later there's an underwater photography sub forum, and decided it may be interesting enough to post a link.

If you say something that's incorrect then I may respond to that. If something is incorrect then it is that, incorrect, and how long I have been around the block is moot.

However first you say it's similar to taking sand as a point for WB, and that's simply not correct. Even if you could accomplish more or less the same by accident using Photoshop and whatnot, it still does not make it the same. It only illustrates you're missing the point.

Then you also say you wait for the paper then I give you the paper. Which still isn't good enough (don't ask for it then). Now you want before and after shots. A ready to use tool. That fits your workflow. Well, good for you, all I did was post a link I deemed to be interesting, period. And all I did was respond to your incorrect assumptions made on a dentist's chair. Sure you decide how useful it is to you. I'm not telling you what to do.

Click me to see before / after Sea Thru

Joep van Steen wrote:

However first you say it's similar to taking sand as a point for WB, and that's simply not correct. Even if you could accomplish more or less the same by accident using Photoshop and whatnot, it still does not make it the same. It only illustrates you're missing the point.

Is there a point?   Seems to be a lot of whining for a guy who also proclaims to not care.

Color correcting for depth and lighting is the point.   Sand, skin, and sometimes the tank can serve as a substitute for the grey card, and then the photographer can adjust as subjectively appropriate.  It's fairly simple, works now, and works with one shot.   As someone who spends much more time shooting moving objects, one shot is important.   Shooting a coral reef every 4 months to watch trends - might be able to work with the requirements.

I will remain skeptical that there is a 'correct' color result.   Going past the obvious variability in the environment and the lighting used, even the camera sensors vary.  Canon colors vs Sony's shadow pulling.

Progress comes when there is a repeatable (by others) experiment.  We're not there here.

kelpdiver wrote:

Joep van Steen wrote:

However first you say it's similar to taking sand as a point for WB, and that's simply not correct. Even if you could accomplish more or less the same by accident using Photoshop and whatnot, it still does not make it the same. It only illustrates you're missing the point.

Is there a point?

Yes, that it's not the same.

Seems to be a lot of whining for a guy who also proclaims to not care.

If I feel like it I reply. And it seems to me, you do the whining by criticizing without understanding. By complaining about words being chosen, al-go-rithm, yes that's what it's called. By asking for a paper and then ignoring it. By accusing a researcher of self promotion, setting of your BS meter while most of the BS comes from you.

Color correcting for depth and lighting is the point. Sand, skin, and sometimes the tank can serve as a substitute for the grey card

See? You still don't get it. Also it does not require a card.

, and then the photographer can adjust as subjectively appropriate. It's fairly simple, works now, and works with one shot.

Yes. no one argues that. No one claims you can or should throw that overboard.

As someone who spends much more time shooting moving objects, one shot is important. Shooting a coral reef every 4 months to watch trends - might be able to work with the requirements.

Yes, sure. So? IOW, it's not for you. Okay.

I will remain skeptical that there is a 'correct' color result.

Maybe write your own paper then.

Going past the obvious variability in the environment and the lighting used, even the camera sensors vary. Canon colors vs Sony's shadow pulling.

You didn't even glance at the paper you asked for, did you? The researchers admit to that, but it's also data that can be fed to the model.

Progress comes when there is a repeatable (by others) experiment. We're not there here.

The progress is there no matter how you try to frame it. A first step is already progress even if the method would proven to be wrong at a later point.

I would have imagined me being a marine photographer, being interested in this type of developments whether it is useful for you right away or not.