Previous news story    Next news story

Researchers follow the noise to find composite images

By dpreview staff on Jul 9, 2012 at 16:24 GMT

Researchers at the University of Albany have developed an efficient and automatic process for identifying composite images, based on the different noise patterns between the two images. In a paper presented at the IEEE's International Conference on Computational Photography, a team led by Siwei Lyu showed they were able to find and locate composited material in images from an online 'Photoshopping' contest site worth1000.com. The team's algorithm exploits the tendency for image noise (regardless of source) to have a characteristic shape (kurtosis). Scanning the image for areas with different noise patterns allows the system to identify non-original content.

The original image from the contest website Worth1000.com The same image, overlaid with the added areas identified by the algorithm.

Because the process is based on analysis of noise, the authors concede it can't identify images with cloning or similar shots from the same camera, and struggles with downsized and heavily-compressed images that have lost their underlying noise characteristics. However, they hope to extend their work by improving its ability to distinguish between local image detail and noise, to improve its accuracy and provide improved noise reduction methods.

The group's research paper can be read here. Non statisticians may wish to look at Figure 4 to see how successful the system was in identifying added-in regions. As the authors point out,'Note that these forgeries are carefully manipulated and processed and have realistic appearance, many can only be exposed based on conceptual knowledge of the physical world (e.g., hippopotamus is unlikely to be found in arctic regions).' Figure 2, which shows the system's success at isolating intentionally introduced noise, hints at the benefits the system could bring to context-sensitive noise-reduction, with further work.

Comments

Total comments: 38
Bill Bezzant
By Bill Bezzant (Jul 15, 2012)

OJ's shoes ???

0 upvotes
DVT80111
By DVT80111 (Jul 10, 2012)

Can it detect faked boobs?

5 upvotes
Parsek
By Parsek (Jul 10, 2012)

A few global blending layers of PS grain, noise etc. should wipe any detectable differences in noise profiles out if done properly. Obviously with some cost, but hardly for web usage.

3 upvotes
Shivaess
By Shivaess (Jul 13, 2012)

Glad to see someone is thinking along the same lines I am... wonder if they've tested that...

1 upvote
J R R S
By J R R S (Jul 14, 2012)

added noise will not mask diffrences in original noise patterns - yes it will help hide them slightly but I doubt it will stop this working... as even with "new noise" the original noise will stilll create a differing pattern when mixed with the new noise!

0 upvotes
falconeyes
By falconeyes (Jul 10, 2012)

As said by Karl Gnter Wnsch and AluKd, this method is known. The details (kurtosis) may be new, but even then I would have expected they used black + shot noise parametrizations. The contribution to the state-of-the-art is little.

Photoshoppers and detectors know this. They hide (or search) for different noise profiles or (ommitted in the work) different compression artefacts. Esp. wrong JPG blocks are detectable if they aren't aligned modulo eight.

I would have assumed that corresponding software became standard in forensic work a long time ago. It probably did.

2 upvotes
Laurentiu Todie
By Laurentiu Todie (Jul 10, 2012)

Photoshop has its uses and strengths in the hands of professionals who, like magicians don't dispel all their procedures : )

0 upvotes
gsum
By gsum (Jul 10, 2012)

This paper hints at the reason why proprietary noise reduction software such as Nikon's NX2 is so much more effective than e.g. Photoshop; proprietary software is inherently context-sensitive.

0 upvotes
Karl Gnter Wnsch
By Karl Gnter Wnsch (Jul 10, 2012)

There should be an easy way around this detection. Simply take a noise reference shot of your camera (grey card) and overlay that over the final image (maybe after a pass of noise reduction on the shopped image. Why use the noise reference shot? Well it's a better representative than any algorithmic noise and would be harder to detect...

0 upvotes
MarkInSF
By MarkInSF (Jul 10, 2012)

I wouldn't rely on any of these methods destroying the evidence without damaging the IQ so badly the image was unusable. My partner was doing research on digital watermarking years ago (the invisible kind), and those watermarks would survive all kinds of transformations and adulterations. The confidence in a match would decline, bit still be quite high even after the pictures has been messed around with quite a lot, including applying noise reduction and adding noise. Watermarks are just very subtle alterations that are effectively noise of a sort that can't be readily perceived and can't be identified as a watermark. The characteristic noise of an image is much stronger than a subtle watermark and would be even harder to eliminate without messing up the image completely.

Watermarking can also be used to detect if an image has been altered, and by approximately how much. Law enforcement was one of the only customers for the technology a decade ago, for evidence photos.

0 upvotes
AluKd
By AluKd (Jul 10, 2012)

Watermarking is a completely different beast. It works by encrypting information by modulating somehow the carrier data, usually with strong reliance on redundancy.

No kind of watermarking survives "all kinds of transformations", either. Some fare better, some fare worse - some are even made weak on purpose to catch tampering.

The thing to understand here is that watermarking is (fairly) strong because it's made on purpose. It's not reliant on visible aspects, but on the underlying data. It's also very redundant.

0 upvotes
huyzer
By huyzer (Jul 10, 2012)

Advancement in technology is amazing.

0 upvotes
Bullsnapper
By Bullsnapper (Jul 10, 2012)

Excellent. Maybe we will again be able to rely on photos as evidence ("the camera never lies") by being able to prove a photo has NOT been manipulated.

This meshes with an item on Australian ABC-TV Media Watch last night (www.abc.net.au/mediawatch) about Google Images being able to match almost any image with other images on the web, so proving or disproving an image's veracity. They used an image asserted by a newspaper to be of Osama bin Laden to prove that it was a fake - it was of another similar looking person.

So, two new tools come together to provide truth. This is good.

0 upvotes
theRBK
By theRBK (Jul 10, 2012)

not likely anytime soon... the technology mentioned in the article can prove an image has been manipulated, but it can't definitely prove an image has not been manipulated... even the researchers admit that there are processes that their technology can't pick up...

0 upvotes
Martin Datzinger
By Martin Datzinger (Jul 10, 2012)

Manipulation begins even before capturing a single image (choice of subject, context, composition, time, perspective, exposure, model make-up, pose, etc... ). Unless you are a forensic photographer maybe which are trained to keep such things neutral, I suppose.

3 upvotes
CameraLabTester
By CameraLabTester (Jul 10, 2012)

That is one great looking putter.

No wonder the algos zeroed in on that one...

.

3 upvotes
bizi clop
By bizi clop (Jul 9, 2012)

Similar techniques aren't entirely new:

http://www.hackerfactor.com/blog/index.php?/archives/322-Body-By-Victoria.html

1 upvote
AV Janus
By AV Janus (Jul 9, 2012)

The things people are paid to develop these days...

4 upvotes
Martin Kulhavy
By Martin Kulhavy (Jul 9, 2012)

"tendency for image noise" - is that vertical banding?

0 upvotes
Richard Butler
By Richard Butler (Jul 10, 2012)

No. It's the tendency for the statistical distribution of image noise to have a certain shape.

0 upvotes
forensicscientist
By forensicscientist (Jul 9, 2012)

Might help Tigers putting....

1 upvote
Dan4321
By Dan4321 (Jul 9, 2012)

If there are algorithms to detect it, then the same algorithms can be used to hide it, what's the point really?

7 upvotes
micahmedia
By micahmedia (Jul 9, 2012)

Much like the TSA deters only the stupid terrorists, this will deter those with weak photoshop mojo. I can't think of anything bad about deterring a bunch of people who are bad a photo manipulation from even trying.

4 upvotes
J R R S
By J R R S (Jul 14, 2012)

Not that I bet you can recite, reverse engeneer or apply these algorithms.... thats the point!

Like algorythms of this complexity are a simple and easy thing to make?

Comment edited 1 minute after posting
1 upvote
Tom Goodman
By Tom Goodman (Jul 9, 2012)

Unless I am looking at pictures supplied by Joseph Stalin or Joe McCarthy, I could care less if photoshopping can be detected.

2 upvotes
RPJG
By RPJG (Jul 10, 2012)

You mean, you *couldn't* care less?

1 upvote
xpatUSA
By xpatUSA (Jul 10, 2012)

No, that's the American way of saying wot you just said and is perfectly understandable to those who live in the USA. But I do have to say that this fine example of being "divided by a common language" makes me cringe whenever I see or hear it ;-)

Comment edited 39 seconds after posting
2 upvotes
Tom Goodman
By Tom Goodman (Jul 10, 2012)

RPJG is correct and I couldn't care less about the pshp'g. Language, on the other hand, matters greatly and I couldn't care more about it!

3 upvotes
mediokre
By mediokre (Jul 14, 2012)

The Americans I know would consider "I could care less" wrong. "I could care less" implies you do care somewhat, which is contrary to the intended meaning that you care very little or not at all. Idiom and usage override logic sometimes, but this is too much for me.

0 upvotes
AluKd
By AluKd (Jul 9, 2012)

Anyone who works professionally with image compositing already knew this for quite a long time.

Most of the time compositing actually goes torwards making sources match in photographic aspects and that, yes, usually includes dosing BOTH photographs with heavy loads of NR.

After the composite is done, a few passes of random noise (with different intensities in different channels, usually Blue gets 4x, Red 2x and Green 1x), global color balancing, global adjustments and - a favorite of mine - upsizing by 33% and downsizing by the same amount - usually makes it so any trace of compositing is obliterated. I have a REALLY hard time believing this algorithm of theirs would be able to identify a composite properly done.

15 upvotes
noirdesir
By noirdesir (Jul 9, 2012)

Wouldn't such images show a loss of detail (at the 100% level) due to the NR of the first step? Sure, it would need a different algorithm to detect the removal of noise and its replacement and you probably would not be able to tell the different areas apart but you would be able to detect that somebody was covering their tracks.

Of course, once images are downsized enough it might be hard to separate the effect of NR done before the downsizing.

0 upvotes
AluKd
By AluKd (Jul 9, 2012)

All NR incurs in loss of real detail, so yeah, of course. It's pretty hard to differentiate this loss of real detail from the one you get from general, day-to-day editing, though.

The noise added will cover it, though, as any algorithm that looks for high frequency patterns will mistake noise for relevant data at any significant detection threshold. I mean, WE humans are fooled by random noise into thinking we're seeing more detail than there actually is.

One thing that COULD give away a composite would be large scale patterned noise, bit if they were searching for this kind of pattern mismatch, I'd guess they'd have quite a lot of false positives and wouldn't really catch all the real stuff.

Comment edited 1 minute after posting
2 upvotes
yudhir
By yudhir (Jul 9, 2012)

http://www.tineye.com/
does something like this but not this close though

http://www.tineye.com/search/c0f581fa1dab0b4820513164b5d067b21a553857/
i searched the above picture with this.

1 upvote
love_them_all
By love_them_all (Jul 9, 2012)

What if after you ps'd the pic, do some NR, and then add a bunch of random noise evenly on the pic, rinse and repeat. Would the software still be able to pick up the unmatched noise pattern?

0 upvotes
floppie86
By floppie86 (Jul 9, 2012)

I don't need an algorythm to see that picture is photoshopped.. And I'm not talking about the absurd subject. Or is that just me?

The bird is on a whole different plane of sharpness for instance.

0 upvotes
liveagain
By liveagain (Jul 9, 2012)

Maybe, but the objective is to automate the detection process so that we can go through two million images to quickly narrow down composite ones for manual inspection, a task that would otherwise be impractical due to the labor and time involved.

1 upvote
wb2trf
By wb2trf (Jul 9, 2012)

So now the ante has been upped for compositing. Now one needs to get the new "uniform noise signature" filter posted by some hacker who just wants to meet the challenge. Next.

0 upvotes
abadona
By abadona (Jul 10, 2012)

Good idea, but not bulletproof.

By printing and image on the transparency and doing alcohol pattern, you can undermine the method to the point it is non relevant.

What if you add noise to the image? It might also screw up this process

0 upvotes
Total comments: 38