K100D RAW is actually RAW+JPEG !

Ok, so I shoot pretty much RAW but I don't like to post-process every single image. So I've seen that in Pentax Photo Browser there's an interesting option which functions with RAW images: "Extract JPEG...". The result is instantaneous, and you can get the JPEG results from a bunch of RAW images in a couple of seconds. Strange, as the CPU doesn't work, so the JPEG must be embedded in the RAW image.

Here's some schematics:



After a closer look at the RAW file, I've seen that there are actually 3 JPEG files embedded in it: a thumbnail, a mid-size image, and the full JPEG, developed with the actual parameters used when shooting the photo.

They are always situated at the following offsets (given as hexadecimals):
$3394 => mid-size 640x480 (around 50kb)
$1005E => thumbnail, 160x120 (around 8kb)
$8DF4DE => full JPEG, 3008x2000

That's pretty encouraging, I can shoot RAW now and post-process only the images I want, for the rest I can extract the JPEG directly. The quality of the resulting JPEG seems to be two stars ( ), I compared its size with 3 shots at , and and it's pretty similar to the one at two stars ( ).

(Ok, all this makes me think at the K10D, which has RAW and RAW+JPEG function. If JPEG is still embedded in the RAW file, RAW+JPEG means then RAW+JPEG+JPEG... there's some redundancy in-here).

Pentax RAW has always had an embedded jpeg - even on my "antique" *istD. That is what you see when you review on the camera lcd.

--
Steve

http://www.pbase.com/steephill

Atoma said:

After a closer look at the RAW file, I've seen that there are
actually 3 JPEG files embedded in it: a thumbnail, a mid-size
image, and the full JPEG, developed with the actual parameters used
when shooting the photo.

They are always situated at the following offsets (given as
hexadecimals):
$3394 => mid-size 640x480 (around 50kb)
$1005E => thumbnail, 160x120 (around 8kb)
$8DF4DE => full JPEG, 3008x2000

Click to expand...

thanks for sharing, this is something new indeed. At least to me (but I'm newbie here, anyway, and my current camera is istDS not K100D).

Atoma said:

(Ok, all this makes me think at the K10D, which has RAW and
RAW+JPEG function. If JPEG is still embedded in the RAW file,
RAW+JPEG means then RAW+JPEG+JPEG... there's some redundancy
in-here).

Click to expand...

I do not think K10D employs the same scheme. After all, it has RAW+JPEG mode and dedicated button to quickly turn it on. Futhermore, its RAW (PEF) files are only 10MB large, from 10Mpx camera that's pretty small.

steephill said:

Pentax RAW has always had an embedded jpeg - even on my "antique"
*istD. That is what you see when you review on the camera lcd.

Click to expand...

I never knew that. Well, that's what forums are good for - learning from others

I seriously doubt the K10d RAW+jpg mode has two copies in it. The full size jpg is the one that Pentax Photo Browser uses to display the raw files. I thought that it was low quality, but your observations are more encouraging. I've used the function before on snapshots that really didn't need anything and were not worth keeping the RAW file. I also tested that jpg vs. the one I created from the Pentax Photo Laboratory and I must say I couldn't tell the difference. In fact, the camera does a great job of adjusting the colors in high contrast shots. If you open a RAW file of a sunset, you often have to slide the bars to preserve satuation and preserve color to get the picture back to where the camera had it in the jpg version.
--

Verbage is not a word, but a mangling of verbiage - differing in both print and meaning. It replaces more cumbersome words such as copy and text.

steephill said:

Pentax RAW has always had an embedded jpeg - even on my "antique"
*istD. That is what you see when you review on the camera lcd.

Click to expand...

... and it can be extracted to a JPEG file using the current Pentax software or the excellent PEF2JPEG utility if you're on a PC (see: http://www.kit.hi-ho.ne.jp/yj_okawa/pef2jpeg_readme-e.html ) or with command line tools like Exiftool.

It's amazing how many timed this keeps being "discovered"

--
John Bean [GMT - is there any other?]

PAW Week 45:
http://waterfoot.smugmug.com/gallery/1082841/4/111655527/Large



Index page: http://waterfoot.smugmug.com
Latest walkabout (4 April): http://waterfoot.smugmug.com/gallery/1348582

The full size Jpeg is highly compressed. The IQ is really bad. I won't call that RAW+JPeg.

It is indeed "highly compressed". The designation of "good" that Pentax photo browser gives the file is the same one generated by a highly compressed image generated through Pentax Photo laboratory. The quality is still good enough for a lot of things. I just pixel peeped a test on this and found that I couldn't tell any real difference between the two. The extracted image apparently had +1 on the sharpening as it actually looked better than the highest quality I extracted due to this. The only other noticable differences were the comparible file sizes - the extracted file was about 70% larger than the equivalent photo generated through Photo lab program. I assume the camera is not using as complicated of an encoding scheme as the software program and cannot make as efficient of a conversion, but there could be something else going on as well. Also the height of the extracted image was 2000 pixels while the converted RAW was 2008 pixels - just weird.
--

Verbage is not a word, but a mangling of verbiage - differing in both print and meaning. It replaces more cumbersome words such as copy and text.

This is a great feature for workflow, especially if one post a lot of images on the net. Already having existing reduced-sized shots would save time and would actually encourage me, for instance, to switch from JPEG to RAW for my general photography needs.

Will be interesting to see how the K10 handles this in the final-release firmware.

TOny

Foxbat121 said:

The full size Jpeg is highly compressed. The IQ is really bad. I
won't call that RAW+JPeg.

Click to expand...

The quality of the embedded JPEG is actually similar to two stars ( ), so it's pretty useful.

The 2000 pixels height is odd, could this be related to the Bayer filter?
--
Andy
http://andreit.free.fr/photos/Featured/

Atoma said:

The 2000 pixels height is odd, could this be related to the Bayer
filter?

Click to expand...

No it's related to the JPEG compression. 2000 (and 3008) are divisible by 16, 2008 isn't, so the JPEGs are always that size. Raw files record the full sensor data but the crop size is tagged as 2008x2008.

--
John Bean [GMT - is there any other?]

PAW Week 45:
http://waterfoot.smugmug.com/gallery/1082841/4/111655527/Large



Index page: http://waterfoot.smugmug.com
Latest walkabout (4 April): http://waterfoot.smugmug.com/gallery/1348582

I use the raw-embedded JPGs from my DS now for a more than 1.5 year for previewing and sometimes they're so good I don't need to develop the raw anymore.
--
Menno

John Bean (UK) said:

Atoma said:

The 2000 pixels height is odd, could this be related to the Bayer
filter?

Click to expand...

No it's related to the JPEG compression. 2000 (and 3008) are
divisible by 16, 2008 isn't, so the JPEGs are always that size. Raw
files record the full sensor data but the crop size is tagged as
2008x2008.

Click to expand...

Is this need to be divisible by 16 to do with image rotation?

I remember seeing that some raw converters output very slightly different size jpegs compared to other converters. I noticed this when flipping through a number of variations of images, processed with different raw converters. Can't remember which converter did what, but I seem to remember that at least one of them cropped the image just a fraction. It probably wouldn't make any difference to anything.

RustyHinges said:

John Bean (UK) said:

Atoma said:

The 2000 pixels height is odd, could this be related to the Bayer
filter?

Click to expand...

No it's related to the JPEG compression. 2000 (and 3008) are
divisible by 16, 2008 isn't, so the JPEGs are always that size. Raw
files record the full sensor data but the crop size is tagged as
2008x2008.

Click to expand...

Is this need to be divisible by 16 to do with image rotation?

Click to expand...

Yes and no - a JPEG image is compressed (and can be losslessly rotated) using the individual "tiles". They don't need to be 16x16 but that size is often chosen as a good compromise of size and efficiency so it's very common to find the raw dimensions trimmed so as to be divisible by 16.

--
John Bean [GMT - is there any other?]

PAW Week 45:
http://waterfoot.smugmug.com/gallery/1082841/4/111655527/Large



Index page: http://waterfoot.smugmug.com
Latest walkabout (4 April): http://waterfoot.smugmug.com/gallery/1348582

John Bean (UK) said:

RustyHinges said:

Is this need to be divisible by 16 to do with image rotation?

Click to expand...

Yes and no - a JPEG image is compressed (and can be losslessly
rotated) using the individual "tiles". They don't need to be 16x16
but that size is often chosen as a good compromise of size and
efficiency so it's very common to find the raw dimensions trimmed
so as to be divisible by 16.

Click to expand...

Ahhh! Presumably, very slight differences between one converter's JPEG dimensions and another's are just a question of whether there has been any rounding to 16. The knowledge (and patience!) of the good people on this forum frequently astounds me. Thank you!

It depends on what you're look at. You won't tell a difference between differen compression ratio JPEGs when images are smooth. The compression artifacts are shown in high contrast edges. That's why you feel the embedded jpeg are sharpened. It is not. It's the compression artifacts.

I feel even the ' ' Jpeg mode on my DL to be subpar as the files are usually around 2MB. I'd prefer 3~4MB+ Jpeg file size for highest quality Jpeg to eliminate the JPeg compression artifacts.

Yes it is useful to serve as thumbnail or general photo browsing. But to use it as your main photo is a bit stretch.

Atoma said:

Foxbat121 said:

The full size Jpeg is highly compressed. The IQ is really bad. I
won't call that RAW+JPeg.

Click to expand...

The quality of the embedded JPEG is actually similar to two stars
( ), so it's pretty useful.

Click to expand...

Andy, the embedded full size JPEG is well known over hundreds of very finely detailed images (that are thus less efficient to compress) to average about 1.2 MBytes as compared to the 18 MBytes a 6 MP 8 bit per channel TIFF image would take for a compression ratio of 1:15 or so. It takes a compression ratio of about 1:6 or so to pretty much preserve detail in finely detailed JPEG images without perceptible image degradation. That makes the embedded full size JPEG's unreliable as to IQ when there are fine details in the image. However, it's fine for quickly reviewing composition, exposure, proper focus, etc., as in quickly reviewing on the camera's LCD monitor or your computer.

Regards, GordonBGood

In weddings, to print contact sheets, i use JPEG files as they are much faster to process than RAW files and the application that i use doesn't open .PEFs.

It works like a charm, the mid/high compressed JPEGs have more that enought quality to be printed in contact sheets ( 5cmx7,5cm) and then i can post process in my home lab (my iMac ) the RAW files for the final prints.

So, for little size prints or for web, this embedded JPEG files can do the job. But only for this.

------------------------------------------------------
http://www.flickr.com/photos/patrickx