Reading mode: Light Dark

READ: A77 true max ISO and something disturbing about A77 RAW

Go to last post
Iliah Borg said:
Iliah Borg said:
As mentioned in my post, there are cameras that have both. They are older Sony models, like the A700, that have both RAW and cRAW formats.
Click to expand...
No, a700 makes cRAW after the fact, from linear raw, in digital domain. In current cameras compression is accomplished in analogue domain.

--
http://www.libraw.org/
Click to expand...
You have evidence that gain is applied non-linearly in the analogue domain (i.e. on chip?). It would be a highly unusual technique.
 
Kilrah said:
TimberWolfQX said:
mastroalex: You need to convert them to DNG first. Remember the creator died before SLT cameras were released. Converting them to DNG will allow you to open them in Rawanalyze.
Click to expand...
Waitasecond... so you did those tests on DNGs?

That's something that should have been said in the first post... You do realise that converting to DNG is affecting data, right? In order to fit manufacturer-specific data format into the DNG standard there's a series of operations taking place...

From the DNG spec:
Kilrah said:
Mapping Raw Values to Linear Reference Values

The section describes DNG's processing model for mapping stored raw sensor values into linear reference values.
The processing model follows these steps:
•Linearization
•Black Subtraction
•Rescaling
•Clipping
Click to expand...
And so on... some being done according to the "receipt" included in the RAW file, some being "guessed" by Adobe...

Can't see any kind of objective evaluation done on non-native DNGs, as we have no idea what comes from the RAW file and what comes from the DNG conversion!!

Have you at least also converted the other cameras' files you used for comparison into DNG?

--
http://www.flickr.com/photos/kilrah/
http://www.youtube.com/user/kilrahvp
Click to expand...
I did say that in first post. However, under conversion settings, there is a "Preserve RAW Image" setting, which preserves the raw mosaic data. I did not choose "Linear", which would have destroyed the original RAW data.
 
Iliah Borg said:
SteveGJ said:
Iliah Borg said:
Iliah Borg said:
The dynamic range of all sensors decline from base ISO onwards
Click to expand...
Hm. Really of all the sensors?
Click to expand...
Every sensor that I've ever seen recorded on DXOmark has had that characteristic.
Click to expand...
You misunderstand the way they test. Amplification does not affect the dynamic range the way you think it is.
Click to expand...
I disagree - apply gain and the brightest level that can be measured without saturation is reduced in proportion. The gain applied at the low level is effectively going to amplify noise. That raises the noise floor and reduces the DR. ISO in digital sensors is simply the degree of gain applied over the native sensitivity (whether applied in the analogue or digital domains).
Iliah Borg said:
Iliah Borg said:
The base ISO is essentially that level at which no gain is applied
Click to expand...
Nope. Some gain in the form of conversion coefficient is always applied, and for base ISO is can be negative gain.
Click to expand...
Base ISO is not the same as the lowest ISO (or call it native ISO if you prefer). I know that on some cameras negative gain is applied for lower ISOs, but in those instances highlight headroom will be reduced. The saturation is just down to how many photons are received and result in excited electrons. Halve the gain has no effect on that at all - it's just the camera's exposure calculations would allow twice the light in. As it is, it seems that the measured ISOs are often higher at the low end. For instance, the A77's 50 ISO appears to be around 83 ISO, presumably to minimise the danger of staturation.

Also, I'm sure that the sensors still do pass through the amplification stages, even if it's to provide buffering. However, the point is that there is no boost for sensitivity - just what's required to interface to the A/D conversion. Probably I should make clear that the gain I'm talking about is that above any interfacing level when working above native or base ISO.
Iliah Borg said:
Iliah Borg said:
It's not true that all the bits are rerquired. The "data" (as opposed to the noise) is only recorded in the more significant bits of the actual value.
Click to expand...
It was demonstrated that even the LSB of 14-bit raw contains image data.
Click to expand...
Demonstrated where? It might be that on a very high quality sensor running at base or native ISO that there may be data in the 14th bit (maybe the D7000 or K5 gets close to that level). However, it manifestly isn't the case at 1600 ISO when there most definitely won't be anything by noise in the last few bits. Values yes, image information, no.
Iliah Borg said:
--
http://www.libraw.org/
Click to expand...
 
hum interesting article... i downloaded the program and i have many high iso shots with my e-p1 and e-pl1 cameras and i want to find out whats their maximum real analogue iso...

i converted the files at different isos from raw but couldnt find a way to tell.. except that there is a value when histogram is at fine setting that goes from 64,675 to 65,575 and 66,575 from iso 800 to 1600...

would like some help to detect cameras real iso...

thanks
 
TimberWolfQX wrote:
\
TimberWolfQX said:
seilerbird666 said:
What a load of BS, you must be M!55ls brother.
Click to expand...
Whoa whoa whoa. Big difference here: I own the A77 and have used it extensively over the past few days. I have shot events, posted my impressions on this forum here: http://forums.dpreview.com/forums/readflat.asp?forum=1037&message=39634020
Click to expand...
I am sorry to have said that, it was extreme. I don't think you are looking at the RAW file.
TimberWolfQX said:
The ARW files were converted to DNG and processed using Photobola Raw Analysis Version 2.10.4.0.
Click to expand...
You have converted the RAW file twice.
 
compositor20 said:
hum interesting article... i downloaded the program and i have many high iso shots with my e-p1 and e-pl1 cameras and i want to find out whats their maximum real analogue iso...

i converted the files at different isos from raw but couldnt find a way to tell.. except that there is a value when histogram is at fine setting that goes from 64,675 to 65,575 and 66,575 from iso 800 to 1600...

would like some help to detect cameras real iso...

thanks
Click to expand...
Shoot a series of test pictures at different high ISO. Also, shoot one at the base ISO of your camera.

Open the base ISO dng/raw file using Photobola Raw Image Analyzer. Click "Histogram" option, check the "Fine" box. You will see the channel data as in the first picture of the original post. Next, try loading the rest of the high ISO pics. The highest ISO picture that does not exhibit "combing" (for extreme example, see my ISO 12800 histogram) is the true high ISO.

QX
 
OK where is this evidence? Non-linear gain is used for things like audio dynamic range compression. I'm not sure why you'd do such a thing in the analogue domain for image sensors when there's a digital processor to hand.
 
Hi QX,

I have to agree with SteveGJ on this one, in that the data you have shown, whilst, interesting is not as damaging as it first appears. Nikon use, "lossy" NEF files as a matter of routine and a good explanation of why it is not that important can be found here,
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#bitdepth .

Also, ETTR is not the panacea that it is claimed to be and the, "extra raw levels", it purports to achieve is a bit of a myth, again a full explanation of this point can be found at the above site.

Best regards, Howard
 
SteveGJ said:
apply gain and the brightest level that can be measured without saturation is reduced in proportion.
Click to expand...
Yes. But simultaneously you are recovering some shadows. In certain cases the situation is balanced and photographic dynamic range is preserved.
SteveGJ said:
Base ISO is not the same as the lowest ISO (or call it native ISO if you prefer).
Click to expand...
Who says it is?
SteveGJ said:
I know that on some cameras negative gain is applied for lower ISOs,
Click to expand...
You have a well of 70 thousand, and 14 bit output. So you have to divide. That is what causes negative gain (or, if you prefer, the conversion ratio of less than 1).
SteveGJ said:
SteveGJ said:
It was demonstrated that even the LSB of 14-bit raw contains image data.
Click to expand...
Demonstrated where?
Click to expand...
Here, on DPR. ISO 800 night scene.

--
http://www.libraw.org/
 
Fraxinus excelsior said:
I want RAW not some kind of lossy cRAW or like a77. Let me deside Sony.

Hope it will be adressed in a firmwareupdate and that the comming FF cameras are able to shoot RAW not some kind of cRAW.
Click to expand...
As soon as the image sensor signal went through the ADC it got 'compressed' in exaclty the same way, the signal was truncated to the nearest discrete level. The only issue is how finely spaced those discrete levels need to be in order to get sufficiently accurate image data. Due to photon noise, which grows as the light level increases, the spacing need not be as fine in highlights as it is in shadows because of the inherent inaccuracy introduced by the noise. (An analogy -- there is no reason to have a stopwatch accurate to 1/1000sec if your reflexes in operating it are only accurate to 1/10 sec.)

Sony cRAW is not 'lossily' compressed in any meaningful way, any more than 14-bit raw from DSLR's is more 'compressed' than 16-bit raw from MFDB's, provided the noise in the image data is always sufficiently large compared to the tonal spacing of raw levels.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/
 
Iliah Borg said:
Iliah Borg said:
Iliah Borg said:
It was demonstrated that even the LSB of 14-bit raw contains image data.
Click to expand...
Demonstrated where?
Click to expand...
Here, on DPR. ISO 800 night scene.
Click to expand...
Iliah is referring to his test shown here:
http://forums.dpreview.com/forums/read.asp?forum=1042&message=39630479

though the original thread is much longer ago. His interpretation of the test is incorrect however as is explained here:
http://forums.dpreview.com/forums/read.asp?forum=1021&message=28196804
http://forums.dpreview.com/forums/read.asp?forum=1021&message=28209024

When blacks are not clipped, and noise dominates the last bit, that bit is equally likely to be 1 or 0. When blacks are clipped in raw data, as Nikon does, then the last bit locally encodes the percentage of pixels that were shifted to negative values by the noise (which can be either positive or negative), because in that percentage of pixels the last bit is artificially set to zero by the clipping. With a broad noise distribution, such as one has at higher ISO, the average signal can be rather far above black when the lower tail of the local histogram starts getting clipped; as the local average moves up and down, less and more of the tail dips below zero and so the last bit mirrors the average signal level (see the graphic at my links above). But it is wholly a nonlinear effect due to the clipping, that imprints data from higher bits into the last bit; it is not that the last bit encodes any independent image information.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/
 
All this Analysis Paralysis !

So when you sold that image or won 1st place in a competition did anybody ask you how many levels you used? Go out there and enjoy the camera

:)
 
theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html
TimberWolfQX said:
The fact that level quantization much finer than the noise is superfluous, since noise erases the perceptual impact of abrupt tonal transitions, means that many of the levels used to record higher exposure zones are unneeded and indeed wasteful -- photon shot noise is much larger than the level spacing in midtones and highlights on a typical DSLR.
Click to expand...
TimberWolfQX said:
That is, if the average photon count is 10000, there will be fluctuations from pixel to pixel of as much as sqrt[10000]=100 photons in the sample. Suppose each increase by one in the raw level corresponds to counting ten more photons (i.e. the gain g=10 photons/ADU); then noise for this signal is 100/10=10 raw levels. The linear encoding of the raw signal wastes most of the raw levels, since the noise is much larger than the quantization step.
Click to expand...
TimberWolfQX said:
In shadows, it's a different story. Suppose our average signal is 100 photons; then the photon fluctuations are sqrt[100]=10 photons, which translates to + - one raw level for the supposed gain g=10. At low signal level, none of the raw levels are "wasted" in digitizing the noise.
Click to expand...
TimberWolfQX said:
The idea behind "lossy" NEF compression is thus to thin the levels in highlights where levels are too finely distributed relative to the inherent noisiness of light, but keep most or all of the levels in shadows where the magnitude of the noise fluctuations is smaller. This is why the NEF compression table is linear at the low end, and quadratic further up. The quadratic part is a reflection of the quadratic relationship between noise and signal of light (the noise squared is proportional to signal) -- the higher the signal the more levels can be thinned out. The linear part of the compression table kicks in when the signal is low enough that the noise fluctuations are less than or of order one raw level -- then no compression is possible and the output value must equal the input value.
Click to expand...
TimberWolfQX said:
Quantitatively, the appropriate criterion is that a jump in the compressed value C by one should correspond to a span of raw values S that is less than the noise (which is at least as big as the shot noise sqrt[S/g]), so that the noise continues to dither the larger jumps of the compressed image data where levels have been eliminated. In other words,
Click to expand...
dC > dS/sqrt[S/g]
TimberWolfQX said:
A bit of calculus then leads to the relation between compressed values and raw values
Click to expand...
S less than ((C-C0)^2)/4g
TimberWolfQX said:
where C0 is a constant offset equal to half the length of the linear part of the table, where compression is not possible. The best fit parabola to the nonlinear part of the lookup table, which is the red curve in Figure 21, has C0=134 and g=18. The criterion that the jumps in compressed values not exceed the photon noise is that this value of g should be more than the actual gain of the camera at its base ISO. In fact, the typical base ISO gain for the Nikon models that use this particular lookup table is about 6-8, so the condition is met with a good margin of safety.
Click to expand...
TimberWolfQX said:
Bottom line: The "lossy" form of NEF compression is a clever use of information theory to save space by eliminating redundant raw levels. The noise which is unavoidably present in light effectively dithers tonal transitions so that the compression is lossless in that the image is still encoded without loss of visual information. In this sense, "lossy" compression is perhaps an inappropriate appelation.
Click to expand...
I stand corrected on the issue of RAW compression. There is indeed sense in the nonlinear encoding algorithm.

QX
 
HL53 said:
Nikon use, "lossy" NEF files
Click to expand...
It does not matter what Nikon are "using" (they are manufactures and designers, shooting is not their bread and butter), it matters that:
  • Nikon raw shooters have a choice
  • Nikon raw shooters I know do not use lossy or visually lossless compression.
You can investigate the quality of bokeh using different raw shooting modes if you wish.

--
http://www.libraw.org/
 
You must log in or register to reply here.

Keyboard shortcuts

f
Forum
m
My threads
Mobile site
About
Editorial content
Cameras & Lenses
Community
All content, design, and layout are Copyright © 1998–2025 Digital Photography Review All Rights Reserved.
Reproduction in whole or part in any form or medium without specific written permission is prohibited.
When you use DPReview links to buy products, the site may earn a commission.
©GPS Media - Guides, Products, Services.
Back
Top