Fuji GFX 50S over rated by dpr...

  • Thread starter Thread starter Christof21
  • Start date Start date
I agree, my semi-automatic process misses the "notches" at ISO 1600.
My hand analysis yields something like ISO 1308.
Perhaps at some point I'll try to make my semi-automatic process "smarter".
How about a 1D FFT of the histogram and threshold the result for periodicities some moderate distance away from zero frequency?
There are quite a few things I'd like to add to my software that require 1D or 2D FTs.
It's on my list but I don't use MatLab and need to integrate an appropriate library into my software (currently using c# any specific ideas are welcome).
What I do is "quick and dirty" and with some fine-tuning would do even better.
(The current approach only "sees" gaps that are zero not notches.)

Regards,
 
I don't think we'd be getting that close to 1 electron.

But let's look at it from the standpoint of PDR Shadow Improvement :

9d6aa00ed6da43b8bf904c061072aa4b.jpg.png

I think the red linear extrapolation is too aggressive but comes out at 1/2 stop.
I think the blue curved extrapolation is more like what we typically see and comes out at about 1/6 stop.
I'll buy 1/6 of a stop. Thanks.

Jim
Beautiful work Bill. I'd like to replace my earlier phrase 'imaginary noise benefit I can't even see' with '1/6 of a stop noise benefit' then. ;)

-Rishi
 
Christof21 wrote: That is why the good question of this thread is: Is this sensor ISOLess over ISO1600. The last conclusions of this thread could be that it is ISOLess in fact (or almost), contrary to what I may think when comparing with Pentax...
I was personally going under the assumption that Fuji engineers came to that conclusion and that's why they stopped amplifying the signal above ISO1600.
To be clear on the semantics, which we have not always been so far in this thread; the assumption is that the engineers made it ISO-less at the point that it became ISO Invariant.

The threshold for ISO Invariance is subjective; people apply different standards.

I suspect the engineers were influenced (maybe in a circular fashion) by something practical.
Perhaps they reached the top end of the amplifier they chose.
(Just guessing, I'm not a sensor designer.)
For reference, by looking at the Pentax645z ISO3200 and 6400 histograms it appears to me that they stop raising analog gain and go digital earlier than ISO1600, I would say about ISO1350. From then to ISO10k they simply chip away at DR with no apparent other benefit. Then at ISO12800 it seems that they reconfigure the pixel (I am not sure how they do that).
On the 645Z I see noise reduction starting at ISO 3200 and digital scaling starting at ISO 6400.

Just this morning I have introduced additional symbols to my charts to convey this information, for example :

beff37dc70b1470c8678b5d4c9018f36.jpg.png

The determinations may be imperfect and are subject to change although for the 645Z these look solid.

(Scaling is determined by inspecting histogram gaps and noise reduction by visual inspection of 2D FTs. With over 150 data series this is subject to human error :-) )

Regards,

--
Bill ( Your trusted source for independent sensor data at http://www.photonstophotos.net )
This is fantastic and exciting, Bill. Sorry if I missed it, but do you have a key to the symbols? For example, filled in circle vs clear circle, triangle pointing upwards, etc.?

Thanks,
Rishi
 
Christof21 wrote: That is why the good question of this thread is: Is this sensor ISOLess over ISO1600. The last conclusions of this thread could be that it is ISOLess in fact (or almost), contrary to what I may think when comparing with Pentax...
I was personally going under the assumption that Fuji engineers came to that conclusion and that's why they stopped amplifying the signal above ISO1600.
To be clear on the semantics, which we have not always been so far in this thread; the assumption is that the engineers made it ISO-less at the point that it became ISO Invariant.

The threshold for ISO Invariance is subjective; people apply different standards.

I suspect the engineers were influenced (maybe in a circular fashion) by something practical.
Perhaps they reached the top end of the amplifier they chose.
(Just guessing, I'm not a sensor designer.)
For reference, by looking at the Pentax645z ISO3200 and 6400 histograms it appears to me that they stop raising analog gain and go digital earlier than ISO1600, I would say about ISO1350. From then to ISO10k they simply chip away at DR with no apparent other benefit. Then at ISO12800 it seems that they reconfigure the pixel (I am not sure how they do that).
On the 645Z I see noise reduction starting at ISO 3200 and digital scaling starting at ISO 6400.

Just this morning I have introduced additional symbols to my charts to convey this information, for example :

beff37dc70b1470c8678b5d4c9018f36.jpg.png

The determinations may be imperfect and are subject to change although for the 645Z these look solid.

(Scaling is determined by inspecting histogram gaps and noise reduction by visual inspection of 2D FTs. With over 150 data series this is subject to human error :-) )

Regards,

--
Bill ( Your trusted source for independent sensor data at http://www.photonstophotos.net )
This is fantastic and exciting, Bill. Sorry if I missed it, but do you have a key to the symbols? For example, filled in circle vs clear circle, triangle pointing upwards, etc.?

Thanks,
Rishi
One of the tricks to see the breaking points in handling the raw data is to look at the changes in black level and suggested maximum, for example:

exiftool -filename -StandardOutputSensitivity -whitelevel -blackpoint -ext pef -T /Volumes/Raw_Collection/Pentax/KP/

pentax_kp_01.pef 100 16318 64 64 64 64

pentax_kp_02.pef 200 16254 128 128 128 128

pentax_kp_03.pef 400 16126 256 256 256 256

pentax_kp_04.pef 800 15870 512 512 512 512

pentax_kp_05.pef 1600 15871 512 512 512 512

pentax_kp_06.pef 3200 15359 1024 1024 1024 1024

pentax_kp_07.pef 6400 14335 2048 2048 2048 2048

pentax_kp_08.pef 12800 14335 2048 2048 2048 2048

pentax_kp_09.pef 25600 14217 2048 2048 2048 2048

pentax_kp_10.pef 51200 12054 4092 4092 4092 4092

pentax_kp_11.pef 102400 11817 4092 4092 4092 4092

pentax_kp_12.pef 204800 11342 4092 4092 4092 4092

pentax_kp_13.pef 409600 10392 4092 4092 4092 4092

pentax_kp_14.pef 819200 8491 4092 4092 4092 4092

--
http://www.libraw.org/
 
Last edited:
On the 645Z I see noise reduction starting at ISO 3200 and digital scaling starting at ISO 6400.

Just this morning I have introduced additional symbols to my charts to convey this information, for example :

beff37dc70b1470c8678b5d4c9018f36.jpg.png

The determinations may be imperfect and are subject to change although for the 645Z these look solid.

(Scaling is determined by inspecting histogram gaps and noise reduction by visual inspection of 2D FTs. With over 150 data series this is subject to human error :-) )
This is fantastic and exciting, Bill. Sorry if I missed it, but do you have a key to the symbols? For example, filled in circle vs clear circle, triangle pointing upwards, etc.?
It's at the top of the chart, but we don't read everything when we see a pretty picture, do we :-)

ba5f6460b7894f9a8612de0549be4499.jpg.png

See the caveats in bold above.

Regards,

--
Bill ( Your trusted source for independent sensor data at http://www.photonstophotos.net )
 
There are quite a few techniques, none is perfect (for a number of interesting reasons), and eventually you're faced with what the true definition of "scaling" is.

I'm not interested in small amount such as for WB-preconditioning.
Mostly I care if analog stops and additional gain is solely digital scaling.

Regards,
 
On the 645Z I see noise reduction starting at ISO 3200 and digital scaling starting at ISO 6400.

Just this morning I have introduced additional symbols to my charts to convey this information, for example :

beff37dc70b1470c8678b5d4c9018f36.jpg.png

The determinations may be imperfect and are subject to change although for the 645Z these look solid.

(Scaling is determined by inspecting histogram gaps and noise reduction by visual inspection of 2D FTs. With over 150 data series this is subject to human error :-) )
This is fantastic and exciting, Bill. Sorry if I missed it, but do you have a key to the symbols? For example, filled in circle vs clear circle, triangle pointing upwards, etc.?
It's at the top of the chart, but we don't read everything when we see a pretty picture, do we :-)

ba5f6460b7894f9a8612de0549be4499.jpg.png

See the caveats in bold above.

Regards,

--
Bill ( Your trusted source for independent sensor data at http://www.photonstophotos.net )
Haha Bill! You're right - I'm easily distracted by beautiful graphs :)

Caveats noted. Thanks, as always.

-Rishi
 
How about a 1D FFT of the histogram and threshold the result for periodicities some moderate distance away from zero frequency?
There are quite a few things I'd like to add to my software that require 1D or 2D FTs.
It's on my list but I don't use MatLab and need to integrate an appropriate library into my software (currently using c# any specific ideas are welcome).
If I were you I would drop C# like a hot potato. And, use the freely available FFTW (possibly the fastest and mature freely available and open source FFT library).
 
There are quite a few techniques, none is perfect
"Breaks" are important when one needs to establish the exposure meter offset. Breaks coincide with switching modes of analog circuitry and multiplication. Combined with histogram gaps analysis, read noise on black frames, levels at which the non-linearity and clipping start, and Fourier metadata gives a very clear picture. However one may want to start with metadata to plan the experiments better. If the other data contradicts metadata, it is when I double-check my experiments. Vignetting or CA compensation may skew experimental results.
I'm not interested in small amount such as for WB-preconditioning.
WB preconditioning may give false positives for noise reduction when calculating Fourier transforms, same as other manipulations. Those manipulations are also ugly in many other respects.

--
http://www.libraw.org/
 
Last edited:
I'm not interested in small amount such as for WB-preconditioning.
WB preconditioning may give false positives for noise reduction when calculating Fourier transforms, same as other manipulations. Those manipulations are also ugly in many other respects.
All digital manipulations applied to raw data before recording it are unnecessary and nasty. So are some analog manipulations like that negative gain applied to fake 2^n - 1/3 ISO settings.
 
I'm not interested in small amount such as for WB-preconditioning.
WB preconditioning may give false positives for noise reduction when calculating Fourier transforms, same as other manipulations. Those manipulations are also ugly in many other respects.
All digital manipulations applied to raw data before recording it are unnecessary and nasty. So are some analog manipulations like that negative gain applied to fake 2^n - 1/3 ISO settings.
While I see where you're coming from, and generally agree, I think there are some digital manipulations that are OK. Dead and stuck pixel mapping, for instance, although in a perfect world the dead and stuck pixel locations would be encoded in the metadata and the mapping done in post -- does anyone do it that way?

With respect to "negative gain", you have me confused. It is possible to lower the pre-ADC gain so that the pixel reaches non-linearity before ADC full scale, but I don't see how that gain could be considered negative. On the other hand, it might be possible to implement all the analog gain settings in an inverting amplifier and setting up the ADC accordingly, but that would be negative gain for all the ISO settings.

Jim
 
I'm not interested in small amount such as for WB-preconditioning.
WB preconditioning may give false positives for noise reduction when calculating Fourier transforms, same as other manipulations. Those manipulations are also ugly in many other respects.
All digital manipulations applied to raw data before recording it are unnecessary and nasty. So are some analog manipulations like that negative gain applied to fake 2^n - 1/3 ISO settings.
While I see where you're coming from, and generally agree, I think there are some digital manipulations that are OK. Dead and stuck pixel mapping, for instance, although in a perfect world the dead and stuck pixel locations would be encoded in the metadata and the mapping done in post -- does anyone do it that way?
I do not think anybody does it this way, or they need to disclose metadata format for third-party converters to work.
With respect to "negative gain", you have me confused. It is possible to lower the pre-ADC gain so that the pixel reaches non-linearity before ADC full scale, but I don't see how that gain could be considered negative. On the other hand, it might be possible to implement all the analog gain settings in an inverting amplifier and setting up the ADC accordingly, but that would be negative gain for all the ISO settings.
I think he means something like this (look how at ISO 320, ISO 1250, and ISO 5000 the specular white level drops):

ISO black specular_white

100 511 11892

200 511 14338

320 2048 10755

400 2048 14338

500 2048 14338

800 2048 14338

1250 2048 10755

1600 2048 14338

2000 2048 14338

3200 2048 14338

5000 2048 10755

6400 2048 14338

12800 2049 14338

25600 2048 14338

51200 2041 14338
 
While I see where you're coming from, and generally agree, I think there are some digital manipulations that are OK. Dead and stuck pixel mapping, for instance, although in a perfect world the dead and stuck pixel locations would be encoded in the metadata and the mapping done in post -- does anyone do it that way?
I do not think anybody does it this way, or they need to disclose metadata format for third-party converters to work.
Also, I think that dead and stuck pixel stats are probably considered trade secrets, like yields.

It's been years (decades, actually), but I vaguely remember disks coming with bad sector maps.

Jim
 
With respect to "negative gain", you have me confused. It is possible to lower the pre-ADC gain so that the pixel reaches non-linearity before ADC full scale, but I don't see how that gain could be considered negative. On the other hand, it might be possible to implement all the analog gain settings in an inverting amplifier and setting up the ADC accordingly, but that would be negative gain for all the ISO settings.
I think he means something like this (look how at ISO 320, ISO 1250, and ISO 5000 the specular white level drops):

ISO black specular_white

100 511 11892

200 511 14338

320 2048 10755

400 2048 14338

500 2048 14338

800 2048 14338

1250 2048 10755

1600 2048 14338

2000 2048 14338

3200 2048 14338

5000 2048 10755

6400 2048 14338

12800 2049 14338

25600 2048 14338

51200 2041 14338
Yes. The gain is smaller than what a simple model would say it would be, but it's not negative.

Nice table. As to the data therein, ugh.

Jim
 
With respect to "negative gain", you have me confused. It is possible to lower the pre-ADC gain so that the pixel reaches non-linearity before ADC full scale, but I don't see how that gain could be considered negative. On the other hand, it might be possible to implement all the analog gain settings in an inverting amplifier and setting up the ADC accordingly, but that would be negative gain for all the ISO settings.
I think he means something like this (look how at ISO 320, ISO 1250, and ISO 5000 the specular white level drops):

ISO black specular_white

100 511 11892

200 511 14338

320 2048 10755

400 2048 14338

500 2048 14338

800 2048 14338

1250 2048 10755

1600 2048 14338

2000 2048 14338

3200 2048 14338

5000 2048 10755

6400 2048 14338

12800 2049 14338

25600 2048 14338

51200 2041 14338
Yes. The gain is smaller than what a simple model would say it would be, but it's not negative.
I do not know how it is implemented in all the cameras, and certainly overall gain is not negative, but one of the several gain registers or tables may be programmed with a negative value.

I can't exclude this is done in digital domain, expansion in digital domain is easy to detect, but compression is not that easy to detect, and results may be inconclusive.
Nice table. As to the data therein, ugh.
The data looks unnecessary manipulated, to avoid even a hint of ISO-less process in the camera ;)

The interesting part is that specular white levels are changing, even depending on the assembly temperature.
 
All digital manipulations applied to raw data before recording it are unnecessary and nasty. So are some analog manipulations like that negative gain applied to fake 2^n - 1/3 ISO settings.
With respect to "negative gain", you have me confused.
That's my poor choice of words, sorry, Jim. Lowering gain is what I meant.
We're on the same page now. Thanks for getting back to me.

Jim
 
There has been mention in this thread that indicates that Fuji is cheating the users out of choice that is by all rights theirs by ceasing the increase of analog gain after a certain ISO setting and giving the raw developer instructions about ite ISO setting so that it can apply (linear, or hopefully non-linear) gain in post.

Maybe so, although I don't think Fuji's leaving much on the table.

But what about the many camera manufacturers that apply digital gain at high ISOs in the camera before they write the data to the raw file? Surely, they're depriving the user of analog gain in exactly the same way.
I didn't notice this thread until a bit late, but my $0.02 is that analog and digital gain are completely separate things that can change the recorded tonal properties in ways that are not all that difficult to measure and understand -- and are pretty easy to model.

I wouldn't mind having the option of manually (explicitly) turning those two dials separately, and as Jim was pointing out, you can sort-of do that with cameras that apply analog gain and not with cameras that destructively apply digital gain (as this Fuji apparently does in some cases). However, I think the real answer is to automate the tradeoff selection using smarter exposure logic. I suggested a couple of ways to do this in my Electronic Imaging 2015 paper, ISO-less? (and here's a preprint of it you can read for free).

BTW, literally ISO-less (as Rishi has taken to calling ISO-invariant) behavior almost never happens -- there are almost always tradeoffs in the tonal qualities. For example, it isn't "ISO-invariant" if you get better dynamic range using digital vs. analog gain. In the end, I used "ISO-less" to describe the process of determining exposure with analog and digital gains as outputs of exposure computation, not inputs.
 
Last edited:

Keyboard shortcuts

Back
Top