Anyone know DXO's sigma sensor test?

Started Apr 27, 2012 | Discussions
xfz
xfz
Regular MemberPosts: 129
Like?
Anyone know DXO's sigma sensor test?
Apr 27, 2012

I don't see it on their website. Any one with their testing suite care to measure it? I am really interested in seeing LOW ISO results.

FritsThomsen
Senior MemberPosts: 2,759
Like?
DXO boycuts Sigma
In reply to xfz, Apr 27, 2012
-- hide signature --

They have said several times that they have never measured a Foveon sensor ...and never will ...
Makes sense since DXO are only there for serving Nikons marketing
Please visit my galleries at :
http://www.flickr.com/photos/yoicz/
or
http://www.naturephotos.dk/NaturePhotos_ejergalleri.php?menu=3&_Ejer=57

Reply   Reply with quote   Complain
Kendall Helmstetter Gelner
Forum ProPosts: 17,300Gear list
Like?
Not exactly right...
In reply to FritsThomsen, Apr 27, 2012

FritsThomsen wrote:

-- hide signature --

They have said several times that they have never measured a Foveon sensor ...and never will ...

For many months they said a SD-1 test was coming in July.

Some time after July, that announcement was dropped - no explanation.

Could be you are right that someone else did not like what they found and nixed the test.

 Kendall Helmstetter Gelner's gear list:Kendall Helmstetter Gelner's gear list
Sigma 70-200mm F2.8 EX DG OS HSM Sigma 8-16mm F4.5-5.6 DC HSM Sigma 24-70mm F2.8 EX DG HSM Sigma 50-500mm F4.5-6.3 DG OS HSM Sigma 85mm F1.4 EX DG HSM +2 more
Reply   Reply with quote   Complain
DaSigmaGuy
Forum ProPosts: 12,275Gear list
Like?
Re: Not exactly right...
In reply to Kendall Helmstetter Gelner, Apr 27, 2012

Kendall Helmstetter Gelner wrote:

FritsThomsen wrote:

-- hide signature --

They have said several times that they have never measured a Foveon sensor ...and never will ...

For many months they said a SD-1 test was coming in July.

Some time after July, that announcement was dropped - no explanation.

Could be you are right that someone else did not like what they found and nixed the test.

Or the result was so bad that they felt sorry for Sigma and so didn't publish it?

 DaSigmaGuy's gear list:DaSigmaGuy's gear list
Fujifilm FinePix F660EXR Sigma SD10 Sigma SD14 +25 more
Reply   Reply with quote   Complain
colinbm
Senior MemberPosts: 1,219
Like?
Re: Not exactly right...
In reply to DaSigmaGuy, Apr 27, 2012

OR it was so good that someone paid for it to be pulled ?
Col

Reply   Reply with quote   Complain
zodiacfml
Regular MemberPosts: 456Gear list
Like?
Re: Not exactly right...
In reply to colinbm, Apr 28, 2012

It was so bad that someone paid for it to be pulled. Just kidding.

In terms of DXOmark's method, I predict the SD1 will sit below, but not much, of current APS-C sensors but it will best all the others at base ISO.

colinbm wrote:

OR it was so good that someone paid for it to be pulled ?
Col

 zodiacfml's gear list:zodiacfml's gear list
Sigma DP2
Reply   Reply with quote   Complain
Roland Karlsson
Forum ProPosts: 19,548Gear list
Like?
Re: Lots of theories - all wrong?
In reply to zodiacfml, Apr 30, 2012

Lots of theories why they dont test Foveon sensors.

The true reason is somewhat different though. And that is - their test procedure does not fit Foveon sensors.

They measure RAW data. And for Bayer sensors thats red, green and blue pixels/detectors. Each color is its own channel and does not need any color conversion.

So - when they measure DR and whatever they can look at each channel separately.

But Foveon doe not have any RGB channels - its three X3 channels. The RAW data has to be converted in software to be useful in their tests. But - then they dont test the sensor.

OK - they could test the X3 layers instead. But ... no one would understand the result ... not themselves either.

 Roland Karlsson's gear list:Roland Karlsson's gear list
Canon PowerShot G10 Pentax K-5 Pentax smc DA 15mm F4 ED AL Limited Pentax smc FA 50mm F1.4 Pentax smc DA 21mm F3.2 AL Limited +6 more
Reply   Reply with quote   Complain
xfz
xfz
Regular MemberPosts: 129
Like?
Re: Lots of theories - all wrong?
In reply to Roland Karlsson, Apr 30, 2012

That make sense. However, I think they definitely should be able to do at least SNR, or maybe some DR.

Roland Karlsson wrote:

Lots of theories why they dont test Foveon sensors.

The true reason is somewhat different though. And that is - their test procedure does not fit Foveon sensors.

They measure RAW data. And for Bayer sensors thats red, green and blue pixels/detectors. Each color is its own channel and does not need any color conversion.

So - when they measure DR and whatever they can look at each channel separately.

But Foveon doe not have any RGB channels - its three X3 channels. The RAW data has to be converted in software to be useful in their tests. But - then they dont test the sensor.

OK - they could test the X3 layers instead. But ... no one would understand the result ... not themselves either.

Reply   Reply with quote   Complain
DMillier
Forum ProPosts: 16,969
Like?
Re: Lots of theories - all wrong?
In reply to xfz, Apr 30, 2012

Does Imatest work with Foveon? Given that it has become the de facto industry standard test package used by manufacturers as well as test sites, it ought to. I ought to check out IR's ever delayed SD1 test as they use Imatest.

ps

Imatest is one of the those unexpected run away success stories. Norman Koren who developed it is a former Kodak scientist who was "unexpectedly retired" to use his words. I would imagine that he had no idea that Imatest would go from a private little experiment to world dominating test software overnight!
--
Galleries and website: http://www.whisperingcat.co.uk/default.shtml

Reply   Reply with quote   Complain
DMillier
Forum ProPosts: 16,969
Like?
Re: Lots of theories - all wrong?
In reply to DMillier, Apr 30, 2012

Answer my own question.

Short answer:

Resolution - the SD1 has high rez but is still beaten here slightly by the NEX 7.

Dynamic image - at the highest quality level SD1 one stop worse than NEX 7 and 2 stop worse than K5/D7000. This is not bad actually, as the EXMOR sensors have raised the bar. This means SD1 is about on a par with older sensor designs like Canons (and they were not considered bad at all until the EXMOR appeared).

SD1 imatest results in detail:

colour

Muted color with slightly below average hue accuracy. Hue error increases and saturation falls rapidly at ISOs above 800. we noticed saturation falls off rather rapidly at ISOs above 800 to the point where colors have lost almost 50% saturation at ISO 6,400. It's not unusual to see a drop in saturation at higher ISOs, but we're not used to seeing it as dramatic as this.

Outdoors, daylight

Good exposure but muted, slightly warm color outdoors. The Sigma SD1 struggled a bit under harsh sunlight, yielding muted colors as well as a lot of clipped highlights, though exposure accuracy was better than average. Default exposure was a little hot, clipping some highlights in the white building, though shadow detail was quite good if a touch noisy.

Resolution

Very high resolution, 2,100 lines of strong detail from JPEGs, about the same from converted RAW files.

c/f to NEX 7: Resolution

Very high resolution, about 2,200 to 2,300 lines of strong detail from JPEGs, up to 2,400 from RAW files.

Sharpness & Detail

Excellent sharpness straight from the camera, with very minor edge-enhancement on high-contrast subjects. Moderate noise suppression artifacts at base ISO.

Sharpness. The Sigma SD1 produced images with excellent sharpness and very good detail, doing much better than a typical 16-megapixel Bayer sensor would do at base ISO.

Detail. The crop above right shows some detail loss due to noise suppression, as darker areas and areas of low contrast in the model's hair show some smudging and loss of definition.

ISO & Noise Performance

Very good handling of noise vs detail at low to moderate ISOs, but image quality quickly deteriorates at higher ISOs.

Images were reasonably clean at ISO 100 through 400, with just a touch of luminance and chrominance noise visible in the shadows. As mentioned, though, colors are already quite muted at base ISO. Chrominance noise becomes noticeably stronger at ISO 800 in the form of large pink and green blotches, though luminance noise was still pretty good.

Extremes: Sunlit, dynamic range and low light tests~

Very high resolution but with poor highlight preservation. Fair low-light performance, capable of capturing bright images in near darkness at low ISOs, though autofocus struggled and higher ISOs were very noisy.

dynamic range

The Sigma SD1's RAW file scored essentially the same in total dynamic range compared to the JPEG (9.78 vs 9.76 f-stops) however the score at the highest quality level increased about 1.8 f-stops from 6.1 to 7.92 f-stops. This is still a below average score, though, and it's worth noting here is that Photo Pro's default noise reduction settings reduced overall noise relative to the levels in the in-camera JPEG (compare the noise plots in the graphs), which tends to boost the dynamic range numbers for the High Quality threshold.

c/f to NEX 7 etc: the score at the highest quality level increased from 7.52 to 9.38 f-stops, which is almost a two f-stop improvement, while total dynamic range increased just over one f-stop, to 13 from 11.9. The High Quality results are very good, though not quite as good as the best APS-C sensors we've tested, hampered somewhat by slightly higher noise levels from the NEX-7's denser sensor. (The Nikon D7000 for example scored 10.1 f-stops at the highest quality level, while the Pentax K-5 scored 10.2 f-stops.)

-- hide signature --
Reply   Reply with quote   Complain
Roland Karlsson
Forum ProPosts: 19,548Gear list
Like?
Re: Lots of theories - all wrong?
In reply to xfz, Apr 30, 2012

xfz wrote:

That make sense. However, I think they definitely should be able to do at least SNR, or maybe some DR.

No - DR and SNR for the X3 RAW channels are totally uninteresting for the resulting DR and SNR.

What you could do is measure whats coming out of SPP. But ... thats processed so its npt possible to compare with Bayer cameras. And another version of SPP or with other settings will give another result.

 Roland Karlsson's gear list:Roland Karlsson's gear list
Canon PowerShot G10 Pentax K-5 Pentax smc DA 15mm F4 ED AL Limited Pentax smc FA 50mm F1.4 Pentax smc DA 21mm F3.2 AL Limited +6 more
Reply   Reply with quote   Complain
PrebenR
Senior MemberPosts: 1,957Gear list
Like?
Re: DXO boycuts Sigma
In reply to FritsThomsen, Apr 30, 2012

FritsThomsen wrote:

Makes sense since DXO are only there for serving Nikons marketing

Canon fanboy?
--
AF takes away the Zen in photography
Photos: http://www.flickr.com/photos/prebenr/
M42 on Foveon: http://www.flickr.com/groups/m42-foveon/

Reply   Reply with quote   Complain
FritsThomsen
Senior MemberPosts: 2,759
Like?
Re: DXO boycuts Sigma
In reply to PrebenR, May 1, 2012

PrebenR wrote:

FritsThomsen wrote:

Makes sense since DXO are only there for serving Nikons marketing

Canon fanboy?

No..but clearly Nikon adds "grey"-information to its files to improve the Signal/noise coefficient , thus getting extraordinarily high points in the DXO tests ...that are designed to reward this kind of fiddeling....

-- hide signature --

AF takes away the Zen in photography
Photos: http://www.flickr.com/photos/prebenr/
M42 on Foveon: http://www.flickr.com/groups/m42-foveon/

Reply   Reply with quote   Complain
DaSigmaGuy
Forum ProPosts: 12,275Gear list
Like?
Re: Not exactly right...
In reply to zodiacfml, May 1, 2012

zodiacfml wrote:

It was so bad that someone paid for it to be pulled. Just kidding.

 DaSigmaGuy's gear list:DaSigmaGuy's gear list
Fujifilm FinePix F660EXR Sigma SD10 Sigma SD14 +25 more
Reply   Reply with quote   Complain
DaSigmaGuy
Forum ProPosts: 12,275Gear list
Like?
Re: Lots of theories - all wrong?
In reply to DMillier, May 1, 2012

DMillier wrote:

Answer my own question.

Short answer:

Resolution - the SD1 has high rez but is still beaten here slightly by the NEX 7.

Dynamic image - at the highest quality level SD1 one stop worse than NEX 7 and 2 stop worse than K5/D7000. This is not bad actually, as the EXMOR sensors have raised the bar. This means SD1 is about on a par with older sensor designs like Canons (and they were not considered bad at all until the EXMOR appeared).

SD1 imatest results in detail:

colour

Muted color with slightly below average hue accuracy. Hue error increases and saturation falls rapidly at ISOs above 800. we noticed saturation falls off rather rapidly at ISOs above 800 to the point where colors have lost almost 50% saturation at ISO 6,400. It's not unusual to see a drop in saturation at higher ISOs, but we're not used to seeing it as dramatic as this.

Outdoors, daylight

Good exposure but muted, slightly warm color outdoors. The Sigma SD1 struggled a bit under harsh sunlight, yielding muted colors as well as a lot of clipped highlights, though exposure accuracy was better than average. Default exposure was a little hot, clipping some highlights in the white building, though shadow detail was quite good if a touch noisy.

Resolution

Very high resolution, 2,100 lines of strong detail from JPEGs, about the same from converted RAW files.

c/f to NEX 7: Resolution

Very high resolution, about 2,200 to 2,300 lines of strong detail from JPEGs, up to 2,400 from RAW files.

Sharpness & Detail

Excellent sharpness straight from the camera, with very minor edge-enhancement on high-contrast subjects. Moderate noise suppression artifacts at base ISO.

Sharpness. The Sigma SD1 produced images with excellent sharpness and very good detail, doing much better than a typical 16-megapixel Bayer sensor would do at base ISO.

Detail. The crop above right shows some detail loss due to noise suppression, as darker areas and areas of low contrast in the model's hair show some smudging and loss of definition.

ISO & Noise Performance

Very good handling of noise vs detail at low to moderate ISOs, but image quality quickly deteriorates at higher ISOs.

Images were reasonably clean at ISO 100 through 400, with just a touch of luminance and chrominance noise visible in the shadows. As mentioned, though, colors are already quite muted at base ISO. Chrominance noise becomes noticeably stronger at ISO 800 in the form of large pink and green blotches, though luminance noise was still pretty good.

Extremes: Sunlit, dynamic range and low light tests~

Very high resolution but with poor highlight preservation. Fair low-light performance, capable of capturing bright images in near darkness at low ISOs, though autofocus struggled and higher ISOs were very noisy.

dynamic range

The Sigma SD1's RAW file scored essentially the same in total dynamic range compared to the JPEG (9.78 vs 9.76 f-stops) however the score at the highest quality level increased about 1.8 f-stops from 6.1 to 7.92 f-stops. This is still a below average score, though, and it's worth noting here is that Photo Pro's default noise reduction settings reduced overall noise relative to the levels in the in-camera JPEG (compare the noise plots in the graphs), which tends to boost the dynamic range numbers for the High Quality threshold.

c/f to NEX 7 etc: the score at the highest quality level increased from 7.52 to 9.38 f-stops, which is almost a two f-stop improvement, while total dynamic range increased just over one f-stop, to 13 from 11.9. The High Quality results are very good, though not quite as good as the best APS-C sensors we've tested, hampered somewhat by slightly higher noise levels from the NEX-7's denser sensor. (The Nikon D7000 for example scored 10.1 f-stops at the highest quality level, while the Pentax K-5 scored 10.2 f-stops.)

-- hide signature --

So essentially, they are simply confirming what I have been saying on here ever since the first SD1 samples were released!...The higher pixel density has reduced DR and caused major problems with capturing highlights, compared to previous Sigma models.

Even more reason for Sigma to think far more conservatively when deciding how many photosites to cram onto any FF Foveon sensor they may be developing.

 DaSigmaGuy's gear list:DaSigmaGuy's gear list
Fujifilm FinePix F660EXR Sigma SD10 Sigma SD14 +25 more
Reply   Reply with quote   Complain
Roland Karlsson
Forum ProPosts: 19,548Gear list
Like?
Re: Lots of theories - all wrong?
In reply to DaSigmaGuy, May 1, 2012

DaSigmaGuy wrote:

So essentially, they are simply confirming what I have been saying on here ever since the first SD1 samples were released!...The higher pixel density has reduced DR and caused major problems with capturing highlights, compared to previous Sigma models.

I could not read that from the results.

Even more reason for Sigma to think far more conservatively when deciding how many photosites to cram onto any FF Foveon sensor they may be developing.

That does not work. Sigma has to keep the MP count up. It has to be able to compete resolution wise with the better cameras out there.

 Roland Karlsson's gear list:Roland Karlsson's gear list
Canon PowerShot G10 Pentax K-5 Pentax smc DA 15mm F4 ED AL Limited Pentax smc FA 50mm F1.4 Pentax smc DA 21mm F3.2 AL Limited +6 more
Reply   Reply with quote   Complain
DMillier
Forum ProPosts: 16,969
Like?
Re: Lots of theories - all wrong?
In reply to DaSigmaGuy, May 2, 2012

Alf

There's nothing here that points to reduced DR compared to previous models. It's just that the EXMOR sensors have moved the bar. SD1 looks to be in the same ballpark as previous generation bayer sensors.

-- hide signature --
Reply   Reply with quote   Complain
DaSigmaGuy
Forum ProPosts: 12,275Gear list
Like?
Re: Lots of theories - all wrong?
In reply to DMillier, May 2, 2012

DMillier wrote:

Alf

There's nothing here that points to reduced DR compared to previous models.

My eyes tell me the SD1 has reduced DR compared to previous models!
Its visible in the vast majority of SD1/SD1M samples.

 DaSigmaGuy's gear list:DaSigmaGuy's gear list
Fujifilm FinePix F660EXR Sigma SD10 Sigma SD14 +25 more
Reply   Reply with quote   Complain
Roland Karlsson
Forum ProPosts: 19,548Gear list
Like?
Re: Lots of theories - all wrong?
In reply to DaSigmaGuy, May 2, 2012

DaSigmaGuy wrote:

My eyes tell me the SD1 has reduced DR compared to previous models!
Its visible in the vast majority of SD1/SD1M samples.

DSG with the golden eyes!

I think I want some measurements though.

 Roland Karlsson's gear list:Roland Karlsson's gear list
Canon PowerShot G10 Pentax K-5 Pentax smc DA 15mm F4 ED AL Limited Pentax smc FA 50mm F1.4 Pentax smc DA 21mm F3.2 AL Limited +6 more
Reply   Reply with quote   Complain
FritsThomsen
Senior MemberPosts: 2,759
Like?
Re: Lots of theories - all wrong?
In reply to DaSigmaGuy, May 2, 2012

DaSigmaGuy wrote:

DMillier wrote:

Alf

There's nothing here that points to reduced DR compared to previous models.

My eyes tell me the SD1 has reduced DR compared to previous models!
Its visible in the vast majority of SD1/SD1M samples.

My ...very subjektive and non-scientific DR- ranking of various Sigma cameras

SD 10 = very high lets call it = 100 points
SD 14 = high = 90
DP1 = extremely high= 110
DP2s = low = 60
SD15 = low 70
SD 1 = medium = 80 ????
They hit a low when they introduced the new TRUE prossesor ...
--
Please visit my galleries at :
http://www.flickr.com/photos/yoicz/
or
http://www.naturephotos.dk/NaturePhotos_ejergalleri.php?menu=3&_Ejer=57

Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads