A decent ISO test should be indoors or out or doesn't matter?

I don't plan on making this standardized for multiple cameras, I just want to give a fair idea to those considering the camera. Multiple shots of the same subject at different ISO settings.

I'm wondering if a decent ISO test should be done indoors in average room lighting, or outdoors in shade / in sunlight? Or does it not matter as long as I give details regarding the lighting in the review?

Regards

damian5000 said:

I don't plan on making this standardized for multiple cameras, I just want to give a fair idea to those considering the camera. Multiple shots of the same subject at different ISO settings.

I'm wondering if a decent ISO test should be done indoors in average room lighting, or outdoors in shade / in sunlight? Or does it not matter as long as I give details regarding the lighting in the review?

Click to expand...

Rather than testing different cameras at the same ISO, you should consider testing them with equivalent exposure (shutter speed, fnumber) parameters and let the ISO fall where it needs to in order to achieve the correct brightness. Why? Because who shoots in ISO priority mode?

Heh.

damian5000 said:

I don't plan on making this standardized for multiple cameras, I just want to give a fair idea to those considering the camera. Multiple shots of the same subject at different ISO settings.

I'm wondering if a decent ISO test should be done indoors in average room lighting, or outdoors in shade / in sunlight? Or does it not matter as long as I give details regarding the lighting in the review?

Regards

Click to expand...

I put my camera with a good lens on a decent tripod and set it to look at the view of trees and buildings from my balcony. I use a 3 second delay to make sure that the camera is as still as possible.

I prefer sunny conditions and I try to use a fixed aperture which is optimum for the lens, say f/5.6.

Starting at the highest ISO I make sure that exposure is correct at my top shutter speed. I then take shots, reducing shutter speed as I decrease ISO in steps to its minimum value.

I may have to change aperture at least once to keep exposure correct as I don't have a neutral density filter.

Then I open the shots in Lightroom and compare noise in a neutral area - say a roadway. I optimize noise for each ISO and make noise reduction pre-sets versus ISO for this camera.

Some people say that you should do this test in low light conditions since that is when you will use high ISO values.

However, I use medium to high ISO in a variety of good light conditions such as macro with f/8-11, action shots requiring high shutter speeds, and long telephoto shots of moving targets which require image stabilization to be turned off plus high shutter speeds.

I take a few dusk shots at high ISO to look at low light performance and shadow noise but my main test is as described above.

If you shoot ISO 12800 outdoors at noon on a sunny day, it will still be relatively low noise because there is still so much light that it cancels out most of the noise. If you shoot 12800 in a dim room, then the noise is going to be a stronger signal than the dim light, so the noise cancels out the limited light.

So if you want to test cameras to see how they will perform under the conditions you shoot in, you have test them in those conditions. Most people only use super high ISO when it's quite dark, so that's why you don't want to test high ISO in bright conditions and make conclusions based on that test.

Test the camera, where you want to use them. If your real images would be more outdoor in night handholding, then do the night shot testings. If you do bird photography outdoor, then test the iso under these condition. Same for indoor sport or whatever you want to achieve. There is not single best idea how to test the iso performance.

tkbslc said:

If you shoot ISO 12800 outdoors at noon on a sunny day, it will still be relatively low noise because there is still so much light that it cancels out most of the noise. If you shoot 12800 in a dim room, then the noise is going to be a stronger signal than the dim light, so the noise cancels out the limited light.

So if you want to test cameras to see how they will perform under the conditions you shoot in, you have test them in those conditions. Most people only use super high ISO when it's quite dark, so that's why you don't want to test high ISO in bright conditions and make conclusions based on that test.

Click to expand...

Yes, noise is worse in the shadows or low level signal regions. You could shoot outside in dark conditions that have a high level of detail and deep shadows fading to black. To test something you need to test the extremes.

As a reality check you could also test what you consider to be low light. Performance between iso's can be non linear.

Wow, big variety of answers. Perhaps I'll do several tests. One in shade outdoors on a sunny day, one indoors in "normal" lighting and one poor lightning. All of the same subject at the same distance.

My purpose isn't so much for myself, but for giving an others an idea what it's capable of (or not).

damian5000 said:

Wow, big variety of answers. Perhaps I'll do several tests. One in shade outdoors on a sunny day, one indoors in "normal" lighting and one poor lightning. All of the same subject at the same distance.

My purpose isn't so much for myself, but for giving an others an idea what it's capable of (or not).

Click to expand...

You should definitely read this on the topic of fixed exposure vs fixed ISO: http://www.dpreview.com/forums/post/53996784

The former is far more relevant when it comes to assessing real-world performance.

damian5000 said:

I don't plan on making this standardized for multiple cameras, I just want to give a fair idea to those considering the camera. Multiple shots of the same subject at different ISO settings.

I'm wondering if a decent ISO test should be done indoors in average room lighting, or outdoors in shade / in sunlight? Or does it not matter as long as I give details regarding the lighting in the review?

Regards

Click to expand...

Standardized test = standardized lighting = you must be in control of lighting

= indoors.

You don't want to mislead someone by taking a shot of, say, a tree with shade, and 15 minutes later you're on camera #3 and a cloud passes in front of the sun, making the shadows subtly deeper and making noise stand out more in the dark areas.

tkbslc said:

If you shoot ISO 12800 outdoors at noon on a sunny day, it will still be relatively low noise because there is still so much light that it cancels out most of the noise. If you shoot 12800 in a dim room, then the noise is going to be a stronger signal than the dim light, so the noise cancels out the limited light.

So if you want to test cameras to see how they will perform under the conditions you shoot in, you have test them in those conditions. Most people only use super high ISO when it's quite dark, so that's why you don't want to test high ISO in bright conditions and make conclusions based on that test.

Click to expand...

I agree and that's one of the problems I have with dpreview's sample images.

Almost any camera from the cheapest to the most expensive will provide an acceptable outdoor picture, in good light, even at higher ISO settings (1600-6400).

The only way to judge a camera's low light/high ISO performance is indoors, under low light conditions.

Anything else is just fluff.

Then again, I'm probably not the best person to answer your question. I'm one of those people who still thinks good photography is all about good light.

Unless it's virtually impossible, I'll chose a flash attachment over high ISO any day. I hate raccoon eyes and shadows that aren't supposed to be there.

Where are you getting all these test cameras?

Hope you're not one of those people who orders, "tests" and returns cameras.

Ed B said:

Where are you getting all these test cameras?

Hope you're not one of those people who orders, "tests" and returns cameras.

Click to expand...

I guess I'm being misunderstood. Language barrier? I'm *not* looking to test multiple cameras or set up some standardized test for multiple cameras. I'm just looking to test this one camera.

damian5000 said:

damian5000 said:

Where are you getting all these test cameras?

Hope you're not one of those people who orders, "tests" and returns cameras.

Click to expand...

I guess I'm being misunderstood. Language barrier? I'm *not* looking to test multiple cameras or set up some standardized test for multiple cameras. I'm just looking to test this one camera.

Click to expand...

Unless it's some obscure camera, I'd guess that there are already other people testing it and doing a better job than you (and I only say that because they're doing it and you're asking questions about how to do it).

ISO is a funny thing. First, it's not standardized, and if you need it, you probably need it to achieve some specific aperture/shutter speed combo. So if I'm shooting in low light, I'm shooting at whatever ISO I need to get 1/125s (or 1/500s or 1/30s or whatever I need) wide open, and if it's ISO 3200 on one camera and 2500 on another camera, then I need to know that. (I like DXOMark for that reason). Second, you have people who shoot raw and post process and people who shoot jpeg, and the people who shoot jpeg might like minimal noise reduction or might want high noise reduction. Some people use high ISO in crummy, indoor lighting, which may be daylight balanced or incandescent or fluourescent; others use it in better lighting to get 1/2000s shutter speed shots of athletes in action.

And finally, these days, all you really know is the sensor size to have a decent idea of how a given camera is going to do, noise-wise.

So IMO, either dig into it and do it right, don't do it at all, or just throw up some sample shots because it really doesn't matter.

• Dennis
  --
  Gallery at http://kingofthebeasts.smugmug.com

damian5000 said:

damian5000 said:

Where are you getting all these test cameras?

Hope you're not one of those people who orders, "tests" and returns cameras.

Click to expand...

I guess I'm being misunderstood. Language barrier? I'm *not* looking to test multiple cameras or set up some standardized test for multiple cameras. I'm just looking to test this one camera.

Click to expand...

Sorry. I misread your original post.

My fault, not yours.

1. If you want only to compare two cameras, if you care the tests are the same and done under the very same conditions, it could have some validness independent of the details of your tests.

2. On the other hand, for a more significant results, if you insists to analyze images, I would consider two points on how those images were generated:

• The light. Ideally you have to balance your light according to the native color of the sensor (levering all colors at the same value - something like UniWB reverse), otherwise you can have some "interesting" results: make your tests under water and you could cut almost or all red channel, of course the level of noise you can get in this channel is enormous if you try to force your data for more equilibrated WB. It would mean lots of noise even at very low ISO. Tungsten tends to reduce blue channel tremendously, so when you try force the channels at same level for an equilibrated WB, again you will get lots of noise come from blue channel
• The tonal curve. Data from sensor are linear (or very close) but any conversion to create an image will apply a conversion tonal curve for each channel. Most of the times I have seen the shadows pushed way more than the remaining data. The noise level you will find in the shadows of the image can absolutely different from an image where you do not need to push the shadows at such level for images made at the same conditions, including ISO level

3. Personally I would prefer do not evaluate noise using images but from the raw files directly if it is possible to you. It is not difficult if you have some base in Math/Statistics and a tool like RawDigger or similar. The idea is to analyze raw data and create suitable Statistics. Start with blank (or black?) images at different ISO setup. It can revels lots of very interesting information on noise and eventually even the policy some manufacturers have to cope with noise (as clipping shadows deliberately eventually lost also some information). In or forums Illiah Borg and Jim Kasson (and they are not alone) have wrote very interesting observations on how raw data from camera sensors impact on the final images - actually you can visit their sites for additional and rich information.

4. At last, your last resource is to look for in the internet for several comparisons already done. DxO, DPreview, and lots of others have very interesting comparison tools for different cameras.

All the best,

cainn24 said:

damian5000 said:

I don't plan on making this standardized for multiple cameras, I just want to give a fair idea to those considering the camera. Multiple shots of the same subject at different ISO settings.

I'm wondering if a decent ISO test should be done indoors in average room lighting, or outdoors in shade / in sunlight? Or does it not matter as long as I give details regarding the lighting in the review?

Click to expand...

Rather than testing different cameras at the same ISO, you should consider testing them with equivalent exposure (shutter speed, fnumber) parameters and let the ISO fall where it needs to in order to achieve the correct brightness. Why? Because who shoots in ISO priority mode?

Heh.

Click to expand...

anyone that shoots concerts will use iso priority WHY ? because if I shoot a dance concert I set the shutter speed to 125/160 to freeze motion almost and use f 4.5 to have a little more dof so the camera can hit focus . but heh, might as well us a highend compact at f1.8 and 4 times less iso !

cheers don

Donald B said:

cainn24 said:

damian5000 said:

I don't plan on making this standardized for multiple cameras, I just want to give a fair idea to those considering the camera. Multiple shots of the same subject at different ISO settings.

I'm wondering if a decent ISO test should be done indoors in average room lighting, or outdoors in shade / in sunlight? Or does it not matter as long as I give details regarding the lighting in the review?

Click to expand...

Rather than testing different cameras at the same ISO, you should consider testing them with equivalent exposure (shutter speed, fnumber) parameters and let the ISO fall where it needs to in order to achieve the correct brightness. Why? Because who shoots in ISO priority mode?

Heh.

Click to expand...

anyone that shoots concerts will use iso priority WHY ? because if I shoot a dance concert I set the shutter speed to 125/160 to freeze motion almost and use f 4.5 to have a little more dof so the camera can hit focus . but heh, might as well us a highend compact at f1.8 and 4 times less iso !

Click to expand...

Do you ever go to a concert and say to yourself "I'd really like to use 1/125 sec and f/4.5 but I can't because I really need ISO400"?

As you yourself have demonstrated you prioritize shutter speed and fnumber.

Anyway my point is that if you want to find out how different cameras perform with a particular amount of available light you don't fix the ISO and work from there, you fix the shutter speed and the fnumber. But since the OP is apparently only going to be testing a single camera in isolation it probably doesn't matter much.

cainn24 said:

Donald B said:

cainn24 said:

damian5000 said:

I don't plan on making this standardized for multiple cameras, I just want to give a fair idea to those considering the camera. Multiple shots of the same subject at different ISO settings.

I'm wondering if a decent ISO test should be done indoors in average room lighting, or outdoors in shade / in sunlight? Or does it not matter as long as I give details regarding the lighting in the review?

Click to expand...

Rather than testing different cameras at the same ISO, you should consider testing them with equivalent exposure (shutter speed, fnumber) parameters and let the ISO fall where it needs to in order to achieve the correct brightness. Why? Because who shoots in ISO priority mode?

Heh.

Click to expand...

anyone that shoots concerts will use iso priority WHY ? because if I shoot a dance concert I set the shutter speed to 125/160 to freeze motion almost and use f 4.5 to have a little more dof so the camera can hit focus . but heh, might as well us a highend compact at f1.8 and 4 times less iso !

Click to expand...

Do you ever go to a concert and say to yourself "I'd really like to use 1/125 sec and f/4.5 but I can't because I really need ISO400"?

Click to expand...

of coarse, because you have no other choice, the lighting at a dance school concert is so variable it ranges from iso 400 to 3200. at the given shutter speeds . another reason to shoot pentax.

cainn24 said:

As you yourself have demonstrated you prioritize shutter speed and fnumber.

Anyway my point is that if you want to find out how different cameras perform with a particular amount of available light you don't fix the ISO and work from there, you fix the shutter speed and the fnumber. But since the OP is apparently only going to be testing a single camera in isolation it probably doesn't matter much.

Click to expand...

what about Noise test? I have not seen many noise tests with long exposures.

Regards,

Achyut Hatimuria

Donald B said:

cainn24 said:

Donald B said:

cainn24 said:

damian5000 said:

I don't plan on making this standardized for multiple cameras, I just want to give a fair idea to those considering the camera. Multiple shots of the same subject at different ISO settings.

I'm wondering if a decent ISO test should be done indoors in average room lighting, or outdoors in shade / in sunlight? Or does it not matter as long as I give details regarding the lighting in the review?

Click to expand...

Rather than testing different cameras at the same ISO, you should consider testing them with equivalent exposure (shutter speed, fnumber) parameters and let the ISO fall where it needs to in order to achieve the correct brightness. Why? Because who shoots in ISO priority mode?

Heh.

Click to expand...

anyone that shoots concerts will use iso priority WHY ? because if I shoot a dance concert I set the shutter speed to 125/160 to freeze motion almost and use f 4.5 to have a little more dof so the camera can hit focus . but heh, might as well us a highend compact at f1.8 and 4 times less iso !

Click to expand...

Do you ever go to a concert and say to yourself "I'd really like to use 1/125 sec and f/4.5 but I can't because I really need ISO400"?

Click to expand...

of coarse, because you have no other choice, the lighting at a dance school concert is so variable it ranges from iso 400 to 3200. at the given shutter speeds . another reason to shoot pentax.

Click to expand...

That's not "ISO priority". In fact I was basically making a joke by using that term.

If I ever go to a concert and choose a shutter speed that is too low to freeze the on-stage action because I want to use some particular ISO, feel free to confiscate all my cameras because my stupidity would warrant it.