Sony NEX-6 Benchmark Test?

Started Sep 1, 2013 | Questions
Mel Snyder
Veteran MemberPosts: 3,603Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to justincarlson, Sep 2, 2013

justincarlson wrote:

I'm about half a month out of the return period for Amazon, but I've emailed Sony to see if they have any sort of warranty for these lenses.

Justin:

I don't think there's anything wrong with your lens.

Read my comments - neither of your test images were good tests. One was shot at too high an ISO, and one was shot way over water in diffraction range for this lens. I see evidence of both errors on the two images you bravely submitted. Most who complain about this and other lenses don't have the courage to submit more than their unsubstantiated comments.

You could, of course, like so many here, begin the snipe hunt for a better copy, although as I explained, in this era of software-designed lenses, the repeatability of manufacturing is astonishing.

As I noted, there are those who would postulate that lens making is like souffle making, and some made by the same recipe come out differently. While possible, not probable. If you hear hoofbeats outside your window at night, it could be a zebra, or even a unicorn. But neither is a probable explanation.

Those who finally get a good copy are learning more about the camera and lens as they pursue their Quixotic search for a "better copy." And as a result of their testing and testing, get better. And what they find at the end of their search is not truly a better lens, but a better photographer.

Should you choose to contact Sony, they will likely ask you to send them your lens for testing. That leads to one of two probable and a third possible result:

1. They will tell you there's nothing wrong with your lens - after 2-3 weeks.

2. They will send you a "refurbished" or "recertified" replacement. That's one that really had something wrong with it, and they "fixed" it. You could end up with a worse lens than you sent them.

3. Least probable, but possible - they will send you a brand spanking new lens

Before I'd go down that path, Justin, I'd want to be darn sure there was something really wrong with your lens. I don't think you're quite there yet.

If you'd like to send me some original OOC jpegs or raw images not processed by the DPREVIEW display algorithms, send me a private message and I'll respond with my email address.

Mel

 Mel Snyder's gear list:Mel Snyder's gear list
Sony Alpha NEX-6 Sony Alpha 7 Sony E 16mm F2.8 Pancake Tokina AT-X Pro 11-16mm f/2.8 DX Leica Summicron-M 50mm f/2 +13 more
Reply   Reply with quote   Complain
santiclaws
Contributing MemberPosts: 684
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to justincarlson, Sep 2, 2013

I have a suggestion: read up on basic photography settings and learn why anything but lousy images at the settings you used would have been a shock. If you're used to driving a Toyota Civic and then get into a Porsche 911, stomp on the gas and end up in a tree it ain't the Porsche's fault.

Learn to use the tool you have. Then, if you still think there's something wrong with it, come back and post your results with images taken at reasonable settings.

Reply   Reply with quote   Complain
hogeye
Regular MemberPosts: 102Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to justincarlson, Sep 2, 2013

justincarlson wrote:

I've got a Sony NEX-6 with the 16-50mm kit lens, and I've really been having difficulty getting good quality shots from it. I know the 1650 isn't the sharpest of lenses in the least, but my photos seem to be coming out with way more blur (I'm not sure if blur is the right word) than you should expect with an $800 camera. If I'm working with a lot of light I seem to be able to get o.k. shots.

I'm wondering if there is some sort of benchmark at standardized focal length/aperture/ISO that DPreview has to test if I have a bad lens copy or if there is even possibly something wrong with my camera's sensor. It seems more likely that it would be the lens. I've had the camera for about a month and a half, purchased from Amazon.

I've attached an image I shot today, perhaps it's not the best sample image because of the low light and high ISO, but it seems like the lower-lit areas are just not as clear as they should be. I can provide other samples as well if it's helpful.

I was getting a little paranoid myself after hearing so many complaints of the 1650 focus quality and found this simple procedure posted in one of the threads. I copied it for later since it would be difficult to locate the thread it came from. Kudos to the poster!

I bought the Bracket Pro app and performed the test a few days ago and found my lens was good. It won't hurt to make sure before the warranty runs out.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

"Best way is to buy the focus bracketing app for a few $, leave it on default bracketing spacing and then stand about 10 feet away from a sign ( building works warning signs are good as they have different sizes of writing on the sign so its easy to judge focus) .. then with the aperture WIDE OPEN (so thats f5.6 at 50mm) take a few shots (when you press the shutter the NEX takes its focus shot then one either side of its focus point) .. make sure your focus is set to spot and targetted on the sign. The depth of field (DOF) with these settings is i think about 2.5 feet (dark lens) so if you are not getting good focus in this situation the AF is badly off.

In my case one of the bracketed focus points was sharper than the NEX's so Sony took it back to repair straight away once emailed them the shots ...

If you are getting a decent focus just move your bracketing spacing to smaller and see how it goes .. this way you will find out if its your lens thats crap or your expectations".

 hogeye's gear list:hogeye's gear list
Fujifilm FinePix S6000fd Sony Alpha NEX-6 Sony E 55-210mm F4.5-6.3 OSS Sony E 16-50mm F3.5-5.6 PZ OSS Sony E 35mm F1.8 OSS
Reply   Reply with quote   Complain
justincarlson
Forum MemberPosts: 88Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to hogeye, Sep 2, 2013

Thanks!!

 justincarlson's gear list:justincarlson's gear list
Sony Alpha NEX-6 Sony Carl Zeiss Sonnar T* E 24mm F1.8 ZA Sony E 50mm F1.8 OSS Sony E 16-50mm F3.5-5.6 PZ OSS HDRsoft Photomatix Pro +1 more
Reply   Reply with quote   Complain
GaryW
Veteran MemberPosts: 6,985Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to Mel Snyder, Sep 2, 2013

Mel Snyder wrote:

wfektar wrote:

justincarlson wrote:

I've attached an image I shot today, perhaps it's not the best sample image because of the low light and high ISO, but it seems like the lower-lit areas are just not as clear as they should be. I can provide other samples as well if it's helpful.

Think of it this way -- light is information. At 1/4000 sec exposure time you're getting very little data in the shadows -- there's practically nothing there to work with. At 37mm focal length you should be able to handhold that at 1/125th easily and you'd have 32 times more data in the shadows to work with.

Brilliant response! You can see the noise reduction artifacts clearly in the grass just to the right of the top of the waterfall - what Henry ("blue skies") would call a failure of the "fur test."

Yeah, it's the noise reduction that is harming the low-level detail the most, I'd bet.  I turned my NR to "Low", although that isn't even the right answer -- the RAW processors on the computer will do a better job, and I can fine-tune the results.

When it comes to the waterfall, instead of trying to freeze the action, most people would want some motion.  I would have tried for as slow of a shutter as I could.  But without an ND filter, 1/125 at ISO 100 and f5.6 might have been worth trying.  Maybe something like f8 and 1/60?  A little blur to the waterfall, plenty of DOF with f8, and a slow enough shutter to hand-hold.  I bet the results would look a lot better.

But even if you wanted the water motion frozen, I would go for, say, 1/1000 or something to get something more reasonable for the ISO.  1/4000 just seems needlessly fast.

-- hide signature --

Gary W.

 GaryW's gear list:GaryW's gear list
Sony Alpha NEX-6 Sony E 16-50mm F3.5-5.6 PZ OSS Sony E PZ 18-105mm F4 G OSS Sony Cyber-shot DSC-V3 Sony Cyber-shot DSC-HX5 +12 more
Reply   Reply with quote   Complain
justincarlson
Forum MemberPosts: 88Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to GaryW, Sep 2, 2013

Thanks for the tips. I was afraid that since I didn't have an ND filter I wouldn't be able to get enough blur for the shot to look nice, but it would have been worthwhile to try something like 1/60 like you said.

 justincarlson's gear list:justincarlson's gear list
Sony Alpha NEX-6 Sony Carl Zeiss Sonnar T* E 24mm F1.8 ZA Sony E 50mm F1.8 OSS Sony E 16-50mm F3.5-5.6 PZ OSS HDRsoft Photomatix Pro +1 more
Reply   Reply with quote   Complain
justincarlson
Forum MemberPosts: 88Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to hogeye, Sep 3, 2013

So how exactly would I run this test? Should it be done outdoors where lighting is better? If I printed off some targets and taped them to my garage door would that work for testing the lens with the bracketing app? How do I know if the lens performs well vs. poorly with this particular test?

 justincarlson's gear list:justincarlson's gear list
Sony Alpha NEX-6 Sony Carl Zeiss Sonnar T* E 24mm F1.8 ZA Sony E 50mm F1.8 OSS Sony E 16-50mm F3.5-5.6 PZ OSS HDRsoft Photomatix Pro +1 more
Reply   Reply with quote   Complain
verybiglebowski
Senior MemberPosts: 3,024Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to justincarlson, Sep 3, 2013

justincarlson wrote:

So how exactly would I run this test? Should it be done outdoors where lighting is better? If I printed off some targets and taped them to my garage door would that work for testing the lens with the bracketing app? How do I know if the lens performs well vs. poorly with this particular test?

You don't need any application IMHO to test your lens. What you need though is to proceed systematically.

First you probably want to check if your lens is optically correct.

You need some flat target with a pattern that will allow you to judge resolution and will fill the frame at at least 20x of the focal length. (32 -50cm suggested for 16mm up to 1-2m for 50mm). Any 100% static target that please your eye can do. (Newspapers taped to garage doors i.e.) Be sure not to test the lens on some moving targets - such as trees or water.

Try to place your target under even light, such as shade or overcast.

Put your lens on the sturdy tripod in self timer (if not in hurry 10s is better than 2s) mode or use remote shutter.

Switch off the OSS.

Switch to Manual Focus.

Software corrections makes this tests hardly relevant, but without'em it will be even harder for You to judge corners. So at least set the sharpness in JPEG to zero.

Use large magňification to acquire proper focus. If not sure, try to adjust it for more than 1 capture and select the sharpest image later. You might use peaking, but don't relay only on it.

Take several shots at several FL and apertures and check them carefully in the computer.

After this test, you should made a valuable conclusion about overall resolution behavior of your lens.

From there, you can proceed further depending on the results...

If you find one side softer than other (ot one corner) your lens might be decentered (or you didn't align your target well, or your sensor isn't properly alligned). If you have tripod with reversable middle column, reverse it and take another shot of your target with the camera upside-down. If not, turn it in portrait mode and take another shot of the target. In the computer check if the soft part moved with the image or not. Make the conclusion based on that.

If you got good resolution results with the above tests, you might have a problem with AF. Here you can use the bracketing application, but you can also simply switch between MF and AF and compare results. Be sure to make large enough contrast area for AF.

If you find your MF images to be sharper than AF ones, you should take your camera and lens to the service claiming AF problems.

-- hide signature --

Don't trust your eyes or mind, they might betray you! Trust only comments posted on the forums, because there is the absolute truth!
http://www.flickr.com/photos/viktor_viktor/
http://verybiglobo.blogspot.com/

Reply   Reply with quote   Complain
hyenadog
Contributing MemberPosts: 724
Like?
Re: Sony NEX-6 Benchmark Test? before and after
In reply to justincarlson, Sep 3, 2013

i had a similar problem .. best way is to buy the focu bracketing app and set it up on default focus bracketing .. set the zoom fully open at 50mm and take some shots at about 10-15 feet away from a building sign (different sized text so good test of sharpness and focus)

I sent mine back to sony because the outlier focus bracketing was better focus than the lens AF point (on default spacing) .. which explains its bad perceived softness ! .. got it back a few days ago after havung a new lens block fitted .. nice and sharp focus now (for a kit lens)

a lot of people have slated the lens as being soft but in fact its just crap Sony lens AF QC on this lens in many cases

here are a few test shots from a couple of days ago on the way back from the post office

http://www.dpreview.com/forums/post/52068989

Reply   Reply with quote   Complain
justincarlson
Forum MemberPosts: 88Gear list
Like?
Re: Sony NEX-6 Benchmark Test? before and after
In reply to hyenadog, Sep 3, 2013

Were those taken with a 50mm prime?

 justincarlson's gear list:justincarlson's gear list
Sony Alpha NEX-6 Sony Carl Zeiss Sonnar T* E 24mm F1.8 ZA Sony E 50mm F1.8 OSS Sony E 16-50mm F3.5-5.6 PZ OSS HDRsoft Photomatix Pro +1 more
Reply   Reply with quote   Complain
justincarlson
Forum MemberPosts: 88Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to verybiglebowski, Sep 3, 2013

Thanks so much, this is exactly what I was looking for. I'll try this tomorrow and see what I come up with.

 justincarlson's gear list:justincarlson's gear list
Sony Alpha NEX-6 Sony Carl Zeiss Sonnar T* E 24mm F1.8 ZA Sony E 50mm F1.8 OSS Sony E 16-50mm F3.5-5.6 PZ OSS HDRsoft Photomatix Pro +1 more
Reply   Reply with quote   Complain
Sosua
Senior MemberPosts: 2,259Gear list
Like?
ISO 3200 and 1/4000th?
In reply to justincarlson, Sep 3, 2013

What mode are you in?

Far better results could be attained with ISO 200 and 1/250th (4 stops less ISO and shutter speed).

Cheers,

-- hide signature --

www.samwaldron.co.nz

 Sosua's gear list:Sosua's gear list
Olympus OM-D E-M5 Panasonic Lumix G 20mm F1.7 ASPH Olympus M.Zuiko Digital ED 40-150mm 1:4-5.6 R Olympus M.Zuiko Digital 45mm 1:1.8 Olympus M.Zuiko Digital ED 12mm 1:2
Reply   Reply with quote   Complain
verybiglebowski
Senior MemberPosts: 3,024Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to justincarlson, Sep 3, 2013

justincarlson wrote:

Thanks so much, this is exactly what I was looking for. I'll try this tomorrow and see what I come up with.

Let us know what you'll find. I forgot to add - put the camera in A mode and 100 ISO. Good luck.

-- hide signature --

Don't trust your eyes or mind, they might betray you! Trust only comments posted on the forums, because there is the absolute truth!
http://www.flickr.com/photos/viktor_viktor/
http://verybiglobo.blogspot.com/

Reply   Reply with quote   Complain
nevada5
Veteran MemberPosts: 4,792Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to Mel Snyder, Sep 3, 2013

Mel Snyder wrote:

nevada5 wrote:

justincarlson wrote:

I've got a Sony NEX-6 with the 16-50mm kit lens, and I've really been having difficulty getting good quality shots from it. I know the 1650 isn't the sharpest of lenses in the least, but my photos seem to be coming out with way more blur (I'm not sure if blur is the right word) than you should expect with an $800 camera. If I'm working with a lot of light I seem to be able to get o.k. shots.

I'm wondering if there is some sort of benchmark at standardized focal length/aperture/ISO that DPreview has to test if I have a bad lens copy or if there is even possibly something wrong with my camera's sensor. It seems more likely that it would be the lens. I've had the camera for about a month and a half, purchased from Amazon.

I've attached an image I shot today, perhaps it's not the best sample image because of the low light and high ISO, but it seems like the lower-lit areas are just not as clear as they should be. I can provide other samples as well if it's helpful.

Some people have posted some very nice images from the SELP1650. Others, like me, have not had positive experiences with the lens. I've come to believe there are serious fluctuations between various copies. Mt experience was that I purchased two NEX-6/SELP1650 kits. In both instances the lens performed very poorly and I sent back/sold the lens.

I decided to go with the SEL1855 for the times that I want a standard zoom. The first one I bought was no better than the 1650's I had. The second one is a very good copy - images taken with it are as sharp and contrasty as any kit zoom I've had. Again I was taught first hand that with Sony's inexpensive lenses, quality control is lacking.

I would not be pleased with the image you posted either. If you are within a return period, I might suggest putting the lens through some more testing, especially at lower ISO settings. I don't shoot JPEG so I can't say that the camera isn't using some heavy-handed noise-reduction at ISO 3200.

If you can't get substantially better IQ from the lens, it's quite possible you have a poor copy and you might think about returning it. You may or may not get a better one in an exchange.

Good luck.

-- hide signature --

Life is short - drive a convertible.
Tolerance is the last virtue of a dying society. Aristotle

There's nothing wrong with his lens, and possibly, nothing was really wrong with yours. In his case, as an earlier poster pointed out, he shot at too high an ISO and the in-camera noise reduction killed the image.

This poster actually showed an image that he thought demonstrated a lens issue, and we were able to analyze it and save him a snipe hunt for a better copy.

What too many people fail to realize is that modern lenses are designed by computers and matched to in-camera software. They aren't turned out like souffles, where one made with the same recipe can be different from the ones made before and after. I know it's comforting to many to believe that myth, but like so much on this forum, it's pretty easy to slay some beautiful hypotheses with an ugly fact, if we're just shown the images.

Which few posters who complain about a "bad copy" rarely do.

Maybe you're overdue for a little vacation there Mel.  Some place quiet.  Maybe write some new lines.

-- hide signature --

Life is short - drive a convertible.
Tolerance is the last virtue of a dying society. Aristotle

Reply   Reply with quote   Complain
justincarlson
Forum MemberPosts: 88Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to verybiglebowski, Sep 3, 2013

Here are some manual focus shot comparisons showing the sharpness at varying focal lengths and apertures. I compared them with autofocused shots, and the autofocus seems to be fine. I should have increased the exposure compensation for these, I didn't realize they would be so dark in the shade. Metering mode was set to center-weighted average.

16mm f6.3 MF

16mm f14 MF

35mm f6.3 MF

35mm f14 MF

50mm f6.3 MF

50mm f11 MF

 justincarlson's gear list:justincarlson's gear list
Sony Alpha NEX-6 Sony Carl Zeiss Sonnar T* E 24mm F1.8 ZA Sony E 50mm F1.8 OSS Sony E 16-50mm F3.5-5.6 PZ OSS HDRsoft Photomatix Pro +1 more
Reply   Reply with quote   Complain
Lightshow
Senior MemberPosts: 2,475Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to justincarlson, Sep 3, 2013

justincarlson wrote:

Here's a shot I took at a smaller aperture and ISO-100. Once again with this shot I was not happy with the sharpness, but maybe I'm being too critical.

I've looked at your 2 shots, the first has the details obliterated by the high ISO noise reduction, go with the lowest ISO you can to get the shutter speed you need to get the shot, if you want to stop the water, 1/250 of a sec should be fast enough, and if you want to show movement, 1/2 sec should do.

The 2nd does seem a bit soft @ 100%, I'm not sure you needed f13 to get the shot, f 8 should be good enough and could be closer to the lenses sweet spot.

Another thing you can do, is take your camera and lens to a camera store, and take some shots with your lens on your camera, then more shots with the store's lens to compare sharpness, hopefully the results are obvious so you have a clear path forward.

good luck.

 Lightshow's gear list:Lightshow's gear list
Sony Alpha NEX-7 Sony Alpha 7R +1 more
Reply   Reply with quote   Complain
justincarlson
Forum MemberPosts: 88Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to Lightshow, Sep 3, 2013

Thanks for the feedback. That's a good point, using a larger aperture would have likely increased the sharpness by at least a small amount.

 justincarlson's gear list:justincarlson's gear list
Sony Alpha NEX-6 Sony Carl Zeiss Sonnar T* E 24mm F1.8 ZA Sony E 50mm F1.8 OSS Sony E 16-50mm F3.5-5.6 PZ OSS HDRsoft Photomatix Pro +1 more
Reply   Reply with quote   Complain
Lightshow
Senior MemberPosts: 2,475Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to justincarlson, Sep 3, 2013

justincarlson wrote:

Here are some manual focus shot comparisons showing the sharpness at varying focal lengths and apertures. I compared them with autofocused shots, and the autofocus seems to be fine. I should have increased the exposure compensation for these, I didn't realize they would be so dark in the shade. Metering mode was set to center-weighted average.

16mm f6.3 MF

16mm f14 MF

35mm f6.3 MF

35mm f14 MF

50mm f6.3 MF

50mm f11 MF

Were you standing off to the side of the center of the door? It definitely feels that way.

try it again with the camera centered and square to the door on a tripod, use the histogram to get your exposure better, and post crops of the 4 corners and centre.

 Lightshow's gear list:Lightshow's gear list
Sony Alpha NEX-7 Sony Alpha 7R +1 more
Reply   Reply with quote   Complain
justincarlson
Forum MemberPosts: 88Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to Lightshow, Sep 3, 2013

I was pretty close to the center, however the garage door is slightly caved in in the middle from wind damage. I'll try to center the targets better.

Should I do all three FL's again, or just one or two? Should I do two different apertures per FL as well?

 justincarlson's gear list:justincarlson's gear list
Sony Alpha NEX-6 Sony Carl Zeiss Sonnar T* E 24mm F1.8 ZA Sony E 50mm F1.8 OSS Sony E 16-50mm F3.5-5.6 PZ OSS HDRsoft Photomatix Pro +1 more
Reply   Reply with quote   Complain
boardsy
Senior MemberPosts: 2,197Gear list
Like?
Re: Sony NEX-6 Benchmark Test?
In reply to justincarlson, Sep 3, 2013

justincarlson wrote:

I was pretty close to the center, however the garage door is slightly caved in in the middle from wind damage. I'll try to center the targets better.

Should I do all three FL's again, or just one or two? Should I do two different apertures per FL as well?

For a simple test I typically just turn off OSS, choose A mode, EV = 0, base ISO, sit the camera on a table in front of a dollar bill or wine bottle label or something, carefully manual focus on the center at max magnification with the shutter self-timer, not allowing anyone to walk past at the crucial moment! Quick and simple, and will show you how sharp any lens can be e.g. Canon FD 35/2 at f2:

http://www.flickr.com/photos/59079068@N02/8077686538/

Any shot less sharp than this at f2 is now scientifically proven to be my fault, not the lenses!

I genuinely don't care too much about lens de-centering, it's never been an issue - even if I'm shooting a flat surface I'll do it at f5.6 or f8 where most or enough of it will be in focus regardless. YMMV

http://www.flickr.com/photos/59079068@N02/8636232964/

-- hide signature --

Regards,
Alan
my Flickr

Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads