How we rate and score cameras
In 2010 we undertook a major overhaul of the way we rate and score cameras for the final conclusion of our in-depth reviews. This page explains how we produce those scores, how to interpret them and how they compare with the old system (which had run essentially unchanged since the late 1990's).
Camera ratings: at a glance
What the scores mean
The category scores (Rating bars) are designed to give you an idea of where the camera's strengths and weaknesses lie - few cameras are bad (or excel) at everything, so there's a tendency for the final scores to 'even out'. Although no replacement for actually reading the review, the results box is designed to give you an 'at a glance' view of the camera based on our findings, and how it compares to its competitors. Very short bars mean below the average for the category, very long lines mean above average. The scores behind these ratings are then combined to produce the single '%' score at the end
Although, taken on its own, this overall rating figure has little real 'meaning', it does give you an idea of how the camera ranks - when taken as a whole - compared with the other cameras in the same category.
The overall scores sit in the following very rough bands:
- 0-40% Totally unacceptable. Run away
- 41-50% Poor to Below average, avoid
- 51- 60% At best average, treat with caution
- 61-70% Average to Good
- 71-80% Very Good to Excellent
- Anything over 80%: Outstanding
This is all you really need to know:
- All scores are relative to the other cameras in the same category
- For compacts the scores are also broadly comparable across categories
- The lengths of the bars represent the weighted average of a range of measurements and scores
- The final score is calculated using a weighted average of the main category scores
- A camera's score represents 'a moment in time' - the date the review is published
- Since the scores, weights and ratings contain elements of opinion (these are, after all, reviews) you should take the time to read the entire review if you feel our opinions and priorities may not match yours.
- The ratings are produced after extensive consultation between all the editors and reviewers and after careful tabulation of all available information.
A word on weighting
You can read more about it down the page if you really want, but if all you want is the basics then read on. The final score is, as mentioned above, produced by weighting the scores of the various categories then taking an average. The weighting is the 'dpreview' opinion of what matters most (based on a lot of experience, it should be said).
Broadly speaking our main priority is image quality, so the categories that make up that get the highest weight. In future we will allow you to apply your own weights if you have more specific needs.
Want to know roughly how the final score is calculated? The weights applied break down as shown below (you can see more detail later on this page if you want):
The new awards
We've created two new awards (Gold and Silver) to replace the retiring 'Recommended' and 'Highly Recommended' gongs. There are only two things to know about the awards:
- They are not awarded to every camera, just those we feel deserve one
- There is no direct link between the overall score and the awards: they are not given automatically to cameras reaching a certain threshold. Crucially a camera can get an award even if a camera with a higher overall score didn't.
Whilst we have retrospectively applied our new stricter, more comprehensive scoring system to all cameras reviewed last year, we will not be retrospectively giving out awards (which is why you'll see the original rating on the page too).
How do the new scores relate to those in older (pre-2009) reviews?
The short answer is that they don't: the new scores represent a 'reset' of both scoring and how we give out our awards. The new SILVER and GOLD awards replace the outgoing Recommended and Highly Recommended awards, but they're not directly comparable. The old scores are still valid, but they will almost certainly be numerically higher than those produced by the new system. Long term we intend to take this scoring system back to cover all currently available cameras, but this is a huge undertaking so will take a long time.
Where the scores come from
Once a camera has been reviewed (a process that takes around 4-6 weeks for an SLR and involves 1000's of shots inside and outside the lab) all the measured data (from our lab tests) is gathered together and tabulated to establish how well it performed compared to its competitors (the other cameras in its category).
By the end of the process we scores based on measured values, specification and less easily quantifiable aspects (such as handling) for nearly 60 aspects of camera performance and specification (see below for more details). These are then combined and weighted in various ways to produce the final 11 'top level' scoring categories you see. These are then weighted before being used to calculate the final percentage rating. This information - the top level scores and the final % score - is what you see in the table at the bottom of the conclusion page.
The situation for compact (fixed lens) compacts is slightly different: there is a single category for most 'compact' cameras, and scoring in this category is based on the performance relative to a wide range of models, even those you may not consider to be direct competitors. As you'll see below, however, we have pulled out four specific categories towards the top of the market, and it is these models that we most often review.
Understanding the scores and ratings
Every camera is placed into a broad category based on its market position, feature set and price point. The scores and ratings we assign are based on the camera's performance relative to the other cameras in that category (though there is a natural tendency for average scores to rise slightly as you move up the category range - see below). If a camera is very near the boundary of the next category (in either direction) we also consider the nearest competitors in the next category if it makes sense to do so.
SLR / interchangeable lens camera categories:
- Mid Range
- High End Enthusiast / Semi-Pro
Compact (fixed lens) cameras are also categorized, though one of the categories (the first) covers a far, far wider range than any other. Scores for compacts are essentially comparable across categories simply because that's the nature of the compact camera market (where features and styling are the main differentiators), but when we're scoring we're only taking into account sensible competitors (including those in other categories where applicable).
Fixed lens camera categories:
- Compact camera - this is most point-and-click models on the market today
- Compact 'superzoom' - small cameras with big zooms (over 8x)
- Superzoom / Bridge - SLR-styled cameras with large zooms and electronic viewfinders
- Enthusiast compacts - High-end models with extensive photographic control and (usually) raw mode
Scoring categories (SLR / interchangeable lens camera)
There are 11 top level scoring categories on the conclusion page - each represents a (lightly) weighted combination of scores and measurements. Some of the aspects of performance that contribute to each score are shown below:
|Category||Constituent scores and measurements|
|Build quality||Construction, materials, finish and sealing|
Ergonomics & Handling
|Physical handling, user interface|
|Features||The features score is based on an extensive comparison of (in order of importance) core features (those we would consider essential), useful but non-standard features (such as in-body IS) and cool new features (such as art filters or in-camera HDR). Note that minor differences in shutter speed, ISO ranges etc will not affect this score.|
|Metering & Focus accuracy||Auto focus and default metering accuracy|
|Image Quality (JPEG)||Resolution, noise (up to ISO 800), noise reduction, auto white balance*, default parameters (optimization)*, pixel-level sharpness and demosaicing, dynamic range, tone curve, color and tonality, flash performance*.
|Image Quality (RAW)||Resolution, noise (up to ISO 800), dynamic Range, pixel-level sharpness|
|Low light / High ISO performance||Noise (over ISO 800), JPEG noise reduction, low light focus and exposure|
|Viewfinder / Screen||Screen size, resolution, viewfinder size / type / usability|
|Performance (speed)||Focus speed, burst rate, overall performance scores, buffer, live view performance|
|Movie / video mode||Movie format, resolution, quality and functionality|
|Value||Price / performance compared to peers|
Note that for compacts the list is slightly different (it includes separate scores for Flash performance and Optics - how good/capable the lens is).
The scores awarded in each category are shown by a bar representing the range from poor (way below average) to excellent - in comparison to the other cameras in the category at the point the review is published. Note that in order to magnify the small differences between cameras the bars do not actually start at zero. In future we will make the display of our findings considerably more interactive and informative.
The final rating is calculated using the weighted average of the 11 (or 12) scoring categories. At the moment we're using fairly mild weighting (there's enough image quality categories to ensure IQ is the number one even without weighting) that represents our view of what's most important to us, and of most import to people looking to buy a camera. Note that the bars used to represent the scores of the individual categories do not represent the weighted values - these are the absolute scores.
The weighting distribution used for compacts and SLRs (including non-reflex interchangeable lens cameras) is shown below. If you don't like our weightings simply ignore the final score - by then you should've read enough about the camera to make your own mind up!
To ensure that - at this moment in time - the absence of a movie mode in an SLR doesn't disproportionately affect the score we use a slightly different calculation for these cameras (removing the movie section from the calculation entirely), though the features score will be reduced slightly if a movie mode is considered standard for the category.
How we score - in detail
The basic scoring process is as follows:
- Review camera
- Collate all measurements and sit down with other editors to score 'unmeasurable' aspects
- Combine all these (with weighting where appropriate) into 27 basic rated attributes
- Apply weightings and combine these into the 11 (or 12) rating categories published
- Produce the final % score from the weighted average of the category scores
In future we intend to allow users to override the last two stages, applying their own weightings (to produce bespoke, personalized rankings/ratings).
All our cameras are put through a series of standardized tests - most (but not all) of which are reported in the reviews themselves. Virtually all these tests produce some kind of measured value, and these measured values are used to establish the average performance across an entire category. A very similar process is used to score attributes that are essentially part of the spec (viewfinder size, feature comparisons etc).
This allows us to establish a benchmark for producing our scores - cameras with measured results at, or near, the average get a certain score.
The exact score (out of 10) an 'average' result in a category for a particular generation of cameras gets depends on several factors, including the following:
- The spread of results (how much better the best is than the worst in the entire category)
- How 'good' the average is (based on our own long-developed criteria of acceptability)
- How rapid the trend toward improvement is (based on our historical test results)
- How accurately we consider our controlled tests reflect real world experiences
This means that the 'average' score (that given to cameras near the middle of the pack) for SLR auto white balance, for example, will be slightly higher than the average score for SLR movie mode - we see little change or variance in AWB performance (and it's normally pretty good overall), but we do see a lot of difference - and rapid change - in movie mode functionality, so the average performance is expected to rise, and differences to grow between models.
This system allows us to produce a series of scores that reflect not only how well a particular camera performs against its peers, but how good the average camera in any particular category does. This aspect of the scoring system means that, although there's no absolute relationship between the scores in different categories, there is a general increase in the average score as you move up the range (presuming, of course, that the cameras do, on average, get better as you move up the categories).
We hope in the future to offer the ability to re-score SLR cameras based on the entire market (i.e. to get an absolute score), but we've taken the decision that it's more useful when purchasing to see how cameras compare to their peers.
The big plan for the future is that now we are adding the review measurements and scores to a database we'll be able to use our findings to offer new and exciting ways to explore the relative performance of a wide range of cameras. This is only the first phase - representing the 'dpreview' score (which, yes, has an element of opinion in it - get over it!). In future we'll offer custom ranking and scoring based on preset criteria or your own preferences, priorities and needs. We're a long way off being able to deploy anything but our plans include:
- Suitability ranking / scoring based on pre-set usage profiles (sports, kids, etc)
- Custom scores (personal weighting options)
- Absolute (weighting-free) ratings/rankings
- New user interfaces for exploring cameras based on review scores
© 2010, www.dpreview.com
Dec 27, 2015
Dec 16, 2015
Nov 23, 2015
Oct 7, 2015
|2014_1211_140657AA by old shutter bugger|
from The Bride
|Overloaded by NZ Scott|
from Your City - Delivery Boy
|Petals by Flor Tempra|
from Petal Portraits
|Barley by Will B Milner|
|APPLE & ROACH by TX Photo Doc|
from Delicious - Unpalatable
Take a quick tour of Nikon's new D850 in our 'First Look' video and find out what makes this new pro-level DSLR so exciting. Hint: a lot of things.
Nikon appears to have pulled out all the stops for its D850. It combines high resolution and speed: a full-frame 45.7MP BSI CMOS sensor capable of 7 fps bursts. The D5's 153-point AF system, a tilting touchscreen and 4K/24p video are also on board. It arrives in September for $3300.
The Nikon D850 is a 45.7MP full-frame DSLR that can shoot at seven frames per second. Supporting this is an autofocus system lifted wholesale from the company's pro-sports D5 model. Add in a bigger viewfinder and full-width 4K capture and you've got a lot more than a warmed-up D810.
Cinema equipment powerhouse Arri has introduced a new line of full spectrum neutral density filters that drawn on its years of experience creating internal filtration systems for its ALEXA Mini and the AMIRA movie cameras.
The 'pocket powerhouse' Godox Wistro AD200 flash just got a new accessory. Now, you can swap out the speedlight or bare bulb heads for a 3.6-watt LED lamp.
Photographers Andrew Studer and Ted Hesser captured some of the most iconic images of this week's eclipse, showing a climber standing in the middle of the glowing corona. This is the story behind those images.
Intrigued by those ultra-cheap, fast lenses coming out of China? This video review of the $550 Sainsonic Kamlan 55mm F1.2 lens points out some of the performance you sacrifice to save an almighty dollar (or $800).
Canon has expanded its PIXMA TS-series Wireless Inkjet printer lineup with five new models, two of which contain an improved ink system that adds a sixth color ('Photo Blue') to help reduce graininess and improving overall quality.
Oprema Jena is on a roll. After a wildly successful Kickstarter to bring back the legendary Zeiss Biotar 75mm F1.5 lens, the company is sweetening the pot by resurrecting the Biotar 58mm F2 as well.
Nikon has issued a delay and apology regarding their 100th Anniversary D5, D500, and Triple Lens sets. Due to a logo issue, the company is being forced to delay shipments until October.
Yet another reason to always shoot Raw. These two shots are actually the same photo, photographer Dan Plucinski simply pulled up the shadows in post.
The Galaxy Note 8 is the first Samsung smartphone to feature a dual-cam setup. The 2x tele lens allows for a background-blurring portrait mode and comes with optical image stabilization.
Cloud backup service CrashPlan has announced that it will permanently shutter it's "for home" service by the end of October. If you use CrashPlan to back up your photos, you'll want to find an alternative ASAP.
Equivalence is much-discussed, but still often misunderstood. Here's a simplified explanation of the concept of equivalent apertures, which is just another way of talking about light received by your camera.
Try your hand at this blind portrait shootout between the Canon 1DX Mark II, Nikon D5 and Sony a9. With all bias removed, you might just rank your favorite camera brand worst.
Photo sharing site 500px has just added support for wide-gamut color profiles such as AdobeRGB and ProPhotoRGB, even allowing users to filter their searches by color profile.
DJI just released a mandatory firmware update for the DJI Spark. If you own a Spark and don't update your firmware by September 1st, DJI will remotely ground your drone.
Affordable flash manufacturer Godox has updated its smartphone app so that it can be used to control all of its wireless X flash units, not just the A1 smartphone flash.
Western Digital's new My Book Duo external desktop storage system offers up to 20TB of storage capacity, and comes with RAID-optimized WD Red hard drives.
Version 1.04 of the Sony a6500 firmware can be downloaded from the Sony Support website now.
Not sure how to choose your first drone? In this article, the second of a 3-part series, we discuss what factors you should consider when deciding what drone is right for you.
NASA photo editor Joel Kowsky didn't just capture the solar eclipse from his vantage point in Wyoming, he also managed to capture the ISS buzzing across what remained of the sun.
In these videos, talented photographer and filmmaker Daniel DeArco breaks down several tips that will help flash photography newbies start experimenting with artificial light.
Photographer and master potter Steve Irvine makes incredibly intricate, functional ceramic pinhole cameras that look like robots and monsters.
Chinese gimbal manufacturer Gudsen has released a firmware update for its Moza Air that lets you control the direction and angle of the head remotely just by moving a small handlebar-mounted control unit.
Curious how the Sony a9 performs underwater? Our friends at Backscatter took the camera diving off the Baja California coast, to find out how it handled shooting great white sharks.
While most of the DPReview crew put away our cameras and just watched the celestial event, Rishi decided last-minute to hack together a rig and capture a few shots.
Defunct Russian camera maker Zenit is making a comeback, and they're planning to release a full-frame mirrorless camera in 2018.
The days where you're more or less locked into premium or first-party flash units has gone. They're less than $50 now, so there's one less excuse not to get one. Here's our case for adding one to your kit, and a few pointers to get you going.
If you're shooting the solar eclipse here's a hint: don't fry your camera's sensor. Use a proper solar filter that offers at least 16 stops of light filtration, along with UV and IR filtering. More important? Don't look at it unless you've got solar filters. Sensors can be replaced, your retinas can't.