RAW files getting ruined by Apple Photos

[truth1ness]

I shot this RAW photo for the first time on my Canon G7x Mark II, imported it into photos and it looked fine until I pressed Edit and the photo instantly went from this (I created this jpeg from the RAW before pressing Edit):

Jpeg from Raw before pressing "Edit"

to this terribly messed up/posterized/noisy version:

[IMG width="400px" alt="Jpeg from Raw after entering "Edit" and how it looks even after exiting Edit mode (but no edits actually made)"]

https://imgur.com/s8Cjc7y

Jpeg from Raw after entering "Edit" and how it looks even after exiting Edit mode (but no edits actually made)

This happened simply by entering Edit mode, and to be clear I made no actual edits. After I exited Edit mode the picture continues to look like the bastardized version and when I export the photo to a jpeg it also looks bastardized. Fortunately the original .cr2 RAW file is not affected, but what the heck is going on? Photos/OSX is clearly able to create a clean jpeg from the RAW initially up until I actually click "Edit" then it all goes to hell.

The only thing I could think of was Apple's Camera RAW conversion software doesn't support my G7x Mark II as it's a fairly recent camera but I checked https://support.apple.com/en-us/HT205272 and it is supported and I re-installed the very latest version from their site again just to be sure. Furthermore, I tried opening it in Aperture and the same exact thing happened in edit mode.

Can Photos edit RAW files at all? Or do I need to buy Lightroom now or something? I'm completely new to RAW editing so I don't know if I'm doing something wrong or if I'm using the wrong software for the job. Can anyone with Photos try opening the .cr2 RAW file on your mac to see if it does the same thing?

 

[Iliah Borg]

There was very little light; embedded JPEG has noise reduction applied, rather heavy-handed. The raw itself contains quite a lot of noise. You may try applying noise reduction in Photos; or try free Canon DPP converter, the results will be at least not worse than the embedded JPEG. But I would be thinking of using more light in the shot, faces need boosting about 4 stops.

 

[LouCioccio]

Refer to pg 79 camera manual. Check the metadata of the RAW or the JPEG to see which metering mode. The far background skewed the reading hence the under exposure. You should be able to get it back with a RAW converter even LR, ACR or Canon's own.

It look like no flash so I would have moved the light meter mode to spot.

Press the Q set button and select spot metering [•].

Lou Cioccio

 

[truth1ness]

The photo is extremely dark but this doesn't seem to explain why these crazy artifacts are happening. First of all, as I said in my original post when I output to JPEG before entering edit mode it produces a perfectly fine JPEG that you see above. Doesn't that infer the actual RAW>JPEG converter in Photos/Aperture itself is not the culprit?

You can try this yourself, put the RAW cr2 in Photos then drag it to your desktop to output a JPEG and you should get a pretty decent RAW conversion. But do that after you press Edit and it gets destroyed. Something beyond a simple RAW>JPEG conversion seems to be happening after pressing Edit otherwise the output would look like the first picture. I'm curious to see if this happens for you because that will at least help me determine if this is a glitch specific to my computer or not.

What I also don't understand is why would the application convert the RAW file to a JPEG before I finish editing it? Isn't the whole point of RAW that you edit it first in RAW and only when it's time to export you actually output a jpeg? I understand Photos is a consumer level application but Aperture was/still is used by professionals/enthusiasts. I'm having trouble believing this is how a formerly industry leading image editing program works. Are you guys saying when I edit a RAW file in Photos or Aperture I am actually editing a JPEG or am I misunderstanding something?

 

[truth1ness]

Also I am aware this shot is way too dark, I chose it to illustrate the artifact issue, but I think that is a separate unrelated issue. There are RAW photos taken with extremely dark areas taken all the time but we don't see a rash of JPEGs with crazy green posterized noise bands all around it.

As another point of reference, I just opened the .cr2 file in Preview and exported it as a JPEG and it also came out fine just like Photos before entering Edit mode.

 

[Saint 112]

I can't help you with Photos since I avoid using Apple's softwares altogether. So I'll give you some general insights.

On that extremely underexposed shot it did a heavy handed recovery of the dark parts. You must understand that with such a dark shot Photos is trying to make a "decent" picture and it retrieves whatever shades it finds in the shadows and the blacks. How smart the job is is another story.

I use DxO Optics Pro for my RAW treatment but, even though it does probably a smarter job, when I shoot low light, indoor events like concerts I disable the Smart Light tool because of the similar issue as yours. It tends also to go overboard and even black parts, that should remain black, become dark, dirty grey, sometimes with strange shades of colors and get a lot of noise. In these cases I use only the Selective Tone tools (which has 4 sliders). I just correct Shadows and leave the Blacks slider alone so blacks remain black.

Photos probably has Selective Tone corrections tools with Highlights and Blacks and you would probably get better results by doing manual corrections. Yet you shouldn't expect miracles because your shot is hardly recoverable altogether.

Nick

 

[truth1ness]

Ok big update, my suspicions in my last two posts were correct. This is not a RAW>JPEG issue nor a dark picture issue. I gained access to another Mac and loaded the picture into both Photos and Aperture, went in and out of Edit mode and exported and the picture looked normal every step of the way, no crazy artifacts at all (I am assuming you did not get a terribly distorted image like mine if you tried this either?). So something is wrong with my mac it seems and some system wide thing that Photos and Aperture both use to display RAW files. Even stranger, the distorted version seems to propogate to by iPhone, but not on the new mac I just synced. Unfortunately I'm completely lost on what is making my particular Mac do this and what the next steps are to remedy this.

 

[Iliah Borg]

This is not a RAW>JPEG issue nor a dark picture issue

Well, raw data is extremely noisy and lacks shadow details, here is the green channel from raw:



--

About LibRaw | LibRaw

www.libraw.org

 

[noirdesir]

This is such a classic, it should be a sticky in this forum:

What you see in the first photo is the raw conversion undertaken by the camera and saved as (full-size) JPEG preview inside the raw file. The second version is the result of raw conversion done by Photos. Initially, Photos is using the embedded JPEG because it is faster to read that than to read the raw data (multiples of the amount of data for the latter) and requires no raw conversion processing (again saves time). One would think that this would be restricted to only the first few seconds one is viewing an image but apparently Photos keeps using the embedded JPEG for a couple of other simple tasks as well.

As to why the raw conversion by Photos results in something worse than the raw conversion in the camera, Iliah already tried to answer that: Your camera is by default simply applying more/better noise reduction. That might also include a more pleasing tone curve correction with less colour artefacts.

The easiest way to show when and where the embedded JPEG is used instead of the result of a raw conversion is to set your camera to B&W. That doesn't affect the raw data but makes it perfectly clear when the embedded JPEG is used: You'll see a B&W image and when the raw data are used: You'll see a colour image.

 

[Eric Nepean]

Simple answer - don't use Apple SW for photography. Their tools are not very good anymore, since they have started to focus their development entirely on the snapshot crowd and have dropped the professional/serious hobbyist market.

 

[truth1ness]

I don't mean to be argumentative but I think the evidence speaks for itself in the two images I'm going to present below that this is not an issue of "the picture is too dark" or "apple's Raw>Jpeg engine is not good". I've controlled for both of those variables in these pictures and the result is completely different. These are the same photo in the same program with the same exposure increase using the same digital camera raw version with the only difference them being done on two different Macs. Something is not adding up. The worse one is on a macbook pro 2010 and the other a mac mini I think 2009. Could this be due to a hardware/video card difference?

Surely you can see the difference here, the distortion in my Macbook Photos edited pic is 100x worse than any Apple created RAW>JPEG converter could ever do (which I've shown does a fine job in Preview on my macbook and Photos/Aperture on a second mac mini) and I've shown that that terrible distortion is not happening on a different mac and in other Apple programs like Preview when doing the same computer based RAW>JPEG.

A non-ruined jpeg of the same RAW file editing in the same Photos program on a different mac:

Compare that to this using the same exact settings on my mac in Photos:

 

[noirdesir]

Interesting, I had no idea the Raw files have a jpeg inside of them especially since I'm shooting in RAW only, not RAW+JPEG. I thought RAW is just RAW unless RAW+JPEG is on, is that incorrect?

Almost all raw files have a JPEG version inside. Many years ago this might have been a smaller resolution version but for a long time now it's usually a full-size version, just with a higher compression (aka lower quality). It's what you see when you flip through images on your camera's back display. Doing a real-time conversion from raw (including reading the full raw data) as you jump from image to image would be way too slow. Even on a real computer, using that embedded jpeg is noticeably faster and is thus used in a number of situations.

The question, 'When I view an image in Aperture it changes looks after a few seconds' is a classic and exactly due to Aperture initially using the embedded JPEG and then after it had time to read the full raw data and do the conversion it flips to using the image based on the raw conversion instead of the JPEG.

Really, if you have any doubts about this: Take your camera, set it to B&W, take a shot, and import the raw file into Photos (or view it with Preview). If you see a B&W version, you are seeing the embedded JPEG and not an image based on the raw conversion done by Photos.

I'm not sure if you read my last update post but I tried this photo on another Mac and it did not create these horrible artifacts. I was able to put it in Photos, crank up the exposure, and it came out at least usable, not like the completely bastardized version in my first post.

I didn't see the difference properly in your original post (the image was too dark and small). But what I see now, is some sort of corruption. On my system, when I import the photo into Photos, do some editing to brighten it, and export it as a JPEG, I don't get any of the posterisation (as you didn't get on your other Mac), meaning it is something that is specific to your Mac that is causing this corruption. It could be that some internal data on this image are corrupted and deleting the image an re-importing it could fix that. If that doesn't do it, do some basic troubleshooting like restarting Photos and your computer.

 

[Iliah Borg]

I don't mean to be argumentative but I think the evidence speaks for itself in the two images I'm going to present below that this is not an issue of "the picture is too dark" or "apple's Raw>Jpeg engine is not good". I've controlled for both of those variables in these pictures and the result is completely different. These are the same photo in the same program with the same exposure increase using the same digital camera raw version with the only difference them being done on two different Macs. Something is not adding up. The worse one is on a macbook pro 2010 and the other a mac mini I think 2009. Could this be due to a hardware/video card difference?

Hardly it is hardware difference, or there would be thousands of complaints.

The second, posterized version, looks like it is resulting from a low quality JPEG, not from raw. Maybe somehow your workflow wasn't the same on those two computers?

Surely you can see the difference here, the distortion in my Macbook Photos edited pic is 100x worse than any Apple created RAW>JPEG converter could ever do (which I've shown does a fine job in Preview on my macbook and Photos/Aperture on a second mac mini) and I've shown that that terrible distortion is not happening on a different mac and in other Apple programs like Preview when doing the same computer based RAW>JPEG.

A non-ruined jpeg of the same RAW file editing in the same Photos program on a different mac:

Compare that to this using the same exact settings on my mac in Photos:

--

About LibRaw | LibRaw

www.libraw.org

 

[Jim in Hudson]

I don't mean to be argumentative but I think the evidence speaks for itself in the two images I'm going to present below that this is not an issue of "the picture is too dark" or "apple's Raw>Jpeg engine is not good". I've controlled for both of those variables in these pictures and the result is completely different. These are the same photo in the same program with the same exposure increase using the same digital camera raw version with the only difference them being done on two different Macs. Something is not adding up. The worse one is on a macbook pro 2010 and the other a mac mini I think 2009. Could this be due to a hardware/video card difference?

Surely you can see the difference here, the distortion in my Macbook Photos edited pic is 100x worse than any Apple created RAW>JPEG converter could ever do (which I've shown does a fine job in Preview on my macbook and Photos/Aperture on a second mac mini) and I've shown that that terrible distortion is not happening on a different mac and in other Apple programs like Preview when doing the same computer based RAW>JPEG.

A non-ruined jpeg of the same RAW file editing in the same Photos program on a different mac:

Compare that to this using the same exact settings on my mac in Photos:

It seems there's some kind of conversion setting on your own Mac that is way off. A few questions come to mind:

Is the JPEG output quality and file size the same between the two?

Are you running the same version of Mac OS X?

Is a custom setting for conversion being used on your Mac? I ask because it appears you have the Shadows pushed to something close to 100%.

 

[graybalanced]

The only thing I could think of was Apple's Camera RAW conversion software doesn't support my G7x Mark II as it's a fairly recent camera but I checked

The symptom of an unsupported camera is that you can't open the raw files at all. The camera is supported, it is only a question of how well it is supported.

The posterized example of the raw conversion almost looks like what you get if you overcorrect the shadows in a JPEG. Almost looks like clipping combined with insufficient color noise reduction . It makes me wonder if Photos is actually editing the JPEG instead of the raw, or if the raw conversion is just flawed for this camera in Photos/Aperture.

Another side of this is that if you made no edits, you're only viewing the default raw conversion, and no one sticks with the default, otherwise there's no point in having a raw. Where I am going with this is that it is worth revealing the advanced editing controls in Photos and seeing if you can improve quality that way (click Adjust, then Add). This will give you access to sliders like Noise Reduction, White Balance, and Levels. If you are only using the top-level (basic) editing controls in Photos, you're missing out on a lot!

(Expanding the advanced controls is the only way I can stand to use Photos. I do all raw editing in Lightroom.)

 

[noirdesir]

The only thing I could think of was Apple's Camera RAW conversion software doesn't support my G7x Mark II as it's a fairly recent camera but I checked

The symptom of an unsupported camera is that you can't open the raw files at all. The camera is supported, it is only a question of how well it is supported.

The posterized example of the raw conversion almost looks like what you get if you overcorrect the shadows in a JPEG. Almost looks like clipping combined with insufficient color noise reduction . It makes me wonder if Photos is actually editing the JPEG instead of the raw, or if the raw conversion is just flawed for this camera in Photos/Aperture.

Posterisation generally means insufficient bit depth. Pushing a JPEG file (by definition only being 8-bit) significantly will result in something like this. But this looks even worse, even the 'unpushed' image in the original post has already some posterisation and colour artefacts. It could be that for some reason Photos was using the embedded JPEG as the basis for the editing (given that the embedded JPEG usually has a pretty high compression, adjusting that embedded JPEG will more easily show artefacts). But it is equally possible that Photos was erroneously using an internal thumbnail/preview copy as the basis for the adjustment. Whatever the reason, it is because something went wrong with the OP's Photos Library, copy of the Photos installation or the OS itself as the problem manifests itself only on one particular Mac and one particular Photos library (and not, eg, on my Mac using the same raw file).

 

[Bruce McL]

“RAW files getting ruined by Apple Photos”

“Fortunately the original .cr2 RAW file is not affected”

I think you should change the headline of this thread - or change your user name.

There is nothing even close to truthful in the headline.

On the plus side, you can use your writing skills to get a good job writing headlines for petapixel.

 

[truth1ness]

I'm not sure if you read my last update post but I tried this photo on another Mac and it did not create these horrible artifacts. I was able to put it in Photos, crank up the exposure, and it came out at least usable, not like the completely bastardized version in my first post.

I didn't see the difference properly in your original post (the image was too dark and small). But what I see now, is some sort of corruption. On my system, when I import the photo into Photos, do some editing to brighten it, and export it as a JPEG, I don't get any of the posterisation (as you didn't get on your other Mac), meaning it is something that is specific to your Mac that is causing this corruption. It could be that some internal data on this image are corrupted and deleting the image an re-importing it could fix that. If that doesn't do it, do some basic troubleshooting like restarting Photos and your computer.

Thank you for verifying this. I have deleted and re-imported the image dozens of times, re-installed apples digital camera raw, rebooted, and nothing is working so far. Even when I load the image into my Mac Mini which does not create the distortion then sync it to my Macbook through iCloud the distortion then reappears. This is really puzzling me. That's why I'm going so far as to suspect even a hardware graphics card issue.

Here's another thing. Watching closely I noticed the minor change in the image that you are describing when Photos processes its own jpeg instead of the default. I notice this happen on both Macs. However, on the Macbook (the one with the problem) it then does a second shift a few seconds later to the bastardized one, so it is as if Photos is slapping on another layer of terrible processing which I see no way to control. The thing is, sometimes this happens almost instantly and other times it doesn't happen for about 10 seconds. I have software that can switch between the Integrated and Discrete video card on my macbook and I notice when discrete is on it takes towards the longer end to happen whereas on Integrated it happens almost instantly. I don't know what this means but perhaps something video card related is going on?

 

[Jim in Hudson]

I'm not sure if you read my last update post but I tried this photo on another Mac and it did not create these horrible artifacts. I was able to put it in Photos, crank up the exposure, and it came out at least usable, not like the completely bastardized version in my first post.

I didn't see the difference properly in your original post (the image was too dark and small). But what I see now, is some sort of corruption. On my system, when I import the photo into Photos, do some editing to brighten it, and export it as a JPEG, I don't get any of the posterisation (as you didn't get on your other Mac), meaning it is something that is specific to your Mac that is causing this corruption. It could be that some internal data on this image are corrupted and deleting the image an re-importing it could fix that. If that doesn't do it, do some basic troubleshooting like restarting Photos and your computer.

Thank you for verifying this. I have deleted and re-imported the image dozens of times, re-installed apples digital camera raw, rebooted, and nothing is working so far. Even when I load the image into my Mac Mini which does not create the distortion then sync it to my Macbook through iCloud the distortion then reappears. This is really puzzling me. That's why I'm going so far as to suspect even a hardware graphics card issue.

Here's another thing. Watching closely I noticed the minor change in the image that you are describing when Photos processes its own jpeg instead of the default. I notice this happen on both Macs. However, on the Macbook (the one with the problem) it then does a second shift a few seconds later to the bastardized one, so it is as if Photos is slapping on another layer of terrible processing which I see no way to control. The thing is, sometimes this happens almost instantly and other times it doesn't happen for about 10 seconds. I have software that can switch between the Integrated and Discrete video card on my macbook and I notice when discrete is on it takes towards the longer end to happen whereas on Integrated it happens almost instantly. I don't know what this means but perhaps something video card related is going on?

This certainly seems like a setting or profile in OS X that is causing the problem. Have you tried running Color Sync Profile and doing a repair?

Also, from Photos, when you do a "get info" (the "i" button), what file size is reported for the posterized image? Lastly, when you hit "edit" and the problem occurs, what happens when you hit the "restore original" button next to the "edit" button?

 

[LouCioccio]

Have you run AHT Apple Hardware Test. If anyone uses AHT on laptop it needs to be plugged in an outlet, others like mac mini and iMac's are plugged in an outlet.

This may show a graphics problem but not an intermittent one unless it catches it.

Lou Cioccio