current state of producing photos for HDR display?

Started Oct 1, 2022 | Discussions
Brian Kimball
Brian Kimball Contributing Member • Posts: 759
current state of producing photos for HDR display?
1

As we all know, HDR displays are generally capable of much brighter peak output than traditional standard dynamic range, or SDR, displays. A few years ago Richard Butler wrote about the exciting potential of these displays right here on DPR.

So how do we take advantage of this display technology in our photo editing? How do we create a final output file that "turns on", so to speak, HDR display of that photo? Assuming of course one is using a viewer that supports such files and a display capable of displaying them.

Please note I'm not asking about HDR capture. I'm very familiar with blending bracketed exposures, but even that still produces final images that are displayed in SDR.

While searching the web I see a lot of discussion about video color spaces (P3, bt.2020, etc) with regard to HDR displays, but that seems to be a distraction. For years we've been able to produce files in wide gamut color spaces like AdobeRGB, ProPhotoRGB, etc, and yet they still only display as "SDR" photos on HDR displays.

So what else is needed?

Is there a mainstream photographic editing tool made today that allows you to edit for display on HDR devices?

For reference, iPhones have been taking and displaying HDR-capable photos for a few years now (again, not HDR multi-shot capture, which they also have, but single-shot HDR display). They display properly in HDR on the iPhone itself in the Photos app, and on HDR monitors like the new Macbook Pros and Apple's Pro Display XDR, again using the Photos app.

But interestingly, those same photos brought into Lightroom or other macOS image viewers like Preview do not display as HDR, even when viewed on an HDR screen. So what key metadata is turning HDR on in Photos, and how do we edit with an actual HDR screen as our viewing target?

This is not limited to just Apple.  Spider-mario has produced some special .avif files here that Google Chrome can open and display properly on an HDR monitor.  But they were made in Davinci Resolve, a professional video editor that has no concept of raw still photo editing.

Thanks for any insight and guidance anyone can provide.

Bikeridr Forum Member • Posts: 76
Re: current state of producing photos for HDR display?

Maybe I'm far out fishing here, but..

What container are you using? JPEG can only display 8-bit colour images.
Have you tried saving in HEIF (10 bit) or TIFF 16 bit? Or even the now obsolete flop JPEG2000? Those formats should be able to be displayed properly on an HDR monitor in higher than 8 bit.

-- hide signature --

Bikeridr - Happy amateur photographer

 Bikeridr's gear list:Bikeridr's gear list
Canon EOS 5D Mark II Canon EOS M100 Canon EOS R5 Canon EF 50mm F1.4 USM Canon EF 400mm f/5.6L USM +9 more
Brian Kimball
OP Brian Kimball Contributing Member • Posts: 759
Re: current state of producing photos for HDR display?

Bikeridr wrote:

Maybe I'm far out fishing here, but..

What container are you using? JPEG can only display 8-bit colour images.
Have you tried saving in HEIF (10 bit) or TIFF 16 bit? Or even the now obsolete flop JPEG2000? Those formats should be able to be displayed properly on an HDR monitor in higher than 8 bit.

So it turns out high bit depth is not a requirement for HDR display of HDR-capable photos.

HDR display capable photos taken on an iPhone and stored in a HEIF container are, surprisingly, 8-bit, even though the container format allows for up to 16 bits (not just 10).

There's something else in the file that triggers HDR display, and I don't know what it is.

Looking at the wikipedia page for the HEIF format, I see in the Hardware section:

So it seems the PQ & HLG tone curves are the key points here.  Now how in the world do we incorporate them into photos?

spider-mario
spider-mario Senior Member • Posts: 1,143
Re: current state of producing photos for HDR display?
5

Brian Kimball wrote:

So it seems the PQ & HLG tone curves are the key points here. Now how in the world do we incorporate them into photos?

Indeed, they are.

Until recently (see below), ICC profiles were unable to signal the use of PQ or HLG, because the way that ICC color management fundamentally works assumes that output light scales linearly with “media white”. But that is not how PQ and HLG work.

PQ works by representing absolute display light in the 0-10000 cd/m² range. So, regardless of the output display’s peak luminance, the RGB triplet (1, 1, 1) always represents 10 000 cd/m², (0.5081, 0.5081, 0.5081) always represents 100 cd/m², and so on. In a typical PQ workflow, you would adjust the image to look good on the mastering display that you are using, and then encode the absolute light that comes out of it as a PQ signal.

HLG is slightly closer to (but still different from) ICC-based color management in that it is relative instead of absolute like PQ, but unlike in ICC, the way that output light scales with the peak luminance of the display is non-linear. Instead, the HLG signal is interpreted as representing relative “scene light” to which a further transform (the “opto-optical transfer function” or “OOTF”) is supposed to be applied to produce display light. For HLG, that transform takes the form of a gamma curve applied to the luminance channel, with the value of the exponent varying according to the peak display luminance and the surround luminance. This video by an author of HLG explains it very well.

Traditionally, the focus of HLG has been on live broadcast (it was, after all, developed by the BBC and NHK), and therefore, one way to produce an HLG signal is to encode the linear camera output directly to HLG (with the convention being to place diffuse white ~2 stops below the maximum) and count on the OOTF to make it look subjectively similar regardless of the viewing device/environment. But it is also possible to adopt a workflow closer to that associated with PQ, in which you make the image look good on a given device, then apply the appropriate inverse OOTF to what is displayed on-screen (to go “back” to scene light) and encode the result to HLG.

Now, back to the question of how to signal the use of PQ or HLG in an image file. The solution adopted by AVIF comes from the video world and consists in transmitting four numbers (“coding-independent code points” or “CICP”) indicating which color primaries, transfer function, matrix coefficients and range (“full” vs. “limited” or “narrow”) are used. The values that those numbers can take are described in H.273. PQ is transfer function 16 and HLG is 18. The Rec. 2020 primaries are 9, and storing the signal as RGB without transforming it to Y′CbCr or similar is with “matrix coefficients” = 0 (identity matrix). So, full-range RGB Rec. 2020 + PQ would be represented by CICP values 9/16/0/1.

Support for transmitting CICP values has recently been added to the PNG specification and current versions of Chrome support the new chunk. And while JPEG does not support CICP directly, ICC v4.4 has also added support for a CICP tag, and the current development version (108) of Chrome can read that tag.

This version, ICC.1:2022.is an update to ICC.1:2010. The main technical changes to ICC.1:2010 are:

  • cicpTag has been added to enable HDR metadata to be carried in the profile

I hope that my comment was at least somewhat clear. 😁

 spider-mario's gear list:spider-mario's gear list
Canon G1 X III Olympus TG-6 Olympus E-M1 II Olympus E-M1 III Canon EOS R6 +23 more
robgendreau Forum Pro • Posts: 11,693
Re: current state of producing photos for HDR display?
1

spider-mario wrote:

Brian Kimball wrote:

So it seems the PQ & HLG tone curves are the key points here. Now how in the world do we incorporate them into photos?

Indeed, they are.

Until recently (see below), ICC profiles were unable to signal the use of PQ or HLG, because the way that ICC color management fundamentally works assumes that output light scales linearly with “media white”. But that is not how PQ and HLG work.

PQ works by representing absolute display light in the 0-10000 cd/m² range. So, regardless of the output display’s peak luminance, the RGB triplet (1, 1, 1) always represents 10 000 cd/m², (0.5081, 0.5081, 0.5081) always represents 100 cd/m², and so on. In a typical PQ workflow, you would adjust the image to look good on the mastering display that you are using, and then encode the absolute light that comes out of it as a PQ signal.

HLG is slightly closer to (but still different from) ICC-based color management in that it is relative instead of absolute like PQ, but unlike in ICC, the way that output light scales with the peak luminance of the display is non-linear. Instead, the HLG signal is interpreted as representing relative “scene light” to which a further transform (the “opto-optical transfer function” or “OOTF”) is supposed to be applied to produce display light. For HLG, that transform takes the form of a gamma curve applied to the luminance channel, with the value of the exponent varying according to the peak display luminance and the surround luminance. This video by an author of HLG explains it very well.

Traditionally, the focus of HLG has been on live broadcast (it was, after all, developed by the BBC and NHK), and therefore, one way to produce an HLG signal is to encode the linear camera output directly to HLG (with the convention being to place diffuse white ~2 stops below the maximum) and count on the OOTF to make it look subjectively similar regardless of the viewing device/environment. But it is also possible to adopt a workflow closer to that associated with PQ, in which you make the image look good on a given device, then apply the appropriate inverse OOTF to what is displayed on-screen (to go “back” to scene light) and encode the result to HLG.

Now, back to the question of how to signal the use of PQ or HLG in an image file. The solution adopted by AVIF comes from the video world and consists in transmitting four numbers (“coding-independent code points” or “CICP”) indicating which color primaries, transfer function, matrix coefficients and range (“full” vs. “limited” or “narrow”) are used. The values that those numbers can take are described in H.273. PQ is transfer function 16 and HLG is 18. The Rec. 2020 primaries are 9, and storing the signal as RGB without transforming it to Y′CbCr or similar is with “matrix coefficients” = 0 (identity matrix). So, full-range RGB Rec. 2020 + PQ would be represented by CICP values 9/16/0/1.

Support for transmitting CICP values has recently been added to the PNG specification and current versions of Chrome support the new chunk. And while JPEG does not support CICP directly, ICC v4.4 has also added support for a CICP tag, and the current development version (108) of Chrome can read that tag.

This version, ICC.1:2022.is an update to ICC.1:2010. The main technical changes to ICC.1:2010 are:

  • cicpTag has been added to enable HDR metadata to be carried in the profile

I hope that my comment was at least somewhat clear. 😁

Very clear! Thanks for the explanation.

One of those scenarios where video is ahead of the stills curve.

I've viewed a few .exr HDR stills on my HDR capable display, and it is very nice. Sunsets are amazing If one is on macOS, here's a tool for comparison and viewing, tev:https://github.com/Tom94/tev/releases

 robgendreau's gear list:robgendreau's gear list
Pentax 645Z
Brian Kimball
OP Brian Kimball Contributing Member • Posts: 759
Re: current state of producing photos for HDR display?
1

Thank you, that was incredibly helpful in getting me oriented to this technology. You've given me a bunch of great jumping off points to learn more. Really appreciate it.

It seems like the video world has this more or less already figured out, but the stills world is floundering and very late to the party.

Here's a high-level summary of what I think I know about photos made for HDR display. It would be much appreciated if you could tell me if I'm still misunderstanding things:

Generally speaking, HDR-enabled still photos...

  • have no specific color space requirement
  • have no specific bit depth requirement
  • employ a special gamma encoding that either generates an absolute intended brightness of a pixel (PQ) or a brightness that is relative to whatever display and viewing environment is being used (HLG)
  • or alternatively are in a special 16 or 32-bit .EXR format which is linear
  • or alternatively are in even more obscure formats that are unlikely to be usable outside of very specialized viewers

Also..

  • PQ and HLG were both developed for HDR video first, many years ago. Only now is support for them starting to show up in bleeding-edge versions of JPG and PNG specs. Who knows when or if Apple/Google/Microsoft/Adobe will ever support HDR JPGs or PNGs.
  • .HEIF and .AVIF already support PQ & HLG, but software support for producing one's own photos in these filetypes is still quite limited
    • this may be the result of patent/royalty/political issues between large tech companies

So on a practical level, to produce a photo ready for HDR display today, one must...

  • shoot SOOC in HEIF format on a modern iPhone from the past 2-3 years. This will use HLG
  • shoot SOOC in HEIF format on a Canon 1DXmIII, R5, or R6. This will use PQ
  • possibly shoot SOOC in HEIF format on some Android phones. Unclear here.
  • shoot raw/jpg/whatever, edit in Affinity Photo, export as .EXR
  • shoot raw/jpg/whatever:
    1. with any camera
    2. edit & export in any editor
    3. open that file in Davinci Resolve
    4. perform magic
    5. end up with an HDR-enabled .AVIF file that is only viewable in... Chrome?

Any insight you can give with these last few points would be much appreciated! 🙂

And finally, for proper viewing of HDR-enabled photos, one must:

  • have an HDR display
  • view them in Apple Photos (if HEIF), or
  • view them in Google Chrome (if AVIF), or
  • somehow view them on an HDR TV, or
  • if .EXR, view them in in tev or some other compatible viewer or
  • view them in .... what else?

Thank you! 🙏

Edit: my ultimate goal, I should add, is to edit in Lightroom Classic, grade to HDR in whatever app allows it, and eventually end up with a HEIF file I can view in HDR on my Apple devices (using Apple Photos) and HDR TV (using Apple TV).

spider-mario
spider-mario Senior Member • Posts: 1,143
Re: current state of producing photos for HDR display?
2

Brian Kimball wrote:

Here's a high-level summary of what I think I know about photos made for HDR display. It would be much appreciated if you could tell me if I'm still misunderstanding things:

Generally speaking, HDR-enabled still photos...

  • have no specific color space requirement

Correct, they often use Rec. 2020 primaries but they don’t have to. In fact, even when encoded in Rec. 2020, HDR content is often restricted to the P3 gamut anyway as that’s what current mastering displays are capable of.

  • have no specific bit depth requirement

Right, although at least 10 bits, ideally 12, is usually recommended to avoid quantization artifacts, as illustrated in this graph from this thread.

  • employ a special gamma encoding that either generates an absolute intended brightness of a pixel (PQ) or a brightness that is relative to whatever display and viewing environment is being used (HLG)
  • or alternatively are in a special 16 or 32-bit .EXR format which is linear
  • or alternatively are in even more obscure formats that are unlikely to be usable outside of very specialized viewers

This all matches my understanding.

  • PQ and HLG were both developed for HDR video first, many years ago. Only now is support for them starting to show up in bleeding-edge versions of JPG and PNG specs. Who knows when or if Apple/Google/Microsoft/Adobe will ever support HDR JPGs or PNGs.

The current version of Chrome should already support HDR PNGs (such as this one), and Chrome 108, currently in “Canary” and due for release on 29 November 2022, should support HDR ICC profiles (such as the one in this JPEG).

  • .HEIF and .AVIF already support PQ & HLG, but software support for producing one's own photos in these filetypes is still quite limited> * this may be the result of patent/royalty/political issues between large tech companies

If one has a PQ or HLG image already, encoding it to AVIF can be done with free and open-source software. Assuming that “input.png” contains pixel data in Rec. 2020 PQ (not necessarily with any HDR signaling at all), the following command compresses it to AVIF:

$ avifenc --ignore-icc --cicp 9/16/9 -d 12 input.png output.avif [other compression options…]

For JPEG XL, it would be more convenient to start from a PPM file, and do:

$ cjxl -x color_space=RGB_D65_202_Rel_PeQ input.ppm output.jxl

For HEIF, I haven’t really found a satisfactory solution yet.

So on a practical level, to produce a photo ready for HDR display today, one must...

  • shoot SOOC in HEIF format on a modern iPhone from the past 2-3 years. This will use HLG
  • shoot SOOC in HEIF format on a Canon 1DXmIII, R5, or R6. This will use PQ
  • possibly shoot SOOC in HEIF format on some Android phones. Unclear here.
  • shoot raw/jpg/whatever, edit in Affinity Photo, export as .EXR

A note on the last point: having produced such an EXR file, the reference JPEG XL implementation contains a command-line tool that can be used to convert such OpenEXR files to PQ. (It also includes tools for conversion between PQ and HLG.)

  • shoot raw/jpg/whatever:> 1. with any camera
    1. edit & export in any editor
    2. open that file in Davinci Resolve
    3. perform magic
    4. end up with an HDR-enabled .AVIF file that is only viewable in... Chrome?

What DaVinci Resolve gives me, more specifically, is a 16-bit TIFF file in the colorspace of my choosing. From there, it is up to me to compress it to AVIF or something else, with the appropriate metadata.

And finally, for proper viewing of HDR-enabled photos, one must:

  • have an HDR display
  • view them in Apple Photos (if HEIF), or
  • view them in Google Chrome (if AVIF), or
  • somehow view them on an HDR TV, or

The only solution I have found to view them on my TV is to encode them as a video. FFmpeg can do it, e.g. this creates a 30-second video of just one image, albeit without tone-mapping metadata:

$ ffmpeg -loop 1 -i input.ppm -t 30 -c:v libx265 -crf 12 -pix_fmt yuv420p10le -color_primaries bt2020 -color_trc smpte2084 -colorspace bt2020nc output.mkv

  • if .EXR, view them in in tev or some other compatible viewer or
  • view them in .... what else?

Personally, tev is indeed my EXR viewer of choice. Sadly, I don’t know of many options other than Chrome and tev to view HDR stills at the moment.

 spider-mario's gear list:spider-mario's gear list
Canon G1 X III Olympus TG-6 Olympus E-M1 II Olympus E-M1 III Canon EOS R6 +23 more
Brian Kimball
OP Brian Kimball Contributing Member • Posts: 759
Re: current state of producing photos for HDR display?
1

spider-mario wrote:

[...]

Assuming that “input.png” contains pixel data in Rec. 2020 PQ (not necessarily with any HDR signaling at all), the following command compresses it to AVIF:

$ avifenc --ignore-icc --cicp 9/16/9 -d 12 input.png output.avif [other compression options…]

[...]

For JPEG XL, it would be more convenient to start from a PPM file, and do:

$ cjxl -x color_space=RGB_D65_202_Rel_PeQ input.ppm output.jxl

[...]

The only solution I have found to view them on my TV is to encode them as a video. FFmpeg can do it, e.g. this creates a 30-second video of just one image, albeit without tone-mapping metadata:

$ ffmpeg -loop 1 -i input.ppm -t 30 -c:v libx265 -crf 12 -pix_fmt yuv420p10le -color_primaries bt2020 -color_trc smpte2084 -colorspace bt2020nc output.mkv

Thank you so, so much! You've been a goldmine of useful starter info. _REALLY_ appreciate it. If I have any luck I'll be sure to report back with samples.

And speaking of HDR TVs, I can confirm the Apple Photos app on Apple TV fails to show HDR photos properly.  That's a real bummer.

Finally, I can perhaps offer something interesting: JPGs exported by either Apple Photos or by Apple Preview from an Apple-produced HLG HEIC are surprisingly still HDR.  I don't believe they're following the new JPG/ICC specs you've described since they aren't displaying as HDR in Chrome Dev.  See samples here, including Lightroom's non-HDR export from the source HEIC:

https://www.dropbox.com/sh/o3noy2b1myxawfx/AAC7XZBzU96e8VXcSWBzLPuqa?dl=0

robgendreau Forum Pro • Posts: 11,693
Re: current state of producing photos for HDR display?
1

Brian Kimball wrote:

Thank you so, so much! You've been a goldmine of useful starter info. _REALLY_ appreciate it. If I have any luck I'll be sure to report back with samples.

And speaking of HDR TVs, I can confirm the Apple Photos app on Apple TV fails to show HDR photos properly. That's a real bummer.

Finally, I can perhaps offer something interesting: JPGs exported by either Apple Photos or by Apple Preview from an Apple-produced HLG HEIC are surprisingly still HDR. I don't believe they're following the new JPG/ICC specs you've described since they aren't displaying as HDR in Chrome Dev. See samples here, including Lightroom's non-HDR export from the source HEIC:

https://www.dropbox.com/sh/o3noy2b1myxawfx/AAC7XZBzU96e8VXcSWBzLPuqa?dl=0

I think Preview can show HDR properly, in reference to your earlier comment. I also use tev, which I linked. It has a GUI btw.

And thanks for the summary. I think a lot of people don't realize that there are quite a lot of HDR capable displays in people's hands, even if not in TVs, out there. Seems rather odd that will all the fuss over "low light shooting" and DR specs in cameras and whatnot that this very obvious improvement in stills isn't implemented better in the stills world. But it took them quite a while to get wider acceptance of LUTs in still processing software as well.

And again, good you brought this up. DPR should do another article about it.

 robgendreau's gear list:robgendreau's gear list
Pentax 645Z
pippo27 Contributing Member • Posts: 531
Re: current state of producing photos for HDR display?

Hi,

Is there a mainstream photographic editing tool made today that allows you to edit for display on HDR devices?

Not quite mainstream (actually far from it!), but recent versions of ART can do this if properly configured. Here is an SDR jpg and the "corresponding" HDR (with PQ transfer function) AVIF file with peak at 1000 nits. They were obtained from this raw file, and they should ideally look similar on a proper HDR display with a proper HDR viewer. I say should because I do not have an HDR display myself, so I'm not really sure it that actually works as intended

But it would be great if someone could test and confirm or correct my claim. Also if said someone is interested in the workflow, (s)he is welcome to send me a pm for more details.

HTH

Brian Kimball
OP Brian Kimball Contributing Member • Posts: 759
ACR 15 Adds High Dynamic Range Output
1

well well well

https://gregbenzphotography.com/photography-tips/acr-15-adds-high-dynamic-range-output/

It's a "technology preview" and it's currently limited to macOS only.  Looking forward to trying it.  Unclear if LrC will also have this preview.

MonstieurVoid New Member • Posts: 16
Re: current state of producing photos for HDR display?
1

iOS 16 uses HEIF wide gamut 8-bit gamma + proprietary EXIF data to render HDR in the Camera / Photos app. It’s not compatible with any non-Apple device. There is no support for HEIF HLG.

Ellis Vener
Ellis Vener Forum Pro • Posts: 20,172
Processing photos for HDR display?
1

You might find the following tutorial helpful

https://photoshopcafe.com/new-hdr-features-in-photoshop-2023/

-- hide signature --

Ellis Vener
To see my work, please visit http://www.ellisvener.com
I am on Instagram @EllisVenerStudio
“It's not about the f-stop." -Jay Maisel

 Ellis Vener's gear list:Ellis Vener's gear list
Nikon Z6 Nikon Z7 II Sigma 70-200mm F2.8 EX DG OS HSM Nikon Z 24-70mm F4 Zeiss Batis 40mm F2 CF +4 more
Keyboard shortcuts:
FForum MMy threads