Fewer words more pictures.
For capturing a scene's visual information into a JPEG by controlling jpeg creation, which is what this thread is about, it is better to use a less contrasty scene transfer curve. Here is why:
Above you see a screen capture of a test image shadow area encoded by the Nikon standard profile on the left side and the flat profile that the newer Nikons have built in them on the right. The shadow area of the standard profile was pulled up to match approximately the luminance of the area from the flat profile. If you look at the boulders one can see that the standard profile does not contain much shadow detail, but the shadow area encoded by the flat profile does in fact contain a lot on shadow detail.
Nikon explains the flat profile:
graph
“With minimal artificial manipulation of color and brightness, Flat maintains a subject’s information as much as possible and faithfully reproduces it. As you can see in the graph, images or movies taken with Flat contain rich information extending from shadow to the medium brightness range. Also, from the medium to high brightness range, artificial processing is minimized and the subject’s information is maintained. Therefore, Flat is optimal for capturing raw materials to be used for post-shooting adjustment.”
So flat is useful for capturing the scene visual info into a jpeg, but will typically need some adjustments before final presentation. Fortunately there are many gradations of tonality so tones can be moved around.
Another option which seems to work quite well is to use the neutral profile, with or without the embedded automatic adaptive tonal control from Apical imaging (aka Active-D lighting) which can adaptively change the transfer curve over each section of the image.
Above you see crops of the same shadow area from a contrasty image. On the left is a crop of a JPEG which was encoded from the standard transfer curve, in the middle from the neutral curve with Active-D lighting on auto (auto Active D did a very little tone curve adjustment in this picture), and on the right you see a JPEG crop encoded using the flat profile. The shadows of the standard and neutral profiles were pulled up to match the luminance of the flat profile crop. The detail in the shadow areas with the neutral profile is quite good; and in addition, the neutral profile can create images which require very few adjustments whereas the flat profile image would need more manipulation to add more contrast.
Above you see the whole test image: standard profile jpeg encode on the upper left, flat profile upper right, lower left a neutral profile with a preset which moves it very close to the standard profile, lower right and unembellished neutral profile image.
I think the holy Grail for me would be to have the raw processor create a JPEG image which would typically be very close to the final output image, but less contrasty. The raw processor would use a preset to create this JPEG, which would be quite close to the final desired output image, but still have adequate tonality for editing. When ingesting this JPEG into digital asset management (DAM) software, another preset could make or transform the look of this JPEG to whatever look I desire, but even though this DAM preset may have very dark shadows, and thus appear at first glance like a JPEG which would have little tonality in the shadows, it is backed by a JPEG designed to maintain adequate tonality throughout the image. Thus I can edit the image for final output, and output to tiff or printing would have the same number of tonality levels as the original jpeg image.
Workflow 1:
RAW Sensor data > Preset (auto-adaptive) processing > visually rich highest quality jpeg storage > Option, DAM ingest viewed with a more contrasty preset > Output (if needed): manipulate tones, size, color palette sRGB, into jpeg, tiff, print, email.
The above workflow can have the raw processing happen in the camera, with final tweaks prior to output on a general purpose computer. In addition, things in the EXIF maker notes, like Adaptive D lighting, can be implemented.
Workflow 2:
RAW Sensor data > Raw File > Preset processing, manual-adaptive processing > Output (if needed): manipulate tones, size, color palette sRGB, jpeg, tiff, print, email.
Workflow 2 is probably how most folks roll. It uses an intel processor instead of the expeed and as most use a third party raw processor the instructions in the EXIF maker notes are ignored, the extra features are not available.
In short, to capture the scene visual info into a jpeg, the scene transfer curve determines what gets dropped or squished from the original data, so a less contrasty transfer curve insures less info is thrown away. This is true whenever a jpeg is created, but becomes especially important if the jpeg is the sole source of the scene.
BTW, reference 3 is misleading if you think of jpeg images as 8 bit images. Jpeg images are more like 11 bit images, as there are 64 coefficients which must be summed for the final output. Typically this summation adds 2-3 bits to the 8 bit coefficients.