What is Source-Based Tone Mapping?
Source-Based Tone Mapping (SBTM) is a new technology developed by the HDMI Forum to solve a specific problem. The problem is an inconsistent viewing experience when the source device (set-top box, gaming console, PC, etc.) has some combination of HDR, SDR, dynamic HDR and graphics overlays on the same screen simultaneously. The result can be an image that doesn’t look right with overly bright or dark areas and distorted colors.
HDMI mentions an example which is a menu of video thumbnails from a streaming service provider. Some of the thumbnails might be HDR, and others might be SDR, and the menu is rendered using graphics. Another example is a game with a mixture of HDR game action, SDR video, and graphics overlays.
While HDR displays can do their own tone mapping internally to adjust the incoming luminance and color range to the luminance and color capabilities of the display device, they cannot correctly perform tone mapping when presented with such a mixed-format image. The source must do such optimization, so SBTM is needed.
SBTM does not replace existing HDR technologies like HDR10 or HLG. It works in conjunction with these technologies.
According to a written response from the HDMI Forum, SBTM can help in the mixed situation and the HDR-on-HDR case. In those cases, the source must perform the compositing operation in an HDR color volume that it selects. Metadata can’t help in these situations because the source does not have and cannot generate metadata for composite pictures. Without SBTM, the source’s selected color volume might not be optimal because it doesn’t represent the display’s characteristics. If it does, the display might still reserve some of its capability to render signals beyond its capacity.
We also asked the HDMI Forum to confirm the step-by-step process.
- Source device reads TV EDID data
The EDID contains a color volume specification that the Sink is expected to render accurately.
- Source evaluates current content to determine class of content (SDR, HDR, dynamic HDR, graphic overlay, etc.). How does it do this?
Streams are coded as HDR or SDR. Static or dynamic HDR metadata (if present) is also encoded within a content stream. Graphic overlays can be encoded differently, but the dynamic range is generally known to the application.
- Based on class of content, it sets a flag in the HDMI stream to signal the TV about the type of content.
Yes, for each frame, the Source signals to the Sink device that the source is tone mapping the content to the Sink’s advertised color volume (i.e., the one specified in the EDID).
- Based on the class of content and the capabilities of the TV, tone mapping is performed by the source device.
Correct. In SBTM mode, the TV expects to receive HDR signals in a color volume of its choosing. The source utilizes the metadata to perform the tone mapping operation BEFORE the compositing operation occurs. This tone mapping can be from an SDR, HDR, HDR10, or dynamic HDR content source. The TV specifies the color volume the source uses for compositing, and the TV knows not to expect signals outside of that color volume. Note that no additional static or dynamic metadata is transmitted to the TV when SBTM is used since the tone mapping in the source already used the metadata.
TV/display makers and content creators would have concerns about a source device altering the video because they feel their TVs will do a better job. Would filmmaker mode override SBTM?
Tone mapping performance can vary from Source to Source. However, a Source product’s performance can easily be compared to another by providing the same EDID to two Sources and comparing how they look. There is a controlled way of doing this defined inside the spec. Content providers, display makers, or industry specifications (such as Dolby Vision or HDR10+) could eventually impose their additional constraints on how content is tone mapped. UHDA defines “Filmmaker Mode”, so it would be up to them to specify any behavioral rules that are needed. Even though the Source tone maps to a color volume of the display’s choosing, in some situations, the display might perform its additional tone mapping due to ambient light levels, power constraints, etc.
I understand SBTM can be implemented in HDMI 2.1 interfaces by a firmware update but doesn’t tone mapping require processing and memory, which are not part of the HDMI chipset?
Source devices in the market today generally need to perform tone mapping to a color volume of their choosing when they support mixed SDR/HDR content. Sometimes this color volume can be adjusted via a user control or slider. If the chipset allows this color volume to be sufficiently programmable, a Source device might be firmware-upgraded to read the EDID and program that color volume appropriately. Similarly, a TV or display might need to add a small number of EDID bytes and adjust its internal tone mapping when the Extended Metadata Packet is received to not expect any colors outside the EDID-specified color volume.
How is SBTM useful in gaming?
SBTM also helps enable “plug and play” HDR gaming without the need for calibration screens. Because the gaming device knows the TV’s preferred color volume, the gaming engine can perform appropriate tone mapping to ensure that every gamer can see the essential graphical elements. In addition, none of the TV’s capability is wasted since none of its dynamic range needs to be reserved for pixel values outside the specified color volume.
SBTM will be available with the release of HDMI 2.1a in Q1 2022.
