Taming the curves of Smart TV
A story of color correcting the TV that started with adjusting some settings with remote, then escalated to doing vector multiplications by hand and ended up with small modifications to the firmware of the TV. Read on if you’re interested in color and embedded Android Linux.
Quick intro to color correction
It doesn’t take much to produce color from any screen now, but it takes a lot to produce accurate colors. What does the term accurate colors actually mean?
There is an assumption that all humans perceive the color in the same way (or at least we’re targeting an average person) and the first quantifiable study gave birth to XYZ color space or CIE 1931, the largest color space there is (for humans). It is based on the physiology of our eye and that we basically perceive color with three different types of sensors. Y in XYZ acts as luminance while Z is kind of a response to blue stimulation (S cone) and X is a mix of cone response curves. Interesting enough is that the perception is not linear for each color component and our cones do not perceive only one specific color.
All the other color spaces mostly are a subset within the average human eye capabilities. But the important difference is the medium which is used to create color: paper print and your average desk monitor have completely different capabilities and challenges in terms of color reproduction. That is why in 1996 HP and Microsoft came up with a standard for color reproduction for use in printing and monitors under average light conditions (home and office) that allowed us to see the same colors regardless of the device or medium used. This doesn’t come cheap because all the colors have to be reproducible on a lot of different mediums and that’s why sRGB is significantly smaller that CIE 1931.
So by the term accurate colors we usually mean accurate color reproduction of sRGB (unless you’re a specialist in the color field, in which case I don’t know what you’re doing here, hehe). This means that when you request the color RGB=255,0,0 you get the most red color in the sRGB color space, not the most red color your eye can see. So the result of proper color reproduction means that RED=(255,0,0), BLUE=(0,0,255), MAGENTA=(255,0,255) or any other color will look the same on your office monitor, on your phone and also on your fridge (in case it has a display, if no — just print a sticker).
Now that we get the basics of the task let’s get to the problem at hand.
The TV
Let’s see where we are and do the initial calibration.
As you can see the TV is quite capable of producing all sRGB colors. The problem comes when you want to use that color reproduction on all your devices connected to the TV (that shiny Xbox or PlayStation, or external Android TV/Apple TV box, right?).
Usually the job of the colorimeter ends with support from the OS for embedding the color profile information. Let’s take a look at some of the information embedded into that color profile file:
...
BEGIN_DATA_FORMAT
RGB_I RGB_R RGB_G RGB_B
END_DATA_FORMATNUMBER_OF_SETS 256
BEGIN_DATA
0.00000000 0.00000000 0.00000000 0.00000000
0.00392157 0.00432829 0.00270345 0.00134085
0.00784314 0.00866792 0.00543103 0.00269029
...
0.99215700 0.99317600 0.80856900 0.69093900
0.99607800 0.99710700 0.81207300 0.69394600
1.00000000 1.00000000 0.81557800 0.69695200
END_DATA
...
This basically is a normalized representation of the per-channel curves above, or a 3x1DLUT where LUT stands for Look Up Table. In simple terms it’s just mapper from input value to the output. If we apply this transformation we get close to the target sRGB color space representation.
Unfortunately this isn’t totally true if you account also for hue changes. 1 dimensional LUTs only change the brightness of each color, but real world is a bit more complex and sometimes you need to change the hue of the color for a specific output. Below are examples of changing the hue of the blue color which is not possible without changing the green value.
This is usually implemented using 3DLUTs which are basically written mappings for some of the original points. Problem here is the space it takes (if you map all points (R,G,B) -> (R’,G’,B’) then you get basically 256x256x256x3 = 50Mb of data which is too hard for low-level hardware. so this is usually compressed to represent only some points in the RGB space and then do trilinear interpolation between them for values not specified.
Unfortunately most TV’s lack this capability not only in the regular settings menu but also in the underlying software implementation.
TV controls
Let’s see what we have in the menu of the TV.
Usual settings include some presets and also some manual corrections for brightness and contrast. Tint and Color Temperature are usually not found, but it’s nice to have them anyway.
Before going into any deeper calibration it’s best to try to adjust all these settings first to get to a better initial state: target a specific white point luminance (120 cd/m2 for example) and black point luminance (0.2 cd/m2) which will lead to contrast ratio of 600:1 (which marketers usually like to exaggerate). Measure with your calibration device and adjust the settings until you get to a good start with white and black point then move on to adjusting the RGB components separately to target a specific color temperature, usually 6500K. 6500K is roughly white light of the sun with cloudy conditions, remember that white is not always same white: it changes with conditions, you just perceive it as white, that’s why there is such a thing as white balance. Even under the same conditions you perceive black and white in a strange way, see below for example.
After all the correction you end up with 3x1DLUT’s for channels and no idea how to use this data in the TV. Let’s get to the interesting part:
The firmware
Everything next will be specific to my TV that I have at home.
I noticed that in the app store for the TV there was ES File Explorer which is an Android app. Strange. So this TV runs Android? Cool. Let’s see if I’m able to get a terminal using adb. After executing the connection command and trusting my computer from the TV I was in.
Let’s see if there is a specific config mount on this device
The structure of this volume revealed that there are multiple models running this firmware. Each model corresponds to a project and also to a specific panel id. Inside the configuration .ini file for each model:
[GAMMA_TABLE]
gamma_path = " /config/gamma/Gamma_1_ST5461D01-3.ini";I’ve found the gamma tables as follows:
###########################
# Gamma Table -0 #
###########################
[gamma_table_0]
parameter_r = \
{ \
...
0x24,0xFC,0xFD,0x01,0xFE,0xFF,0x00,0xFF \
} \
;parameter_g = \
{ \
0x90,0x00,0x01,0x71,0x03,0x04,0x2D,0x05,0x07, \
...
0x24,0xFC,0xFD,0x01,0xFE,0xFF,0x00,0xFF \
} \
;parameter_b = \
{ \
0x90,0x00,0x01,0x71,0x03,0x04,0x2D,0x05,0x07, \
...
0x24,0xFC,0xFD,0x01,0xFE,0xFF,0x00,0xFF \
} \
;//Repeats for 8 more tables.
Aha! This corresponds to the settings for gamma that I have in my UI of the TV:
So what are these tables? Separation by R,G and B is obvious but what are the values? 386 bytes doesn’t sound like a 1 byte -> 1 byte mapping. At first I visualized the data.
Doesn’t look like a curve yet. One thing to note is that these curves are the same for all channels at least for this specific gamma file.
Judging from the pictures I deduced that it’s actually three components in the data:
All the gamma files had the same overall picture. I’ve been on this for a couple of days until I tried to search for the parser code for something similar and I’ve found that there is a format that saves gamma curves in 12bit format.
On the left are basically two lower 4 bits concatenated. On the right are just higher bits in the measurement. This lead me to the following for the base presets for gamma:
Now that I knew how there curves are structured I wrote a converter from the .icc 3x1DLUT’s to these parameter_x curves.
The only thing left was how to embed them into the firmware. I’ve found several projects that unpacked the format of the update binary but upon trying to flash the firmware the internal updater rejected the update with error message. Okay, we’re not going down official firmware update route then. I had two options after this: root or custom recovery image like TWRP. I started digging around the firmware and found tclsu binary which gave me interesting output:
Trying to remount the tclconfig partition lead to an Operation not permitted error though. So after I installed proper root I’ve replaced one of the original gamma curves with my corrected one:
crossed my fingers and rebooted.
Just as before the TV rebooted and greeted me to the home screen. The first gamma table that I replaced happened to be mapped to the value -4 in the UI settings. As soon as I changed I saw that it wasn’t just contrast changes, but actual color mappings.
After that I did several adjustment calibrations to get to the point where the calibration curves were as close to linear as possible and then I was back to enjoying the content on the TV instead of researching and coding.
Hope this was an interesting read and feel free to clap and share.
If someone from Android Graphics team is reading this then I have a question for you: when are we gonna have custom calibration profiles loading with all the fancy color management in Oreo? It’s basically useless for people who are interested in color unless you have a perfect factory calibration setup (which is let’s be hones is not happening) or loading a custom icc profile which you create yourself.
