Colour Matrices And Why Typesetters Make Encoders Look Bad
It seems that a lot of people still have issues with when and how colour matrix conversions should be used. After talking to some fellow encoders and a few typesetters I figured it was about time to write something useful here again.
There are two main colour matrices used in pretty much everything: ITU-Rec.709 and ITU-Rec.601. Contrary to popular belief, both use what we call TV levels, that is, on the luma scale, black is 16 and white is 235 as opposed to the PC levels of 0 and 255 respectively. Rec.709 (also called bt709 which I will use from now on) is used on HD material, that is anything with MORE than 576 pixels vertically, while bt601 is used on SD material, or anything up to and including 576 vertical pixels.
The trouble people seem to have is WHERE to use these. You can say that for anything at each of those resolutions, you will use what is appropriate, but what happens when someone takes a screenshot? Video is almost always encoded in the YV12 colourspace, yet screenshots are RGB. A conversion there needs to use the correct matrix, however most mediaplayers will assume bt601. Playback itself they will either read the stream info or assume based on resolution, but it isn’t so for screenshots most of the time.
As another example, what happens if you have something broadcast on an HD stream at bt709 but it’s really upscaled, and you decide to encode it at say 480p, which would you use? Effectively you can use the fancy colormatrix() plugin for avs to convert bt709 to bt601, but why not just flag the stream? It’s my own personal practice to convert SD material because I know people will screenshot it, and if it’s an SD encode, there is no point keeping the other matrix.
Furthermore, what happens when you need to process video? Most programs that use RGB outputs, say for encoding overlay files with an alpha channel, will assume bt601, I know VirtualDub and Adobe After Effects do. AutoDesk Inferno has an option to set it, although I’d assume AE does as well, but then again it isn’t something the industry likes to think about. So lets assume we have a source at bt709, and it’s HD. We then do some processing on it and output a lossless file for someone else to typeset onto, which is then encoded as RGB32, and overlayed with an alpha channel back onto the original clip. The following is a flowchart of sorts in how this normally goes, and results in incorrect colours:
Source (bt709 YV12) -> Processing (bt709 YV12) -> Lossless AVC (bt709 YV12) -> Typeset (bt601 RGBA) -> Overlayed (bt709 YV12)
You could of course encode your lossless as for example, Lagarith, at RGB24, but are you going to use a colourspace conversion in that? Yes. Are you going to do it yourself? No. The encoder will do it and assume bt601. The colours are then off. The correct way to do it would be to set your source going into the LAGS encoder as already RGB, pass that to the typesetter who has correct colours and outputs RGBA for you correctly, and everything is hunky dory. Example avs code that goes right at the end of the script:
ConvertToRGB24(matrix="Rec709")
That’s it. LAGS now has RGB data directly, and there is no problem. Because people get that wrong though, typesetters bork colours slightly (it’s really quite unnoticable to most people, and even then only on certainhues) and then the encoder looks stupid.
I think deciding which matrix to use in final encodes is really up to personal preference, as long as the stream has it correct there is no problem. All that matters is that the same colourspace is used consistently or if changed, any conversions are done correctly. Using the Colormatrix() filter is optional, if you intend to output bt601 then it’s a great way to start off in that colourspace (it goes in the header of your avs file, before anything that changes frames in any way is run) and if not, making sure to flag bt709 in x264 (–colormatrix bt709) should be done instead. At encoding time, flagging for bt601 is also good. I’d say for SD shows doing a bt601 encode is a better idea due to the aforementioned screenshot issue, but for HD stuff it’s ok to use bt601 as well really. Rec.709 is of course preferred for HD content.
I hope this has made it a little clearer about WHERE to put conversions and maybe one or two people actually learn something.
Comments
2 Responses to “Colour Matrices And Why Typesetters Make Encoders Look Bad”
Leave a Reply
Or alternatively don’t have AFX typesetting because hardsubbed typesetting is gay.
Even ASS typesetting can have mismatched colours due to bad colourimetry. It uses BGR overlaying while the video is YV12. I have seen this happening on quite a few clips.