Differences

This shows you the differences between two versions of the page.

--- isp:ccm [2020/12/17 22:34] – [CCM] Igor Yefmov
+++ isp:ccm [2023/09/12 23:07] (current) – [Support] Igor Yefmov
@@ Line 1: / Line 1: @@
-===== Support =====
+======= Color Correction Matrix =======
-SUB2r camera starts supporting CCM (a.k.a. CMX) with **FX3 v. ''58''** and **FPGA v. ''73''**.
+A.k.a. "CCM" or "CMX"
-===== Pixel layout =====
+===== Purpose =====
-With the OV10823 %%OmniVision%% image sensor we use the "standard" Bayer pattern that looks like this:
+Why do the color correction at all? A picture is worth a thousand words so here's a visual to demonstrate the original (left) and color-corrected (right) values. Click to enlarge:
+{{ :isp:color_correction_sample.jpg?direct&200 |}}
-{{ :isp:ov10823-bayer.png?direct |}}
 ===== CCM =====
-Color Correction Matrix is often used as an "add-on" matrix during YUV->RGB conversion. In our case, since the de-bayering and RGB->YUV conversions happen literally on opposite sides of the imaging pipeline, we only use the portion of the corrections designed to compensate for the sensor's cross-talk, converting pre-debayer values \(R_0, Gr_0, Gb_0, B_0\) into \(R, Gr, Gb, B\) that are then used for conversion into RGB:
+Color Correction Matrix is often used as an "add-on" matrix during YUV->RGB conversion. In our case, since the de-bayering and RGB->YUV conversions happen literally on opposite sides of the imaging pipeline, we only use the portion of the corrections designed to compensate for the sensor's cross-talk, converting post-debayer values \(R_0, G_0, B_0\) into \(R, G, B\):
 \[
- \begin{bmatrix} B & Gb & Gr & R\end{bmatrix}
+ \begin{bmatrix} R & G & B\end{bmatrix}
  =
  \begin{bmatrix}
- CCM_{00} & CCM_{01} & CCM_{02} & CCM_{03} \\
+ CCM_{00} & CCM_{01} & CCM_{02} \\
- CCM_{10} & CCM_{11} & CCM_{12} & CCM_{13} \\
+ CCM_{10} & CCM_{11} & CCM_{12} \\
- CCM_{20} & CCM_{21} & CCM_{22} & CCM_{23} \\
+ CCM_{20} & CCM_{21} & CCM_{22} \\
- CCM_{30} & CCM_{31} & CCM_{32} & CCM_{33}
  \end{bmatrix}
  \cdot
  \begin{bmatrix}
- B_0 \\
+ R_0 \\
- Gb_0 \\
+ G_0 \\
- Gr_0 \\
+ B_0
- R_0
  \end{bmatrix}
 \]
-In this matrix \(Gb_0\) is a value of a "green pixel in a blue row" (i.e. in a B-G-B-G-B-... row) and, similarly, \(Gr_0\) is a value of a "green pixel in a red row".
+Regular linear algebra rules apply, of course. The calculation itself (once expanded) looks like this:
+\[
+R = CCM_{00} * R_0 + CCM_{01} * G_0 + CCM_{02} * B_0 \\
+G = CCM_{10} * R_0 + CCM_{11} * G_0 + CCM_{12} * B_0 \\
+B = CCM_{20} * R_0 + CCM_{21} * G_0 + CCM_{22} * B_0
+\]
+===== 3x3 -> 4x4 =====
+Industry papers on CCM use a \(3\times3\) matrix yet we tried to do a full \(4\times4\) so there was a need to find a way to convert from one to another (and also from an RGGB into BGGR ordering). Here's an [[https://math.stackexchange.com/questions/3957955/ccm-3x3-into-4x4-image-processing|article on Math StackExchange]] on that subject.
+And if you'd like to play with the values in an Excel-like environment - here's a [[https://docs.google.com/spreadsheets/d/1X1n5hVDLHd-Z6ZLEcwPZijU2WQZ40QAtt9Z-ExZlRG0/edit?usp=sharing | link to Google Sheets]] for your enjoyment.
+After some research and a bit of math crunching it turned out that such an approach is invalid. Primarily because it ignores a very important step - de-bayering, which affects the result in a critical manner.
+Therefore we go with the "standard" approach of using a \(3\times3\) matrix in \(RGB\) space.
-The matrix is applied to a \(2\times2\) block of \(\begin{bmatrix}B_0 & Gb_0 \\ Gr_0 & R_0\end{bmatrix}\) values represented by a "flattened out" vector \(\begin{bmatrix}B_0 & Gb_0 & Gr_0 & R_0\end{bmatrix}\) to produce the same \(2\times2\) block of corrected values \(\begin{bmatrix}B & Gb \\ Gr & R\end{bmatrix}\).