Both sides previous revisionPrevious revisionNext revision | Previous revision |
isp:ccm [2020/12/22 04:39] – [Support] Igor Yefmov | isp:ccm [2023/09/12 23:07] (current) – [Support] Igor Yefmov |
---|
===== Support ===== | ======= Color Correction Matrix ======= |
SUB2r camera starts supporting CCM (a.k.a. CMX) with **FX3 v. ''58''** and **FPGA v. ''73''**. | A.k.a. "CCM" or "CMX" |
| |
===== 3x3 -> 4x4 ===== | ===== Purpose ===== |
Industry papers on CCM use a \(3x3\) matrix yet we do a full \(4x4\) so there needs to be a way to convert from one to another (and also from an RGGB into BGGR ordering). Here's an [[https://math.stackexchange.com/questions/3957955/ccm-3x3-into-4x4-image-processing|article on Math StackExchange]] on that subject. | Why do the color correction at all? A picture is worth a thousand words so here's a visual to demonstrate the original (left) and color-corrected (right) values. Click to enlarge: |
| |
And if you'd like to play with the values in an Excel-like environment - here's a [[https://docs.google.com/spreadsheets/d/1X1n5hVDLHd-Z6ZLEcwPZijU2WQZ40QAtt9Z-ExZlRG0/edit?usp=sharing | link to Google Sheets]] for your enjoyment. | |
===== Why? ===== | |
Why do the color correction at all? A picture is worth a thousand words so here's a visual to demonstrate the original (left) and color-corrected (right) values: | |
{{ :isp:color_correction_sample.jpg?direct&200 |}} | {{ :isp:color_correction_sample.jpg?direct&200 |}} |
| |
===== Pixel layout ===== | |
With the OV10823 %%OmniVision%% image sensor we use the "standard" Bayer pattern that looks like this: | |
| |
{{ :isp:ov10823-bayer.png?direct&200 |}} | |
| |
===== CCM ===== | ===== CCM ===== |
Color Correction Matrix is often used as an "add-on" matrix during YUV->RGB conversion. In our case, since the de-bayering and RGB->YUV conversions happen literally on opposite sides of the imaging pipeline, we only use the portion of the corrections designed to compensate for the sensor's cross-talk, converting pre-debayer values \(R_0, Gr_0, Gb_0, B_0\) into \(R, Gr, Gb, B\) that are then used for conversion into RGB: | Color Correction Matrix is often used as an "add-on" matrix during YUV->RGB conversion. In our case, since the de-bayering and RGB->YUV conversions happen literally on opposite sides of the imaging pipeline, we only use the portion of the corrections designed to compensate for the sensor's cross-talk, converting post-debayer values \(R_0, G_0, B_0\) into \(R, G, B\): |
| |
\[ | \[ |
\begin{bmatrix} B & Gb & Gr & R\end{bmatrix} | \begin{bmatrix} R & G & B\end{bmatrix} |
= | = |
\begin{bmatrix} | \begin{bmatrix} |
CCM_{00} & CCM_{01} & CCM_{02} & CCM_{03} \\ | CCM_{00} & CCM_{01} & CCM_{02} \\ |
CCM_{10} & CCM_{11} & CCM_{12} & CCM_{13} \\ | CCM_{10} & CCM_{11} & CCM_{12} \\ |
CCM_{20} & CCM_{21} & CCM_{22} & CCM_{23} \\ | CCM_{20} & CCM_{21} & CCM_{22} \\ |
CCM_{30} & CCM_{31} & CCM_{32} & CCM_{33} | |
\end{bmatrix} | \end{bmatrix} |
\cdot | \cdot |
\begin{bmatrix} | \begin{bmatrix} |
B_0 \\ | R_0 \\ |
Gb_0 \\ | G_0 \\ |
Gr_0 \\ | B_0 |
R_0 | |
\end{bmatrix} | \end{bmatrix} |
\] | \] |
| |
In this matrix \(Gb_0\) is a value of a "green pixel in a blue row" (i.e. in a B-G-B-G-B-... row) and, similarly, \(Gr_0\) is a value of a "green pixel in a red row". | Regular linear algebra rules apply, of course. The calculation itself (once expanded) looks like this: |
| |
| \[ |
| R = CCM_{00} * R_0 + CCM_{01} * G_0 + CCM_{02} * B_0 \\ |
| G = CCM_{10} * R_0 + CCM_{11} * G_0 + CCM_{12} * B_0 \\ |
| B = CCM_{20} * R_0 + CCM_{21} * G_0 + CCM_{22} * B_0 |
| \] |
| ===== 3x3 -> 4x4 ===== |
| Industry papers on CCM use a \(3\times3\) matrix yet we tried to do a full \(4\times4\) so there was a need to find a way to convert from one to another (and also from an RGGB into BGGR ordering). Here's an [[https://math.stackexchange.com/questions/3957955/ccm-3x3-into-4x4-image-processing|article on Math StackExchange]] on that subject. |
| |
| And if you'd like to play with the values in an Excel-like environment - here's a [[https://docs.google.com/spreadsheets/d/1X1n5hVDLHd-Z6ZLEcwPZijU2WQZ40QAtt9Z-ExZlRG0/edit?usp=sharing | link to Google Sheets]] for your enjoyment. |
| |
| After some research and a bit of math crunching it turned out that such an approach is invalid. Primarily because it ignores a very important step - de-bayering, which affects the result in a critical manner. |
| |
| Therefore we go with the "standard" approach of using a \(3\times3\) matrix in \(RGB\) space. |
| |
The matrix is applied to a \(2\times2\) block of \(\begin{bmatrix}B_0 & Gb_0 \\ Gr_0 & R_0\end{bmatrix}\) values represented by a "flattened out" vector \(\begin{bmatrix}B_0 & Gb_0 & Gr_0 & R_0\end{bmatrix}\) to produce the same \(2\times2\) block of corrected values \(\begin{bmatrix}B & Gb \\ Gr & R\end{bmatrix}\). | |