SUB2r - isp

automatic_exposure

Anonymous (anonymous@undisclosed.example.com) — 2022-04-04T23:32:03+00:00

Automatic exposure control In the modern age of digital cameras and rolling shutter sensor designs the variables that affect the luminal quality of the picture have shifted from their traditional set of aperture, shutter speed, film sensitivity (ISO) into Black Level, Gains, and

automatic_white_balance

Anonymous (anonymous@undisclosed.example.com) — 2022-04-04T23:32:03+00:00

Automatic white balancing Quite often end users don't have the patience or the expertise (or both) to properly set up the white balance of the captured scene and prefer to rely on camera “just knowing what to do”. Many algorithms are available today to deal with color normalization, some very sophisticated (and computationally expensive) and others, way less computationally expensive, but so simple that they are easily thrown off by things like a huge green screen background or a bright super-…

brightness

Anonymous (anonymous@undisclosed.example.com) — 2023-09-10T21:49:12+00:00

Brightness In the simplest terms the “brightness” of a pixel is its luminosity value as perceived by a human eye (making infrared and gamma rays \(0\) “bright”). There are quite a few competing standards and definitions of what the means exactly, but for our purposes we will use the \(L\)\(R, G, B \in [0..4095]\)\(br \in [-1024..+1023]\)\([0..4095]\)\[L = \frac{min(R, G, B)+max(R, G, B)}{2}\]\[ L` = L + br \\ L` \in [0..4095] \]\(k_R\)\(L\)\[ k_{R} = \begin{cases} R / L & \text{if} \; L \leq 20…

ccm

Anonymous (anonymous@undisclosed.example.com) — 2023-09-12T23:07:36+00:00

Color Correction Matrix A.k.a. “CCM” or “CMX” Purpose Why do the color correction at all? A picture is worth a thousand words so here's a visual to demonstrate the original (left) and color-corrected (right) values. Click to enlarge: CCM \(R_0, G_0, B_0\)\(R, G, B\)\[ \begin{bmatrix} R & G & B\end{bmatrix} = \begin{bmatrix} CCM_{00} & CCM_{01} & CCM_{02} \\ CCM_{10} & CCM_{11} & CCM_{12} \\ CCM_{20} & CCM_{21} & CCM_{22} \\ \end{bmatrix} \cdot \begin{bmatrix} R_0 \\ G_0 \\ B_0 …

contrast

Anonymous (anonymous@undisclosed.example.com) — 2023-09-10T21:53:10+00:00

Contrast Increasing the contrast increases the separation between dark and bright, making shadows darker and highlights brighter. Increasing it too much may clip both limits' values. Decreasing the contrast brings the shadows up and highlights down to make them closer to one another.\(L\)\(L\)\[ component = (component' - 0.5) * k^2 + 0.5 \\ component' \in [0..1.0] \\ contrast \in ]0..2.0]%% \]\(contrast\)\([0..2047]\)\[ \frac{ \left(\begin{bmatrix} R \\ G \\ B \end{bmatrix} - \begin{bmatrix} …

gamma

Anonymous (anonymous@undisclosed.example.com) — 2023-09-06T00:37:21+00:00

Gamma More precisely - this is a “gamma encoding” that is originally used to correct for the old-school CRT-type monitors' \(x^{2.2}\) voltage correction. Read more details on Wikipedia. Gamma encoding is calculated as a power function, where each pixel's \(L\) component (in \(N\)\(L \in [0..1]\)\(\_gamma \leq 1\)

gen5-brightness

Anonymous (anonymous@undisclosed.example.com) — 2026-01-13T23:47:45+00:00

YUV 4:4:4 UVC Brightness At this stage of the imaging pipeline the pixel data is represented in full-range YCbCr 4:4:4 color space with 12-bit precision per component. The purpose of this block is to apply UVC Brightness adjustment to the image by modifying the luma component while preserving chroma.\[ Y12 \in [0..4095]\\ C_b12 \in [0..4095]\text{, neutral at }2048\\ C_r12 \in [0..4095]\text{, neutral at }2048 \]\[ Y12^{\text{bright}} \in [0..4095]\\ C_b12^{\text{bright}} = C_b12\\ C_r12…

gen5-contrast

Anonymous (anonymous@undisclosed.example.com) — 2026-01-13T23:53:07+00:00

YUV 4:4:4 UVC Contrast At this stage of the imaging pipeline the pixel data is represented in full-range YCbCr 4:4:4 color space with 12-bit precision per component. The purpose of this block is to apply UVC Contrast adjustment by modifying the luma component relative to a neutral midpoint, while preserving chroma.\[ Y12 \in [0..4095]\\ C_b12 \in [0..4095]\text{, neutral at }2048\\ C_r12 \in [0..4095]\text{, neutral at }2048 \]\[ Y12^{\text{contrast}} \in [0..4095]\\ C_b12^{\text{contrast…

gen5-gamma

Anonymous (anonymous@undisclosed.example.com) — 2026-01-27T05:40:20+00:00

RGB UVC Gamma Adjustment At this stage of the imaging pipeline the pixel data is represented in full-range RGB color space with 12-bit precision per component, as produced by the CMX stage (gen5-cmx). The purpose of this block is to apply UVC gamma adjustment\[ R12 \in [0..4095]\\ G12 \in [0..4095]\\ B12 \in [0..4095] \]\[ R12^{\gamma} \in [0..4095]\\ G12^{\gamma} \in [0..4095]\\ B12^{\gamma} \in [0..4095] \]\(R12\)\(G12\)\(B12\)\[ R12^{\gamma} = LUT[R12]\\ G12^{\gamma} = LUT[G12]\\ B12^…

gen5-hue-rotation

Anonymous (anonymous@undisclosed.example.com) — 2026-01-14T00:11:35+00:00

UVC Hue rotation (YCbCr 4:4:4) This block applies UVC Hue adjustment by rotating the chroma vector \((C_b, C_r)\) around the neutral point, while leaving luma \(Y\) unchanged. This block does not clamp the output values. Chroma values outside the nominal 12-bit full-range are allowed and are expected to be handled later in the pipeline.\[ Y12 \in [0..4095]\\ C_b12 \in [0..4095]\text{, neutral at }2048\\ C_r12 \in [0..4095]\text{, neutral at }2048 \]\(H\)\(\times 100\)\[ H \in [-18000..+1800…

gen5-rgb-to-yuv

Anonymous (anonymous@undisclosed.example.com) — 2026-01-13T23:24:05+00:00

RGB to YCbCr 4:4:4 Pixels are coming out of de-bayer as 12-bit per channel RGB triplets (36 bits total per pixel). For now what we get from the sensor is assumed to be linear-light data (so we have to apply a transform function first to be aligned with the \(RGB\)\([0..4095]\)\[ Y12 \in [0..4095]\\ C_b12 \in [0..4095]\text{, neutral at }2048\\ C_r12 \in [0..4095]\text{, neutral at }2048 \]\(RGB\)\(L\)\(R'G'B'\)\(E\)\[ E = \begin{cases} 4.5 \times L & \quad \text{if } L < 0.018\\ 1.099 …

hsl-_rgb

Anonymous (anonymous@undisclosed.example.com) — 2022-04-04T23:32:04+00:00

HSL->RGB The text below is based on a [ SUB2r ISP Excel workbook]. That workbook provides a sort of playground where you can try various input data and see the final results, both “precise” and “optimized”, as well as the discrepancy between the two methods. Preface \(H\)\([-180°..+180°]\)\(S\)\([0\%..100\%]\)\(L\)\([0\%..100\%]\)\(saturation\)\(RGB\)\([0..255]\)\(x\)\(^1/_x\)\(^1/_x\)\(x\)\[\frac{1}{x} = \frac{1*N}{x*N}\]\(N\)\(x*N\)\(2\)\(Z\)\[\frac{C}{x} = C*\frac{1}{x} = C*\frac{1*N}{x*N}\…

hue_rotation

Anonymous (anonymous@undisclosed.example.com) — 2023-09-12T12:25:02+00:00

Hue rotation “Hue rotation” is a global (every pixel) shift of the image's color spectrum by a given radial offset (considering the color wheel). In RGB color space To rotate the color vector by an angle \(\Theta\) is to rotate the RGB cube around the \((0,0,0)-(1,1,)\) diagonal axis, for which the \[ cos_1 = cos(\Theta) \\ cos_2 = \frac{1 - cos_1}{3} \\ sin_1 = sin(\Theta)\sqrt{1/3} \\ \begin{bmatrix} cos_1 + cos_2 & cos_2 - sin_1 & cos_2 + sin_1 \\ cos_2 + sin_1 & cos_1 + cos_2 & cos_2 - s…

luminance

Anonymous (anonymous@undisclosed.example.com) — 2023-09-06T00:34:32+00:00

Luma BT. 709 Luminance of a pixel (read more on Wikipeia) is a measure of perceived brightness and is calculated in SUB2r code using the following formula: \[ luma = 0.2126R + 0.7152G + 0.0722B \] This is in accordance with the HDTV's BT.709 recommendation (and not the “expected” CCIR 601) and the \[ luma = 0.299R + 0.587G + 0.114B \]\[ luma = 0.2627R + 0.678G + 0.0593B \]

sample_ae_implementation_in_c

Anonymous (anonymous@undisclosed.example.com) — 2022-04-04T23:32:04+00:00

Sample AE implementation in C++ The code below provides a sample implementation of the AE algorithm described in Automatic exposure. It relies on a few external dependencies outlined below: * S2R::IImgSensor from the SUB2r-lib - a conveniency library to access image sensor's chip and control its registers

sample_awb_implementation_in_c

Anonymous (anonymous@undisclosed.example.com) — 2022-04-04T23:32:04+00:00

Sample AWB implementation in C++ The code below provides a sample implementation of the AWB algorithm described in Automatic white balance. It relies on a few external dependencies outlined below: * S2R::IImgSensor from the SUB2r-lib - a conveniency library to access image sensor's chip and control its registers

sharpness

Anonymous (anonymous@undisclosed.example.com) — 2022-04-04T23:32:04+00:00

Sharpness We define sharpness as the difference of brightness (the pixel's \(L\) component in HSL color space) from a pixel to its neighboring pixels. Sharpness for each pixel is calculated based on the values of immediately surrounding it pixels, only the luma component \(L\)\(coef_5 = 5 + sharpness\)\(sharpness\)\([-1..+1]\)\[ \left[ {\begin{array}{ccc} 0 & -1 & 0 \\ -1 & coef_5 & -1 \\ 0 & -1 & 0 \\ \end{array} } \right] \]

start

Anonymous (anonymous@undisclosed.example.com) — 2026-01-13T21:25:18+00:00

Image signal processing (ISP) Auto-functions * White balance, a.k.a. “color normalization” * Automatic white balance or AWB * Automatic exposure control UVC adjustments * Brightness * Contrast * Hue rotation * Vibrancy or “smart saturation” * Sharpness * Gamma Color grading Scopes * Histograms * Parade Adjustments * Hue vs. Hue * Hue vs. Saturation * Hue vs. Luma *

vibrancy

Anonymous (anonymous@undisclosed.example.com) — 2023-09-05T06:22:43+00:00

Vibrancy A vibrancy of a color is a perceived quality that is somewhat similar to Saturation. One could think of it as pure, bright, high chroma color. It is not an absolute measure (like Saturation) and there's no agreed upon definition of what it means to change the vibrancy of a color.\(\in[-127..+128]\)\(1.0\)\((0, 0)\)\([-127..+128]\)\((0, 0)\)\(S\)\(S\)\[ scale = 1 + \frac{100 - saturation}{100} * (vibrancy-1) \\ vibrancy \in [0..1]\\ saturation \in [0..100]\% \]

white_balance

Anonymous (anonymous@undisclosed.example.com) — 2022-04-04T23:32:05+00:00

White Balance What is white color temperature A “white” color has different chromatic characteristics, depending on its “temperature”. That temperature is traditionally expressed in Kelvin and corresponds to light, emitted by an ideal black body radiator, when heated to that temperature. Think of it as the color of a burning stick of wood at a campsite or a \(6000°K\)\(1024\)\(B = 4205.4 - T*0.4087\)\(3500°K\)\[ R = \begin{cases} 467.6 + T * 0.166 & \quad \text{if } T < 3500°K \\ 82 + T *…