In the modern age of digital cameras and rolling shutter sensor designs the variables that affect the luminal quality of the picture have shifted from their traditional set of aperture, shutter speed, film sensitivity (ISO) into Black Level, Gains, and “exposure” (the “exposure” is in quotes because its value doesn't affect the physical blocking of light but rather how the chip reads and re-sets data values in individual sensor elements).

Below are a few sections that provide a bit of information on what effect each of the 3 types of digital settings have on the output image

Black level basically tells the sensor which level of brightness to consider “true black”. If you take a look at the brightness histogram of a given image the value of the Black Level will directly correspond to the gap at the left edge of that histogram, where there are no pixels reported to be of such low brightness value. Increasing the Black Level too high will cause the image to look washed out while using a too low of a setting causes some grey pixels to be mistakenly read as pitch black.

Remember: the brighter the pixel the lower it's read charge. So increasing the Black Level causes more pixels to be considered “brighter than black” and lowering the Black Level value causes more off-black (grey) pixels to be reported as solid black as in “these pixels were not bright enough to be considered lit”.

Gains are constant values that are applied to the analog data read from the sensor before running through an ADC and can be either additive or multiplicative.

There are 3 individual Gain settings for each of the 3 color channels (Red, Green, Blue) and those are applied to pixel data corresponding to that color. Naturally changing these values has a profound effect on the output color.

Global Gain is a Gain that is applied to every pixel. Adjusting this setting greatly affects the brightness of the picture while only minimally affecting the color balance.

The timing generator outputs clocks to access the rows of the imaging array, precharging and sampling the rows of the array sequentially. In the time between precharging and sampling a row, the charge in the pixels decrease with exposure to incident light. This is the exposure time in rolling shutter architecture.

The exposure time is controlled by adjusting the time interval between pre-charging and sampling. After the data of the pixels in the row has been sampled, it is processed through analog circuitry to correct the offset and multiply the data with corresponding Gain. Following analog processing is the ADC which outputs 10, 12, or 14 bits¹⁾ of data for each pixel in the array.

Ignoring the effects of UVC's “Brightness”, “Contrast”, and “Gamma” controls we are only concentrating on the following sensor chip level adjustments when trying to configure the settings for an optimal picture luminocity: Black level, Global Gain, Exposure.

So what do we mean when we set off to adjust the exposure? In short - we are trying to hit these vague goals:

1. No overexposed pixels²⁾
2. “Bright enough” overall picture
3. Natural progression from dark to bright

In terms of quantification of the above goals we are striving to:

1. Limit the number of overexposed pixels to less than a given percentage, say 2%
2. The global brightness average (arithmetic mean) should be in a comfortable zone around 90-100 (assuming [0..255] range for brightness)
3. Black level should be set to correctly reflect the visual perception of the scene and the image's rendering on a display

As the sensor processes input its electrical characteristics could fluctuate (due to, for example, heating up) requiring a manual human intervention to set an appropriate Black level (or an automated “Black level Calibration” procedure, if implemented by the sensor chip).

The following steps describe the auto-exposure procedure in a top-down approach:

1. Collect statistics
2. Decide the action to take and based on that:
   1. No adjustment needed - loop back to step one
   2. Increase the brightness
   3. Reduce the brightness
3. Loop back to step one

Pretty straight-forward - just collect the standard histogram data for the “brightness” of the pixel, like a Y component of the Y'UV triplet.

For better results the collection should be limited to the central region of the image (where the face usually is) specified by a pair of values that denote the relative vertical and horizontal sizes of that window in percentage points. As an example: a window's horizontal relative size of 50% would mean that the left edge of that window starts at 25% of the total frame's pixel count on the left and continues for 50% of the total frame's pixel count. Both values should be in [5..100] range.

The decision is based on two factors:

1. A ratio of target brightness to the actual average brightness (e.g. if the current image's average brightness is 100 and the target brightness is 70, then the ratio is 0.7 - meaning we need to dim the picture)
2. How right-hand-side heavy is the histogram. To determine that number we calculate how many right-most bars of the histogram does it take to accumulate the number of pixels that our overexposure tolerance is set to. Given an image with a total of 2M pixels and an overexposure tolerance set at 2% if the result is 240, then that means that there are over 40,000 pixels with their brightness at level 241 and above.

Once we have the above two factors figured out the decision tree looks as follows:

enum class ExpCorrAction{STAY, INC, DEC};
ExpCorrAction _needExpCorrection(int _right, double _adjY){
    if(_right > 60 && _adjY < 0.97){
        return ExpCorrAction::DEC;  // there are pixels at level 100+ and we were asked to dim the image
    }
    if(_right >= 250 && _adjY <= 1){
        return ExpCorrAction::DEC;  // there are more overexposed pixels than the threshold and dimming the image
    }
    if(_right < 240 && _adjY > 1.2){
        return ExpCorrAction::INC;  // there's room to grow and asked to increase brightness
    }
    return ExpCorrAction::STAY;
}

1. If we have not reached the limit on Exposure setting - increase it by multiplying by the target/real brightness ratio
2. Else, if we have not hit the Global Gain limit - increase it by multiplying by the target/real brightness ratio
3. Else (special case!) only if the auto-white balance is currently turned on and if the brightness adjustment is very high (above 4) and if the Red and Blue channel's Gains are not maxed out then carefully increase the Green Gain. By “carefully” I mean the following:
   1. Never increase the Green Gain beyond 1180
   2. Only increase the Green Gain as much as to not cause the overflow for the Gains in Red or Blue channels (take a conservative guess and then lower it by another 5%)

This is pretty much the opposite of the Increase the brightness described above with the actions prioritized in the opposite order:

1. First try to decrease the Green Gain by 10%, but never below 1024 and only if the auto-white balance is currently enabled
2. If the Green Gain is at 1024 - decrease the Global Gain by multiplying it by the target/real brightness ratio
3. If the Global Gain is already at 0 - decrease the Exposure by multiplying it by the target/real brightness ratio