In the modern age of digital cameras and rolling shutter sensor designs the variables that affect the luminal quality of the picture have shifted from their traditional set of aperture, shutter speed, film sensitivity (ISO) into Black Level, Gains, and “exposure” (the “exposure” is in quotes because its value doesn't affect the physical blocking of light but rather how the chip reads and re-sets data values in individual sensor elements).
Below are a few sections that provide a bit of information on what effect each of the 3 types of digital settings have on the output image.
Black level basically tells the sensor which level of brightness to consider “true black”. If you take a look at the brightness histogram of a given image, the value of the Black Level will directly correspond to the gap at the left edge of that histogram, where there are no pixels reported to be of such low brightness value. Increasing the Black Level too high will cause the image to look washed out, while using a too low of a setting causes some grey pixels to be mistakenly read as pitch black.
Remember: the brighter the pixel, the lower its read charge. So increasing the Black Level causes more pixels to be considered “brighter than black” and lowering the Black Level value causes more off-black (grey) pixels to be reported as solid black, as in “these pixels were not bright enough to be considered lit”.
Gains are constant values that are applied to the analog data read from the sensor before running through an ADC and can be either additive or multiplicative.
There are 3 individual Gain settings for each of the 3 color channels (Red, Green, Blue) and those are applied to pixel data corresponding to that color. Naturally changing these values has a profound effect on the output color. The offset to increasing gain is that it will increase the noise.
Global Gain is a Gain that is applied to every pixel. Adjusting this setting greatly affects the brightness of the picture while only minimally affecting the color balance. The offset to increasing gain is that it will increase the noise.
The timing generator outputs clocks to access the rows of the imaging array, pre-charging and sampling the rows of the array sequentially. In the time between pre-charging and sampling a row, the charge in the pixels decreases with exposure to incident light. This is the exposure time in rolling shutter architecture.
The exposure time is controlled by adjusting the time interval between pre-charging and sampling. After the data of the pixels in the row has been sampled, it is processed through analog circuitry to correct the offset and multiply the data with corresponding Gain. Following analog processing is the ADC which outputs 10, 12, or 14 bits1) of data for each pixel in the array.
Ignoring the effects of UVC's “Brightness”, “Contrast”, and “Gamma” controls we are only concentrating on the following sensor chip level adjustments when trying to configure the settings for an optimal picture luminocity: Black level, Global Gain, Exposure.
So what do we mean when we set off to adjust the exposure? In short - we are trying to hit these vague goals:
In terms of quantification of the above goals we are striving to:
2%
The following steps describe the auto-exposure procedure in a top-down approach:
Pretty straight-forward - just collect the standard histogram data for the “brightness” of the pixel, like a Y
component of the Y'UV
triplet.
For better results the collection should be limited to the central region of the image (where the face usually is) specified by a pair of values that denote the relative vertical and horizontal sizes of that window in percentage points. As an example: a window's horizontal relative size of 50%
would mean that the left edge of that window starts at 25%
of the total frame's pixel count on the left and continues for 50%
of the total frame's pixel count. Both values should be in [5
..100
] range.
The decision is based on two factors:
100
and the target brightness is 70
, then the ratio is 0.7
- meaning we need to dim the picture)2
M pixels and an overexposure tolerance set at 2%
if the result is 240
, then that means that there are over 40,000
pixels with their brightness at level 241
and above.Once we have the above two factors figured out the decision tree looks as follows:
enum class ExpCorrAction{stay, inc, dec}; ExpCorrAction _needExpCorrection(int _right, double _adjY){ if(_right > 60 && _adjY < 1){ return ExpCorrAction::dec; // there are pixels at level 100+ and we were asked to dim the image } if(_right >= 250 && _adjY <= 1){ return ExpCorrAction::dec; // there are more overexposed pixels than the threshold and dimming the image } if(_right < 240 && _adjY > 1){ return ExpCorrAction::inc; // there's room to grow and asked to increase brightness } return ExpCorrAction::stay; }
4
) and if the Red and Blue channels' Gains are not maxed out, then carefully increase the Green Gain. By “carefully” I mean the following:1180
5%
)This is pretty much the opposite of the Increase the brightness described above with the actions prioritized in the opposite order:
10%
, but never below 1024
and only if the auto-white balance is currently enabled1024
- decrease the Global Gain by multiplying it by the target/real brightness ratio0
- decrease the Exposure by multiplying it by the target/real brightness ratioAs the sensor processes input its electrical characteristics could fluctuate (due to, for example, heating up) requiring a manual human intervention to set an appropriate Black level (or an automated “Black level Calibration” procedure, if implemented by the sensor chip).
In order to automate this process one could think of Black Level as the counterpart to the overexposure. The goal is to have the Black Level set at such value as to ensure that there are no more than specified overexposure pixel count that are pitch black.
Sample AE implementation in C++ shows one possible implementation of the approach described above. It does use (without much explanation) some of the external structures, but those should not be hard to comprehend from their usage