| Both sides previous revisionPrevious revision | |
| code:fx3_hvci_and_fpga_i_c_commands [2024/02/17 15:41] – [General image stats and adjustments] Igor Yefmov | code:fx3_hvci_and_fpga_i_c_commands [2024/02/17 15:49] (current) – Igor Yefmov |
|---|
| FX3 API specification has been moved to [[fx3_api|this location]] | FX3 API specification has been moved to [[fx3_api|this location]] |
| |
| ====== FPGA I²C bridge (registers' map) ====== | FPGA register map specifications have been moved to [[fpga_registers_map|this location]] |
| The following tables provide information on how to access the camera's functionality for an FPGA I²C bridge. | |
| | |
| Here's a sample code (skipping all error checking) that sets the LED to bright-yellow color FIXME: | |
| <code c++>S2R::FX3 dev; // auto-open device #0 | |
| using S2R::FX3; | |
| dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 255, 0x08); // red | |
| dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 255, 0x0A); // green | |
| dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 0, 0x0C); // blue | |
| </code> | |
| | |
| The address space is broken down into smaller chunks, grouped by common functionality: | |
| ===== FPGA basic status and controls ===== | |
| ^Name ^Offset ^Bytes ^Access ^Bit mapping ^Notes ^ | |
| |:!: FPGA Version|''0x00''| 4 |R/O| see [[code#firmware_version_info|Firmware Version Info]] | | | |
| |:!: FPGA status|''0x01''| 1 |R/O|''7:2'' reserved| | | |
| |:::|:::|:::|:::|:!: ''1'' DPC calibration done| ''1'' indicates that DPC calibration is completed | | |
| |:::|:::|:::|:::|:!: ''0'' SFP+| ''1'' indicates there is an active device in SFP+ cage | | |
| |:!: FPGA control|''0x02''| 1 |R/W|''7:2'' reserved| | | |
| |:::|:::|:::|:::|:!: ''1'' enable DPC| enable DPC correction | | |
| |:::|:::|:::|:::|:!: ''0'' FPGA config enable|if set to ''1'', the GPIF becomes read only and waits for an FPGA update bitstream| | |
| |:!: FPGA config status|''0x03''| 2 |R/O|see [[#FPGA config status - SPI codes|SPI codes]] for details| | | |
| |:!: FPGA core temperature|''0x04''| 1 | R/O | ''7:0'' - temperature in Farenheits | Reading of the FPGA's internal temperature sensor in degrees of Farenheits | | |
| |:!: LED RED|''0x05''| 2 |R/W| |LED's red intensity in range \([0..65535]\)| | |
| |:!: LED GREEN|''0x06''| 2 |R/W| |LED's green intensity in range \([0..65535]\)| | |
| |:!: LED BLUE|''0x07''| 2 |R/W| |LED's blue intensity in range \([0..65535]\)| | |
| | |
| ===== AWB ===== | |
| Info for performing Auto White-Balance adjustments on the image | |
| | |
| ^Name ^Offset ^Bytes ^Access ^Notes ^ | |
| |:!: AWB Red adjustment|''0x08''| 2 |R/W|a signed 16-bit value of an adjustment to apply to every pixel in Red channel. Default is \(0\)| | |
| |:!: AWB Green adjustment|''0x09''| 2 |R/W|a signed 16-bit value of an adjustment to apply to every pixel in Green channel. Default is \(0\)| | |
| |:!: AWB Blue adjustment|''0x0A''| 2 |R/W|a signed 16-bit value of an adjustment to apply to every pixel in Blue channel. Default is \(0\)| | |
| |Reserved| ''0x0B'' | | | | | |
| |:!: AWB Red total| ''0x0C'' | 4 | R/O | | | |
| |:!: AWB Green total| ''0x0D'' | 4 | R/O | | | |
| |:!: AWB Blue total| ''0x0E'' | 4 | R/O | | | |
| |:!: AWB count total| ''0x0F'' | 4 | R/O | A count of pixels that were used to calculate the \(R/G/B\) totals | | |
| | |
| | |
| | |
| ===== Basic UVC controls ===== | |
| ^Name ^Offset ^Bytes ^Access ^Range ^Range description ^Neutral value ^ | |
| |:!: Brightness|''0x10''| 2 |R/W|\([-1024..1023]\) |\(-1024\) makes the image very dark\\ \(1023\) makes the image very bright |\(0\) | | |
| |:!: Contrast|''0x11''| 2 |R/W|\([1..2047]\) |\(1\) turns image into grayscale\\ \(2047\) makes all pixels either black or white |\(1023\) | | |
| |:!: Saturation|''0x12''| 2 |R/W|\([0..900]\) |\([0\%..900\%]\) or grayscale to 9x |\(100\) | | |
| |:!: Sharpness|''0x13''| 1 |R/W|\([0..255]\) | |\(0\) | | |
| |:!: Gamma|''0x14''| 1 |R/W|\([0..15]\) | |\(0\) | | |
| |:!: Hue|''0x15''| 2 |R/W|\([-8192..8191]\) |\([-180°..180°)\) |\(0\) | | |
| |Reserved|''0x16-0x1F''| | | | | | |
| | |
| | |
| | |
| ===== General image stats and adjustments ===== | |
| | |
| ^ Defective pixel cancellation ^^^^^ | |
| ^Name ^Offset ^Bytes ^Access ^Notes ^ | |
| |:!: DPC Threshold|''0x20''| 2 |R/W| A write into this register triggers the start of DPC calibration | | |
| |:!: DPC count|''0x21'' | 2 |R/O|Once the DPC calibration is done the 16-bit value is stored in here| | |
| | ||||| | |
| ^ General image stats ^^^^^ | |
| |:!: flags|''0x22''| 1 |R/O| ''7:2'' reserved\\ :!: ''1'' - indicates whether a red overexposure is detected\\ :!: ''0'' - set if there is a general overexposure detected | | |
| |:!: FPS|''0x23''| 2 |R/O|16-bit unsigned value representing number of 10μs units between frame start signals from the image sensor, e.g. a value of \(1000\) means it took 10ms between 2 frame start signals, which corresponds to 100FPS| | |
| |:!: Y average|''0x24''| 2 |R/O|Average "brightness" value| | |
| | |
| ===== Color Correction Matrix (a.k.a. CCM or CMX) ===== | |
| See [[isp:ccm|Color correction matrix]] article in this Wiki's ISP section for more details. The 16-bit (MSB-LSB) value is defined as 7+9 bits, where MSB[7:1] are the integer part and MSB[0]LSB[7:0] is the fractional part (effectively that value is 512 times larger than the original fractional part) | |
| | |
| ^Name ^Offset ^Bytes ^Access ^Notes ^ | |
| |:!: CCM_00|''0x25''| 2 |R/W|\(CCM_{00}\)| | |
| |:!: CCM_01|''0x26''| 2 |R/W|\(CCM_{01}\)| | |
| |:!: CCM_02|''0x27''| 2 |R/W|\(CCM_{02}\)| | |
| |:!: CCM_10|''0x28''| 2 |R/W|\(CCM_{10}\)| | |
| |:!: CCM_11|''0x29''| 2 |R/W|\(CCM_{11}\)| | |
| |:!: CCM_12|''0x2A''| 2 |R/W|\(CCM_{12}\)| | |
| |:!: CCM_20|''0x2B''| 2 |R/W|\(CCM_{20}\)| | |
| |:!: CCM_21|''0x2C''| 2 |R/W|\(CCM_{21}\)| | |
| |:!: CCM_22|''0x2D''| 2 |R/W|\(CCM_{22}\)| | |
| | |
| ===== Color grading ===== | |
| | |
| ^Name ^Offset ^Access ^Bit mapping ^Notes ^ | |
| |:!: switch|''0x2E''|W|''15:10'' reserved |Controls what information is being read/written by accessing the register ''0x002F''| | |
| |:::|:::|:::|''9:7'' table switch|''000'' - Hue vs. Hue (''14'' bits)\\ ''001'' - Hue vs. Saturation (''12 bits'')\\ ''010'' - Lightness vs. Saturation (''12'' bits)\\ ''011'' - Saturation vs. Saturation (''12'' bits)\\ ''100'' - Lightness vs. Lightness (''12'' bits)\\ ''101'' - Hue vs. Lightness (''12'' bits)\\ ''110-111'' - reserved| | |
| |:::|:::|:::|''6:1'' index LSB| index into a page in the table | | |
| |:::|:::|:::|''0'' access mode| ''0'': "normal mode", in which all the subsequent accesses to the register ''0x002F'' are governed by the values in ''0x002E''\\ ''1'': "bulk access", where after a read or write access to register ''0x002F'' the "Index" value will auto-increment by one so that the next read/wrie will access the subsequent table slot| | |
| |:!: Value|''0x2F''|R/W|''15:0'' value| 16 bits of either signed or unsigned integer value\\ - for a "Hue vs. Hue" table the 14 bits signed value is in range ''[-8192..+8192]'' which maps linearly into a Hue angle range ''-180°..+180°''\\ - for a "Hue vs. Saturation" table (as well as for similar tables %%LvS%% and %%SvS%%) the 12 bit unsigned value in range ''[0..+1280]'' maps linearly into a Saturation range ''[0%..1000%]'' where ''100%'' is the neutral position and ''0%'' produces a greyscale image\\ - (until FPGA v.72) for a "Lightness vs. Lightness" table (as well as for similar table %%HvL%%) the 12 bit unsigned value in range ''[0..+4095]'' maps linearly into a Lightness absolute range ''[0..255]'' where ''0'' is pitch black and ''255'' is the maximum possible pixel luminosity value\\ - (starting with FPGA v.73) for a "Lightness vs. Lightness" table (as well as for similar table %%HvL%%) the 13 bit signed value in range ''[-4096..+4095]'' maps linearly into a Lightness *relative* (adjustment) range ''[-256..255]'' where ''0'' is no adjustment to pixel luminosity value| | |
| | |
| ===== Media setup ===== | |
| ==== Media output and transform blocks ==== | |
| ^Name ^Offset ^Access ^Bit mapping ^Notes ^ | |
| |:!: Media output modules|''0x30''|R/W|''7'' - res\\ ''6'' - res (headphones)\\ ''5'' - res (UAC)\\ ''4'' - res\\ :!: ''3'' - HDMI video\\ :!: ''2'' - SDI video\\ :!: ''1'' - SFP+ video\\ :!: ''0'' - UVC|Enable/disable state of individual media output modules for video and audio streams| | |
| |:!: Video transform blocks|''0x31''|R/W|:!: ''7'' - sharpness\\ ''6'' - res\\ ''5'' - res\\ ''4'' - res\\ ''3'' - res\\ ''2'' - res\\ :!: ''1'' - CCM\\ :!: ''0'' - (UVC Gamma)|Enable/disable individual transformation blocks in video pipeline| | |
| | |
| ==== Video output formats ==== | |
| ^Name ^Offset ^Access ^Bit mapping ^Notes ^ | |
| |:!: Video output format|''0x32''|R/W|:!: ''7:4'' - UVC\\ :!: ''3:0'' - FPS code for SDI, SFP+, SDI|Bit depth for all video formats is set in register ''0x49''\\ UVC video formats:\\ :!:''0'' - "RAW" greyscale pre-debayer pixels\\ :!:''1'' - 4:4:4 RGB\\ :!:''2'' - (res) packed YCbCr 4:4:4\\ :!:''3'' - packed YCbCr 4:2:2\\ :!:''4'' - (res) packed YCbCr 4:2:0\\ :!:''5'' - (res) planar YCbCr 4:4:4\\ :!:''6'' - (res) planar YCbCr 4:2:2\\ :!:''7'' - planar YCbCr 4:2:0\\ :!:''8-15'' - (res) MJPEG, MPEG-x/H.26x, etc\\ \\ SDI/HDMI and SFP+ video output formats/FPS are always in unison. See [[code:fx3_hvci_and_fpga_i_c_commands#sdi_fps|SDI FPS]] table below for codes. SDI output is always in a packed (not planar) YUV 4:2:2 format| | |
| |:!: Video output pixel bit depth|''0x33''|R/W|:!: ''7:6'' - HDMI\\ :!: ''5:4'' - SDI\\ :!: ''3:2'' - SFP+\\ :!: ''1:0'' - UVC|Pixel bit depths \(d_p\) is calculated from a 2-bit value \(N\) as: \[d_p = (N+4)*2\] Not all values are valid, for example SDI and SFP+ both do not support ''8''-bit output and UVC **only** supports ''8''-bit color depth, at least for now| | |
| | |
| ==== Image sensor config ==== | |
| ^Name ^Offset ^Access ^Bit mapping ^Notes ^ | |
| |:!: Image sensor configuration|''0x34''|R/W|''7:2'' - res\\ :!: ''1:0'' - de-mosaicing strategy|**De-mosaicing strategy** directs the use of a specific implementation of color reconstruction:\\ ''3'', ''2'' - reserved\\ ''1'' - use "branching 5x5", for example the one [[http://www.siliconimaging.com/RGB%20Bayer.htm|described here]]\\ ''0'' - use "branchless 5x5", like the one [[https://www.ipol.im/pub/art/2011/g_mhcd/article.pdf|described in here]]| | |
| |Reserved|''0x35-0x39''| | | | | |
| | |
| ==== Audio setup ==== | |
| ^Name ^Offset ^Access ^Bit mapping ^Notes ^ | |
| |:!: Audio transform blocks|''0x3A''|R/W|''7:0'' - res|Enable/disable individual transformation blocks in audio pipeline| | |
| |Reserved|''0x3B-0x3F''| | | | | |
| | |
| ==== FOURCC formats (for UVC) ==== | |
| A combination of data in ''0x0033[1:0]'' (pixel bit depth) and ''0x32[7:4]'' (video format) used for UVC is mapped into standard FOURCC codes as summarized in the following table: | |
| ^''0x0033[1:0]''\''0x0032[7:4]'' ^''0'' (RAW) ^''1'' (RGB) ^''2'' (packed YUV 4:4:4((ordering is UYV))) ^''3'' (packed YUV 4:2:2((macropixel byte ordering: Y0U0Y1V0))) ^''7'' (planar YUV 4:2:0((chroma plane is a interleaved set of U/V samples))) ^ | |
| ^''0'' (8 bit) |:!: BA81/BYR1/GREY/Y8/Y800 (8bpp)|:!: BI_RGB/RGB (24bpp) |:!: Y444/IYU2 (24bpp) |:!: YUY2/YUYV (16 bpp)|:!: NV12 (12bpp)| | |
| ^''1'' (10 bit) |:!: Y10((need to register with MS)) (16bpp((not 10)))|:!: BI_BITFIELDS (48bpp) |:!: Y410 (32bpp((includes 2 bit alpha at [31:30]))) |:!: Y210 (32bpp)\\ YUVP?/Y42T (24bpp?)|:!: P010 (32bpp) | | |
| ^''2'' (12 bit) |:!: BYR2 (16pbb((not 12)))|:!: BI_BITFIELDS (48bpp) |:!: Y412((need to register with MS)) (40bpp((or 36?))) |:!: Y212((need to register with MS)) (32bpp((or 24?))) |:!: P012((need to register with MS)) (32bpp((or 24, or 18?))) | | |
| ^''3'' (14 bit) |:!: Y16((yes, 16, not 14)) (16bpp((not 14)))|:!: BI_BITFIELDS (48bpp) |:!: Y414((need to register with MS)) (48bpp((or 42?))) |:!: Y214((need to register with MS)) (32bpp((or 28?))) |:!: P014((need to register with MS)) (32bpp((or 21?))) | | |
| | |
| ==== SDI FPS ==== | |
| Below is the table that lists codes for supported SDI FPS settings set in ''0x0032[3:0]'': | |
| ^ FPS ^ code ^ | |
| | 23.98 | 0x01 | | |
| | 24 | 0x02 | | |
| | 25 | 0x04 | | |
| | 29.97 | 0x05 | | |
| | 30 | 0x06 | | |
| | 50 | 0x08 | | |
| | 59.94 | 0x09 | | |
| | 60 | 0x0A | | |
| | 120 | 0x0C | | |
| | |
| ==== SDI pixel clock frequency ==== | |
| ^ Frame geometry ^ MHz ^ FPS ^ | |
| | 1920x1080 | 74.18 | 23.98 | | |
| | ::: | ::: | 29.97 | | |
| | ::: | 74.25 | 24 | | |
| | ::: | ::: | 25 | | |
| | ::: | ::: | 30 | | |
| | ::: | 148.35 | 59.94 | | |
| | ::: | 148.5 | 50 | | |
| | ::: | ::: | 60 | | |
| | ::: | 297 | 120 | | |
| | 3840x2160 | 296.70 | 23.98 | | |
| | ::: | ::: | 29.97 | | |
| | ::: | 297 | 24 | | |
| | ::: | ::: | 25 | | |
| | ::: | ::: | 30 | | |
| | ::: | 593.41 | 59.94 | | |
| | ::: | 594 | 50 | | |
| | ::: | ::: | 60 | | |
| | ::: | 1188 | 120 | | |
| | 7680x4320 | 1188 | 24 | | |
| | ::: | ::: | 30 | | |
| | |
| | |
| | |
| | |
| ===== Green Screen enhancer ===== | |
| //"Green screen", a.k.a. "Chroma Key" on-board optimization is designed to replace a range of colors with a single solid one// | |
| ==== band #0 ==== | |
| If enabled these setting take precedence over other 3 bands | |
| ^Name ^Offset ^Bytes ^Access ^Notes ^ | |
| |:!: CK control|''0x80''| 1 |R/W|''7:1'' reserved\\ ''0'' enable/disable Chroma Key control| | |
| |:!: CK saturation min|''0x81''| 1 |R/W|[0..255] to specify the minimum saturation threshold| | |
| |:!: CK saturation max|''0x82''| 1 |R/W|[0..255] to specify the maximum saturation threshold| | |
| |:!: CK luma min|''0x83''| 1 |R/W|[0..255] to specify the minimum brightness threshold| | |
| |:!: CK luma max|''0x84''| 1 |R/W|[0..255] to specify the maximum brightness threshold| | |
| |:!: CK hue|''0x85''| 2 |R/W|14 bits of a signed hue value, its [-8K..+8K] range is mapped into [-180°..+180°]| | |
| |:!: CK tolerance|''0x86''| 2 |R/W|13 bits of an unsigned hue tolerance value, valid range is [0..8K], which is mapped into [0°..180°].\\ That value specifies how far to stretch the ''CK hue'' value both ways (symmetrically). If the ''CK tolerance'' is above 90° the covered color space is over 50% of values| | |
| |:!: CK red|''0x87''| 2 |R/W| | | |
| |:!: CK green|''0x88''| 2 |R/W| | | |
| |:!: CK blue|''0x89''| 2 |R/W| | | |
| |Reserved|''0x8A-0x8F''| | | | | |
| | |
| ==== band #1 ==== | |
| Color substitution (if enabled) takes place after the first band had a chance to process the pixels | |
| | |
| The layout of the settings is identical to that of band #0 just shifted down by a paragraph and occupying address block ''0x0090-0x009F'' | |
| | |
| ==== band #2 ==== | |
| Color substitution (if enabled) takes place after the first and second bands had a chance to process the pixels | |
| | |
| The layout of the settings is identical to that of band #0 just shifted down by 2 paragraphs and occupying address block ''0x00A0-0x00AF'' | |
| | |
| ==== band #3 ==== | |
| Color substitution (if enabled) takes place after other bands had a chance to process the pixels | |
| | |
| The layout of the settings is identical to that of band #0 just shifted down by 3 paragraphs and occupying address block ''0xB0-0xBF'' | |
| | |
| | |
| ==== mFT lense access ==== | |
| | |
| ^Name ^Offset ^Bytes ^Access ^Notes ^ | |
| |:!: mFT control register|''0xC0''| 1 |R/W|''7:2'' returned data count\\ ''1'': (R/O) command completion flag \\ ''0'': (W/O) setting this to ''1'' sends the command to mFT| | |
| |:!: mFT command setup|''0xC1''| 1 |W/O|See below for the 32-bit command structure| | |
| |:!: mFT returned data|''0xC2''| 1 |R/O|Once the command completion flag is set in control register returned data can be retrieved through this register| | |
| | |
| FPGA "register" ''0xC1'' serves as a service point to access mFT protocol and allows FX3 to communicate with the mFT by forwarding requests and replies between the two. | |
| | |
| Micro Four Thirds System (mFT) defines a communication protocol in which the "body" (host) issues 4 bytes of data (command type, command code, and two 1-byte parameters) and gets a response of variable number of bytes, depending on the command (including a 0-length data response). See the mFT spec for each individual command's request/response. | |
| | |
| FPGA acts as a two-way command repeater and in such capacity it expects a 4 byte command supplied to it as part of an I²C "write" operation and responds with the appropriate return data buffer to a subsequent "read" I²C operation on the same address. | |
| | |
| The 4 bytes of the command are laid out as follows: | |
| <code>struct MftCmd{ | |
| uint8_t cmd_type; | |
| uint8_t cmd_code; | |
| uint8_t param1, param2; | |
| }; | |
| </code> | |
| | |
| or in a more visual format (taking into consideration the little-endian memory layout typical of the architectures we use): | |
| | MSW || LSW || | |
| | MSB | LSB | MSB | LSB | | |
| | param2 | param1 | cmd_code | cmd_type | | |
| | |
| which is the equivalent of this code: | |
| <code>MftCmd cmd{1, 2, 3, 4}; | |
| static_assert(std::bit_cast<uint32_t>(cmd) == 0x04030201); | |
| </code> | |
