This is an old revision of the document!
FX3/FPGA API spec
FX3 API specification has been moved to this location
FPGA I²C bridge (registers' map)
The following tables provide information on how to access the camera's functionality for an FPGA I²C bridge.
Here's a sample code (skipping all error checking) that sets the LED to bright-yellow color :
S2R::FX3 dev; // auto-open device #0 using S2R::FX3; dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 255, 0x08); // red dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 255, 0x0A); // green dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 0, 0x0C); // blue
The address space is broken down into smaller chunks, grouped by common functionality:
FPGA basic status and controls
Name | Offset | Bytes | Access | Bit mapping | Notes |
---|---|---|---|---|---|
FPGA Version | 0x00 | 4 | R/O | see Firmware Version Info | |
FPGA status | 0x01 | 1 | R/O | 7:2 reserved | |
1 DPC calibration done | 1 indicates that DPC calibration is completed |
||||
0 SFP+ | 1 indicates there is an active device in SFP+ cage |
||||
FPGA control | 0x02 | 1 | R/W | 7:2 reserved | |
1 enable DPC | enable DPC correction | ||||
0 FPGA config enable | if set to 1 , the GPIF becomes read only and waits for an FPGA update bitstream |
||||
FPGA config status | 0x03 | 2 | R/O | see SPI codes for details | |
FPGA core temperature | 0x04 | 1 | R/O | 7:0 - temperature in Farenheits | Reading of the FPGA's internal temperature sensor in degrees of Farenheits |
LED RED | 0x05 | 2 | R/W | LED's red intensity in range \([0..65535]\) | |
LED GREEN | 0x06 | 2 | R/W | LED's green intensity in range \([0..65535]\) | |
LED BLUE | 0x07 | 2 | R/W | LED's blue intensity in range \([0..65535]\) |
AWB
Info for performing Auto White-Balance adjustments on the image
Name | Offset | Bytes | Access | Notes |
---|---|---|---|---|
AWB Red adjustment | 0x08 | 2 | R/W | a signed 16-bit value of an adjustment to apply to every pixel in Red channel. Default is \(0\) |
AWB Green adjustment | 0x09 | 2 | R/W | a signed 16-bit value of an adjustment to apply to every pixel in Green channel. Default is \(0\) |
AWB Blue adjustment | 0x0A | 2 | R/W | a signed 16-bit value of an adjustment to apply to every pixel in Blue channel. Default is \(0\) |
Reserved | 0x0B | |||
AWB Red total | 0x0C | 4 | R/O | |
AWB Green total | 0x0D | 4 | R/O | |
AWB Blue total | 0x0E | 4 | R/O | |
AWB count total | 0x0F | 4 | R/O | A count of pixels that were used to calculate the \(R/G/B\) totals |
Basic UVC controls
Name | Offset | Bytes | Access | Range | Range description | Neutral value |
---|---|---|---|---|---|---|
Brightness | 0x10 | 2 | R/W | \([-1024..1023]\) | \(-1024\) makes the image very dark \(1023\) makes the image very bright | \(0\) |
Contrast | 0x11 | 2 | R/W | \([1..2047]\) | \(1\) turns image into grayscale \(2047\) makes all pixels either black or white | \(1023\) |
Saturation | 0x12 | 2 | R/W | \([0..900]\) | \([0\%..900\%]\) or grayscale to 9x | \(100\) |
Sharpness | 0x13 | 1 | R/W | \([0..255]\) | \(0\) | |
Gamma | 0x14 | 1 | R/W | \([0..15]\) | \(0\) | |
Hue | 0x15 | 2 | R/W | \([-8192..8191]\) | \([-180°..180°)\) | \(0\) |
Reserved | 0x16-0x1F |
General image stats and adjustments
Defective pixel cancellation | ||||
---|---|---|---|---|
Name | Offset | Bytes | Access | Notes |
DPC Threshold | 0x20 | 2 | R/W | A write into this register triggers the start of DPC calibration |
DPC count | 0x21 | 2 | R/O | Once the DPC calibration is done the 16-bit value is stored in here |
General image stats | ||||
flags | 0x22 | 1 | R/O | 7:2 reserved1 - indicates whether a red overexposure is detected0 - set if there is a general overexposure detected |
FPS | 0x23 | 2 | R/O | 16-bit unsigned value representing number of 10μs units between frame start signals from the image sensor, e.g. a value of \(1000\) means it took 10ms between 2 frame start signals, which corresponds to 100FPS |
Y average | 0x24 | 2 | R/O | Average “brightness” value |
Color Correction Matrix (a.k.a. CCM or CMX)
See Color correction matrix article in this Wiki's ISP section for more details. The 16-bit (MSB-LSB) value is defined as 7+9 bits, where MSB[7:1] are the integer part and MSB[0]LSB[7:0] is the fractional part (effectively that value is 512 times larger than the original fractional part)
Name | Offset | Bytes | Access | Notes |
---|---|---|---|---|
CCM_00 | 0x25 | 2 | R/W | \(CCM_{00}\) |
CCM_01 | 0x26 | 2 | R/W | \(CCM_{01}\) |
CCM_02 | 0x27 | 2 | R/W | \(CCM_{02}\) |
CCM_10 | 0x28 | 2 | R/W | \(CCM_{10}\) |
CCM_11 | 0x29 | 2 | R/W | \(CCM_{11}\) |
CCM_12 | 0x2A | 2 | R/W | \(CCM_{12}\) |
CCM_20 | 0x2B | 2 | R/W | \(CCM_{20}\) |
CCM_21 | 0x2C | 2 | R/W | \(CCM_{21}\) |
CCM_22 | 0x2D | 2 | R/W | \(CCM_{22}\) |
Color grading
Name | Offset | Access | Bit mapping | Notes |
---|---|---|---|---|
switch | 0x2E | W | 15:10 reserved | Controls what information is being read/written by accessing the register 0x002F |
9:7 table switch | 000 - Hue vs. Hue (14 bits)001 - Hue vs. Saturation (12 bits )010 - Lightness vs. Saturation (12 bits)011 - Saturation vs. Saturation (12 bits)100 - Lightness vs. Lightness (12 bits)101 - Hue vs. Lightness (12 bits)110-111 - reserved |
|||
6:1 index LSB | index into a page in the table | |||
0 access mode | 0 : “normal mode”, in which all the subsequent accesses to the register 0x002F are governed by the values in 0x002E 1 : “bulk access”, where after a read or write access to register 0x002F the “Index” value will auto-increment by one so that the next read/wrie will access the subsequent table slot |
|||
Value | 0x2F | R/W | 15:0 value | 16 bits of either signed or unsigned integer value - for a “Hue vs. Hue” table the 14 bits signed value is in range [-8192..+8192] which maps linearly into a Hue angle range -180°..+180° - for a “Hue vs. Saturation” table (as well as for similar tables LvS and SvS) the 12 bit unsigned value in range [0..+1280] maps linearly into a Saturation range [0%..1000%] where 100% is the neutral position and 0% produces a greyscale image- (until FPGA v.72) for a “Lightness vs. Lightness” table (as well as for similar table HvL) the 12 bit unsigned value in range [0..+4095] maps linearly into a Lightness absolute range [0..255] where 0 is pitch black and 255 is the maximum possible pixel luminosity value- (starting with FPGA v.73) for a “Lightness vs. Lightness” table (as well as for similar table HvL) the 13 bit signed value in range [-4096..+4095] maps linearly into a Lightness *relative* (adjustment) range [-256..255] where 0 is no adjustment to pixel luminosity value |
Media setup
Media output and transform blocks
Name | Offset | Access | Bit mapping | Notes |
---|---|---|---|---|
Media output modules | 0x30 | R/W | 7 - res6 - res (headphones)5 - res (UAC)4 - res3 - HDMI video2 - SDI video1 - SFP+ video0 - UVC | Enable/disable state of individual media output modules for video and audio streams |
Video transform blocks | 0x31 | R/W | 7 - sharpness6 - res5 - res4 - res3 - res2 - res1 - CCM0 - (UVC Gamma) | Enable/disable individual transformation blocks in video pipeline |
Video output formats
Name | Offset | Access | Bit mapping | Notes |
---|---|---|---|---|
Video output format | 0x32 | R/W | 7:4 - UVC3:0 - FPS code for SDI, SFP+, SDI | Bit depth for all video formats is set in register 0x49 UVC video formats: 0 - “RAW” greyscale pre-debayer pixels1 - 4:4:4 RGB2 - (res) packed YCbCr 4:4:43 - packed YCbCr 4:2:24 - (res) packed YCbCr 4:2:05 - (res) planar YCbCr 4:4:46 - (res) planar YCbCr 4:2:27 - planar YCbCr 4:2:08-15 - (res) MJPEG, MPEG-x/H.26x, etcSDI/HDMI and SFP+ video output formats/FPS are always in unison. See SDI FPS table below for codes. SDI output is always in a packed (not planar) YUV 4:2:2 format |
Video output pixel bit depth | 0x33 | R/W | 7:6 - HDMI5:4 - SDI3:2 - SFP+1:0 - UVC | Pixel bit depths \(d_p\) is calculated from a 2-bit value \(N\) as: \[d_p = (N+4)*2\] Not all values are valid, for example SDI and SFP+ both do not support 8 -bit output and UVC only supports 8 -bit color depth, at least for now |
Image sensor config
Name | Offset | Access | Bit mapping | Notes |
---|---|---|---|---|
Image sensor configuration | 0x34 | R/W | 7:2 - res1:0 - de-mosaicing strategy | De-mosaicing strategy directs the use of a specific implementation of color reconstruction:3 , 2 - reserved1 - use “branching 5×5”, for example the one described here0 - use “branchless 5×5”, like the one described in here |
Reserved | 0x35-0x39 |
Audio setup
Name | Offset | Access | Bit mapping | Notes |
---|---|---|---|---|
Audio transform blocks | 0x3A | R/W | 7:0 - res | Enable/disable individual transformation blocks in audio pipeline |
Reserved | 0x3B-0x3F |
FOURCC formats (for UVC)
A combination of data in 0x0033[1:0]
(pixel bit depth) and 0x32[7:4]
(video format) used for UVC is mapped into standard FOURCC codes as summarized in the following table:
0x0033[1:0] \0x0032[7:4] | 0 (RAW) | 1 (RGB) | 2 (packed YUV 4:4:41)) | 3 (packed YUV 4:2:22)) | 7 (planar YUV 4:2:03)) |
---|---|---|---|---|---|
0 (8 bit) | BA81/BYR1/GREY/Y8/Y800 (8bpp) | BI_RGB/RGB (24bpp) | Y444/IYU2 (24bpp) | YUY2/YUYV (16 bpp) | NV12 (12bpp) |
1 (10 bit) | Y104) (16bpp5)) | BI_BITFIELDS (48bpp) | Y410 (32bpp6)) | Y210 (32bpp) YUVP?/Y42T (24bpp?) | P010 (32bpp) |
2 (12 bit) | BYR2 (16pbb7)) | BI_BITFIELDS (48bpp) | Y4128) (40bpp9)) | Y21210) (32bpp11)) | P01212) (32bpp13)) |
3 (14 bit) | Y1614) (16bpp15)) | BI_BITFIELDS (48bpp) | Y41416) (48bpp17)) | Y21418) (32bpp19)) | P01420) (32bpp21)) |
SDI FPS
Below is the table that lists codes for supported SDI FPS settings set in 0x0032[3:0]
:
FPS | code |
---|---|
23.98 | 0x01 |
24 | 0x02 |
25 | 0x04 |
29.97 | 0x05 |
30 | 0x06 |
50 | 0x08 |
59.94 | 0x09 |
60 | 0x0A |
120 | 0x0C |
SDI pixel clock frequency
Frame geometry | MHz | FPS |
---|---|---|
1920×1080 | 74.18 | 23.98 |
29.97 | ||
74.25 | 24 | |
25 | ||
30 | ||
148.35 | 59.94 | |
148.5 | 50 | |
60 | ||
297 | 120 | |
3840×2160 | 296.70 | 23.98 |
29.97 | ||
297 | 24 | |
25 | ||
30 | ||
593.41 | 59.94 | |
594 | 50 | |
60 | ||
1188 | 120 | |
7680×4320 | 1188 | 24 |
30 |
Green Screen enhancer
“Green screen”, a.k.a. “Chroma Key” on-board optimization is designed to replace a range of colors with a single solid one
band #0
If enabled these setting take precedence over other 3 bands
Name | Offset | Bytes | Access | Notes |
---|---|---|---|---|
CK control | 0x80 | 1 | R/W | 7:1 reserved0 enable/disable Chroma Key control |
CK saturation min | 0x81 | 1 | R/W | [0..255] to specify the minimum saturation threshold |
CK saturation max | 0x82 | 1 | R/W | [0..255] to specify the maximum saturation threshold |
CK luma min | 0x83 | 1 | R/W | [0..255] to specify the minimum brightness threshold |
CK luma max | 0x84 | 1 | R/W | [0..255] to specify the maximum brightness threshold |
CK hue | 0x85 | 2 | R/W | 14 bits of a signed hue value, its [-8K..+8K] range is mapped into [-180°..+180°] |
CK tolerance | 0x86 | 2 | R/W | 13 bits of an unsigned hue tolerance value, valid range is [0..8K], which is mapped into [0°..180°]. That value specifies how far to stretch the CK hue value both ways (symmetrically). If the CK tolerance is above 90° the covered color space is over 50% of values |
CK red | 0x87 | 2 | R/W | |
CK green | 0x88 | 2 | R/W | |
CK blue | 0x89 | 2 | R/W | |
Reserved | 0x8A-0x8F |
band #1
Color substitution (if enabled) takes place after the first band had a chance to process the pixels
The layout of the settings is identical to that of band #0 just shifted down by a paragraph and occupying address block 0x0090-0x009F
band #2
Color substitution (if enabled) takes place after the first and second bands had a chance to process the pixels
The layout of the settings is identical to that of band #0 just shifted down by 2 paragraphs and occupying address block 0x00A0-0x00AF
band #3
Color substitution (if enabled) takes place after other bands had a chance to process the pixels
The layout of the settings is identical to that of band #0 just shifted down by 3 paragraphs and occupying address block 0xB0-0xBF
mFT lense access
Name | Offset | Bytes | Access | Notes |
---|---|---|---|---|
mFT control register | 0xC0 | 1 | R/W | 7:2 returned data count1 : (R/O) command completion flag 0 : (W/O) setting this to 1 sends the command to mFT |
mFT command setup | 0xC1 | 1 | W/O | See below for the 32-bit command structure |
mFT returned data | 0xC2 | 1 | R/O | Once the command completion flag is set in control register returned data can be retrieved through this register |
FPGA “register” 0xC1
serves as a service point to access mFT protocol and allows FX3 to communicate with the mFT by forwarding requests and replies between the two.
Micro Four Thirds System (mFT) defines a communication protocol in which the “body” (host) issues 4 bytes of data (command type, command code, and two 1-byte parameters) and gets a response of variable number of bytes, depending on the command (including a 0-length data response). See the mFT spec for each individual command's request/response.
FPGA acts as a two-way command repeater and in such capacity it expects a 4 byte command supplied to it as part of an I²C “write” operation and responds with the appropriate return data buffer to a subsequent “read” I²C operation on the same address.
The 4 bytes of the command are laid out as follows:
struct MftCmd{ uint8_t cmd_type; uint8_t cmd_code; uint8_t param1, param2; };
or in a more visual format (taking into consideration the little-endian memory layout typical of the architectures we use):
MSW | LSW | ||
MSB | LSB | MSB | LSB |
param2 | param1 | cmd_code | cmd_type |
which is the equivalent of this code:
MftCmd cmd{1, 2, 3, 4}; static_assert(std::bit_cast<uint32_t>(cmd) == 0x04030201);