This is an old revision of the document!

Preface

SUB2r camera is built on a Cypress FX3 chipset that facilitates the Super-Speed USB 3.0+ communication between the device and a host system. Every component of the camera, be it an FPGA or an image sensor, receives user commands via that Cypress FX3.

On Windows the device is registered with a GUID {36FC9E60-C465-11CF-8056-444553540000} and if you are not planning on using the SUB2r-lib for your development - that would be the GUID to search for to properly connect to the command channel.

Whether you use SUB2r-lib or not you need to install the provided driver for the OS to properly configure the device in order to be able to connect to its control endpoints.

Here's a sample code that shows how to send commands to both the FX3 Host and to FPGA I²C bridge. The code lacks error checking (for clarity) and this should go without saying that if you copy-paste it into your code you must add error handling :)

// set a new auto-functions' update interval to 2x the default
// just issue the command directly to FX3's vendor request interface
void setUUInterval(){
    S2R::I2C fx3;
    fx3.open(0);
    fx3.vrCmd(0xDF, S2R::FX3::write, 6000, 0);
}
 
// run DPC calibration - also just a straight-up vendor request command to FX3
void runDPC(UCHAR _threshold = 240){
    S2R::I2C fx3;
    fx3.open(0);
    fx3.vrCmd(S2R::FX3::calibrate_dpc, S2R::FX3::write, _threshold, 0);
}
 
// increase LED's green brightness by 25%
// utilize the FPGA's I²C bridge
void lightUpTheGreen()
{
    using Cmd = S2R::FX3::Fx3Cmd;
    using OpType = S2R::FX3::VrCmdOpType;
 
    S2R::FX3 fx3;   // `S2R::I2C fx3;` works as well
    fx3.open(0);
    uint8_t        buf[1]{0};
    const uint16_t clrChannel{0x0A};    // LED green
    fx3.vrCmd(Cmd::i2c_bridge, OpType::read, 0, clrChannel, buf, 1);
    buf[0] += buf[0] / 4;  // yes, this can totally overflow
    fx3.vrCmd(Cmd::i2c_bridge, OpType::write, buf[0], clrChannel);
}

FX3 Host Vendor Command Reference

The following tables provide information on how to access the camera's functionality for FX3 Host Vendor Command Interface. The address space is split into smaller chunks, grouped by common functionality:

0x00-0x9F

Name	Offset	wIndex	wValue	Access type	Byte length	Return buffer bits	Notes
Reserved	`0x00`-`0x9F`						Think of this as “system address space”

0xA0-0xA7

Name	Offset	wIndex	wValue	Access type	Byte length	Return buffer bits	Notes
Bootloader check	`0xA0`			R/W			Check if a bootloader is running, the result is in the command's status code (success/failure interpreted as `true`/`false`)
Reserved	`0xA1`
Run DPC calibration	`0xA2`		DPC Threshold	W/O			Start the dynamic DPC calibration with the given DPC Threshold in range [0..255]
Reconfig FGPA	`0xA3`			W/O			Writing anything into this register causes the FPGA to reconfigure itself from SPI Flash
FPGA I²C Bridge	`0xA4`	FPGA register offset	FPGA data (write)	W/O	`0` or `1`	`7:0` - FPGA data	FPGA write returns 0 byte buffer, FPGA read returns 1 byte buffer. Read/write is requested via control endpoint's direction attribute being set to `DIR_FROM_DEVICE`/`DIR_TO_DEVICE`. For more details refer to FPGA I²C bridge
Sensor I²C bridge	`0xA5`			R/W	1
Reserved for future I²C bridge	`0xA6`
RAW Mode Select	`0xA7`			R/W	1	`7:1` - Reserved `0` - RAW Mode	Select RAW Mode ('1') or Processed Video ('0')

0xA8-0xAF - versioning and reprogramming

Name	Offset	Access type	Byte length	Return buffer bits	Notes
Reserved	`0xA8`-`0xA9`
Erase SPI Flash	`0xAA`	W/O			Any write to this location invalidates the FX3 SPI Flash and causes the FX3 to reset itself to a bootloader mode for reprogramming
FX3 version	`0xAB`	R/O	`4`	`31:29` Vendor ID `28:24` HW_CFG_ID `23:16` Product ID `15:11` Release type `10:0` Build number	Get detailed version information of the FX3, for more details refer to FX3 Version Info
FPGA version	`0xAC`	R/O	`4`	`31:29` Vendor ID `28:24` HW_CFG_ID `23:16` Product ID `15:11` Release type `10:0` Build number	Get detailed version information of the FPGA, for more details refer to FPGA Version Info
FPGA config. ctrl.	`0xAD`	W/O			Any write to this location will put the FPGA into configuration mode
FPGA config. status - SPI codes	`0xAE`	R/O	`2`	see below for details	Retrieve detailed status of the FPGA configuration operation
SPI Flash write enable	`0xAF`	W/O			Reconfigure the FX3 IOMatrix to disable GPIF and enable SPI

FPGA config status - SPI codes

Bit name	Description
`15` Program SwitchWord OK
`14` Verify OK	Verification succeeded
`13` Program OK	Programming completed successfully
`12` Erase OK	SPI erase was successful
`11` Erase SwitchWord OK
`10` Check ID OK
`9` Initialize OK
`8` Config started	Config operation has started
`7` CRC error
`6` Timeout error
`5` Program error	Error while programming the SPI
`4` Erase error	Encountered an error while erasing SPI
`3` IdCode error
`2` Config error	Configuration operation errored out
`1` Config done	Configuration operation is complete
`0` Config not busy	Set to `1` while the config is not busy

0xB0-0xCF

Name	Offset	wIndex	wValue	Access type	Byte length	Return buffer bits	Notes
Reserved	`0xB0`-`0xCF`

0xD0-0xD7 - Auto exposure configuration

Name	Offset	wValue	Access type	Byte length	Return buffer bits	Notes
AE Setpoint	`0xD0`	AE Setpoint	R/W	`0` or `1`	`7:0` AE Setpoint	Target Auto Exposure Luminance Setpoint [0..255] Default: `105` This controls the “average luminance” of the whole frame that we are trying to achieve. The higher the value the brighter the result is going to be
AE Hysteresis	`0xD1`	AE Hysteresis	R/W	`0` or `2`	`15:0` AE Hysteresis	Auto Exposure Hysteresis Value UFIX 8.8 Default `3.0` This controls how far can we diverge from the set target luminance before we begin the correction. In other words the higher this value the further we allow the luminance to drift away from the target before correcting it
AE Error Tolerance	`0xD2`	AE Err Tol	R/W	`0` or `2`	`15:0` AE Err Tol	Auto Exposure Error Tolerance Value UFIX 8.8 Default `1.0` Specifies the “close enough” tolerance at which point the correction can be stopped. Normally this value is (at least somewhat) lower than the AE Hysteresis
AE Exposure Scaling	`0xD3`	AE Exp Scale	R/W	`0` or `2`	`15:0` AE Exp Scale	Auto Exposure Exposure Scaling Value UFIX 8.8 Default `100.0` Controls the speed (stepping) at which the correction is happening. Higher values will result in large brightness jumps and a value too high may cause an oscillation while a value that is too low will cause the correction process to be too slow and seamingly unresponsive
AE C Gain Divisor	`0xD4`	AE C Gain Divisor	R/W	`0` or `2`	`15:0` AE C Gain Divisor	Auto Exposure C Gain Divisor Value UFIX 8.8 Default `4.0` Value between 0 and 255 that is inversely proportional to the rate at which the C gain is adjusted in response to exposure errors. (i.e. the larger the value, the slower C gain will adjust)
Reserved	`0xD5`-`0xD7`

0xD8-0xDE - Auto white balance configuration

Name	Offset	wValue	Access type	Byte length	Return buffer bits	Notes
AWB Setpoint	`0xD8`	AWB Setpoint	R/W	`0` or `2`	`15:0` AWB Setpoint	Auto White Balance G Gain Setpoint [0..2047] Default `1024` The pinned value for Green Gain that is used as the basis for the rest of the white balance adjustments. Only change this value if you need to make your picture brighter and you have exhausted both the Exposure and the Global Gain options
AWB Hysteresis	`0xD9`	AWB Hysteresis	R/W	`0` or `2`	`15:0` AWB Hysteresis	Auto White Balance Hysteresis Value UFIX 8.8 Default `3.0` How far can the error drift before we start adjusting it
AWB Error Tolerance	`0xDA`	AWB Err Tol	R/W	`0` or `2`	`15:0` AWB Err Tol	Auto White Balance Error Tolerance Value UFIX 8.8 Default `1.0` A.k.a. “good enough” approximation - controls when we stop the correction process, having achieved a “close enough” result. Generally this setting is at least somewhat lower than the AWB Hysteresis value
AWB Adjustment Scaling	`0xDB`	AWB Adj Scale	R/W	`0` or `2`	`15:0` AWB Adj Scale	Auto White Balance Adjustment Scaling Value UFIX 8.8 Default `4.0` Value between `0` and `255` that is inversely proportional to the rate at which the R and B gains are adjusted in response to white balance errors. (i.e. the larger the value, the slower R and B gains will adjust)
Reserved	`0xDC`-`0xDE`

0xDF - Auto-functions' timing

Name	Offset	wIndex	wValue	Access type	Byte length	Return buffer bits	Notes
Auto Update Period	`0xDF`		Auto Update Period	R/W	`0` or `2`	`15:0` Auto Update Period	Auto Update Period [0..65535]) Default `3000` Determines how long we wait before trying to apply a new update for both exposure and white balance (when auto functions are enabled). This is an asynchronous update rate (i.e. not an absolute time, but larger values should mean slower updates)

0xE0-0xFF

Name	Offset	wIndex	wValue	Access type	Byte length	Return buffer bits	Notes
FX3 reset	`0xE0`			R/W			Cypress vendor command to soft reset FX3
Reserved	`0xE1`-`0xFF`

FPGA I²C bridge

The following tables provide information on how to access the camera's functionality for an FPGA I²C bridge.

Here's a sample code (skipping all error checking) that sets the LED to bright-yellow color:

S2R::FX3 dev; // auto-open device #0
using S2R::FX3;
dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 255, 0x08); // red
dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 255, 0x0A); // green
dev.vrCmd(FX3Cmd::i2c_bridge, VrCmdOpType::write, 0, 0x0C);   // blue

The address space is broken down into smaller chunks, grouped by common functionality:

0x00-0x07 - FPGA general access

Name	Offset	Access	Bit mapping	Notes
FPGA Version #0	`0x00`	R/O	`7:5` Vendor ID `4:0` HW_CFG_ID
FPGA Version #1	`0x01`	R/O	`7:0` Product ID
FPGA Version #2	`0x02`	R/O	`7:3` Release type `2:0` Build number MSB	Build number is split into 2 MSB and 8 LSB for a combined total width of 10 bits
FPGA Version #3	`0x03`	R/O	`7:0` Build number LSB
FPGA config status #0	`0x04`	R/O	see SPI codes for details	LSB (bits `7`-`0`) of the FPGA config status
FPGA config status #1	`0x05`	R/O	see SPI codes for details	MSB (bits `15`-`8`) of the FPGA config status
FPGA control	`0x06`	R/W		Global control of the FPGA's functionality
			`7` FPGA config enable	If bit `7` is set, the GPIF becomes read only and waits for an update bitstream
			`6` Reserved
			`5` RAW Mode	Setting bit `5` and clearing bit `0x06::2` will enable RAW mode output (4K Only)
			`4` Video Format	Bit `4` selects between NV12 ('1') and YUY2 ('0') output formats
			`3` DPC enable	Setting bit `3` along with bit `0x06::0` will perform a DPC correction
			`2` HSUB enable	Bit `2` is to enable/disable Horizontal Subsampling
			`1` Audio enable	Bit `1` is to enable/disable on-board microphones
			`0` Video enable	Bit `0` enables/disables video streaming
FPGA core temperature	`0x07`	R/O	`7:0` - temperature in Farenheits	Reading of the FPGA's internal temperature sensor in degrees of Farenheits

0x08-0x0D - LED

Name	Offset	Access	Bit mapping	Notes
LED_RED_L	`0x08`	R/W	`7:0` Red LED	LSB of LED's red intensity in range [0..255]
LED_RED_H	`0x09`	R/W	Reserved
LED_GREEN_L	`0x0A`	R/W	`7:0` Green LED	LSB of LED's green intensity in range [0..255]
LED_GREEN_H	`0x0B`	R/W	Reserved
LED_BLUE_L	`0x0C`	R/W	`7:0` Blue LED	LSB of LED's blue intensity in range [0..255]
LED_BLUE_H	`0x0D`	R/W	Reserved

0x0E-0x0F - Noise Reduction

Name	Offset	Access	Bit mapping	Notes
NR Control	`0x0E`	R/W	`7:1` Reserved
NR Control	`0x0E`	R/W	`0` NR Enable	'1' ⇒ NR enabled, '0' ⇒ NR disabled
NR Threshold	`0x0F`	R/W	`7:0` NR Threshold	Noise level threshold in range [0..255]

0x10-0x1F - Basic UVC controls

Name	Offset	Access	Bit mapping	Notes
Brightness_L	`0x10`	R/W	`7:0` LSB	16 bits of brightness are split into 8 bits of LSB and MSB
Brightness_H	`0x11`	R/W	`7:0` MSB	16 bits of brightness are split into 8 bits of LSB and MSB
Contrast_L	`0x12`	R/W	`7:0` LSB	16 bits of contrast are split into 8 bits of LSB and MSB
Contrast_H	`0x13`	R/W	`7:0` MSB	16 bits of contrast are split into 8 bits of LSB and MSB
Saturation_L	`0x14`	R/W	`7:0` LSB	16 bits of saturation are split into 8 bits of LSB and MSB
Saturation_H	`0x15`	R/W	`7:0` MSB	16 bits of saturation are split into 8 bits of LSB and MSB
Sharpness_L	`0x16`	R/W	`7:0` LSB	16 bits of sharpness are split into 8 bits of LSB and MSB
Sharpness_H	`0x17`	R/W	`7:0` MSB	16 bits of sharpness are split into 8 bits of LSB and MSB
Gamma_L	`0x18`	R/W	`7:0` LSB	16 bits of gamma are split into 8 bits of LSB and MSB
Gamma_H	`0x19`	R/W	`7:0` MSB	16 bits of gamma are split into 8 bits of LSB and MSB
Hue_L	`0x1A`	R/W	`7:0` LSB	16 bits of hue are split into 8 bits of LSB and MSB
Hue_H	`0x1B`	R/W	`7:0` MSB	16 bits of hue are split into 8 bits of LSB and MSB
Reserved	`0x1C`-`0x1F`

0x20-0x27 - Defective pixel cancellation

Name	Offset	Access	Bit mapping	Notes
DPC Threshold LSB	`0x20`	R/W	`7:0` LSB	16 bit of DPC (defective pixel cancellation) Threshold are split into 8 bits of LSB and MSB
DPC Threshold MSB	`0x21`	R/W	`7:0` MSB
DPC count LSB	`0x22`	R/O	`7:0` LSB	Once the DPC calibration is done the 16-bit value is stored in these 2 registers
DPC count MSB	`0x23`	R/O	`7:0` MSB
Reserved	`0x24`-`0x27`

0x28-0x2F - General image statistics

Name	Offset	Access	Bit mapping	Notes
Y average	`0x28`	R/O	`7:0` value
~~U average~~	~~`0x29`~~	~~R/O~~
~~V average~~	~~`0x2A`~~	~~R/O~~
R average	`0x29`	R/O	`7:0` value	an average RGB value
G average	`0x2A`	R/O	`7:0` value
B average	`0x2B`	R/O	`7:0` value
Reserved	`0x2C`-`0x2F`

0x30-0x3F - GPIO

Name	Offset	Access	Bit mapping
CAM_GPIO_L	`0x30`	R/W	reserved
CAM_GPIO_H	`0x31`	R/W	reserved
CAM_GPIO_DIR_L	`0x32`	R/W	reserved
CAM_GPIO_DIR_H	`0x33`	R/W	reserved
CAM_GPIO_OE_L	`0x34`	R/W	reserved
CAM_GPIO_OE_H	`0x35`	R/W	reserved
Reserved	`0x36`-`0x3F`

0x40-0x7F - On-board compression

Name	Offset	Access	Bit mapping	Notes
H.264 QP	`0x40`	R/W	`7:6` reserved `5:0` value	Valid range for the H.264 QP value is [0..51]
Reserved	`0x41`-`0x7F`

0x80-0xBF - Green Screen enhancer

“Green screen”, a.k.a. “Chroma Key” on-board optimization is designed to replace a range of colors with a single solid one

0x80-0x8F - band #0

If enabled these setting take precedence over other 3 bands

Name	Offset	Access	Bit mapping	Notes
CK control	`0x80`	R/W	`7:1` reserved `0` enable	enable/disable Chroma Key control
CK status	`0x81`		reserved
CK saturation min	`0x82`	R/W	`7:0` sat. min	[0..255] to specify the minimum saturation threshold
CK saturation max	`0x83`	R/W	`7:0` sat. max	[0..255] to specify the maximum saturation threshold
CK luma min	`0x84`	R/W	`7:0` luma min	[0..255] to specify the minimum brightness threshold
CK luma max	`0x85`	R/W	`7:0` luma max	[0..255] to specify the maximum brightness threshold
CK hue LSB	`0x86`	R/W	`7:0` hue LSB	14 bits of a signed hue value are split into 8 LSB and 6 MSB, its [-8K..+8K] range is mapped into [-180°..+180°]
CK hue MSB	`0x87`	R/W	`7:6` reserved `5:0` hue MSB
CK tolerance LSB	`0x88`	R/W	`7:0` tolerance LSB	13 bits of an unsigned hue tolerance value are split into 8 LSB and 5 MSB, valid range is [0..8K], which is mapped into [0°..180°]. That value specifies how far to stretch the `CK hue` value both ways (symmetrically). If the `CK tolerance` is above 90° the covered color space is over 50% of values
CK tolerance MSB	`0x89`	R/W	`7:5` reserved `4:0` tolerance MSB
CK red LSB	`0x8A`	R/W	`7:0` red LSB
CK red MSB	`0x8B`	R/W	reserved
CK green LSB	`0x8C`	R/W	`7:0` green LSB
CK green MSB	`0x8D`	R/W	reserved
CK blue LSB	`0x8E`	R/W	`7:0` blue LSB
CK blue MSB	`0x8F`	R/W	reserved

0x90-0x9F - band #1

Color substitution (if enabled) takes place after the first band had a chance to process the pixels

The layout of the settings is identical to that of band #0 just shifted down by a paragraph and occupying address block 0x90-0x9F

0xA0-0xAF - band #2

Color substitution (if enabled) takes place after the first and second bands had a chance to process the pixels

The layout of the settings is identical to that of band #0 just shifted down by 2 paragraphs and occupying address block 0xA0-0xAF

0xB0-0xBF - band #3

Color substitution (if enabled) takes place after other bands had a chance to process the pixels

The layout of the settings is identical to that of band #0 just shifted down by 3 paragraphs and occupying address block 0xB0-0xBF

0xC0-0xCF - Color grading

Name	Offset	Access	Bit mapping	Notes
switch	`0xC0`	W		Controls what information is being read/written by accessing the next set of registers (`0xC2..0xC3`)
			`7:5` table switch	`111-011` - reserved `010` - Hue vs. Brightness `001` - Hue vs. Saturation `000` - Hue vs. Hue
			`4:1` index MSB	4 MSB of the 12-bit index into the “Hue vs. X” table
			`0` access mode	the only valid value right now is `0`, which is “normal mode”, in which all the subsequent access to the registers in this API block are governed by the values in `0xC0` and `0xC1` `1` would be used for “bulk access” where after a read or write access the register `0xC3` the “Index” value will auto-increment by one so that the next pair will access the subsequent table slot
Index LSB	`0xC1`	W	`7:0` index LSB	8 LSB bits of the 12-bit index into the “Hue vs. X” table
Value L	`0xC2`	R/W	`7:0` LSB	16 bits split into 8 LSB and 8 MSB for a “Hue vs. Hue” table the 14 bits signed value is in range `[-8192..+8192]` which maps linearly into a Hue angle range `-180°..+180°` for a “Hue vs. Saturation” table the 12 bit signed value in range `[-128..+1152]` maps linearly into a Saturation range `[-100%..+900%]` where `0%` is the neutral position and `-100%` produces a greyscale image
Value H	`0xC3`	R/W	`7:0` MSB
Reserved	`0xC4`-`0xCF`

0xD0-0xFF - Reserved

This is where future API will land. Stay tuned

FX3 Version Info

The version id is also encoded into the firmware image file name as:

<VendorID>_<HardwareID>_<ProductID>_<ReleaseType>_<BuildNumber>

Vendor ID

Code	Value
1	Cypress

Hardware ID

Code	Value
1	FX3

Product ID

Code	Value
1	reserved for Gen 1 camera, a.k.a. “Moon landing”
2	reserved for Gen 2 camera, a.k.a. “Piggy”
3	Gen 3 camera (Alpha), a.k.a. “Frankie” (from Frankenstein's Monster)
4	Gen 3 camera, Production

Release type

Code	Name	Meaning
0	Private build	Private build for debugging and similar purposes
1	Alpha	feature-incomplete early development cycle “somewhat stable” build
2	Beta	feature-complete, but not very stable build (lots of bugs)
3	Evaluation	Tech preview
4	Release candidate	feature complete and stable
5	Release	general availability
6	Backport	backport of a feature from next gen camera
7	Emergency bug fix	a critical post-release bugfix

Build number

Code	Value
#	Increments on each build

FPGA Version Info