User Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
Last revisionBoth sides next revision
code:numberformats [2017/12/09 18:18] – [UFIX 8.8] Igor Yefmovcode:numberformats [2017/12/19 05:21] – [UFIX 8.8] Igor Yefmov
Line 14: Line 14:
 ===== UFIX 8.8 ===== ===== UFIX 8.8 =====
  
-This is a 16-bit value in which 8 MSB represent the unsigned integer part of the number and the 8 LSB represent the fractional part. The fractional value is the ''LSB x 256'', i.e. the smallest "increment" for this format is ''1/256'' or ''0.00390625''.+This is a 16-bit value in which 8 MSB represent the unsigned integer part of the number and the 8 LSB represent the fractional part. The fractional value is the ''LSB x 256'', i.e. the smallest "increment" for this format is ''1/256'' or ''0.00390625''.
  
 Numbers in this format represent a decimal range of ''[0.0..256)''. Numbers in this format represent a decimal range of ''[0.0..256)''.
Line 22: Line 22:
   * ''1.0'' = ''0x01'', ''0x00''   * ''1.0'' = ''0x01'', ''0x00''
   * ''1.5'' = ''0x01'', ''0x80''   * ''1.5'' = ''0x01'', ''0x80''
-  * ''127.3'' = ''0x7F'', ''0x4D'' - notice how the actual precise decimal number ''127 + 77/256'' is not **exactly** ''127.3'' but is an approximation with a precise decimal value of ''127.30078125''+  * ''127.3'' = ''0x7F'', ''0x4D'' - notice how the actual precise decimal number ''127 + 77/256'' is not **exactly** ''127.3'' but is an approximation of the precise decimal value of ''127.30078125''
   * ''-1'' - unrepresentable   * ''-1'' - unrepresentable
   * ''256'' = ''0xFF'', ''0xFF'' - once again, the actual decimal number is "very close" and is in fact ''255.99609375''   * ''256'' = ''0xFF'', ''0xFF'' - once again, the actual decimal number is "very close" and is in fact ''255.99609375''

This website uses cookies. By using the website, you agree with storing cookies on your computer. Also, you acknowledge that you have read and understand our Privacy Policy. If you do not agree, please leave the website.

More information