issue with single precision

[jaskin]

Can someone please explain what's going on here? I see "a = 5.1" printed but not "b = 5.1" when I run the following code.

Code: Select all

dim a as double
dim b as single
a = 5.1
b = 5.1
print a,b
if a = 5.1 then print "a = 5.1"
if b = 5.1 then print "b = 5.1"
sleep

[Munair]

b is of type single which is a less precise floating point data type. The actual value represented by b might just be 5.09998.

In general, floating points are the closest binary approximations of the actual values.

[paul doe]

Can someone please explain what's going on here? I see "a = 5.1" printed but not "b = 5.1" when I run the following code.

As Munair told you above, it's the precision of the representation what's preventing to compare them for equality. In general, you don't want to do a direct comparison against a floating-point value, but need to do an 'approximate' comparison:

Code: Select all

const as double EPSILON = 1e-5

#define fcomp( value1, value2 ) _
  iif( abs( value1 - value2 ) < EPSILON, true, false )

dim as double _
  a = 5.1
dim as single _
  b = 5.1

? a, b

? "a = 5.1", fcomp( a, 5.1 )
? "b = 5.1", fcomp( b, 5.1 )

?

a = 5.10001
b = 5.10001
? "a = 5.10001", fcomp( a, 5.1 )
? "b = 5.10001", fcomp( b, 5.1 )

sleep()

FreeBasic already does this under the hood, but only for numbers of the same type (eg. two doubles or two singles), but not for different types. Floating-point numbers are complex beasts, and this is only one of the many gotchas of using them. As a last recomendation, always try to use the same floating-point representation throughout your app:

Code: Select all

'' Or single, if your app can get away with it
type float as double

dim as float _
  a = 5.1
  b = 5.1

? a, b

? "a = 5.1", cbool( a = b )
? "b = 5.1", cbool( a = b )

sleep()

That way, you won't have to worry about these kind of issues.

[dodicat]

Just tag the end of the literal to what it represents (not always double)

Code: Select all

dim a as double
dim b as single
a = 5.1#
b = 5.1!
print a,b
if a = 5.1# then print "a = 5.1"
if b = 5.1! then print "b = 5.1"
sleep  

[paul doe]

Just tag the end of the literal to what it represents (not always double)

That only works with literals:

Code: Select all

const as double EPSILON = 1e-5

#define fcomp( value1, value2 ) _
  iif( abs( value1 - value2 ) < EPSILON, true, false )

dim as double _
  a = 5.1
dim as single _
  b = 5.1

? a, b

? cbool( a = 5.1# ), cbool( b = 5.1! )
? cbool( a = b )

? fcomp( a, b )

sleep()

[dodicat]

literals, yes. that was the format.
But, I would say that with two numbers, one a single and one a double, casting each to the least accurate (single) and comparing would be satisfactory enough.
But singles/doubles have been argued about so many times in the forum.
I have even tried simulating some code in C and Pascal, and freebasic seems not to be in error.

Code: Select all

const as double EPSILON = 1e-5

#define fcomp( value1, value2 ) _
  iif( abs( value1 - value2 ) < EPSILON, true, false )
  
  #define fcomp2(value1,value2) cbool(csng(value1)=csng(value2))

dim as double _
  a = 5.1
dim as single _
  b = 5.1

? a, b

? cbool( a = 5.1# ), cbool( b = 5.1! )
? cbool( a = b )

? fcomp( a, b )

print fcomp2( a, b )
print "----------"



a=rnd/1000
b=rnd/1000
print a,b

? fcomp( a, b )
print fcomp2( a, b )

print
b=a
print a,b
? fcomp( a, b )
print fcomp2( a, b )



sleep()  

[jaskin]

Thank you all for the interesting explanations. Never in my 40+ years of programming experience in a wide variety of languages have I ever come across this "quirk". Perhaps it's because I always used double, except in Fortran where I often used real*16, which is quadruple. I'll just have to stop using single for floating point representations in FB. It tool me a couple of hours to come to that conclusion after I translated code from another language that contained many statements like the one I posted. Needless to say that other language used double by default for all floating point variables. Lesson learnt. The silly thing on my part is I should have realized this from the beginning because for years I have been using FB to write DLLs for the other language and I always exchanged floating point variables as double.

[badidea]

I wouldn't 'trust' doubles blindly as well. The problem isn't limited to freebasic. Before using floating point numbers, be (a least a bit) aware of the mathematics behind it. See, e.g.: https://en.wikipedia.org/wiki/IEEE_754 (I assume freebasic uses this standard, but I couldn't find it in the reference).

[jaskin]

I visited that and other similar sites. I found this one interesting: https://docs.oracle.com/cd/E19957-01/80 ... dberg.html

[jaskin]

I wouldn't 'trust' doubles blindly as well.

You are certainly right. In fact no floating point representation can be exact all the time. The results from my program in another language do not match exactly the results in FB despite using double throughout. I now have to resort to an old trick I used a long time ago and multiply my sums and other computations by a factor and convert to integer. Results now match perfectly. I'm only doing this to make sure the code in FB is functioning the same way as in the other language. Code conversions can be very challenging for a number of reasons.