PI resolution double vs single ?

General FreeBASIC programming questions.
Post Reply
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

PI resolution double vs single ?

Post by D.J.Peters »

Why is PI as double truncated to "3.141592653589793" ?

if "f" is for a single literal what is it for double ?

Joshy

Code: Select all

const as double dPI = 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899
const as single sPI = dPI
? "defined as 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899"
? "double = " & dPI
? "single = " & sPI

if dPI=3.14159265358979323846264338327950288419716939937510582097494459230781640628620899 then
  print "double are the same as 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899"
end if  

if dPI=3.141592653589793 then
  print "double are the same as 3.141592653589793"
end if  

sleep
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: PI resolution double vs single ?

Post by fxm »

Doubles contain at most 53 bits of precision, or about 15 decimal digits.
Singles contain at most 24 bits of precision, or about 6 decimal digits.

Literal suffix for Double is "d".

ProPgLiterals → fxm [Added the literal suffix "D" for a Double in text]
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: PI resolution double vs single ?

Post by D.J.Peters »

Yes I know, floating point numbers are not new for me but I don't know how FreeBASIC works with it.
Why is const as single sPI = 3.14159265358979323846264338327950288419716939937510582097494459f allowed ?
and why PRINT truncated DOUBLE down to "3.141592653589793"
if a DOUBLE internal can store -> 3.14159265358979323846264338327950288419716939937510582097494459d

EDIT: I see it's printf()

Joshy

Code: Select all

' Literal number too big, truncated
const as double dPI  = 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899d
' OK
const as double dPI_ = 3.14159265358979323846264338327950288419716939937510582097494459d
' Why is this OK too ?
const as single sPI  = 3.14159265358979323846264338327950288419716939937510582097494459f
print dPI_ : sleep
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: PI resolution double vs single ?

Post by fxm »

I now understand that your question concerns the criterion used by the compiler to generate the warning message "Literal number too big, truncated", this criterion seeming both independent and superior to the precision capability of the declared floating type. But I don't know.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: PI resolution double vs single ?

Post by marcov »

D.J.Peters wrote:Yes I know, floating point numbers are not new for me but I don't know how FreeBASIC works with it.
Why is const as single sPI = 3.14159265358979323846264338327950288419716939937510582097494459f allowed ?
and why PRINT truncated DOUBLE down to "3.141592653589793"
if a DOUBLE internal can store -> 3.14159265358979323846264338327950288419716939937510582097494459d
Because it can't store arbitrary numbers with that precision. If you convert between number systems, like binary to decimal, you get fractions generating a lot of digits (like 1/3=0.33333333...).

Textual conversion routines now the number of digits of the in-type, so don't generate more digits then necessary.


The "too big" message is probably simply an overflow of some conversion buffer in the compiler, nothing to do with the actual precision of the types.
Post Reply