PI resolution double vs single ?

General FreeBASIC programming questions.
D.J.Peters
Posts: 7667
Joined: May 28, 2005 3:28

PI resolution double vs single ?

Postby D.J.Peters » Dec 28, 2018 7:17

Why is PI as double truncated to "3.141592653589793" ?

if "f" is for a single literal what is it for double ?

Joshy

Code: Select all

const as double dPI = 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899
const as single sPI = dPI
? "defined as 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899"
? "double = " & dPI
? "single = " & sPI

if dPI=3.14159265358979323846264338327950288419716939937510582097494459230781640628620899 then
  print "double are the same as 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899"
end if 

if dPI=3.141592653589793 then
  print "double are the same as 3.141592653589793"
end if 

sleep
fxm
Posts: 8913
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: PI resolution double vs single ?

Postby fxm » Dec 28, 2018 7:31

Doubles contain at most 53 bits of precision, or about 15 decimal digits.
Singles contain at most 24 bits of precision, or about 6 decimal digits.

Literal suffix for Double is "d".

ProPgLiterals → fxm [Added the literal suffix "D" for a Double in text]
D.J.Peters
Posts: 7667
Joined: May 28, 2005 3:28

Re: PI resolution double vs single ?

Postby D.J.Peters » Dec 28, 2018 8:36

Yes I know, floating point numbers are not new for me but I don't know how FreeBASIC works with it.
Why is const as single sPI = 3.14159265358979323846264338327950288419716939937510582097494459f allowed ?
and why PRINT truncated DOUBLE down to "3.141592653589793"
if a DOUBLE internal can store -> 3.14159265358979323846264338327950288419716939937510582097494459d

EDIT: I see it's printf()

Joshy

Code: Select all

' Literal number too big, truncated
const as double dPI  = 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899d
' OK
const as double dPI_ = 3.14159265358979323846264338327950288419716939937510582097494459d
' Why is this OK too ?
const as single sPI  = 3.14159265358979323846264338327950288419716939937510582097494459f
print dPI_ : sleep
fxm
Posts: 8913
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: PI resolution double vs single ?

Postby fxm » Dec 28, 2018 9:19

I now understand that your question concerns the criterion used by the compiler to generate the warning message "Literal number too big, truncated", this criterion seeming both independent and superior to the precision capability of the declared floating type. But I don't know.
marcov
Posts: 2748
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: PI resolution double vs single ?

Postby marcov » Dec 28, 2018 14:34

D.J.Peters wrote:Yes I know, floating point numbers are not new for me but I don't know how FreeBASIC works with it.
Why is const as single sPI = 3.14159265358979323846264338327950288419716939937510582097494459f allowed ?
and why PRINT truncated DOUBLE down to "3.141592653589793"
if a DOUBLE internal can store -> 3.14159265358979323846264338327950288419716939937510582097494459d


Because it can't store arbitrary numbers with that precision. If you convert between number systems, like binary to decimal, you get fractions generating a lot of digits (like 1/3=0.33333333...).

Textual conversion routines now the number of digits of the in-type, so don't generate more digits then necessary.


The "too big" message is probably simply an overflow of some conversion buffer in the compiler, nothing to do with the actual precision of the types.

Return to “General”

Who is online

Users browsing this forum: Richard and 5 guests