max accuracy double

General FreeBASIC programming questions.
Post Reply
dafhi
Posts: 1641
Joined: Jun 04, 2005 9:51

max accuracy double

Post by dafhi »

would anyone with fb 32 care to verify that this produces a value < 1

Code: Select all

function unit_dbl( in as ulongint ) as double
  return in / ( 2^64 + 2^12 )
end function

dim as ulongint max = -1
print unit_dbl( max )
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: max accuracy double

Post by deltarho[1859] »

Yes: 0.9999999999999998
integer
Posts: 408
Joined: Feb 01, 2007 16:54
Location: usa

Re: max accuracy double

Post by integer »

ditto
same result

Also of interest (to me) as double on 32 bit:
(2^64 - 1) / (2^64 + 2^10) = 1
(2^64 - 1) / (2^64 + 2^11) = 1
(2^64 - 1) / (2^64 + 2^12) = 0.9999999999999998
(2^64 - 1) / (2^64 + 2^13) = 0.9999999999999996
(2^64 - 1) / (2^64 + 2^14) = 0.9999999999999991
dafhi
Posts: 1641
Joined: Jun 04, 2005 9:51

Re: max accuracy double

Post by dafhi »

that should be the final version then. i'll post w/ comments for noobs in Tips & Tricks later

Thanks guys!
srvaldez
Posts: 3373
Joined: Sep 25, 2005 21:54

Re: max accuracy double

Post by srvaldez »

double has 53 bits of precision, so the maximum integer that it can hold is 2^53 or 9007199254740992
however the result of the following code surprised me, can you guess the printed values?

Code: Select all

dim as double x

x=9007199254740992 ' 2^53
? x
x+=1#
? x
x+=1#
? x

sleep
or start with the value x=9007199254740998 and see what you get
< edit >
using the Intel Decimal Floating-Point Math Library viewtopic.php?t=29203

Code: Select all

#include once "libbid.bi"

Using BID

dim as Decimal64 x

x=9007199254740992 ' 2^53
? x
x+=1#
? x
x+=1#
? x

x=9007199254740998

? x
x+=1#
? x
x+=1#
? x

sleep
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: max accuracy double

Post by dodicat »

Good with Decimal Floating-Point Math Library.
I tested fb double and g++ double, both give that strange result when adding 1.
Also Pascal, same.
Doubles increment badly above 2^53.
Decimal Floating-Point or string or ulongint to handle these numbers.

Code: Select all


Sub increment(s As String)
    Dim As Integer counts
    Var ls=Len(s)
    Do
        If  s[ls-counts-1]=57 Then
            counts=counts+1
            If counts=ls Then s="1"+String(ls,"0"):Exit Do
        Else
            s=Left(s,ls-counts-1)+Str(s[ls-counts-1]-47)+String(counts,"0")
            Exit Do
        End If
    Loop
End Sub

var x=str(9007199254740992)
print " double";tab(30);"ulongint";tab(60);"string"
for n as long=1 to 20
      increment(x)
      print val(x),valulng(x),x
next
sleep
 
srvaldez
Posts: 3373
Joined: Sep 25, 2005 21:54

Re: max accuracy double

Post by srvaldez »

it seems counterintuitive to me because you can do this
x=9999999999999998
? x
without problem
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: max accuracy double

Post by dodicat »

Just lucky for that one I think.

Code: Select all

  

var x=9999999999999988ull
print " double";tab(30);"ulongint"
for n as long=1 to 20
      x+=1
      print cdbl(x);tab(30);x
next
sleep
 
Post Reply