Arithmetic problem (floating point)

General FreeBASIC programming questions.
MichaelW
Posts: 3500
Joined: May 16, 2006 22:34
Location: USA

Postby MichaelW » May 05, 2007 19:08

as the number is still wrong in memory

The number is not “wrong”; it is just of limited precision. Few real-world applications need a precision anywhere close to what the FPU can provide. Consider how many engineering and scientific calculations are done on handheld calculators, most of which maintain a significantly lower precision than the FPU.
Richard
Posts: 2958
Joined: Jan 15, 2007 20:44
Location: Australia

Postby Richard » May 05, 2007 22:44

If “What You See Is What You Get” is your aim then you are on a wild goose chase, you can never succeed. For example, it will never be possible to use or display Pi or the square root of two since neither of these can be represented exactly in any system considered so far in this thread.

Engineering calculations can be done on a slide rule with only 3 digits. Surveying needs about 7 digits. Only the math of number theory needs extended arbitrary resolution.

All reasonable calculators use floating point but usually only ten or 12 digits not the 15 available in FreeBASIC double precision. In the end you will find that the answer to your problems was “Print Using”.
cha0s
Site Admin
Posts: 5317
Joined: May 27, 2005 6:42
Location: Illinois
Contact:

Postby cha0s » May 05, 2007 23:11

Huh? Where did that "FRAC" library come from? I wasn't talking about it, I was talking about the intrinsic FB function FRAC.
Zamaster
Posts: 1024
Joined: Jun 20, 2005 21:40
Contact:

Postby Zamaster » May 05, 2007 23:50

@Fox

If you set 1/3 as a fraction and choose "frac to real", it should output 0.3333333333.... because 1/3 cannot be expressed as a real number. FRAC is just a regular fraction library, you want to keep everything in fractions before outputting a final value using frac to real. If you wanted to add 1 to a number. You could always set a fraction, and then use Real to Frac to make "1" a fraction. then add and output the answer. Also, after every few FRAC calculations, use the simplify function to simplify your fractcion to ensure no overflows occur.
Fox
Posts: 353
Joined: Aug 08, 2006 13:39
Location: Lille, France
Contact:

Postby Fox » May 06, 2007 9:36

cha0s wrote:Huh? Where did that "FRAC" library come from? I wasn't talking about it, I was talking about the intrinsic FB function FRAC.


I know :)
But Zamaster don't understood us and gived a link to a FRAC.BI library. It's a good thing, as that way I discovered a really handy lib :-)
Fox
Posts: 353
Joined: Aug 08, 2006 13:39
Location: Lille, France
Contact:

Postby Fox » May 06, 2007 9:38

Richard wrote:In the end you will find that the answer to your problems was “Print Using”.

I see that you are all thinking the same, guys :)
As you are surely better programmers than I (I do that only from time to time as a hobby, thought I began on an Atari XL years ago :), I will recode my program once again, and try to avoid any call to FRAC.BI, relying only on FreeBASIC calculations, than reformatting the output with FORMAT "#.##########" and cutting all ending zeroes with a MID(x,y,z) command...

More I work on that issue, better I see that the problem is deeper and deeper...

Zamaster wrote:You could always set a fraction, and then use Real to Frac to make "1" a fraction. then add and output the answer.

The problem is not here, as I do calculation only on FRACTION type numbers now. Anyway, if I try to do something like "x = y AS FRACTION + z AS DOUBLE", the compiler tells me "type mismatch".
Nevermind. I will probably give up with FRAC.BI, and try to use only PRINT USING, as the whole forum is suggesting me :-)
Zamaster
Posts: 1024
Joined: Jun 20, 2005 21:40
Contact:

Postby Zamaster » May 06, 2007 16:04

... Print Using is just a way of displaying numbers! The type 'Fraction' is not 1 data type, it is a string or 'manifest', a numerator and a denominator which are both ulongints. Im not even sure why your using FRAC unless you want fraction support in your program!
Richard
Posts: 2958
Joined: Jan 15, 2007 20:44
Location: Australia

Postby Richard » May 06, 2007 16:41

Fractions were being considered in an attempt to exactly represent the real number in the memory. This was because many of the numbers used when testing a calculator are fractions. Unfortunately because many irrational numbers cannot be represented exactly as fractions, fractions became an unnecessary added complexity, not an advance or universal solution.

Here is some simple code to generate a programable number of decimal places in the output.

Code: Select all

#include once "vbcompat.bi"
' set the number of decimal places to display
dim as integer decs = 2      ' default initialy to currency.cents
dim as double a = 654321.12345678901      ' the number to print

' -----------------------------------------------------------------
' this is a format string to force sign, insert commas and suppress zeros
dim as string f = "+,"& string(14-decs,"#") & "0." & string(decs,"#")
' but it can preserves one zero before the decimal point
print format(a, f)

' -----------------------------------------------------------------
'an alternative would be
f = "+,"& string(15-decs,"#") & "." & string(decs,"#")
print using f; a

' -----------------------------------------------------------------
sleep
Fox
Posts: 353
Joined: Aug 08, 2006 13:39
Location: Lille, France
Contact:

Postby Fox » May 06, 2007 16:54

Richard wrote:Here is some simple code to generate a programable number of decimal places in the output.

Thank you Richard. In fact, I already wrote something similar to your code ;-)
I dropped the FRAC.BI library, although it seemed to be interesting for my case, but gived me more problems than solutions. I finally stayed with a routine which is FORMATing the desired number to a fixed-point value, then displaying it onscreen. You were right, people - it works pretty well :-)
I used the display limitation of 12 decimal digits, and added a short routine which says "IF ABS(Result) <= 0.0000000000001 THEN Result = 0" to avoid any weird surprises :)

Return to “General”

Who is online

Users browsing this forum: No registered users and 1 guest