Is this a senior moment or a bug ?

[Richard]

I get error number 25, "Overflow in const conversion" in line 3 of this code.

Code: Select all

' FB version 1.04.0
Const As Ubyte msb = &h80
Const As Ubyte tst = Not( msb ) ' overflow in const conversion

[St_W]

Seems like NOT returns an integer (and the value is sign-extended *before* the conversion):

Code: Select all

Const As Ubyte msb = &h80
#print typeof(msb)            'UBYTE
#print typeof(not(msb))       'INTEGER
print bin(not(msb))   '1111111111111111111111111111111111111111111111111111111101111111
Const As Ubyte tst = Not( msb ) and &HFF

IMHO it shouldn't behave like that from a user perspective, though.
A possible solution is to explicitly truncate to byte size again e.g. with "and &hFF

[Richard]

Thanks for that. I have implemented the work around “And &hFF” for the Ubyte situation.
But I have possible types of Ubyte, Ushort, Ulong, Uinteger, Ulongint.
So I must somehow define a mask = &h0000FF…FF without using Not().

[dodicat]

I can use a polynomial for ubyte,ushort,ulong.
tested 64 bit.
for uinteger/ulongint the rounding errors finding the polynomial are too much.

I am going to convert my polynomial interpolator to gmp with >20 decimal places.
Soon.

Code: Select all

#macro msk(x)
Culngint( _
1431525375 + _
-2147320320*(x) + _
715795200*(x)*(x) _ 
)
#endmacro

#macro fx(y)
Not y And msk(Sizeof(y))
#endmacro

Const As Ubyte msb = &h80
Const As Ubyte tst = fx(msb)
Print tst

Const As Ushort mss= 65525
Const As Ushort tst2 = fx(mss)
Print tst2

Const As Ulong msl= 4294967290
Const As Ulong tst3 = fx(msl)
Print tst3
'===============================
print
Const As Ubyte Xmsb = &h80
Const As Ubyte Xtst = not Xmsb and 255
Print Xtst

Const As Ushort Xmss= 65525 
Const As Ushort Xtst2 = not Xmss and 65535
Print Xtst2

Const As Ulong Xmsl= 4294967290
Const As Ulong Xtst3 = not Xmsl and 4294967295
Print Xtst3
sleep



 

[Richard]

I can use a polynomial for ubyte,ushort,ulong. tested 64 bit.

Sorry, but here I am trying to test fast Gray code arithmetic without the slow Gray–Binary–Gray conversions. It is for use with absolute position encoders in NC machines.

[adele]

[quote="Richard"]I get error number 25, "Overflow in const conversion" in line 3 of this code.
Hi Richard,

if you only want to avoid the annoying message (the result might be OK),
why not try the good old-fashioned XOR :

Code: Select all

' works fine with fb105/x64/win
' the *1 vars are for being sure the bits are 
' really masked out :)
Const As Ubyte msb  = &h80
Const As Ubyte msb1 = &h82
Const As Ubyte tst  = &hff Xor MSB 
Const As Ubyte tst1 = &hff Xor MSB1 

? Bin(msb,8)
? Bin(tst,8)
? Bin(tst1,8)
? Bin(msb1,8)

Adi

[dodicat]

Sorry Richard, I hadn't a clue what you were up to.
But at least I was inspired to update my interpolator to gmp.

Code: Select all

#Include once "gmp.bi"
type mpf_t as  __mpf_struct 

Dim Shared As Ulongint PRECISION 
'========= Just in case you forget to set_precision ==========
precision=80
mpf_set_default_prec( PRECISION*4 )
'========================================================
Dim Shared As Zstring * 100000000 outtext
Sub set_precision(n As Uinteger)
    PRECISION=n
    mpf_set_default_prec( PRECISION*4 )
End Sub

Function mult(number1 As String,number2 As String) As String'Automatic precision
    Dim As Integer Ln1=Len(number1),Ln2=Len(number2)
    Dim As __mpf_struct num1,num2,FloatAnswer
    mpf_init2( @num1,4*(Ln1+Ln2+1) )
    mpf_init2( @num2,4*(Ln1+Ln2+1) )
    mpf_init2(@Floatanswer,4*(Ln1+Ln2+1))
    Ln1=Instr(1,number1,"."):Ln2=Instr(1,number2,".")
    var decimals=Len(Mid(number1,Ln1+1))+Len(Mid(number2,Ln2+1))+1
    mpf_set_str( @num1,number1,10)
    mpf_set_str( @num2,number2,10)
    mpf_mul(@Floatanswer,@num1,@num2)
    gmp_sprintf( @outtext,"%." & Str(decimals) & "Ff",@FloatAnswer )
    var outtxt=Trim(outtext)
    If Instr(outtxt,".") Then outtxt= Rtrim(outtxt,"0"):outtxt=Rtrim(outtxt,".")
    Return Trim(outtxt)
End Function

'precision parameter
Function divide(number1 As String,number2 As String,decimals As Uinteger=PRECISION) As String
    Dim As Integer Ln1=Len(number1),Ln2=Len(number2),Ln
    If Ln1>=Ln2 Then  Ln=Ln1 Else Ln=Ln2
    Dim As __mpf_struct num1,num2,FloatAnswer
    mpf_init2( @num1,4*(Ln+1) )
    mpf_init2( @num2,4*(Ln+1) )
    mpf_init2(@Floatanswer,4*(Ln+1)+4*decimals)
    mpf_set_str( @num1,number1,10)
    mpf_set_str( @num2,number2,10)
    mpf_div(@Floatanswer,@num1,@num2)
    gmp_sprintf( @outtext,"%." & Str(decimals) & "Ff",@FloatAnswer)
    Return Trim(outtext)
End Function

Function  Power Overload(number As String,n As Uinteger) As String'automatic precision
    #define dp 3321921
    Dim As __mpf_struct _number,FloatAnswer
    Dim As Ulongint ln=Len(number)*(n)*4
    If ln>dp Then ln=dp
    mpf_init2(@FloatAnswer,ln)
    mpf_init2(@_number,ln) 'or 4*len(number)
    mpf_set_str(@_number,number,10)
    mpf_pow_ui(@Floatanswer,@_number,n)
    gmp_sprintf( @outtext,"%." & Str(n) & "Ff",@FloatAnswer )
    Var outtxt=Trim(outtext)
    If Instr(outtxt,".") Then outtxt= Rtrim(outtxt,"0"):outtxt=Rtrim(outtxt,".")
    Return Trim(outtxt)
End Function

Function plus(number1 As String,number2 As String) As String'automatic precision
    Dim As Integer Ln1=Len(number1),Ln2=Len(number2),decimals,Ln
    If Ln1>=Ln2 Then Ln=Ln1 Else Ln=Ln2
    Ln=ln+1
    Dim As __mpf_struct num1,num2,FloatAnswer
    mpf_init2( @num1,4*(Ln1+1) )
    mpf_init2( @num2,4*(Ln2+1) )
    mpf_init2(@Floatanswer,4*(Ln))
    mpf_set_str( @num1,number1,10)
    mpf_set_str( @num2,number2,10)
    Ln1=Instr(1,number1,"."):Ln2=Instr(1,number2,".")
    If Ln1 Or Ln2 Then
        decimals=Len(Mid(number1,Ln1+1))+Len(Mid(number2,Ln2+1))+1
    End If
    mpf_add(@Floatanswer,@num1,@num2)
    gmp_sprintf( @outtext,"%." & Str(decimals) & "Ff",@FloatAnswer )
    var outtxt=Trim(outtext)
    If Instr(outtxt,".") Then outtxt= Rtrim(outtxt,"0"):outtxt=Rtrim(outtxt,".")
    Return Trim(outtxt)
End Function

Function minus(number1 As String,number2 As String) As String'automatic precision
    Dim As Integer Ln1=Len(number1),Ln2=Len(number2),decimals,Ln
    If Ln1>=Ln2 Then  Ln=Ln1 Else Ln=Ln2
    Ln=ln+1
    Dim As __mpf_struct num1,num2,FloatAnswer
    mpf_init2( @num1,4*(Ln1+1) )
    mpf_init2( @num2,4*(Ln2+1) )
    mpf_init2(@Floatanswer,4*(Ln))
    mpf_set_str( @num1,number1,10)
    mpf_set_str( @num2,number2,10)
    Ln1=Instr(1,number1,"."):Ln2=Instr(1,number2,".")
    If Ln1 Or Ln2 Then
        decimals=Len(Mid(number1,Ln1+1))+Len(Mid(number2,Ln2+1))+1
    End If
    mpf_sub(@Floatanswer,@num1,@num2)
    gmp_sprintf( @outtext,"%." & Str(decimals) & "Ff",@FloatAnswer )
    var outtxt=Trim(outtext)
    If Instr(outtxt,".") Then outtxt= Rtrim(outtxt,"0"):outtxt=Rtrim(outtxt,".")
    Return Trim(outtxt)
End Function
function Absolute overload(a As string) as  string
    Dim As __mpf_struct Ab,Floatanswer
     mpf_init2( @FloatAnswer,4*precision )
     mpf_init2( @Ab,4*precision )
     mpf_set_str( @Ab,a,10)
     mpf_abs(@FloatAnswer,@Ab)
     gmp_sprintf( @outtext,"%." & Str(precision) & "Ff",@FloatAnswer )
    var outtxt=Trim(outtext)
    If Instr(outtxt,".") Then outtxt= Rtrim(outtxt,"0"):outtxt=Rtrim(outtxt,".")
    Return Trim(outtxt)
end function

Function equals overload(a As mpf_t,b As mpf_t) As Integer
    If Mpf_cmp(@a,@b) = 0 Then Return -1
End Function
Function greater overload(a As mpf_t,b As mpf_t) As Integer 'a>b
    If equals(a,b) Then Return 0
    Dim As mpf_t Ab,diff
    mpf_init(@Ab)
    mpf_init(@diff)
    mpf_sub(@diff,@b,@a)
    mpf_abs(@Ab,@diff)
    If mpf_cmp(@Ab,@diff)=0 Then Return 0
    Return -1
End Function
function greater overload(a as string,b as string) as integer
   dim as mpf_t ma,mb
   mpf_init2(@ma,4*precision)
   mpf_init2(@mb,4*precision)
    mpf_set_str( @ma,a,10)
    mpf_set_str( @mb,b,10)
    return greater(ma,mb)
end function

function equals overload(a as string,b as string) as integer
       dim as mpf_t ma,mb
   mpf_init2(@ma,4*precision)
   mpf_init2(@mb,4*precision)
    mpf_set_str( @ma,a,10)
    mpf_set_str( @mb,b,10)
    return equals(ma,mb)
end function

function less overload(a as string,b as string) as integer
   if equals(a,b) then return 0
   if greater(a,b) then return 0
   return -1
end function
Function round(num As String,n As Integer) As String
 #macro inc(s)
counts=0
     ls=Len(s)
    Do
        If  s[ls-counts-1]=57 Then
            counts=counts+1
            If counts=ls Then s="1"+String(ls,"0"):Exit Do
        Else
            s=Left(s,ls-counts-1)+Str(s[ls-counts-1]-47)+String(counts,"0")
            Exit Do
        End If
    Loop
#endmacro  
#macro split(stri,char,var1,var2 )
     pst=Instr(stri,char)
    If pst<>0 Then
    var1=Mid(stri,1,pst-1)
    var2=Mid(stri,pst+1)
    Else
    var1=stri
End if
#endmacro 
    if instr(num,".")=0 then return num
    #macro instrback(st)
    len(st)-instr(st,".")
    #endmacro
    if instrback(num)<=n then return num
    var part1="",part2="",sign="",s=num
      s=Mid(num,1,Instr(s,"."))+Mid(s,Instr(s,".")+1,n)
    
    Dim As String ref=Left(num,Len(s)+1)

    #macro insert(s,char,position)
    part1=Mid$(s,1,position-1)
    part2=Mid$(s,position)
    s=part1+char+part2
    #endmacro
    
    If Instr(s,"-") Then 
        sign="-":s=ltrim(s,"-")
    End If
    Dim i As Integer=Instr(s,".")
    Dim As Integer lens=Len(s),position,counts,ls,pst
    dim as string var1,var2
    split(s,".",var1,var2)
    s=var1+var2
    If Right(ref,1)>="5" Then
        inc(s)
    End If
    If Len(s)+1>lens Then 
        position=i+1
    Else
        position=i
    End If
    insert(s,".",position)
    s=Rtrim(s,"0")
    s=Rtrim(s,".")
    s=ltrim(s,"0")
    if s="-" or s="" then return"0"
    If i<>0 Then Return sign+s
    If i=0 Then 
        inc(s)
        s=ltrim(s,"0")
        Return sign+s
    End If
End Function


type bg
    as zstring * 10000 s
    declare operator let(as string)
    declare operator cast() as string
end type
operator bg.cast() as string
return round(this.s,precision*.75)
end operator

operator bg.let(g as string)
this.s=g
end operator

function Absolute overload(a As bg) as  string
    return absolute(a.s)
end function



operator +(a as bg,b as bg) as bg
dim as bg ans
ans.s=plus(a.s,b.s)
return ans
end operator

operator -(a as bg,b as bg) as bg
dim as bg ans
ans.s=minus(a.s,b.s)
return ans
end operator

operator *(a as bg,b as bg) as bg
dim as bg ans
ans.s=mult(a.s,b.s)
return ans
end operator

operator /(a as bg,b as bg) as bg
dim as bg ans
ans.s=divide(a.s,b.s)
return ans
end operator

operator ^(a as bg,b as uinteger) as bg
dim as bg ans
ans.s=power(a.s,b)
return ans
end operator

operator =(a as bg,b as bg) as long
return a.s=b.s
end operator

operator <(a as bg,b as bg) as long
return less(a.s,b.s)
end operator

'solve linear equations
Sub GaussJordan(matrix() As bg,rhs() As bg,ans() As bg)
    dim as bg zero:zero.s="0"
    Dim As Long n=Ubound(matrix,1)
    Redim ans(0):Redim ans(1 To n)
    Dim As bg b(1 To n,1 To n),r(1 To n)
    For c As Long=1 To n 'take copies
        r(c)=rhs(c)
        For d As Long=1 To n
            b(c,d)=matrix(c,d)
        Next d
    Next c
    #macro pivot(num)
    For p1 As Long  = num To n - 1
        For p2 As Long  = p1 + 1 To n  
            If Absolute(b(p1,num))<Absolute(b(p2,num)) Then
                Swap r(p1),r(p2)
                For g As Long=1 To n
                    Swap b(p1,g),b(p2,g)
                Next g
            End If
        Next p2
    Next p1
    #endmacro
    For k As Long=1 To n-1
        pivot(k)              'full pivoting 
        For row As Long =k To n-1
            If b(row+1,k)=zero Then Exit For
            Var f=b(k,k)/b(row+1,k)
            r(row+1)=r(row+1)*f-r(k)
            For g As Long=1 To n
                b((row+1),g)=b((row+1),g)*f-b(k,g)
            Next g
        Next row
    Next k
    'back substitute 
    For z As Long=n To 1 Step -1
        ans(z)=r(z)/b(z,z)
        For j As Long = n To z+1 Step -1
            ans(z)=ans(z)-(b(z,j)*ans(j)/b(z,z))
        Next j
    Next    z
End Sub


'Interpolate through points.
Sub Interpolate(x_values() As bg,y_values() As bg,p() As bg)
    Dim As Long U=Ubound(x_values),L=Lbound(x_values),ctrA=0,ctrB=0
    Dim As bg matrix(1 To (U-L+1),1 To (U-L+1)),rhs(1 To (U-L+1))
    For a As Long=L To U
        ctrA+=1
        rhs(ctrA)=y_values(a)
        ctrB=0
        For b As Long=L To U
            ctrB+=1
            matrix(ctrA,ctrB)=x_values(a)^(ctrB-1)
        Next b
    Next a
    'Solve the linear equations
    GaussJordan(matrix(),rhs(),p())
End Sub

'Evaluate a polynomial at x
Function polyeval(Coefficients() As bg,Byref x As bg) As bg
    Dim As bg acc=("0")
    For i As Long=Ubound(Coefficients) To Lbound(Coefficients) Step -1
        acc=acc*x+Coefficients(i)
    Next i
    Return acc
End Function

dim as bg t=("45")
'======================== SET UP POINTS =============== 
Dim As bg x(1 To ...)={(str(Sizeof(Ubyte))),(str(Sizeof(Ushort))),(str(Sizeof(Ulong))),(str(sizeof(ulongint)))}
Dim As bg y(1 To ...)={("255"),          ("65535"),          ("4294967295"),("18446744073709551615")}

'====================================================


Redim As bg Poly(0)
'Get the polynomial Poly()
Interpolate(x(),y(),Poly())

'print coefficients to console
Print "Polynomial Coefficients:"
Print
For z As Long=1 To Ubound(Poly)
    If z=1 Then
        Print "constant term  ";Tab(20);Poly(z)
    Else
        Print Tab(8); "x^";z-1;"   ";Tab(20);Poly(z)
    End If
Next z
Print
Print "test"

Print polyeval(poly(),type(str(sizeof(ubyte))))
Print polyeval(poly(),type(str(Sizeof(Ushort))))
Print polyeval(poly(),type(str(Sizeof(Ulong))))
Print polyeval(poly(),type(str(Sizeof(Ulongint))))
Sleep 



 

gives:

Code: Select all

Polynomial Coefficients:

constant term      -878416381599222053.571428571428571428571428571428571428571428571428571428571429
       x^ 1        1537228668156487680
       x^ 2        -768614334436108800
       x^ 3        109802047878843428.571428571428571428571428571428571428571428571428571428571429

test
255
65535
4294967295
18446744073709551615
 

[Richard]

But at least I was inspired to update my interpolator to gmp.

I overcame the accumulation of numerical noise due to limited resolution by first fitting a polynomial to the data and getting noisy coefficients as expected. Then I back computed the data using those erratic coefficients and tabulated the error function. I then fitted the same order polynomial to the error function, which gave me a second set of coefficients. Adding the two sets of coefficients removed most of the numerical noise from the fit.

[dodicat]

You mentioned that method a few days ago.
I tried it out with the square wave, but got no real improvement.
I probably made some mistake in your method.

[Richard]

@ adele.
It needs to work for all unsigned integer data types so I cannot handcode the constants.

This code fixes the original warning by replacing the Not() function with a subtract 1.
It then works for all Unsigned except for Ulongint when it generates a different warning on a different line;
“warning 33(0): Shift value greater than or equal to number of bits in data type”

Code: Select all

#Define Uint Ulongint

' constants used as immediate literals in code
Const As Integer bits = 8 * Sizeof( Uint )  ' number of bits in variable
Const As Uint par_bit = 1 Shl ( bits - 1 )  ' the most significant bit is parity
Const As Uint par_lsb = par_bit Or 1        ' flip parity and lsb at same time
Const As Uint msg_bit = par_bit Shr 1       ' rollover, msb of Gray code field 
Const As Uint hi_bits = par_bit Or msg_bit  ' used to roll over terminal count  
Const As Uint rollers = msg_bit - 1         ' used to detect terminal count  

' report the constants to verify
Print "  Bits  = "; bits
Print "par_bit = "; Bin( par_bit, bits )
Print "par_lsb = "; Bin( par_lsb, bits )
Print "msg_bit = "; Bin( msg_bit, bits )
Print "hi_bits = "; Bin( hi_bits, bits )
Print "rollers = "; Bin( rollers, bits )
Print
Sleep

[dodicat]

shouldn't it be:
1ull Shl ( bits - 1 )

for the ulongint , for just 1 is only integer.

[Richard]

@dodicat, you are right
par_bit = 1ull Shl ( bits - 1 ) ' works OK for Ubyte and for Ulongint
I think the const definitions all work around now.

[counting_pine]

Code: Select all

Const As Ubyte tst = Not( msb ) ' overflow in const conversion

This doesn't address your query, but please note that NOT is not a function, and may eventually confound your expectations if you treat it as one :)
I would suggest writing it as '(Not msb)' or '(Not (msb))'. Or just 'Not msb' if there's no risk of the compiler surprising you.

[Richard]

… please note that NOT is not a function …

Thanks for that advice on avoiding surprises. Double negatives have always confused me.

I think I am having trouble understanding the difference between a “bitwise logical unary operator” and a Function.

This must be a time for idiosyncrasies. The Space() function is interesting. For this line I get no compile or runtime errors or warnings, even with Compiler command "<$fbc>" -exx -w pedantic "<$file>"

Code: Select all

Space( -1234567890.0123 ) = 935.678 

Does the FB parser think it is an array ?

[fxm]

'Space( n )' is a function.

For any function 'f( x )' with one parameter, the parser interprets the expression:
f( x ) = y
like:
f( x=y )
and if x and y are compatible types, none error is reported.

Error is detected if x and y are incompatible types:
Space( -1234567890.0123 ) = "935.678"

Error is detected if the parser ambiguity is fixed by surrounding function with parentheses:
( Space( -1234567890.0123 ) ) = 935.678

Normally, wanting to make 'f( x ) = y' is reserved for a function returning by reference (referenced variable assignment).
The above ambiguity of parser is already noted in the BYREF (function results) documentation.