@CODERS "Endian stuff"

General FreeBASIC programming questions.
albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

@CODERS "Endian stuff"

Post by albert »

@CODERS

My computer gets the bytes backwards , ushort ptr "01" = "10" instead of "01" ?
How do you correct for the big / little endian stuff??

Can you guys , make the ptr , so you can specif Big Endian or Little Endian

Like : ushort ptr be ,or , ushort ptr le ?????? Then you can write code that will work on either type of machine..
Then you can test the machine and see which way it goes, and write the code to work for either type machine.

if "le" then , or ,if "be" then ?

Code: Select all


screen 19

dim as string l_r = ""
dim as string r_l = ""

print "left / right             right / left"
for a as longint = 0 to 3
    for b as longint = 0 to 3
        
        l_r = str(a) + str(b)
        r_l = str(b) + str(a)
        
        dim as ushort ptr usp_lr = cptr( ushort ptr , strptr( l_r ) )
        dim as ushort ptr usp_rl = cptr( ushort ptr , strptr( r_l ) )
        
        print "   " ; l_r ; " " ; *usp_lr - 12336  , ,   r_l ; " " ;  *usp_rl - 12336
        
    next
next

sleep
end

caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: @CODERS "Endian stuff"

Post by caseih »

Your PC is probably little endian. The least significant *byte* (not digit) is stored first in memory. Now granted that's probably backwards to how most people think about numbers and how we write them. Intel must have had their reasons. I can think of maybe one or two.

In short, you don't correct for it usually. The computer (or compiler) isn't confused about the endianness. It knows how to do the proper thing with a number of a particular width. There's no need unless you're dealing with a source of data other than your machine's architecture. Typically data sent over a network would be big endian and need to be converted before putting on the wire.

There's no way to declare a ptr as pointing to a value of a particular endian. An interesting idea, if you were dealing with in-memory data from another architecture. But other than that I can't think of any other use case.

If you are parsing a little-endian number one byte at a time, you read it like this pseudo code:

Code: Select all

number = 0
for x=0 to numbytes
number = number or (read_byte() shl 8*x)
next
If one wants to convert numbers for transmission on a network using a binary protocol, there are C library functions like htons(), htonl(), etc that convert a number *to* big endian. And the opposite functions ntohs(), ntohl(), etc. These functions are portable and can make code work across platforms with different endian. n means "network" or big endian and h means "host" which is whatever endian your current processor is using. On big endian machines the hton family are noops.

I remember working with FAT12 disk structures years ago. as I recall, it would store pairs of clusters in a three byte sequence, but of course in little endian. So if you read the pair one byte at a time you'd get 8 bits of cluster one, then 4 bits of cluster two, 4 bits of cluster one, and then the last 8 bits of cluster two. Doing an adhoc byte-based read required a bit of work on the programmer's part. The best bet was to read three bytes at a time, and then map a C struct over it, defining each field as 12 bits, and then the compiler would properly grab the numbers without fuss. Like I say computers don't have any problem with endianness. It's we who get confused.
albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

Re: @CODERS "Endian stuff"

Post by albert »

@caseih

I was talking about forcing a ptr to return a certain endian

Like:

dim as ushort beptr value = cptr( ushort beptr , strptr( str ) ) to force big endian
dim as ushort leptr value = cptr( ushort leptr , strptr( str ) ) to force little endian

So if your on a big endian machine , you can force a little endian result , with leptr
So if your on a little endian machine , you can force a big endian result , with beptr

I tried your shift code

Code: Select all


screen 19

do
    
    dim as string n = right("0000" + bin( int( rnd * 2^16 ) ) , 8)
    dim as string number1 = ""
    dim as string number2 = ""
    for x as longint = len(n)-1 to 0 step -1
        number1 = str( (n[x]-48) shl 8  * x ) + "  "  + number1
        number2 = str( (n[x]-48) shl 1  * (len(n)-1-x) ) + "  " + number2
    next
    
    print
    print n
    print number1
    print
    print number2
    
    sleep

loop until inkey = chr(27)

sleep
end

albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

Re: @CODERS "Endian stuff"

Post by albert »

I fooled & fumbled around , and came up with a formula to return the value , of a ushort ptr ; for nums 00 to 99

Code: Select all


screen 19

dim as string l_r = ""
dim as string r_l = ""

print " left / right            right / left              Albert's formula"
for a as longint = 0 to 9
    for b as longint = 0 to 9
        
        l_r = str(a) + str(b)
        r_l = str(b) + str(a)
        
        dim as ushort ptr usp_lr = cptr( ushort ptr , strptr( l_r ) )
        dim as ushort ptr usp_rl = cptr( ushort ptr , strptr( r_l ) )
         
        print "   " ; l_r ; " " ; (*usp_lr - 12336)  , ,   r_l ; " " ;  (*usp_rl - 12336)   ,,
        
        print (((*usp_lr - 12336) mod 256 mod 10) * 10) + ((*usp_lr - 12336) \ 256) ' Albert's formula for solving ( ushort ptr ) endian
        
    next
next

sleep
end

MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: @CODERS "Endian stuff"

Post by MrSwiss »

albert wrote:I was talking about forcing a ptr to return a certain endian
A ptr has always the same "endianness", as the machine's CPU.

Maybe you'll understand it better in code (than in words).
Two overloaded endian switchers (32bit / 64bit integers):

Code: Select all

' (c) 2019-01-24, MrSwiss
Function sw_endian OverLoad (ByVal v As Ulong) As ULong
    Swap CPtr(UByte Ptr, @v)[0], CPtr(UByte Ptr, @v)[3]
    Swap CPtr(UByte Ptr, @v)[1], CPtr(UByte Ptr, @v)[2]
    Return v
End Function

Function sw_endian OverLoad (ByVal v As UlongInt) As ULongInt
    For i As UInteger = 0 To 3
        Swap CPtr(UByte Ptr, @v)[i], CPtr(UByte Ptr, @v)[7 - i]
    Next
    Return v
End Function


Dim As ULongInt ULI = &h11335577ccddeeff' ULonInt
Dim As ULong    UL  = &h3366bbff           ' Ulong

Print "ULongInt"
Print "original     "; Hex(ULI, 16)
Print "sw_endian()  "; Hex(sw_endian(ULI), 16)
Print
Print "ULong"
Print "original     "; Hex(UL, 8)
Print "sw_endian()  "; Hex(sw_endian(UL), 8)

Print : Print
Print "press a key, to EXIT ... ";

Sleep
albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

Re: @CODERS "Endian stuff"

Post by albert »

@MrSwiss

I got my formula returning the same value as mid( n , ? , 2 ) for *ushort ptr , that's what i wanted to accomplish...
To get the actual value from the pointer..

Code: Select all


screen 19

dim as string l_r = ""
dim as string r_l = ""

print " left / right            right / left              Albert's formula"
for a as longint = 0 to 9
    for b as longint = 0 to 9
        
        l_r = str(a) + str(b)
        r_l = str(b) + str(a)
        
        dim as ushort ptr usp_lr = cptr( ushort ptr , strptr( l_r ) )
        dim as ushort ptr usp_rl = cptr( ushort ptr , strptr( r_l ) )
         
        '12336 = ptr val of "00"
        print "   " ; l_r ; " " ; (*usp_lr - 12336)  , ,   r_l ; " " ;  (*usp_rl - 12336)   ,,
        
        print (((*usp_lr -12336) mod 256 ) * 10) + ((*usp_lr-12336)  \ 256) ' Albert's formula for solving ( ushort ptr ) endian
        
    next
next

sleep
end

Now to move on to ; uinteger ptr and ulonint ptr...
albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

Re: @CODERS "Endian stuff"

Post by albert »

I managed to create a formula to swap the bytes of of a ushort ptr...

So now i can get:
The actual "00" to "99" value of the ushort ptr..
The the byte swapped value of the ushort ptr..

Code: Select all


screen 19

dim as string l_r = ""
dim as string r_l = ""

print " left / right            right / left              Albert's formula"
for a as longint = 0 to 9
    for b as longint = 0 to 9
        
        l_r = str(a) + str(b)
        r_l = str(b) + str(a)
        
        dim as ushort ptr usp_lr = cptr( ushort ptr , strptr( l_r ) )
        dim as ushort ptr usp_rl = cptr( ushort ptr , strptr( r_l ) )
         
        '12336 = ptr val of "00"
        print "   " ; l_r ; " " ; ( *usp_lr - 12336 )  , ,   r_l ; " " ;  ( *usp_rl - 12336 )   ,,
        
         'Albert's formula for solving ( ushort ptr ) endian
         'Returns the same value as MID( str , ? , 2 )
        print ( ( ( *usp_lr - 12336 ) mod 256 ) * 10 ) + ( ( *usp_lr - 12336 ) \ 256 )  ;"    " ;  ' Albert's formula for solving ( ushort ptr ) endian
        
        'Albert's formula for solving ( ushort ptr ) endian
        'Returns opposite of ushort ptr  ( byte flop )
        print ((( *usp_lr - 12336 ) mod 256) * 256 ) + (((*usp_lr-12336)  \ 256)  )
        
    next
next 

sleep
end

Now it's on to do , uintiger ptr and ulongint ptr..
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: @CODERS "Endian stuff"

Post by MrSwiss »

albert wrote:So now i can get:
The actual "00" to "99" value of the ushort ptr..
The the byte swapped value of the ushort ptr..
I don't really understand because,

value of a UShort = 0 To 65535 ... ??? (dereferenced UShort Ptr, is the same)
value of a UByte = 0 To 255 ... ??? (dereferenced UByte Ptr, is the same).

But you are dealing with String, which is by default, big-endian Ubyte (array) like.
albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

Re: @CODERS "Endian stuff"

Post by albert »

@MrSwiss

I got the program returning mid( str , place ,2 ) from *ushort_ptr = "00" to "99" from *ptr
I got the program returning byte swapped value of *ushort_ptr...opposite of my machine.. little endian from big endian
.
Now how do you convert the byte swapped value (little endian) back to big endian ????

Code: Select all


screen 19

dim as string l_r = ""
dim as string r_l = ""

print " left / right            right / left              Albert's formula"
for a as longint = 0 to 9
    for b as longint = 0 to 9
        
        l_r = str(a) + str(b)
        r_l = str(b) + str(a)
        
        dim as ushort ptr usp_lr = cptr( ushort ptr , strptr( l_r ) )
        dim as ushort ptr usp_rl = cptr( ushort ptr , strptr( r_l ) )
         
        '12336 = ptr val of "00"
        print "   " ; l_r ; " " ; ( *usp_lr - 12336 )  , ,   r_l ; " " ;  ( *usp_rl - 12336 )   ,,
        
         'Albert's formula for solving ( ushort ptr ) endian
         'Returns the same value as MID( str , ? , 2 )
        dim as ulongint mids = ( ( ( *usp_lr - 12336 ) mod 256 ) * 10 ) + ( ( *usp_lr - 12336 ) \ 256 ) 
        print mids ; "  " ; 
        
        'Albert's formula for solving ( ushort ptr ) endian
        'Returns opposite of ushort ptr  ( byte flop )
        dim as ulongint swapped = ((( *usp_lr - 12336 ) mod 256) * 256 ) + (((*usp_lr-12336)  \ 256)  )
        print swapped
    
    next
next 

sleep
end

MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: @CODERS "Endian stuff"

Post by MrSwiss »

@albert,
albert wrote:Now how do you convert the byte swapped value (little endian) back to big endian ????
Well, the same way:
Swap CPtr(UByte Ptr, @UShort)[0], CPtr(UByte Ptr, @UShort)[1]
@UShort = address of a UShort variable (UShort Ptr)
albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

Re: @CODERS "Endian stuff"

Post by albert »

I got it working with ( Ushort ptr ) ,

A formula to return the mid( str , ? , 2) from a ushort ptr.
A formula to byte swap the big endian to little endian
A formula to convert the mid , back to big endian ptr..

Code: Select all


screen 19

dim as string l_r = ""
dim as string r_l = ""

print "  Big End    Little End                  Mids       Little End     Big End"
for a as longint = 0 to 3
    for b as longint = 0 to 3
        
        l_r = str(a) + str(b)
        r_l = str(b) + str(a)
        
        dim as ushort ptr usp_lr = cptr( ushort ptr , strptr( l_r ) )
        dim as ushort ptr usp_rl = cptr( ushort ptr , strptr( r_l ) )
         
        '12336 = ptr val of "00"
        print "   " ; l_r ; " " ; ( *usp_lr - 12336 )  ,   r_l ; " " ;  ( *usp_rl - 12336 )   ,,
        
         'Albert's formula for solving ( ushort ptr ) endian
         'Returns the same value as MID( str , ? , 2 )
        dim as ulongint mids = ( ( ( *usp_lr - 12336 ) mod 256 ) * 10 ) + ( ( *usp_lr - 12336 ) \ 256 ) 
        print mids ; "         " ,  
        
        'Albert's formula for solving ( ushort ptr ) endian
        'Returns opposite of ushort ptr  ( byte flop )
        dim as ulongint swapped = ((( *usp_lr - 12336 ) mod 256) * 256 ) + (((*usp_lr-12336)  \ 256)  )
        print swapped ,
        
        'Albert's formula for solving ( ushort ptr ) endian
        'converts ( mids ) above back to a big endian ptr
        dim as ulongint back_to_ptr = ( mids mod 10 ) * 256 + ( mids \ 10 )
        print back_to_ptr
        
    next
next 

sleep
end

albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

Re: @CODERS "Endian stuff"

Post by albert »

@MrSwiss

I played around with your code and created a UShort version..

Code: Select all


' (c) 2019-01-24, MrSwiss
Function sw_endian OverLoad (ByVal v As UShort) As UShort
    Swap CPtr(UByte Ptr, @v)[0], CPtr(UByte Ptr, @v)[1]
    Return v
End Function

Function sw_endian OverLoad (ByVal v As Uinteger) As Uinteger
    Swap CPtr(UByte Ptr, @v)[0], CPtr(UByte Ptr, @v)[3]
    Swap CPtr(UByte Ptr, @v)[1], CPtr(UByte Ptr, @v)[2]
    Return v
End Function

Function sw_endian OverLoad (ByVal v As ULongint) As ULongint
    For i As UInteger = 0 To 3
        Swap CPtr(UByte Ptr, @v)[i], CPtr(UByte Ptr, @v)[7 - i]
    Next
    Return v
End Function

screen 19

do
    
    dim as ubyte val1
    dim as string hex_str = string( 16 , chr(0) )
    dim as ubyte ptr ubp = cptr( ubyte ptr , strptr(hex_str) )
    for a as longint = 1 to 16
        val1 = int( rnd*16 )
        if val1 <= 9 then *ubp = val1 + 48
        if val1 >   9 then *ubp = val1 - 10 + 65
        ubp+=1
    next
     ' 
    print
    print "hex_str = " ; hex_str
    
    Dim As ULongint  ULI  = valulng( "&H" + right( hex_str , 16 ) )  'int( rnd*2^64 )  ' ULonInt
    Dim As UInteger     UI  = valulng( "&H"  + right( hex_str ,   8 ) )  'int( rnd*2^64 )  ' ULonInt
    Dim As UShort       US = valulng( "&H"  + right( hex_str ,   4 ) )  'int( rnd*2^64 )  ' ULonInt
    
    print
    Print "ULongInt"
    Print "original     ";  hex( ULI , 16 )
    Print "sw_endian()  "; hex( sw_endian( ULI ) , 16 )
    Print
    Print "UInteger"
    Print "original     "; hex( UI , 8 )
    Print "sw_endian()  "; Hex( sw_endian( UI ) , 8 )
    Print
    Print "UShort"
    Print "original     "; hex( US , 4 )
    Print "sw_endian()  "; hex( sw_endian( US ) , 4 )
    Print
    Print
    Print "press ( esc )  key, to EXIT ... any other key to advance ";
    print
    
    Sleep

loop until inkey = chr(27)

end

How do you get the Mids value of *uinteger ptr and *ulongint ptr , and convert the mids back to big endian ptr???

Here's my formula for ushort ptr.. how to do uinteger and ulongint
dim as ulongint mids = ( ( ( *usp_lr - 12336 ) mod 256 ) * 10 ) + ( ( *usp_lr - 12336 ) \ 256 )
dim as ulongint mids_back_to_BE_ptr = ( mids mod 10 ) * 256 + ( mids \ 10 )
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: @CODERS "Endian stuff"

Post by MrSwiss »

@albert,
albert wrote:How do you get the Mid() value of *uinteger ptr and *ulongint ptr , and convert the mid() back to big endian???
Mid() is a String Function, therefore I don't understand, what you want to do.
Just picking a Ubyte or UShort, from a larger 'integer type', makes no sense to me.

Btw: your UInteger Function doesn't check the size (4 byte FBC32 / 8 byte FBC64).
To be compatible with both compiler versions, you''ll have to do that.
Btw. this is the reason I've made the ULong version (fixed 32 bit).

Function sw_endian OverLoad (Byval v As UInteger) As UInteger ...

Code: Select all

Function sw_endian OverLoad (ByVal v As UInteger) As UInteger
    If SizeOf(Any Ptr) = 8 Then         ' FBC 64 bit
        For i As UInteger = 0 To 3
            Swap CPtr(UByte Ptr, @v)[i], CPtr(UByte Ptr, @v)[7 - i]
        Next
    Else                                ' FBC 32 bit
        Swap CPtr(UByte Ptr, @v)[0], CPtr(UByte Ptr, @v)[3]
        Swap CPtr(UByte Ptr, @v)[1], CPtr(UByte Ptr, @v)[2]
    End If
    Return v
End Function
albert
Posts: 6000
Joined: Sep 28, 2006 2:41
Location: California, USA

Re: @CODERS "Endian stuff"

Post by albert »

@MrSwiss

for the mids:
i want to get the *ulongint ptr and convert it to little endian , and then and then return the base 10 value of the little endian value.
Then i want to convert the base 10 value back to big endian ptr..


So i can convert the ulongint ptr to base 10 and then play with the number , and then convert the number back to big endian ptr to stick back into the string..
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: @CODERS "Endian stuff"

Post by MrSwiss »

@albert,
First, all numeric data-types are, by default little-endian (Intel/AMD CPU's).
The values are BASE 2 (binary), and not BASE 10, which can only be said, if
converted to String. Additionally with String, you'll have to 'read' the UBytes
from end to start (right to left), thereafter subtract 48 (ASCII-offset), to get
a real BASE 10 digit ... (zero to 9).

Dim As ULongInt uli = {assign a number}
Dim As String s = CULngInt(uli)
' use string[index] - 48 to get UByte,
containing a BASE 10 digit ...
Post Reply