Binary to Hex !!!

New to FreeBASIC? Post your questions here.
Alexa
Posts: 56
Joined: May 01, 2007 20:22

Postby Alexa » May 05, 2007 8:37

My Special Thanks for : Zippy, MystikShadows and counting_pine

I Really Appreciate that, you spend your times for me, for a pointless software, however it'll help us to done my friend's project

and about the MystickShadows, he never want to $charge$ me for this, he just went to help me, he is a nice guy as you are Zippy
MystikShadows
Posts: 606
Joined: Jun 15, 2005 13:22
Location: Upstate NY
Contact:

Postby MystikShadows » May 05, 2007 12:00

Zippy wrote:I've been waiting for MystikShadows to appear again. I don't want to get in the way of $commerce$. I'm doing this to (try to) help and (primarily) as a learning experience.


YEah, this was never about commerce, I was just thinking of helping :-). Since Alexa offered to pay, I wanted to take that out of this thread as the important thing to a forum post is to solve the problem and everyone learns from the solution as I have with your code and counting_pine's code as well. It's not about $commerce$ here ;-). it never really should be. So when there's talk to "pay to get something done" I just have in my nature to take those out of forums even if I don't charge. :-).
Zippy
Posts: 1295
Joined: Feb 10, 2006 18:05

Postby Zippy » May 05, 2007 15:52

Hey, there's nothing wrong with $commerce$. NOTHING. I'm just easy. Or is that cheap.. Ah, nevermind.

counting_pine's indexed string lookup method is faster than mine, on the order of 1.77 times as fast. If you change his input buffer to a string and index that instead of a ubyte array then his method is some 1.94 times faster. So you have code something like this:

Code: Select all

'mod c_p's code
dim shared as string decibuff
decibuff=input(lof(1),#1)
'then in the function..
c=decibuff[i]
'instead of
'c = Buffer(i)

It may look odd to input non-string data to strings, but it is simpler and allows that string pointer indexing (fast). And works as long as you don't want to do anything else with that string (like print it..). It is probably just as fast as as using allocate/memory&pointers, I'll test.

@counting_pine

Kewl lookup method. Thanks for the education. I'd like to see what you'd do with the reverse, hex to decimal.
counting_pine
Site Admin
Posts: 6200
Joined: Jul 05, 2005 17:32
Location: Manchester, Lancs

Postby counting_pine » May 06, 2007 0:49

OK, well, here's another attempt at the reverse function.

Code: Select all

sub HexToBinary(byref HexStr as string, Buffer() as ubyte)
   
    const as integer padlen = 1 'num of spaces between chars
    const as integer lb = 0     'starting position in the Buffer
   
    'H2I is a helper function, effectively: valint("&H" & chr(c))
    'It's a lot faster, but there are no checks, so the results for non-hex values are undefined
    #define H2I(c) ((c and 15) + (9 and -(c and 64) shr 6))
   
    dim as integer l = len(HexStr) \ (2 + padlen)
    dim as integer i, j
    dim as ubyte c1, c2
   
    redim Buffer(lb to lb + l - 1)
   
    j = 0
    for i as integer = 0 to l - 1
       
        c1 = HexStr[j]
        c2 = HexStr[j + 1]
       
        Buffer(lb + i) = H2I(c1) shl 4 or H2I(c2)
       
        j += (2 + padlen)
       
    next i
   
end sub

dim as ubyte Buffer()
dim as string s
dim as integer i

open "1.txt" for binary access read as #1
    s = space(lof(1))
    get #1, 1, s
close #1

HexToBinary( s, Buffer() )

open "1_.mp3" for binary access write as #2
   
    put #2, 1, Buffer()
   
close #2

I would recommend this function solely for converting a text file to hex. I don't recommend converting between the two unnecessarily. Any operations you want to do on the binary data should be done on the byte buffer, because that's how the computer works with it best. The hex format is only better for one reason: it's more convenient for humans to work with, not computer programs.
Alexa
Posts: 56
Joined: May 01, 2007 20:22

Postby Alexa » May 08, 2007 11:54

Thank you Guys..!

if i want to convert bytes to 0 and 1, like Binary -> Hex. what should i do ?

let me explain more : open Microsoft Calculator (calc.exe) in view menu select Scientific, select the HEX and type AB and then click on BIN, it show you something like 10101011, i want to convert whole of a file to 10101011 and then a program to return the 10101011 to original value.

i think it's possible by small changes in your codes

** and i think in the next version of FB, Developer add these Source Codes to FB example, those are very useful ...

Thank you.. =)
counting_pine
Site Admin
Posts: 6200
Joined: Jul 05, 2005 17:32
Location: Manchester, Lancs

Postby counting_pine » May 08, 2007 13:31

In a way, binary is simpler because you don't need to use letters, but it does mean you have to work with 8 characters per byte, instead of just two, so it's probably slower.
I've modified my functions to use binary instead of hex.
It wasn't too hard, but it did mean ripping out the guts of both inner loops, and rewriting them.

Code: Select all

Function BinaryToBinStr(Buffer() As Ubyte, Byval Start As Integer, Byval Length As Integer) As String
   
    Const As Integer padlen = 1 'num of spaces between chars
   
    Dim As String  ret     
    Dim As Integer i, j, k
    Dim As Ubyte   c       
   
    ret = Space( length * (8 + padlen) )
   
    For i = Start To Start + Length - 1
        c = Buffer(i)
        Assert( j + 7 < Len(ret) )
        For k = 0 To 7
            ret[j + k] = Asc("0") + c Shr 7
            c *= 2
        Next k
        j += (8 + padlen)
    Next i
   
    Return ret
   
End Function

Dim As Ubyte Buffer()
Dim As String HexResult

Open "1.mp3" For Binary Access Read As #1
    Redim Buffer(0 To Lof(1) - 1)
    Get #1, 1, Buffer()
    HexResult = BinaryToBinStr(Buffer(), 0, Lof(1) )
Close 1

Open "1.txt" For Binary Access Write As #2
    Print #2, HexResult
Close #2

Code: Select all

Sub BinStrToBinary(Byref HexStr As String, Buffer() As Ubyte)
   
    Const As Integer padlen = 1 'num of spaces between chars
    Const As Integer lb = 0     'starting position in the Buffer
   
    Dim As Integer l = Len(HexStr) \ (8 + padlen)
    Dim As Integer i, j, k
    Dim As Ubyte c
   
    Redim Buffer(lb To lb + l - 1)
   
    j = 0
    For i As Integer = 0 To l - 1
       
        For k = 0 To 7
            c = (c * 2) Xor HexStr[j + k]
        Next k
       
        Buffer(lb + i) = c Xor 16
       
        j += (8 + padlen)
       
    Next i
   
End Sub

Dim As Ubyte Buffer()
Dim As String s
Dim As Integer i

Open "1.txt" For Binary Access Read As #1
    s = Space(Lof(1))
    Get #1, 1, s
Close #1

BinStrToBinary( s, Buffer() )

Open "1_.mp3" For Binary Access Write As #2
   
    Put #2, 1, Buffer()
   
Close #2
tommen
Posts: 16
Joined: Apr 19, 2007 12:41

Postby tommen » May 09, 2007 13:50

Hi!

Just wanted to show you a trick I used to convert bin to hex on the good old c-64. No libraries and stuff to help back then.

Here goes:

A byte read from a file comes out as eight zeeros and ones.

i.e: 00101100

now the first trick is to splitt this byte in two (into two nibbles):

nibble hi= 0010
Nibble lo= 1100

now each of those nibbles gives a number between 0 to 15 or 0 to F in hex

Nibble hi= 0010 is the same as 2 in decimal and 2 in hex
Nibble lo= 1100 is the same as 12 in decimal and C in hex

this means 0010 1100 = 2C hex!

See the connection?

now a simple lookup table will give you the correct "ascii" hex value to output to wherever you want.

example in 6502 asm to give you an idea:

Code: Select all

bintohex:
lda binarybyte                      ;Put byte to convert into A
tax                                      ;Copy it to X
and #%00001111                 ;Delete 4 highest bits in A (Lo nibble)
tay                                      ;Copy Lo Nibble to Y
lda table,y                           ;Get ascii value from lookup table(indexed)
sta hexlo                             ;Store the result
txa                                      ;Get original byte
lsr                                       ;move bits right and ignore stack
lsr                                       ;move bits right and ignore stack
lsr                                       ;move bits right and ignore stack
lsr                                       ;move bits right and ignore stack
tay                                      ;Copy Hi Nibble to Y
lda table,y                           ;Get ascii value from lookup table(Indexed)
sta hexhi                             ;Store the result
rts                                      ;Routine stop

table:
.byte 0,1,2,3,4,5,6,7,8,9.a,b,c,d,e,f

A & X & Y are registers. Lookup table contains incorrect values and would have to be changed out with the corresponding ASCII values for

I don't know if this is any help to you but it should be very easy to convert this routine to FB or X86 asm (Which would fly for sure ;-) )
tommen
Posts: 16
Joined: Apr 19, 2007 12:41

Postby tommen » May 11, 2007 14:50

eeehm.. I actually figgured out how to do it in FB quite easilly:

Code: Select all

screen 1

a = &b10101001      'Binary number you want to convert

b = a and &b00001111
c = (a and &b11110000)/16

print hex(a)+hex(b)

sleep


Maybee it is what you're looking for or maybee not^^
Zippy
Posts: 1295
Joined: Feb 10, 2006 18:05

Postby Zippy » May 11, 2007 16:43

tommen wrote:eeehm.. I actually figgured out how to do it in FB quite easilly:

Code: Select all

screen 1

a = &b10101001      'Binary number you want to convert

b = a and &b00001111
c = (a and &b11110000)/16

print hex(a)+hex(b)

sleep


Maybee it is what you're looking for or maybee not^^

@tommen

This is all it takes to convert to 2-byte hex:

Code: Select all

print hex(251,2)    ' "2" = emit 2-byte hex character
print hex(&b11111011,2)
sleep

using the intrinsic hex() function. Blindly simple. Snail slow when processing megabytes, particularly when you add in the cost (time) of getting the values from an array then storing the emitted values.

counting_pine is using a lookup method here:
http://www.freebasic.net./forum/viewtopic.php?p=69392
via string pointer indexing. His code is blindingly fast, using no asm.

This is what I had come up with in asm (shudder, my ignorance..) earlier:

Code: Select all

#include once "vbcompat.bi"
dim as integer c,i,numvals,res
dim as double b,e
'
dim shared as ubyte ptr decibuff
dim shared as zstring ptr hexabuff
'
declare function DecToHex(numvals as integer) as integer
declare function HexToDec(numvals as integer) as integer
'
open "your very large file" for binary access read as #1
'
numvals=lof(1)
if numvals<1 then beep:print "Open failed":sleep:end
decibuff=allocate(numvals)
get #1,,*decibuff,numvals
close
'
res=DecToHex(numvals)   
'
c=0
for i=0 to 15 step 2
   print i,"Dec: ";hex((decibuff[c])),
   print    chr(asc(hexabuff[i]));chr(asc(hexabuff[i+1]));
   print    " :Hex"
   c+=1
next
print
'
res=HexToDec(numvals)
'
c=0
for i=0 to 15 step 2
   print i,"Hex: ";chr(asc(hexabuff[i]));chr(asc(hexabuff[i+1])),
   print   hex((decibuff[c]));
   print   *iif(len(hex((decibuff[c])))=1,@" ",0);
   print   " :Dec"
   c+=1
next
print
'
deallocate(decibuff)
deallocate(hexabuff)
sleep
end
'
function DecToHex(numvals as integer) as integer
'
dim as integer res
dim as double b,e
'
print "DecToHex",
b=timer
'
hexabuff=allocate(numvals*2)
'
asm
   mov  ecx,dword ptr [numvals]
   mov  esi,dword ptr [decibuff]
   mov  edi,dword ptr [hexabuff]
TOP:
   mov  al,[esi]
   mov  bl,al
   shr  al,4
   cmp  al,9
  jg   NG1
   add  al,48
  jmp  NL1
NG1:
   add  al,55
NL1:
   mov  [edi],al
   inc  edi
   mov  al,bl
   and  al,15
   cmp  al,9
  jg   NG2
   add  al,48
  jmp  NL2
NG2:
   add  al,55
NL2:
   mov  [edi],al
   inc  edi
   inc  esi
   dec  ecx
  jnz  TOP
ZOUT:
  mov  [res],edi
end asm
'
e=timer
print format(numvals,"###,###,### \b"),
print "HexLen = ";format(len(*hexabuff),"###,###,### \b")
print using " #.########## seconds";e-b,
print "       CPS: ";format(int(1/(e-b)*numvals),"###,###,###,###")
print
'
return cast(integer,res-hexabuff)
'
end function
'
function HexToDec(numvals as integer) as integer
'
dim as integer res
dim as double b,e
'
print "HexToDec",
b=timer
'
decibuff=allocate(numvals)
'
asm
   mov  ecx,dword ptr [numvals]
   mov  esi,dword ptr [hexabuff]
   mov  edi,dword ptr [decibuff]
TOPZ:
   mov  al, [esi]
   cmp  al,57
  jg   KG1
   sub  al,48
  jmp  KL1
KG1:
   sub  al,55
KL1:
   shl  al, 4
   mov  bl, al
   inc  esi
   mov  al, [esi]
   cmp  al, 57
  jg   KG2
   sub  al,48
  jmp  KL2
KG2:
   sub  al,55
KL2:
   add  bl,al
   mov  [edi],bl
   inc  edi
   inc  esi
   dec  ecx
  jnz  TOPZ
ZOUTZ:
   mov  [res],edi
end asm
'
e=timer
print format(abs(cast(integer,res-decibuff)*2),"###,###,### \b"),
print "DecLen = ";format(numvals,"###,###,### \b")
print using " #.########## seconds";e-b,
print "       CPS: ";format(int(1/(e-b)*numvals),"###,###,###,###")
print
'
return cast(integer,res-decibuff)
'
end function

Using Allocate() is faster than DIMing/accessing a ubyte array. And then using a ztring ptr, allocated (again faster than a string array "sized" using space(n)), for the hex output would allow searching the hexput using instr() if one wished (I would probably use the crt functions).

asm converts about 50 million values per second (obviously processor-variable) whereas a straight array-to-hex() method converts some 2 million CPS. 2 million CPS is way-fast if you aren't dealing with megabytes.
tommen
Posts: 16
Joined: Apr 19, 2007 12:41

Postby tommen » May 12, 2007 8:58

I figgured the way with hex(&b00000000) but I just wanted to show the idea from my previous post which I see you have in your asm part as well.

One question though: Since this is quite a long loop depending on the file wouldn't it be an idea to try and minimize it as mouch as possible? maybee it is optimized out of hell already but I think I have an idea.

To convert the number into ASCII you use:

Code: Select all

  cmp  al,9
  jg     NG1
  add  al,48
  jmp  NL1
NG1:
  add  al,55
NL1:
  mov  [edi],al


I'm not sure what I suggest is doable since im VERY new to x86 asm but please have a look:

Code: Select all

  mov [edi],[TABLE]+al

TABLE:
  .byte 48,49,50,51,52,53,54,55,56,57,65,66,67,68,69,70
'ASCII values for 1,2,3,4,5,6,7,8,9,a,b,c,d,e,f


This way you only need to do one indexed fetch for the ASCII values instead of calculating them.

If this doesent work you could also unroll your loops to avoid unneccessary jumps as mouch as possible. This will quadruple the length of your routine but that's life^^
Zippy
Posts: 1295
Joined: Feb 10, 2006 18:05

Postby Zippy » May 12, 2007 20:02

@tommen

I'm a 80x86 virgin. My only experience with x86 asm was in a small project in 1996. I wrote code to save to file and then restore CMOS (I was changing boot device order programmatically). I stuck my foot back in the water 2 weeks ago, I'm SO ignorant.

.. Tt took me near 4 hours today to figure out index addressing when the index is not the same type as the register being used as the base.

This:

mov bl, [edx]+al

does NOT WORK. No matter how hard you try, what order, what brackets or not: it won't work. But DUH, al is the lobyte of eax..

Anyhow, I've rewritten both asm conversion functions to use lookups. Here's before and after timings:

Code: Select all

'======================> without lookups
'
'DecToHex      5,582,887 b   HexLen = 11,165,774 b
' 0.1153034813 seconds       CPS: 48,419,067
'
'HexToDec      11,165,774 b  DecLen = 5,582,887 b
' 0.1008130922 seconds       CPS: 55,378,591
'
'======================> with lookups
'
'DecToHex      5,582,887 b   HexLen = 11,165,774 b
' 0.0266547843 seconds       CPS: 209,451,591
'
'HexToDec      11,165,774 b  DecLen = 5,582,887 b
' 0.0357305188 seconds       CPS: 156,249,816

Yes, lookups are a BIT FASTER. Some 5x faster in DecToHex, almost 3x faster for the reverse. Stick a HUGE fork in me, I'm done.

In the interest of completeness, here is the complete revised code:

Code: Select all

#include once "vbcompat.bi"
dim as integer c,i,numvals,res
dim as double b,e
'
dim shared as ubyte ptr decibuff
dim shared as zstring ptr hexabuff
'
declare function DecToHex(numvals as integer) as integer
declare function HexToDec(numvals as integer) as integer
'
open "open my very large file" for binary access read as #1
'
numvals=lof(1)
if numvals<1 then beep:print "Open failed":sleep:end
decibuff=allocate(numvals)
get #1,,*decibuff,numvals
close
'
res=DecToHex(numvals)   
'
c=0
for i=0 to 15 step 2
   print i,"Dec: ";hex((decibuff[c])),
   print    chr(asc(hexabuff[i]));chr(asc(hexabuff[i+1]));
   print    " :Hex"
   c+=1
next
print
'
res=HexToDec(numvals)
'
c=0
for i=0 to 15 step 2
   print i,"Hex: ";chr(asc(hexabuff[i]));chr(asc(hexabuff[i+1])),
   print   hex((decibuff[c]));
   print   *iif(len(hex((decibuff[c])))=1,@" ",0);
   print   " :Dec"
   c+=1
next
print
'
deallocate(decibuff)
deallocate(hexabuff)
sleep
end
'
function DecToHex(numvals as integer) as integer
'
dim as integer res
dim as double b,e
'
dim as string htable = "0123456789ABCDEF"
'
print "DecToHex",
b=timer
'
hexabuff=allocate(numvals*2)
'
asm
   mov ecx,   dword ptr [numvals]
   mov esi,   dword ptr [decibuff]
   mov edi,   dword ptr [hexabuff]
   mov edx,   [htable]
TOP:
   mov eax,   0
   mov al,    [esi]      'byte value
   shr al,    4          'downrev high nibble
   mov al,    [edx][eax] 'hex char
   mov [edi], al
   inc edi
'
   mov eax,   0
   mov al,    [esi]      'byte value
   and al,    15         'low nibble
   mov al,    [edx][eax] 'hex char
   mov [edi], al
   inc edi
   inc esi
'
   dec  ecx
   jnz  TOP
end asm   
'
e=timer
print format(numvals,"###,###,### \b"),
print "HexLen = ";format(len(*hexabuff),"###,###,### \b")
print using " #.########## seconds";e-b,
print "       CPS: ";format(int(1/(e-b)*numvals),"###,###,###,###")
print
'
return cast(integer,res-hexabuff)
'
end function
'
function HexToDec(numvals as integer) as integer
'
dim as integer i,res
dim as double b,e
'
dim as ubyte dtable(48 to 70)
for i=48 to 57
   dtable(i)=i-48
next
for i=65 to 70
   dtable(i)=i-55
next
dim as any ptr dtptr=@dtable(0)
'
print "HexToDec",
b=timer
'
decibuff=allocate(numvals)
'
asm
   mov ecx,   dword ptr [numvals]
   mov esi,   dword ptr [hexabuff]
   mov edi,   dword ptr [decibuff]
   mov edx,   dword ptr [dtptr]
TOPZ:
   mov eax,   0
   mov al,    [esi]      'hex char
   mov al,    [edx][eax] 'byte value
   shl al,    4
   mov bl,    al
   inc esi
'
   mov eax,   0
   mov al,    [esi]      'hex char
   mov al,    [edx][eax] 'byte value
   add bl,    al
   mov [edi], bl
   inc edi
   inc esi
'
   dec ecx
   jnz TOPZ
   mov [res], edi
end asm
'
e=timer
print format(abs(cast(integer,res-decibuff)*2),"###,###,### \b"),
print "DecLen = ";format(numvals,"###,###,### \b")
print using " #.########## seconds";e-b,
print "       CPS: ";format(int(1/(e-b)*numvals),"###,###,###,###")
print
'
return cast(integer,res-decibuff)
'
end function

I'm surely committing a thousand asm-SINS. Note this code hasn't been thrashed much, caveat emptor.
tommen
Posts: 16
Joined: Apr 19, 2007 12:41

Postby tommen » May 13, 2007 9:02

Glad my ranting could help ;-)

F.y.i. I'm totally new to x86 asm aswell but a wizz on 6502 asm. And the tricks seem to be portable.

One more thing to think about is doing say two conversions before doing the loop. This will effectivly half the amount of loops you need to do. The gain wil be bigger on larger files. (I Think)^^
Alexa
Posts: 56
Joined: May 01, 2007 20:22

Postby Alexa » May 23, 2007 12:42

Thank you very much, i done the project, it was a FoxPro Accounting App, and Bin -> Hex was a Part of Data Backup on it, they told us to scramble the backup file, easiest way is converting them to 0 1 or binary to hex .... :D , it was enough for those idiots !!

Return to “Beginners”

Who is online

Users browsing this forum: No registered users and 2 guests