md5 hashing bare bones example

Post your FreeBASIC source, examples, tips and tricks here. Please don’t post code without including an explanation.
Post Reply
thrive4
Posts: 70
Joined: Jun 25, 2021 15:32

md5 hashing bare bones example

Post by thrive4 »

A short preamble:
  • do not use this code for sensitive data, as the rossetacode page points out
    'Rosetta Code is not a place you should rely on for examples of code in
    critical roles, including security.
    Also, note that MD5 has been broken and should not be used in applications
    requiring security. For these consider SHA2 or the upcoming SHA3.'
  • there is a cryptlib included in freebasic (with md5 functionality)
    that said there a some issues on getting it to work see:
    viewtopic.php?t=26408
    So I decided to use the code on rosetta (no library required).
  • most os's have md5 support on the commandline
    nix (md5sum)
    windows (bound to have an api or some other powershell function)
Right with that out the way.
While I was toying around with mpv (https://mpv.io) I came across
an elegant solution for creating bookmarks for video's.
In short the app stuffs ini file(s) in a dir called 'watch_later'
the ini filenames are hashed in md5.

When mpv is initiated with a filename it quickly hashes the name
and checks the 'watch_later' folder for a match.

example content md5 ini file
start=5.583000
pause=yes
fullscreen=no

Of course there are many other ways to utilize md5 but
this example gives some context.
This code creates a file with a md5 hashed version of the filename
usage: -file (filename only) or -full (includes path) <filename>.<ext>

Code: Select all

' MD5 hash from the Wikipedia page "MD5"
' compile with: fbc -s console
' from https://rosettacode.org/wiki/MD5/Implementation#FreeBASIC
' note md5 is not reversible, at least it shouldn't be...
' added basic file i/o thrive4 2022

' macro for a rotate left
#Macro ROtate_Left (x, n) ' rotate left
  (x) = (x) Shl (n) + (x) Shr (32 - (n))
#EndMacro

Function MD5(test_str As String) As String

    Dim As String message = test_str   ' strings are passed as ByRef's

    Dim As UByte sx, s(0 To ...) = { 7, 12, 17, 22,  7, 12, 17, 22,  7, 12, _
    17, 22,  7, 12, 17, 22,  5,  9, 14, 20,  5,  9, 14, 20,  5,  9, 14, 20, _
    5,  9, 14, 20,  4, 11, 16, 23,  4, 11, 16, 23,  4, 11, 16, 23,  4, 11, _
    16, 23,  6, 10, 15, 21,  6, 10, 15, 21,  6, 10, 15, 21,  6, 10, 15, 21 }

    Dim As UInteger<32> K(0 To ...) = { &Hd76aa478, &He8c7b756, &H242070db, _
    &Hc1bdceee, &Hf57c0faf, &H4787c62a, &Ha8304613, &Hfd469501, &H698098d8, _
    &H8b44f7af, &Hffff5bb1, &H895cd7be, &H6b901122, &Hfd987193, &Ha679438e, _
    &H49b40821, &Hf61e2562, &Hc040b340, &H265e5a51, &He9b6c7aa, &Hd62f105d, _
    &H02441453, &Hd8a1e681, &He7d3fbc8, &H21e1cde6, &Hc33707d6, &Hf4d50d87, _
    &H455a14ed, &Ha9e3e905, &Hfcefa3f8, &H676f02d9, &H8d2a4c8a, &Hfffa3942, _
    &H8771f681, &H6d9d6122, &Hfde5380c, &Ha4beea44, &H4bdecfa9, &Hf6bb4b60, _
    &Hbebfbc70, &H289b7ec6, &Heaa127fa, &Hd4ef3085, &H04881d05, &Hd9d4d039, _
    &He6db99e5, &H1fa27cf8, &Hc4ac5665, &Hf4292244, &H432aff97, &Hab9423a7, _
    &Hfc93a039, &H655b59c3, &H8f0ccc92, &Hffeff47d, &H85845dd1, &H6fa87e4f, _
    &Hfe2ce6e0, &Ha3014314, &H4e0811a1, &Hf7537e82, &Hbd3af235, &H2ad7d2bb, _
                                                              &Heb86d391 }

    ' Initialize variables
    Dim As UInteger<32> A, a0 = &H67452301
    Dim As UInteger<32> B, b0 = &Hefcdab89
    Dim As UInteger<32> C, c0 = &H98badcfe
    Dim As UInteger<32> D, d0 = &H10325476
    Dim As UInteger<32> dtemp, F, g, temp

    Dim As Long i, j

    Dim As ULongInt l = Len(message)
    ' set the first bit after the message to 1
    message = message + Chr(1 Shl 7)
    ' add one char to the length
    Dim As ULong padding = 64 - ((l +1) Mod (512 \ 8)) ' 512 \ 8 = 64 char.

    ' check if we have enough room for inserting the length
    If padding < 8 Then padding = padding + 64

    message = message + String(padding, Chr(0))   ' adjust length
    Dim As ULong l1 = Len(message)                ' new length

    l = l * 8    ' orignal length in bits
    ' create ubyte ptr to point to l ( = length in bits)
    Dim As UByte Ptr ub_ptr = Cast(UByte Ptr, @l)

    For i = 0 To 7  'copy length of message to the last 8 bytes
    message[l1 -8 + i] = ub_ptr[i]
    Next

    For j = 0 To (l1 -1) \ 64 ' split into block of 64 bytes

    A = a0 : B = b0 : C = c0 : D = d0

    ' break chunk into 16 32bit uinteger
    Dim As UInteger<32> Ptr M = Cast(UInteger<32> Ptr, @message[j * 64])

    For i = 0 To 63
      Select Case As Const i
        Case 0 To 15
          F = (B And C) Or ((Not B) And D)
          g = i
        Case 16 To 31
          F = (B And D) Or (C And (Not D))
          g = (i * 5 +1) Mod 16
        Case 32 To 47
          F = (B Xor C Xor D)
          g = (i * 3 +5) Mod 16
        Case 48 To 63
          F = C Xor (B Or (Not D))
          g = (i * 7) Mod 16
      End Select
      dtemp = D
      D = C
      C = B
      temp = A + F + K(i)+ M[g] : ROtate_left(temp, s(i))
      B = B + temp
      A = dtemp
    Next

    a0 += A : b0 += B : c0 += C : d0 += D

    Next

    Dim As String answer
    ' convert a0, b0, c0 and d0 in hex, then add, low order first
    Dim As String s1 = Hex(a0, 8)
    For i = 7 To 1 Step -2 : answer +=Mid(s1, i, 2) : Next
    s1 = Hex(b0, 8)
    For i = 7 To 1 Step -2 : answer +=Mid(s1, i, 2) : Next
    s1 = Hex(c0, 8)
    For i = 7 To 1 Step -2 : answer +=Mid(s1, i, 2) : Next
    s1 = Hex(d0, 8)
    For i = 7 To 1 Step -2 : answer +=Mid(s1, i, 2) : Next

    Return LCase(answer)

End Function

' setup file output
dim filename as string

' parse arguments
dim validarg as boolean
if command(1) = "/?" or command(1) = "-man" then
    print "create md5 hash version of filename"
    print "usage: -file (filename only) or -full (includes path) <filename>.<ext>"
    print "output: md5 hash"
    end
end if
select case command(1)
    case "-full"
        validarg = true
    case "-file"
        validarg = true
    case else
        print "error: invalid switch " + command(1) + " valid switch -file or -full"
        'end
end select
if instr(command(2), ".") <> 0 and validarg then        
    if command(1) = "-full" then
        filename = exepath + command(2)
    else
        filename = command(2)
    end if
    validarg = true
else
    print "error: file " + command(2) + " not found or supported"
    'end
end if

if len(filename) > 0 then
    print filename
    print "md5: " + md5(filename)
    open md5(filename) for output as #1
        print #1, filename
        print #1, md5(filename)
    close
else
    print "test the test"
    print "md5: " + md5("test the test")
    While InKey <> "" : Wend
    sleep
endif

end
Vortex
Posts: 118
Joined: Sep 19, 2005 9:50

Re: md5 hashing bare bones example

Post by Vortex »

Hi thrive4,

Thanks for your code. Here is a recursive search with forfiles :

Code: Select all

forfiles /P D:\Directory /M *.* /S /C "cmd /C D:\FreeBasic\FBmd5.exe -file @file >> md5.txt" 
A small modification in your code :

Code: Select all

.
.
if len(filename) > 0 then
    print md5(filename)+" *"+filename
else
    print "test the test"
    print "md5: " + md5("test the test")
    While InKey <> "" : Wend
    sleep
endif

end
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: md5 hashing bare bones example

Post by deltarho[1859] »

Image Preamble
Image Clean and readable code
Image Well commented
Image No libraries needed

On my machine, MD5() runs about 15% faster than using the Windows APIs in 32-bit mode and 6% faster in 64-bit mode.

Great stuff, thrive4.
thrive4
Posts: 70
Joined: Jun 25, 2021 15:32

Re: md5 hashing bare bones example

Post by thrive4 »

A little late, well with whats going on the world ya know...

@ Vortex
You are welcome and that is a nice addition thank you.

@ deltarho[1859]
Thanks for feedback!
I like to adhere to the guide line:
viewtopic.php?t=26993
Always nice to see it appreciated.

Nice speed bump there would have not expected
the code to actually outperform the win api
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: md5 hashing bare bones example

Post by deltarho[1859] »

thrive4 wrote:note that MD5 has been broken
It has been broken in the laboratory, so to speak, but there is no evidence, yet, of collisions in the 'wild'.

MD5-HMAC has not been broken in the laboratory: HMAC: Keyed-Hashing for Message Authentication


Having said that, I would not use less than SHA2 for cryptographic purposes.

I have just started a project for Blake2 which has the same security strength of SHA2 and is faster than MD5 or SHA1. Don't hold your breath on my completing that anytime soon. Blake3 is faster still, but the ink is not dry on that yet.
Post Reply