Using a static library: linker problems

[Vortex]

Hi operator+,

If you have a collection of small functions then a static library can be preferable. As you know, the linker extracts the necessary components from the static librray and links them together with your project modules.

[Munair]

Both dynamic and static linking have advantages and disadvantages. The world of programming isn't just black and white. Almost every day I encounter situations that require compromise.

[jj2007]

Oh. Please just use a DLL! I hate static linking fanboy so much!

That is simply a matter of taste (and Munair is absolutely right). Check this archive with an update on my double-to-string conversion for FreeBasic.

The point is, and I am sure you agree, that FreeBasic does not offer the most immediate user experience. It took me several hours to find out how to statically link that library, with a little help from my friends, and again, yesterday night, several hours to find out how to dynamically link the library. Especially the linker sucks, but both compiler and linker suffer from cryptic and misleading error messages. So you waste precious hours with trial and error, and search the Internet for hints to a solution. At the end, it may look so simple and cute, see below, but this language does not deserve the B in its name.

Code: Select all

Dim As Any Ptr libhandle = DyLibLoad("MbDouble2String.dll")
Dim Double2String As Function(byval arg1 as zstring ptr, byval arg2 as double) As zstring ptr
Double2String = DyLibSymbol(libhandle, "Double2String")
Dim Myd as double=1234567890.1234567890
Dim Mbs As zstring ptr=allocate(40)
  Mbs=Double2String("%Gf", MyD)
  Print "Result=";*Mbs
  Print "Result=";*Double2String("%Gf", MyD)
sleep

Output:
Result=1234567890.123457
Result=1234567890.123457

Btw, instead of the zstring ptr, you can also use a simple string:

Code: Select all

Dim Double2String As Function(byval arg1 as zstring ptr, byval arg2 as double) As string
Dim As Any Ptr libhandle = DyLibLoad("MbDouble2String.dll")
Double2String=DyLibSymbol(libhandle, "Double2String")
Dim Myd as double=1234567890.1234567890
Dim Mbs As string
Mbs=Double2String("%Gf", MyD)
Print "Result A=";Mbs
'Print "Result B=";Double2String("%Gf", MyD)	' crashes
sleep

However, the second Print line crashes. The result B line looks almost identically to the previous snippet, where it works fine, but here it crashes. No compiler or linker warning, it just crashes. I know why, because I have 40 years of experience with BASIC and Assembly, and because I wrote the library. But what about that guy who is a Beginner, as in "Beginners' All-purpose Symbolic Instruction Code"?

[St_W]

I wrote something similar to this in some other thread recently: FreeBasic uses the same technical basis for compilation units and linking as C does, thus we also get the same complexity. Not sure if it can be reasonably simplified (at least as soon as something isn't working you need to dig into the technical details).

I don't really agree that the linker gives cryptic or misleading error messages, at least not in general. In this case for example the linker described pretty well what was wrong.

regarding your dynamic linking example:
A intermediate coder probably won't manually load the library and manually resolve it's methods, but would use an import library. The import library is even automatically created by FreeBasic, if you don't have any. And if you create a DLL with FreeBasic it creates an import library by default as well. An intermediate coder most likely won't use assembly language either, that's for experts only (and even then rarely really needed).

Out of interest: what's wrong with the example code that you say crashes? I can't see anything wrong. Or is the implementation in the library somehow wrong?

[jj2007]

Out of interest: what's wrong with the example code that you say crashes? I can't see anything wrong. Or is the implementation in the library somehow wrong?

Code: Select all

Mbs=Double2String("%Gf", MyD)	' does some automagic z$ to $ conversion
Print "Result A=";Mbs
asm int 3
Print "Result B=";Double2String("%Gf", MyD)	' crashes (Float2Asc returns a zstring)
asm nop

The first Mbs= line assigns the string Mbs, and that's working fine.
The second Print line uses the passed zstring ptr directly, and something bad happens afterwards:

Code: Select all

  int3
  push 0
  push 9                                   ; /Arg2 = 9
  push offset 0040603C                     ; |Arg1 = ASCII "Result B="
  call 00401FB0                            ; \TmpFb.00401FB0
  push eax
  push 0
  call 00401810
  push 1
  push dword ptr [local.4]
  push dword ptr [local.5]
  push offset 0040602C                     ; ASCII "%Gf"
  call near [local.2]
  push eax                                 ; ASCII "1234567890.123457"
  push 0
  call 00401810                            ; calls 004017B0 -> crash
  nop
  push -1                                  ; /Arg1 = -1
  call 00401670                            ; \TmpFb.00401670
  ...
  mov [esp+4], edx                         ; ASCII "1234567890.123457"
  call 004017B0                            ; calls some more procs and crashes miserably

The call near [local.2] (i.e. Double2String) returns a pointer to the string "1234567890.123457", which is passed to FB's Print routine - where it crashes.

[Munair]

but this language does not deserve the B in its name.

Using libraries or inline asm is for intermediate and advanced users. If a BASIC dialect offers these features how do you propose them B-friendly? Even in 1980s BASIC programming books those features were covered at the end (if at all) or in books specifically targeting advanced users. For example, I have two great books for QuickBASIC, one for learning the basics and one for advanced features (including writing TSRs).

[miilvyxg]

A DLL could be loaded by Java's JNA and C#'s P/Invoke. A static library can't. This is the different. I want my stuffs to be able to be used from Java and C#. So I prefer DLL. This is about interpreted language. Yes, I know Java and C# are not interpreted language the same sense as Python. But just treat them as one.

A DLL is pretty much self contained. It has a copy of FB's rtlib in it. A static library, even if you could link with other compiled language, is much more troublesome. You have to link the FB's rtlib yourself. Not a wise option to use static library if you want your stuffs to be used by other compiled language.

[coderJeff]

'string' is not equivalent to 'zstring ptr' - not for beginners if coding in both fb and assembly. Or just use fb only and don't worry about how it works under the hood.

On 32-bit, 'zstring ptr' is a pointer (size 4) and 'string' is a descriptor (size 12).

Have a look at what fbc is doing for a function that returns the argument:

Code: Select all

function z( byval x as zstring ptr ) as zstring ptr
	return x 
end function

function s( byval x as zstring ptr ) as string
	return *x	
end function

Code: Select all

	.intel_syntax noprefix

.section .text
.balign 16

.globl _Z@4
_Z@4:
push ebp
mov ebp, esp
sub esp, 4                    ; allocate local on stack for return value
mov dword ptr [ebp-4], 0      ; initialize return pointer to NULL
.L_0004:
mov eax, dword ptr [ebp+8]    ; copy argument to 
mov dword ptr [ebp-4], eax    ; local return value
.L_0005:
mov eax, dword ptr [ebp-4]    ; copy local return value to return register
mov esp, ebp
pop ebp
ret 4                         ; stdcall clean-up
.balign 16

.globl _S@4
_S@4:
push ebp
mov ebp, esp
sub esp, 12                   ; allocate local descriptor
mov dword ptr [ebp-12], 0     ; allocate local descriptor
mov dword ptr [ebp-8], 0      ; allocate local descriptor
mov dword ptr [ebp-4], 0      ; allocate local descriptor
.L_0006:
push 0
push 0
push dword ptr [ebp+8]
push -1
lea eax, [ebp-12]             
push eax
call _fb_StrInit@20           ; initialize local string with argument
.L_0007:
lea eax, [ebp-12]
push eax
call _fb_StrAllocTempResult@4 ; allocate temp descriptor to return to caller
mov esp, ebp
pop ebp
ret 4                         ; stdcall clean-up

The declaration needs to match whatever is in the library, calling convention, arguments, return registers, etc. And in the case of fb's string type which is not fully contained in a single register, cooperation with the fb runtime string management.

[dodicat]

Oh. Please just use a DLL! I hate static linking fanboy so much!

That is simply a matter of taste (and Munair is absolutely right). Check this archive with an update on my double-to-string conversion for FreeBasic.

...

Hi jj2007
My speed test is one million runs, but your dll chokes at 1000000.

Dim Myd as double=1234567890.1234567890
Dim Mbs As zstring ptr=allocate(40)
dim as double t=timer
for n as long=1 to 1000000
Mbs=Double2String("%Gf", MyD)
next n
print timer-t,*Mbs
sleep

It is OK at 100000 with a good speed.

[jj2007]

My speed test is one million runs, but your dll chokes at 1000000

Yep, you are right, I had erroneously used syscall instead of stdcall.

FreeBasic uses the same technical basis for compilation units and linking as C does, thus we also get the same complexity.

The complexity is very similar, here are three examples using a new version of the DLL:

Code: Select all

#include <windows.h>	// using a DLL in C
typedef char* (__stdcall *CharPlusDouble)(char*, double);
int main(void) {
  double MyD=1234567890.123456789;
  handle_t hLib=LoadLibrary("MbDouble2String.dll");
  CharPlusDouble fwp = (CharPlusDouble) GetProcAddress(hLib, "Double2String");
  char* MyString=fwp("%Gf", MyD);
  int T=GetTickCount();
  for (int i=0; i<1000000; i++){
	MyString=fwp("%Gf", MyD);
  }
  T=GetTickCount()-T;
  printf("Here it is: [%s], %i ms later\n", MyString, T);
  MessageBox(0, MyString, "Test:", MB_OK);
  FreeLibrary(hLib);	// not really needed, but...
}

Code: Select all

Dim Double2String As Function stdcall (byval arg1 as zstring ptr, byval arg2 as double) As zstring ptr
Dim As Any Ptr libhandle = DyLibLoad("MbDouble2String.dll")
Double2String=DyLibSymbol(libhandle, "Double2String")
Dim Myd as double=1234567890.1234567890, Mbs As zstring ptr
Mbs=Double2String("%Gf", MyD)	' does some automagic z$ to $ conversion

Dim as double t=Timer
For n as long=1 to 1000000
	Mbs=Double2String("%Gf", MyD)
Next
Print int((Timer-t)*1000); " ms for ";*Mbs

Dim Fbs as string
t=Timer
For n as long=1 to 1000000
	Fbs=Str(MyD)
Next
Print int((Timer-t)*1000); " ms for ";Fbs

Sleep

Code: Select all

include \masm32\MasmBasic\MasmBasic.inc
SetGlobals REAL8 MyD=1234567890.123456789, My$
Init
  Cls
  PrintCpu 0	; show which CPU you are using

  Dll "MbDouble2String"		; load a DLL
  Declare Double2String, 2	; two arguments
  
  NanoTimer()
  push 999999
  xor ecx, ecx
  .Repeat
	inc ecx
	void Double2String("%Gf", MyD)	; get a string
	dec stack
  .Until Sign?
  pop edx
  PrintLine NanoTimer$(), Str$(" for %i Million conversions", ecx) 
  Let My$=Double2String("%Gf", MyD)	; assign a string
  printf("Here it is: [%s]\n", My$)
  MsgBox 0, My$, "Test:", MB_OK
EndOfCode

Note the complexity of

Code: Select all

  Dll "MbDouble2String"		; load a DLL
  Declare Double2String, 2	; two arguments

This is Assembly

[St_W]

Your example is complicated, but that's just not how you'd do it in FreeBasic.

Consider having a small library compiled with fbc -dll myLib.bas

Code: Select all

function Double2String(format as String, value as Double) as String Export
  'missing real implementation, just for demo purposes
	return str(value)
end function

And this is how an application would use it, compile with fbc myApp.bas

Code: Select all

declare function Double2String lib "myLib" (format as String, value as Double) as String

print Double2String("%g", 12345.6789)

The only added complexity compared to your Assembly-language example is the added type-safety.

[jj2007]

Consider having a small library compiled with fbc -dll myLib.bas

I consider a small library not compiled with FreeBasic. There are thousands of them, mostly written in C or C++, and on Windows, almost all of them use zstrings, not BASIC strings.

[St_W]

Only difference is that you have to do the memory management yourself:

myLib.bas (this could be implemented in any other language with a C-compatible ABI)

Code: Select all

function Double2String(format as ZString ptr, value as Double) as ZString ptr Export
  'missing real implementation, just for demo purposes
	dim tmp as string = str(value)
	dim result as zstring ptr
	result = allocate(len(tmp) + 1)
	*result = tmp
	return result
end function

myApp.bas

Code: Select all

declare function Double2String lib "myLib" (format as ZString ptr, value as Double) as ZString ptr

dim as zstring ptr result
result = Double2String("%g", 12345.6789)
print *result
deallocate(result)  ' depending on the malloc implementation used in the library another deallocation implementation might be needed

[jj2007]

Which memory management?

Code: Select all

Dim Double2String As Function stdcall (byval arg1 as zstring ptr, byval arg2 as double) As zstring ptr
Dim As Any Ptr libhandle = DyLibLoad("MbDouble2String.dll")
Double2String=DyLibSymbol(libhandle, "Double2String")
#define elements 5000000
Dim Shared As zString ptr Mbsa(1 to elements)
Dim Shared As String Fbsa(1 to elements)
Dim as long n
Randomize 12345689
Dim as double t=Timer
For n=1 to elements
	Mbsa(n)=Double2String("%Gf", Rnd(9999999))
Next
Print int((Timer-t)*1000); " ms for ";*Mbsa(n-1)
Randomize 12345689
t=Timer
For n=1 to elements
	Fbsa(n)=Str(Rnd(9999999))
Next
Print int((Timer-t)*1000); " ms for ";Fbsa(n-1)
Sleep

Output on my Core i5-2450M CPU @ 2.50GHz:
1508 ms for 0.2741604959592223
5064 ms for 0.2741604959592223

[St_W]

Someone must handle the memory - either the library or the application using the library.
In the simplest case you could implement a library with a static buffer of fixed length. Then you don't need to care about memory management in the application. (in practice nobody would do something like that for a method where the result length is unrestricted, however)

But that wasn't actually my point in my previous answer. I wanted to show that it doesn't change how the DLL is used - the concept is very much the same, no matter whether the library is written in FB or in C. So one line of declaration is all you need, it can be used like a function defined in the same module afterwards.

and btw: the code is exactly the same when you link a static library instead of a dynamic one. Can't see how it could be easier

Check this archive with an update on my double-to-string conversion for FreeBasic

btw2: unfortunately that link is dead