Runtime location of a Function by name in a string

External libraries (GTK, GSL, SDL, Allegro, OpenGL, etc) questions.
wallyg
Posts: 267
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Runtime location of a Function by name in a string

Post by wallyg »

When I use GTK/Glade and that great tool GladeToBac the file projectname.ui is created by Glade. This file is an XML version of the GUI that I designed via Glade. It contains lines like this in the XML to specify the name of a routine to execute at runtime for a widget to handle an event.


<signal name="activate" handler="MiscMenuStatusIcon_activate_cb" swapped="no"/>


Which indicates the name of the routine to execute when an event occurs.

At runtime, I use a gtk builder routine that reads the *.ui and internally builds the gtk data structure needed for execution and will call the specified routine when needed.

My question is since gtk can find where a routine is physically loaded at runtime from just the string version of the routine that it needs to execute, is there a FB routine I can directly call to get the address of a routine "AddNumber" so that I can call it with the proper arguments? This is for a non-gtk program and I do not want to load all of gtk just to get this capability.

Wally
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Runtime location of a Function by name in a string

Post by caseih »

Correct me if I'm wrong but doesn't gtk_builder_connect_signals() only work if your callback function is defined in C using the G_MODULE_EXPORT macro, which adds runtime code to place a pointer to the function into a symbol table? So you'd need to do something similar. It's not possible to easily obtain the address of a function at runtime otherwise, at least for compiled programs, without doing something special to mark them. Unless you leave in debugging information, the names of functions and variables are usually lost, and only referred to in compiled code by addresses. When making or working with dlls and shared objects, there are special tables the compiler uses to match names to functions. I think this uses the export keyword. But I'm not sure it works at all outside of dlls, or interfacing with dlls.
adeyblue
Posts: 299
Joined: Nov 07, 2019 20:08

Re: Runtime location of a Function by name in a string

Post by adeyblue »

I was usurped but anyway...
wallyg wrote:My question is since gtk can find where a routine is physically loaded at runtime from just the string version of the routine that it needs to execute, is there a FB routine I can directly call to get the address of a routine "AddNumber" so that I can call it with the proper arguments?
I don't think GTK does that. I don't know how it works, but if the XML is turned into a file you then compile and link with your code, it'll just use the name as a function pointer by sticking an @ in front of it. It doesn't 'know' or store its name in any sense, it just use it as a normal function pointer.

Basically, it'll turn this
<signal name="activate" handler="MiscMenuStatusIcon_activate_cb" swapped="no"/>
into something like
Dim activateHandler_ As Function (ByVal As ULong, ByVal As ULong) As Long '' I don't know what the actual type is
activateHandler_ = @MiscMenuStatusIcon_activate_cb
'' and then later
dim result As Long = activateHandler(2, 5)

There probably isn't a gtk function which could take "MiscMenuStatusIcon_activate_cb" as an argument and give you the address back. If there is, or if that's what you want, then that can only be done by creating some mapping between the names and the function addresses. You can't arbitrarily lookup a function name typed on the command line (for example) and call it, it will have to be explicitly accounted for, and only those accounted for will be able to be called like that.

To be able to get it to work yourself, you have to implement a table. The two simplest ways are to create an array of
type MappedFunction
address as MyFunctionType
name as String
end type
And then you search for the name in the table and call address().

or you could export your functions from the exe/dll/so and use DyLibSymbol to get the function address.

Unless you want to stick to just one type of function though (ie Function (ByVal ULong, ByVal ULong) As Long etc) you'll need to also have some way of knowing what arguments the addresses need in order to be able to call them properly. This is what the 'name' attribute in the glade xml fragment is for, so GTK knows what arguments to give your function.

Basically, reflection (the technical term for this) is laborious to implement in languages that don't already have support for it. And the fancier you want your system to be, the more cumbersome and error prone it becomes.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Runtime location of a Function by name in a string

Post by caseih »

adeyblue wrote:I don't think GTK does that. I don't know how it works, but if the XML is turned into a file you then compile and link with your code, it'll just use the name as a function pointer by sticking an @ in front of it. It doesn't 'know' or store its name in any sense, it just use it as a normal function pointer.
gtk_builder_connect_signals() does work at *runtime* and connects the events to the callbacks defined in the XML. If you changed the XML and reloaded it, gtk_builder_connect_signals() would indeed make the connection dynamically, with no compiled shim layer. However, it appears to do this because there is a compiled table in the binary that was created by the macro I spoke of. It does not just work with any old function.

But definitely you are correct. Your methods are the only way to do what the OP wants to do.
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: Runtime location of a Function by name in a string

Post by TJF »

wallyg wrote:My question is since gtk can find where a routine is physically loaded at runtime from just the string version of the routine that it needs to execute, is there a FB routine I can directly call to get the address of a routine "AddNumber" so that I can call it with the proper arguments? This is for a non-gtk program ...
Find info at

https://www.freebasic.net/wiki/ProPgCallback
https://www.freebasic.net/wiki/wikka.ph ... gOpProcptr
https://www.freebasic.net/wiki/KeyPgDylibsymbol

and the linked subpages.

Regards
wallyg
Posts: 267
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Re: Runtime location of a Function by name in a string

Post by wallyg »

Thank you for the excellent responses. After reading them, I looked again at the XML generated by Glade (*.ui file). And the line

<signal name="activate" handler="MiscMenuStatusIcon_activate_cb" swapped="no"/>

Appears in the XML file. I then checked the code generated by GladeToBac that has to be included in the FB code when it is compiled. The name MiscMenuStatusIcon_activate_cb does not appear in that generated code.

I do have a routine MiscMenuStatusIcon_activate_cb in my code, that has the header listed below generated by GladeToBac.

Sub MiscMenuStatusIcon_activate_cb CDECL ALIAS "MiscMenuStatusIcon_activate_cb" ( _
BYVAL widget AS GtkWidget PTR, _
BYVAL user_data AS gpointer) EXPORT
...
End Sub

There is no other reference to MiscMenuStatusIcon_activate_cb anyplace else in the FB code that is successfully compiled.

Maybe the Alias clause or the Export clause causes the symbol MiscMenuStatusIcon_activate_cb to be defined by the loader so that a routine could find the location of the Sub at runtime? The code when compiled and executed works and the routine does get executed. Therefore the gtk routine

gtk_builder_add_from_file(XML, "Designer.ui", @meld)

does properly find the routine MiscMenuStatusIcon_activate_cb at runtime. The variable XML is a pointer to the generated gtk data structure and meld is the location gtk_builder returns any error code.

Again thank you.

Wally
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: Runtime location of a Function by name in a string

Post by TJF »

Correct. fbc creates a symbol table (EXPORT keyword) in the executable. The runtime linker makes that table available for a dynamically loaded library. GTK uses GModule function family in order to lookup the callbacks physical addresses after linking. The FreeBASIC eqivalent is the DYLIB... function family.
wallyg
Posts: 267
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Re: Runtime location of a Function by name in a string

Post by wallyg »

Thank you for your help.

I looked up the DyLib commands. And it seems that what I want to use is:

Declare Function DyLibSymbol ( ByVal library As Any Ptr, ByRef symbol As String ) As Any Ptr

And I think I need to use

Declare Function DyLibLoad ( ByRef filename As String ) As Any Pointer

To get a pointer to pass to the DyLibSymbol routine. But in my case, the routine I need to reference has been already loaded in the *.exe that started execution. See the above Glade example where the name of the routine is in the XML file read by gtk_builder and the routine referenced was compiled by FBC and had ALIAS and EXPORT parameters on it.

Should I use the name of the executable for the DyLibLoad call? Or is there a special pointer value (NULL for instance) that I can place on the DyLibSymbol call?

Thanks again for your help

Wally
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: Runtime location of a Function by name in a string

Post by TJF »

wallyg wrote:Should I use the name of the executable for the DyLibLoad call?
No
wallyg wrote:Or is there a special pointer value (NULL for instance) that I can place on the DyLibSymbol call?
Yes, it seems so (never tested):
KeyPgDylibsymbol wrote:If libhandle is 0, the symbol is searched in the current executable or dll.
You'll neither need DYLIBOPEN nor DYLIBFREE.
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: Runtime location of a Function by name in a string

Post by TJF »

BTW:
EXPORT provides the routine in the symbol table, but fbc uses capital letters for the symbol name.
ALIAS takes care of the correct character cases.
Last edited by TJF on Jun 01, 2021 9:03, edited 1 time in total.
wallyg
Posts: 267
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Re: Runtime location of a Function by name in a string

Post by wallyg »

Thank you all for the information provided. It is such a pleasure to ask a question and get such knowledgeable responses.

I now have an additional question about the DLL file I want to load via dyLibLoad. I have a main program that queries the user via a complex series of dialogs and creates a source code file consisting of a set of FreeBasic Functions. I then want to call FBC via EXEC to compile it into a DLL. Then after a successful return from the compiler to load the generated DLL via dyLibLoad and use dyLibSymbol to execute the functions just compiled and put into the DLL. So far so good.

My question is when I create the source file of functions to be compiled, what if these functions call some utility routines that are already loaded in the main program? (Do they need the EXPORT keyword?) When dyLibLoad is called do references to these utility routines be connected to and actually execute the versions stored in the original main program or does the DLL containing these functions have to #include the source code of all the utilities used and be compiled and placed in the DLL? So each DLL generated has copies of all utilities?

To muddy the waters even more. What about variables/arrays that are declared as:

Static Shared as Integer MaximumEntities
Static Shared as UDT ptr LocationOfABigTable
Static Shared as UDT TheWorldAsISeeIt(1 to 100,1 to 1000)

If these exist in the main program that loads a DLL that contains the same declarations, do both the main program and the DLL routines use the same location in memory, or do both have separate areas? Obviously, I can pass the address of these locations to the DLL routines, but was wondering if I needed to? Or is there something I need to do differently to accomplish what I need to?

Thanks in advance for your consideration and help.

Wally
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Runtime location of a Function by name in a string

Post by caseih »

I'm not sure DLLs can't see symbols in the main program, other than through the "extern" mechanism. Your main program simply did not exist when the DLL was compiled. Certainly if the DLL is dlopen'ed it cannot. If it's dynamically linked, however, you possibly can define symbols in the DLL that are implemented in your main program (such as a declare function or declare sub). If there are conflicts, though, I'm not sure how that works.

As for in-memory location, the DLL's code will occupy its own space in memory. It won't overwrite your main program's variables or functions.
wallyg
Posts: 267
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Re: Runtime location of a Function by name in a string

Post by wallyg »

So if I declare the utility routines EXTERN in the main program being executed and EXECing the FB compiler and I pass the executable currently being executed to the FB compiler then the generated DLL would be able to find the utilities and use them for reference to the utilities from the DLL to the currently executing program which obviously exists at the time the DLL is being created?

How do I specify the currently executing program to the FB compiler so that it knows about it?

Does compiling the main program with -g help? How about the DLL with -g?

Does using EXTERN on the variable declarations help? And/Or EXPORT?
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Runtime location of a Function by name in a string

Post by dodicat »

Regarding your shared variables and array, they can be exported via byref functions.

Code: Select all


type udt
    as long x
end type

static as udt u
u.x=13

function MaximumEntities() byref as integer export
    static as integer i
    return i
end function

function LocationOfABigTable() byref as udt ptr export
    static as udt ptr i
    return i
end function

function TheWorldAsISeeIt(n as integer,m as integer) byref as udt export
    static as udt i(1 to 100,1 to 100)
    return i(n,m)
end function


for n as long=1 to 100
    for m as long=1 to 100
     TheWorldAsISeeIt(n,m)=type(n+m)
 next
next

LocationOfABigTable()=@u
MaximumEntities()=78

'======================================================



print MaximumEntities

print LocationOfABigTable()->x

print TheWorldAsISeeIt(6,8).x

MaximumEntities()=-39
print MaximumEntities

sleep

'Static Shared as Integer MaximumEntities
'Static Shared as UDT ptr LocationOfABigTable
'Static Shared as UDT TheWorldAsISeeIt(1 to 100,1 to 1000) 
Last edited by dodicat on Sep 29, 2021 9:44, edited 1 time in total.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Runtime location of a Function by name in a string

Post by caseih »

wallyg wrote:So if I declare the utility routines EXTERN in the main program being executed and EXECing the FB compiler and I pass the executable currently being executed to the FB compiler then the generated DLL would be able to find the utilities and use them for reference to the utilities from the DLL to the currently executing program which obviously exists at the time the DLL is being created?
If I understand you, and if I'm not mistaken, you would use EXPORT in the main program when defining the utility routine you want the DLL to call. In the dll, you'd just "declare function" the utility routine, and then call it normally. The linker will resolve the connection between the dll and the main program. I think extern is only for variables. At least that's what my testing just now indicates.
How do I specify the currently executing program to the FB compiler so that it knows about it?
You don't. The compiler only knows about what it sees in the source code file you are compiling, and anything you included from .bi files. The compiler has no knowledge of dlls or anything like that. If there are conflicting definitions in a .bi file and your source file, the compiler will flag that as a syntax error.

Later on, if there are symbol collisions between a DLL and your main program, the linker will find those at run time, and if there is a problem you'll get an error before the program even starts running.
Does compiling the main program with -g help? How about the DLL with -g?
-g simply adds debugging symbols to the binary or dll so that if you run your program in a debugger, you can step through your code line by line.
Does using EXTERN on the variable declarations help? And/Or EXPORT?
Extern tells the linker that the current module (be it a DLL or a compiled object module) references an object that is defined externally. So at runtime the linker has to match that up with a symbol and make the connection. Under the hood I'm sure there is some kind of pointer dereferencing going on.

EXPORT tells the compiler to make sure the symbol is visible to the linker later on. DLLs may have private functions for internal use that are not EXPORTed so the user of the DLL cannot see them or call them. DLLs can also have private global variables.
Post Reply