variadic functions and argument lists in fbc

For other topics related to the FreeBASIC project or its community.
coderJeff
Site Admin
Posts: 2964
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

variadic functions and argument lists in fbc

Postby coderJeff » Nov 26, 2018 17:35

fbc has a few macros for working with variadic functions: va_first, va_next(), va_arg(), that worked using a pointer to the argument stack. These macros don't work on all targets (like 64-bit) because arguments can be passed to procedures in cpu registers.

sf.net bug https://sourceforge.net/p/fbc/bugs/881/

Working on a replacement though, it's not quite ready to merge in to master:
github https://github.com/freebasic/fbc/pull/115

In the update, to work with variadic functions and argument lists, here's what we've come up with:

cva_list is added as a default type by fbc to expose the argument list. It works kind of like an object, in that usually we don't need to know what's doing, only how to use it. There are a few methods in it's API for accessing a variable length list of arguments passed in to a procedure..

Code: Select all

dim args as cva_list = any
cva_start( args, param )  '' constructor
cva_copy( dest, args )    '' copy constructor
cva_end( args )           '' destructor
result = cva_arg( args, datatype ) '' method returns data and advances to next argument


Note the constructor/destructor pairs:
- cva_start() needs exactly one matching cva_end()
- cva_copy() needs exactly one matching cva_end()

If it looks familiar, it is because it's just about the same in C. We could have come up with our own api, but then it wouldn't be compatible (link-able) with anything but our own code.

If you are currently using any of the new names being added: cva_list, cva_start, cva_end, cva_copy, cva_arg and depending on target __va_list_tag, __va_list, you will need to use #undef to get it out of your way, or rename the symbol in your code.

- No documentation written yet. This post is the first info.
- No guarantees that it is bug free. Please submit bug reports.
- Need help testing on ARM targets. (Or I have to create a VM for it)

Examples:

Iterate through a list of arguments

Code: Select all

sub proc cdecl( n as integer, ... )
   dim args as cva_list
   cva_start( args, n )
   for i as integer = 1 to n
      print "arg " & i & " = " & cva_arg( args, integer )
   next i
   cva_end( args )
end sub

proc( 3, 11, 22, 33 )


Pass argument list to another procedure

Code: Select all


sub proc2 cdecl( n as integer, byval args as cva_list )
   dim x as cva_list = any
   cva_copy( x, args )
   for i as integer = 1 to n
      print "arg " & i & " = " & cva_arg( x, integer )
   next i
   cva_end( x )
end sub

sub proc1 cdecl( n as integer, ... )
   dim x as cva_list = any
   cva_start( x, n )
   proc2( n, x )
   cva_end( x )
end sub

proc1( 3, 11, 22, 33 )


----

Internal stuff:

The implementation and code emitted for cva_list, cva_start, cva_copy, cva_end, cva_arg varies depending on target, backend, and architecture.

In -gen gas backend, these macros translate to expressions because we are providing our own implementation. With the cva_* macros being similar to the implementation of the fbc's existing va_* macros but having different behaviours.

In -gen gcc backend, the cva_* macros will map to gcc's builtin macros:
cva_list => __builtin_va_list
cva_start => __builtin_va_start
cva_arg => __builtin_va_arg
cva_end => __builtin_va_end
cva_copy => __builtin_va_copy

The cva_list data type is a hairy one, because it's type varies depending on the target:

Code: Select all

'' dos/win/linux x86 - gas backend
type cva_list as any alias "char" ptr

'' dos/win/linux/arm x86 - gcc backend
type cva_list as any alias "__builtin_va_list" ptr

'' win32 x86_64 - gcc backend
type cva_list as any alias "__builtin_va_list" ptr

'' linux x86_64 - gcc backend
type __va_list_tag alias "__va_list_tag"
   as ulong gp_offset
   as ulong fp_offset
   as any ptr overflow_arg_area
   as any ptr reg_save_area
end type 
type cva_list as __va_list_tag alias "__builtin_va_list"

'' arm64/aarch64 - gcc backend
type __va_list alias "__va_list"
   as any ptr __stack
   as any ptr __gr_top
   as any ptr __vr_top
   as long __gr_offs
   as long __vr_offs
end type
type cva_list as __va_list alias "__builtin_va_list"


The intent is that cva_list and cva_* macros will work on all the targets, though I have only tested on x86 & x86_64 for win/lin. I don't have a set-up for testing LLVM backend, or ARM targets.

Also, you should notice some new syntax with the AS datatype ALIAS "modifier". This is valid syntax. A side note here: a while ago added the following syntax: byval as long alias "long". This was specifically added to allow fbc programs to link with win32's 32 bit long int in gcc c++, which was previously impossible without this type modifier

What the ALIAS "modifier" allows, is using a datatype as you would expect in fbc, but then, in the backend emitter, do something special with it; mostly used for name mangling and linking. fbc has to do some other tricky stuff to be compatible across multiple targets because __builtin_va_list is typed differently on various targets.

The documenation for usage of ALIAS "modifier" should get its own new page, maybe KeyPgAliasTypeModifier
coderJeff
Site Admin
Posts: 2964
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Nov 26, 2018 17:48

This update will deal with C like variadic paramaters only.

In this old thread:
viewtopic.php?p=242057#p242057

marcov wrote: I would separate Basic level variadic parameters from C level variadic parameters, and treat that as separate constructs. And maybe not allow defining C level variadic functions in FB. (only FB level variadic parameters)


So, having a more "basic" level of variadic parameters is still possible. Maybe using a func(name...) like syntax or similar, that would allow inspecting the argument list for number of arguments, types, using other complex types like STRING or ARRAY(), etc. But it's not in this update.

For reference, other threads talking about variadic functions:
viewtopic.php?p=241348#p241348
viewtopic.php?f=2&t=22951
viewtopic.php?p=244495#p244495
viewtopic.php?t=21668

There's probably other threads as well.
coderJeff
Site Admin
Posts: 2964
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Mar 10, 2019 19:51

varargs feature now merged into fbc 1.07.0 (for development info, see pull request #115)

New wiki pages:
cva_list | cva_arg | cva_start | cva_copy | cva_end

Also Updated:
Variable Arguments | Functional Keyword List | Alphabetical Keyword List | Procedures
... (Ellipsis) | va_first | va_next | va_arg

UPDATE:
New wiki page for ALIAS (Modifier) syntax, WIP.

Found first bug while writing the doc (dang it!).
dim x as any alias "char" ptr 'OK, is valid syntax
dim x as any alias "char" ' BUG, no error but should be "invalid type" error
fxm
Posts: 9083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: variadic functions and argument lists in fbc

Postby fxm » Mar 11, 2019 9:22

Quickly a build from St_W so that we can try this!
fxm
Posts: 9083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: variadic functions and argument lists in fbc

Postby fxm » Mar 25, 2019 10:00

@coderJeff

From documentation:
.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?
Last edited by fxm on Mar 25, 2019 10:39, edited 2 times in total.
fxm
Posts: 9083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: variadic functions and argument lists in fbc

Postby fxm » Mar 25, 2019 10:17

@coderJeff

Can we use dynamic Cva_List variables (with New/Delete) as following? :

Code: Select all

Sub proc cdecl(byval count As Integer, ... )
    Dim pargs As Cva_List ptr = New Cva_List

    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
        Print Cva_Arg( *pargs, Integer )
    Next
   
    Delete pargs
End Sub

proc( 4, 4000, 300, 20, 1 )

This works, but with warnings when using gcc 32/64-bit instead of gas 32-bit:
    Compiler output:
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.bas(11) warning 4(1): Suspicious pointer assignment
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.bas(11) warning 4(1): Suspicious pointer assignment
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c: In function 'PROC':
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c:39:14: warning: comparison of distinct pointer types lacks a cast
    if( TMP$3$1 == (void**)0u ) goto label$5;
    ^
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c:71:14: warning: comparison of distinct pointer types lacks a cast
    if( PARGS$1 == (void**)0u ) goto label$10;
coderJeff
Site Admin
Posts: 2964
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Mar 26, 2019 0:56

fxm wrote:
.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?


cva_end is never called automatically. cva_start & cva_end must always be used in pairs. I'm open to suggestions for better terminology.

Code: Select all

scope
   dim args as cva_list    '' allocation
   cva_start( args, count) '' initialization, construction, create
   cva_end( args )         '' de-initialization, destruction, destroy, clean-up, tear-down
end scope                  '' deallocation
coderJeff
Site Admin
Posts: 2964
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Mar 26, 2019 1:12

fxm wrote:Can we use dynamic Cva_List variables (with New/Delete)? :

Yes, it's supposed to work with pointer types. Thank-you for testing.

For example:

Code: Select all

Sub proc cdecl(byval count As Integer, ... )
    Dim pargs As Cva_List ptr = allocate( sizeof(Cva_List) )

    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
        Print Cva_Arg( *pargs, Integer )
    Next
   
    if( cast( any ptr, pargs) ) then
      deallocate( pargs )
   end if
End Sub

proc( 4, 4000, 300, 20, 1 )


The issues (bugs) appear to be with null pointer checks, with comparisons (IF statements) or generated (-exx compiler option) or internal pointer checks (NEW/DELETE). Here's a variation of your original test case for new/delete that highlights each case:

Code: Select all

Sub proc cdecl(byval count As Integer, ... )

   '' BUG: warning: comparison of distinct pointer types lacks a cast
   '' due to ptr check in NEW
    Dim pargs As Cva_List ptr = New Cva_List

   '' BUG: suspicious pointer assignment
   '' due to null pointer check with '-exx'
    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
       '' BUG: suspicious pointer assignment
       '' due to null pointer check with '-exx'
        Print Cva_Arg( *pargs, Integer )
    Next

   '' BUG: warning: comparison of distinct pointer types lacks a cast
   '' due to comparison of cva_list ptr with NULL
   if( pargs ) then
      print "OK"
   end if
   
   '' BUG: warning: comparison of distinct pointer types lacks a cast
   '' due to ptr check in DELETE
   Delete pargs

End Sub

proc( 4, 4000, 300, 20, 1 )


I wrote a test for new/delete in tests/functions/va_cva_api.bas, it passes, but obviously, I missed the output being generated. Thanks.
MrSwiss
Posts: 3194
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: variadic functions and argument lists in fbc

Postby MrSwiss » Mar 26, 2019 1:51

Took the example from Documentation, and run it OK (see comments in code):

Code: Select all

'
'   tested with St_W's build: DEV x64 WIN standalone --> OK!
'   whole crt dir, with all files, copied from FBC 1.06.0 x64 WIN
'
'' pass the args list to a function taking an cva_list type argument
#include "crt/stdio.bi"

sub myprintf cdecl(fmt as zstring ptr, ...)
    Dim As cva_list args
    cva_start(args, fmt)
    vprintf(fmt, args)
    cva_end(args)
end Sub

dim as string s = "bar"

myprintf(!"integer = %i, longint = %lli, float = %f\n", _
   1, 1ll shl 32, 3.3)
myprintf(!"string = %s, string = %s\n", "foo", s)

Sleep
fxm
Posts: 9083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: variadic functions and argument lists in fbc

Postby fxm » Mar 26, 2019 6:14

coderJeff wrote:
fxm wrote:
.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?


cva_end is never called automatically. cva_start & cva_end must always be used in pairs. I'm open to suggestions for better terminology.

I did not pay enough attention because "eventually" is for me French a "false friend" of "éventuellement" (possibly).
coderJeff
Site Admin
Posts: 2964
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Apr 12, 2019 23:57

When counting_pine, dkl, and I were discussing the design of the C va_list type, there were 2 ways we could go:
1) an abstract type, like what we are using now, where fbc knows some things about va_list usage, but not the full implementation in the backend.
2) a completely new and separate type (like STRING or BOOLEAN), which would require a full implementation for every platform.

I think doing the abstract type first was relatively easier and does not prevent a full type later if that is desired. The intent is that it solves all the most common use cases, even though, we may find fbc generating some code that the backend compiler might reject.

----

I almost forgot about the feature that counting_pine had worked on leading up to his initial contribution to the cva_list type:

#DUMP(expr) and #ODUMP(expr) preprocessor statements for debugging expressions. The exact format of the AST output might vary depending on if you are running a release version or a debug version of fbc. And only works for expressions, not statements.

Example:

Code: Select all

dim as integer x

#print NATURAL EXPRESSION
#DUMP( 2 * x * 4 * 5 )
 
#print OPTIMIZED EXPRESSION
#ODUMP( 2 * x * 4 * 5 )


Output:

Code: Select all

NATURAL EXPRESSION
0                      * =-=  [integer]
                                     / \
1                    * =-=  [integer]
                                   / \
2                  * =-=  [integer]
                                 / \
3    VAR( X ) (integer) [integer]

3                                   2 (integer) [integer]

2                                     4 (integer) [integer]

1                                       5 (integer) [integer]

OPTIMIZED EXPRESSION
0                      * =-=  [integer]
                                     / \
1        VAR( X ) (integer) [integer]

1                                       40 (integer) [integer]
coderJeff
Site Admin
Posts: 2964
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Apr 13, 2019 0:04

coderJeff wrote:The issues (bugs) appear to be with null pointer checks, with comparisons (IF statements) or generated (-exx compiler option) or internal pointer checks (NEW/DELETE). Here's a variation of your original test case for new/delete that highlights each case:

I created a pull request for the following:

- disable suspicious warnings on NULL pointer checks for pointers to cva_list types
- disable -gen gcc warning of comparison of distinct pointer types lacks a cast when comparing cva_list pointer types
- disallow ANY ALIAS "modifier" unless it is also a pointer type, like ANY PTR. Allocated data types must have non-zero size.
TJF
Posts: 3474
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: variadic functions and argument lists in fbc

Postby TJF » May 29, 2019 15:01

Hi all!

Unfortunately this implementation is not sufficient. CVA_ARG returns the parameter values (a *CPTR(dtype PTR, expr2)). But more often we need the pointer itself (= returning expr2). This is necessary to call functions like

Code: Select all

void
g_object_set_valist (GObject *object,
                     const gchar *first_property_name,
                     va_list var_args);
or

Code: Select all

void
gtk_style_get_valist (GtkStyle *style,
                      GType widget_type,
                      const gchar *first_property_name,
                      va_list var_args);
where first_property_name is a pointer to the parameter name, and var_args is a pointer to further parameters.

So what we really need is a further keywort like CVA_PTR(args, type) that returns a pointer to the value, instead of the value itself. (CVA_ARG could be a macro using CVA_PTR).

Regards
coderJeff
Site Admin
Posts: 2964
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » May 31, 2019 2:15

"args" is the "pointer" to the list. Actually, on some platforms it is a pointer to the stack. On other platforms it is a structure that tracks the current argument number & value. i.e. where arguments are passed in registers and don't have an address exactly.

A variadic procedure takes a variable number of arguments and is indicated by last parameter ellipsis (...)

The cva_list type is a data type that represents that list.

I'm not familiar with the framework but maybe this might help illustrate:

Code: Select all

void _some_action (SomeKindOfObj *obj,
   const gchar *first_property,
   ...)
{
   va_list args;
   va_start (args, first_property);
   g_object_set_valist (G_OBJECT (obj), first_property, args);
   va_end (args);
}


The ellipsis allows passing mutiple arguments to _some_action( obj, "PROP1', "VALUE, "PROP2", 123 ).
The va_list data type abstracts the variable argument list and allows it to be passed to g_object_set_valist ();

Do you have a freebasic example we can work with? Or a more complete C example to translate?
TJF
Posts: 3474
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: variadic functions and argument lists in fbc

Postby TJF » May 31, 2019 11:51

Hi coderJeff, thanks for the anwser!
coderJeff wrote:Do you have a freebasic example we can work with?
Sure, ie. look at line 253. You can find similar code in each of the projects objects.

Code: Select all

The ellipsis allows passing mutiple arguments to _some_action( obj, "PROP1', "VALUE, "PROP2", 123 ).
Not entirely correct. In order to mark the end of the list, the va_list always needs a terminating NULL.
coderJeff wrote:I'm not familiar with the framework but maybe this might help illustrate:
Unfortunately your code is unacceptable. I need 1:1 compatibility to the API of the underlying GooCanvas library. but ...
coderJeff wrote:"args" is the "pointer" to the list.
This is the information I couldn't find in the docs. It helps partialy, but obviously that info is not correct. I was able to compile (and run) the following code on a 32-bit system using -gen gcc

Code: Select all

  'VAR va = VA_FIRST(), arg = VA_ARG(va, ZSTRING PTR)
  'IF arg THEN g_object_set_valist(G_OBJECT(polax), arg, VA_NEXT(va, ANY PTR))
  DIM AS CVA_LIST args : CVA_START(args, Text)
  VAR arg = CVA_ARG(args, gchar PTR)
  IF arg THEN g_object_set_valist(G_OBJECT(polax), arg, args)
  CVA_END(args)
But on a 64-bit system I get the error message

Code: Select all

error 58: Type mismatch, at parameter 3 of G_OBJECT_GET_VALIST()
The compiler doesn't accept args as va_list (previously defined as TYPE AS ANY PTR va_list).

Regards

Return to “Community Discussion”

Who is online

Users browsing this forum: No registered users and 11 guests