variadic functions and argument lists in fbc

For other topics related to the FreeBASIC project or its community.
coderJeff
Site Admin
Posts: 2825
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

variadic functions and argument lists in fbc

Postby coderJeff » Nov 26, 2018 17:35

fbc has a few macros for working with variadic functions: va_first, va_next(), va_arg(), that worked using a pointer to the argument stack. These macros don't work on all targets (like 64-bit) because arguments can be passed to procedures in cpu registers.

sf.net bug https://sourceforge.net/p/fbc/bugs/881/

Working on a replacement though, it's not quite ready to merge in to master:
github https://github.com/freebasic/fbc/pull/115

In the update, to work with variadic functions and argument lists, here's what we've come up with:

cva_list is added as a default type by fbc to expose the argument list. It works kind of like an object, in that usually we don't need to know what's doing, only how to use it. There are a few methods in it's API for accessing a variable length list of arguments passed in to a procedure..

Code: Select all

dim args as cva_list = any
cva_start( args, param )  '' constructor
cva_copy( dest, args )    '' copy constructor
cva_end( args )           '' destructor
result = cva_arg( args, datatype ) '' method returns data and advances to next argument


Note the constructor/destructor pairs:
- cva_start() needs exactly one matching cva_end()
- cva_copy() needs exactly one matching cva_end()

If it looks familiar, it is because it's just about the same in C. We could have come up with our own api, but then it wouldn't be compatible (link-able) with anything but our own code.

If you are currently using any of the new names being added: cva_list, cva_start, cva_end, cva_copy, cva_arg and depending on target __va_list_tag, __va_list, you will need to use #undef to get it out of your way, or rename the symbol in your code.

- No documentation written yet. This post is the first info.
- No guarantees that it is bug free. Please submit bug reports.
- Need help testing on ARM targets. (Or I have to create a VM for it)

Examples:

Iterate through a list of arguments

Code: Select all

sub proc cdecl( n as integer, ... )
   dim args as cva_list
   cva_start( args, n )
   for i as integer = 1 to n
      print "arg " & i & " = " & cva_arg( args, integer )
   next i
   cva_end( args )
end sub

proc( 3, 11, 22, 33 )


Pass argument list to another procedure

Code: Select all


sub proc2 cdecl( n as integer, byval args as cva_list )
   dim x as cva_list = any
   cva_copy( x, args )
   for i as integer = 1 to n
      print "arg " & i & " = " & cva_arg( x, integer )
   next i
   cva_end( x )
end sub

sub proc1 cdecl( n as integer, ... )
   dim x as cva_list = any
   cva_start( x, n )
   proc2( n, x )
   cva_end( x )
end sub

proc1( 3, 11, 22, 33 )


----

Internal stuff:

The implementation and code emitted for cva_list, cva_start, cva_copy, cva_end, cva_arg varies depending on target, backend, and architecture.

In -gen gas backend, these macros translate to expressions because we are providing our own implementation. With the cva_* macros being similar to the implementation of the fbc's existing va_* macros but having different behaviours.

In -gen gcc backend, the cva_* macros will map to gcc's builtin macros:
cva_list => __builtin_va_list
cva_start => __builtin_va_start
cva_arg => __builtin_va_arg
cva_end => __builtin_va_end
cva_copy => __builtin_va_copy

The cva_list data type is a hairy one, because it's type varies depending on the target:

Code: Select all

'' dos/win/linux x86 - gas backend
type cva_list as any alias "char" ptr

'' dos/win/linux/arm x86 - gcc backend
type cva_list as any alias "__builtin_va_list" ptr

'' win32 x86_64 - gcc backend
type cva_list as any alias "__builtin_va_list" ptr

'' linux x86_64 - gcc backend
type __va_list_tag alias "__va_list_tag"
   as ulong gp_offset
   as ulong fp_offset
   as any ptr overflow_arg_area
   as any ptr reg_save_area
end type 
type cva_list as __va_list_tag alias "__builtin_va_list"

'' arm64/aarch64 - gcc backend
type __va_list alias "__va_list"
   as any ptr __stack
   as any ptr __gr_top
   as any ptr __vr_top
   as long __gr_offs
   as long __vr_offs
end type
type cva_list as __va_list alias "__builtin_va_list"


The intent is that cva_list and cva_* macros will work on all the targets, though I have only tested on x86 & x86_64 for win/lin. I don't have a set-up for testing LLVM backend, or ARM targets.

Also, you should notice some new syntax with the AS datatype ALIAS "modifier". This is valid syntax. A side note here: a while ago added the following syntax: byval as long alias "long". This was specifically added to allow fbc programs to link with win32's 32 bit long int in gcc c++, which was previously impossible without this type modifier

What the ALIAS "modifier" allows, is using a datatype as you would expect in fbc, but then, in the backend emitter, do something special with it; mostly used for name mangling and linking. fbc has to do some other tricky stuff to be compatible across multiple targets because __builtin_va_list is typed differently on various targets.

The documenation for usage of ALIAS "modifier" should get its own new page, maybe KeyPgAliasTypeModifier
coderJeff
Site Admin
Posts: 2825
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Nov 26, 2018 17:48

This update will deal with C like variadic paramaters only.

In this old thread:
viewtopic.php?p=242057#p242057

marcov wrote: I would separate Basic level variadic parameters from C level variadic parameters, and treat that as separate constructs. And maybe not allow defining C level variadic functions in FB. (only FB level variadic parameters)


So, having a more "basic" level of variadic parameters is still possible. Maybe using a func(name...) like syntax or similar, that would allow inspecting the argument list for number of arguments, types, using other complex types like STRING or ARRAY(), etc. But it's not in this update.

For reference, other threads talking about variadic functions:
viewtopic.php?p=241348#p241348
viewtopic.php?f=2&t=22951
viewtopic.php?p=244495#p244495
viewtopic.php?t=21668

There's probably other threads as well.
coderJeff
Site Admin
Posts: 2825
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Mar 10, 2019 19:51

varargs feature now merged into fbc 1.07.0 (for development info, see pull request #115)

New wiki pages:
cva_list | cva_arg | cva_start | cva_copy | cva_end

Also Updated:
Variable Arguments | Functional Keyword List | Alphabetical Keyword List | Procedures
... (Ellipsis) | va_first | va_next | va_arg

UPDATE:
New wiki page for ALIAS (Modifier) syntax, WIP.

Found first bug while writing the doc (dang it!).
dim x as any alias "char" ptr 'OK, is valid syntax
dim x as any alias "char" ' BUG, no error but should be "invalid type" error
fxm
Posts: 8805
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: variadic functions and argument lists in fbc

Postby fxm » Mar 11, 2019 9:22

Quickly a build from St_W so that we can try this!
fxm
Posts: 8805
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: variadic functions and argument lists in fbc

Postby fxm » Mar 25, 2019 10:00

@coderJeff

From documentation:
.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?
Last edited by fxm on Mar 25, 2019 10:39, edited 2 times in total.
fxm
Posts: 8805
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: variadic functions and argument lists in fbc

Postby fxm » Mar 25, 2019 10:17

@coderJeff

Can we use dynamic Cva_List variables (with New/Delete) as following? :

Code: Select all

Sub proc cdecl(byval count As Integer, ... )
    Dim pargs As Cva_List ptr = New Cva_List

    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
        Print Cva_Arg( *pargs, Integer )
    Next
   
    Delete pargs
End Sub

proc( 4, 4000, 300, 20, 1 )

This works, but with warnings when using gcc 32/64-bit instead of gas 32-bit:
    Compiler output:
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.bas(11) warning 4(1): Suspicious pointer assignment
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.bas(11) warning 4(1): Suspicious pointer assignment
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c: In function 'PROC':
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c:39:14: warning: comparison of distinct pointer types lacks a cast
    if( TMP$3$1 == (void**)0u ) goto label$5;
    ^
    C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c:71:14: warning: comparison of distinct pointer types lacks a cast
    if( PARGS$1 == (void**)0u ) goto label$10;
coderJeff
Site Admin
Posts: 2825
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Mar 26, 2019 0:56

fxm wrote:
.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?


cva_end is never called automatically. cva_start & cva_end must always be used in pairs. I'm open to suggestions for better terminology.

Code: Select all

scope
   dim args as cva_list    '' allocation
   cva_start( args, count) '' initialization, construction, create
   cva_end( args )         '' de-initialization, destruction, destroy, clean-up, tear-down
end scope                  '' deallocation
coderJeff
Site Admin
Posts: 2825
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Mar 26, 2019 1:12

fxm wrote:Can we use dynamic Cva_List variables (with New/Delete)? :

Yes, it's supposed to work with pointer types. Thank-you for testing.

For example:

Code: Select all

Sub proc cdecl(byval count As Integer, ... )
    Dim pargs As Cva_List ptr = allocate( sizeof(Cva_List) )

    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
        Print Cva_Arg( *pargs, Integer )
    Next
   
    if( cast( any ptr, pargs) ) then
      deallocate( pargs )
   end if
End Sub

proc( 4, 4000, 300, 20, 1 )


The issues (bugs) appear to be with null pointer checks, with comparisons (IF statements) or generated (-exx compiler option) or internal pointer checks (NEW/DELETE). Here's a variation of your original test case for new/delete that highlights each case:

Code: Select all

Sub proc cdecl(byval count As Integer, ... )

   '' BUG: warning: comparison of distinct pointer types lacks a cast
   '' due to ptr check in NEW
    Dim pargs As Cva_List ptr = New Cva_List

   '' BUG: suspicious pointer assignment
   '' due to null pointer check with '-exx'
    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
       '' BUG: suspicious pointer assignment
       '' due to null pointer check with '-exx'
        Print Cva_Arg( *pargs, Integer )
    Next

   '' BUG: warning: comparison of distinct pointer types lacks a cast
   '' due to comparison of cva_list ptr with NULL
   if( pargs ) then
      print "OK"
   end if
   
   '' BUG: warning: comparison of distinct pointer types lacks a cast
   '' due to ptr check in DELETE
   Delete pargs

End Sub

proc( 4, 4000, 300, 20, 1 )


I wrote a test for new/delete in tests/functions/va_cva_api.bas, it passes, but obviously, I missed the output being generated. Thanks.
MrSwiss
Posts: 2996
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: variadic functions and argument lists in fbc

Postby MrSwiss » Mar 26, 2019 1:51

Took the example from Documentation, and run it OK (see comments in code):

Code: Select all

'
'   tested with St_W's build: DEV x64 WIN standalone --> OK!
'   whole crt dir, with all files, copied from FBC 1.06.0 x64 WIN
'
'' pass the args list to a function taking an cva_list type argument
#include "crt/stdio.bi"

sub myprintf cdecl(fmt as zstring ptr, ...)
    Dim As cva_list args
    cva_start(args, fmt)
    vprintf(fmt, args)
    cva_end(args)
end Sub

dim as string s = "bar"

myprintf(!"integer = %i, longint = %lli, float = %f\n", _
   1, 1ll shl 32, 3.3)
myprintf(!"string = %s, string = %s\n", "foo", s)

Sleep
fxm
Posts: 8805
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: variadic functions and argument lists in fbc

Postby fxm » Mar 26, 2019 6:14

coderJeff wrote:
fxm wrote:
.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?


cva_end is never called automatically. cva_start & cva_end must always be used in pairs. I'm open to suggestions for better terminology.

I did not pay enough attention because "eventually" is for me French a "false friend" of "éventuellement" (possibly).
coderJeff
Site Admin
Posts: 2825
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Apr 12, 2019 23:57

When counting_pine, dkl, and I were discussing the design of the C va_list type, there were 2 ways we could go:
1) an abstract type, like what we are using now, where fbc knows some things about va_list usage, but not the full implementation in the backend.
2) a completely new and separate type (like STRING or BOOLEAN), which would require a full implementation for every platform.

I think doing the abstract type first was relatively easier and does not prevent a full type later if that is desired. The intent is that it solves all the most common use cases, even though, we may find fbc generating some code that the backend compiler might reject.

----

I almost forgot about the feature that counting_pine had worked on leading up to his initial contribution to the cva_list type:

#DUMP(expr) and #ODUMP(expr) preprocessor statements for debugging expressions. The exact format of the AST output might vary depending on if you are running a release version or a debug version of fbc. And only works for expressions, not statements.

Example:

Code: Select all

dim as integer x

#print NATURAL EXPRESSION
#DUMP( 2 * x * 4 * 5 )
 
#print OPTIMIZED EXPRESSION
#ODUMP( 2 * x * 4 * 5 )


Output:

Code: Select all

NATURAL EXPRESSION
0                      * =-=  [integer]
                                     / \
1                    * =-=  [integer]
                                   / \
2                  * =-=  [integer]
                                 / \
3    VAR( X ) (integer) [integer]

3                                   2 (integer) [integer]

2                                     4 (integer) [integer]

1                                       5 (integer) [integer]

OPTIMIZED EXPRESSION
0                      * =-=  [integer]
                                     / \
1        VAR( X ) (integer) [integer]

1                                       40 (integer) [integer]
coderJeff
Site Admin
Posts: 2825
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: variadic functions and argument lists in fbc

Postby coderJeff » Apr 13, 2019 0:04

coderJeff wrote:The issues (bugs) appear to be with null pointer checks, with comparisons (IF statements) or generated (-exx compiler option) or internal pointer checks (NEW/DELETE). Here's a variation of your original test case for new/delete that highlights each case:

I created a pull request for the following:

- disable suspicious warnings on NULL pointer checks for pointers to cva_list types
- disable -gen gcc warning of comparison of distinct pointer types lacks a cast when comparing cva_list pointer types
- disallow ANY ALIAS "modifier" unless it is also a pointer type, like ANY PTR. Allocated data types must have non-zero size.

Return to “Community Discussion”

Who is online

Users browsing this forum: No registered users and 2 guests