Basic-Macros in fbc 1.08

Forum for discussion about the documentation project.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Basic-Macros in fbc 1.08

Post by coderJeff »

Recently added to fbc were some new built-ins for macro handling. I was a little hasty in merging the first changes, so there has been some recent fix-ups and the following should all work.

__FB_ARG_COUNT__( args... )
Counts the number arguments in an argument list
- always returns a value
- if argument list is empty, returns zero (0)
- if argument list has at exactly one argument, returns one (1)
- if argument list has commas ',', return number of commas plus 1

Code: Select all

#macro m( args... )
	print __FB_ARG_COUNT__( args )
#endmacro

m()
m(1)
m(1,2)
m(,2)
m(,)
'' etc

__FB_JOIN__( arg1, arg2 )
Joins two token arguments together as one, similar to token pasting operator
- will resolve arguments before joining
- compare differences in following example
- does not accept empty arguments

Code: Select all

#define PREFIX p
#define SUFFIX _T

'' this won't work - arguments not expanded
#define	makename1( x )  PREFIX##x##SUFFIX

'' this will work - can do this in older versions of fbc too
#define join( a, b ) a##b
#define makename2( x ) join( PREFIX, join( x, SUFFIX ) )

'' built in __FB_JOIN__() -- works pretty much like join() above
#define	makename3( x )  __FB_JOIN__( PREFIX, __FB_JOIN__( x, SUFFIX ) )

#macro dump( arg )
	#print #arg
#endmacro

dump( makename1(text) )
dump( makename2(text) )
dump( makename3(text) )
__FB_ARG_LEFTOF__( arg, sep ), __FB_ARG_RIGHTOF__( arg, sep )
Returns left or right tokens based on separator

Code: Select all

#macro count( range )
	scope
		dim x as integer = __FB_ARG_LEFTOF__( range, TO )
		dim y as integer = __FB_ARG_RIGHTOF__( range, TO )
		dim s as integer = sgn(y - x)
		print "Counting " & #range
		for i as integer = x to y step s
			print i
		next
	end scope

#endmacro

count( 4 to 10 )
count( 7 to 2 )

__FB_UNIQUEID_PUSH__(ID), __FB_UNIQUEID__(ID), __FB_UNIQUEID_POP__(ID)

For a good size example see ./examples/misc/trycatch/ in the freebasic examples directory.

Code: Select all

namespace crt
#include once "crt/setjmp.bi"
end namespace

'' example how to implement a try/catch using the C runtime setjmp()/longjmp(), plus the __FB_UNIQUEID_###__() and __FB_ARG_###__() builtin macros

#macro TRY 
	scope
		__FB_UNIQUEID_PUSH__(__tc__)
		dim __FB_UNIQUEID__(__tc__) as TryCatch
		if crt.setjmp(@__FB_UNIQUEID__(__tc__).buf) = 0 then
#endmacro

#macro CATCH(x) 
		elseif *cast(object ptr, __FB_UNIQUEID__(__tc__).ex) is __FB_ARG_RIGHTOF__(x, AS) then
			var __FB_ARG_LEFTOF__(x, AS) = cast(__FB_ARG_RIGHTOF__(x, AS) ptr, __FB_UNIQUEID__(__tc__).ex)
#endmacro

#macro THROW(x) 
		__FB_UNIQUEID__(__tc__).throw_(x, __FILE__, __FUNCTION__, __LINE__)
#endmacro

#macro END_TRY 
		end if 
		__FB_UNIQUEID_POP__(__tc__)
	end scope
#endmacro
Honestly, I haven't experimented enough with these new 'UNIQUEID' built-ins, so I will have to work on a simple write-up for what they do.
counting_pine
Site Admin
Posts: 6323
Joined: Jul 05, 2005 17:32
Location: Manchester, Lancs

Re: Basic-Macros in fbc 1.08

Post by counting_pine »

Wow, just seen these - looks amazing. They should make for some very powerful macros..
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Basic-Macros in fbc 1.08

Post by fxm »

Just a typo in the coderJeff's example for __FB_ARG_COUNT__( args... ):

Code: Select all

#macro m( args... )
   print __FB_ARG_COUNT__( args )
#endmacro

m()
m(1)
m(1,2)
m(,2)
m(,)
'' etc
[edit]
Typo corrected, and another below.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Basic-Macros in fbc 1.08

Post by coderJeff »

Some notes about fbc's macro processor, with a very simple examples.

An overview of compilation process:
EMIT <- IR <- AST <- PARSER[PP] <- LEXER[PP] <- READER, + global SYMBOL table(s)

This is how I look at the order of operations and may be opposite to how many people might think of the process.
- All of these objects work together to compile a program. fbc does not deal with address allocations or linking; that job is passed off to gas|gcc, and ld.

Symbol Table(s):
- anything that has a name, either explicit or automatically created is stored here
- it's a database of 'named things' that are referenced and used by every other part of the compiler.
- each 'thing' contains information about what it is, and how to use it later

Reader:
- gets next character of the input stream
- input can come from a file or included file
- input can come from memory, which is the case when macros are expanded

Lexer:
- gets characters from the reader
- tries to group the characters in to a token, which might a symbol like '=', a literal like '&HFFFF' or "hello", or an identifier.
- if it's an identifier, 2 things might happen.
- either it's the name of a macro and control passess to the preprocessor
- or it's given back to the Parser

Parser:
- gets tokens from the Lexer
- tries to match the sequence of tokens to a pattern
- if the sequence of tokens is valid, the parser updates the AST + SYMBOL tables
- some tokens like '#macro', '#if', can also invoke the pre-processor

Preprocessor [PP]:
- this object is kind of separate but is also tightly integrated with parser and lexer
- It can be invoked from either the context of the LEXER or the context of the PARSER
- the context matters as to what operations can be peformed


MACRO EXAMPLES:
- PARSER->PP->LEXER deal with macro definitions
- PARSER->LEXER->PP deal with macro expansions (usage)

Very simple example:

Code: Select all

#define TEXT "hello"
print TEXT
  1. Parser gets the next token from LEXER (which reads from the reader)
  2. it's a '#define', and Parser-PP is invoked to deal with it
  3. PP gets next token 'TEXT' and next token "hello"
  4. PP will add the symbol 'TEXT' to the SYMBOL table as a macro/define for later use
  5. Parser gets next token it's a 'print'
  6. Parser then invokes the 'PRINT' parsing branch to look for the sequence of expected tokens
  7. Parser asks for the next token from the LEXER, **and here's where something different happens**
  8. The lexer gets the next identifier 'TEXT' and recognizes that it is already defined in the SYMBOL table as a macro/define
  9. So LEXER-PP is invoked to deal with it
  10. The PP looks up the definition of 'TEXT' which is "hello" and injects it in to the LEXER/READER as next text to read
  11. PP then returns to LEXER
  12. LEXER then reads the next token, which is now "hello"
  13. LEXER returns the next token to Parser
  14. Parser then updates the AST with the code to 'print "hello"'
[/list]

For a macro defined with 1 parameter

Code: Select all

#macro M( arg )
	print arg
#endmacro

M( "hello" + "!" )
  1. Parser sees '#macro'
  2. Parser-PP reads tokens 'M' '(' 'arg' ')' ... etc.
  3. PP Stores symbol 'M' and the defintion in the Symbol table along with information that it expects 1 argument names 'arg'
  4. Parser continues, which calls LEXER
  5. LEXER sees 'M', it's a defined macro, so PP from LEXER takes over to expand.
  6. LEXER-PP reads '"hello" + "!"' as a list of tokens and stores as a single argument
  7. PP then looks up 'M' in the symbol table to get the definition and what parameters are expected
  8. as PP injects the defintion back in to the lexer
  9. PP replaces all occurances of 'arg' with the argument saved, which was '"hello" + "!"'
  10. finally, PP returns to LEXER
  11. LEXER returns to parser
  12. Parser reads next token
  13. Parser now sees 'print' as next token to deal with
For a macro taking a variable number of arguments, including empty:

Code: Select all

#macro M( args... )
	print #args
#endmacro
M( 1, 2, 3, 4 )
  1. PARSER/LEXER, etc, add 'M' to the symbol table expecting a variadic list of arguments 'args'
  2. LEXER-PP sees 'M' on line 4
  3. PP then read the entire list of arguments as if it were 1 argument '1 + 2 + 3 + 4'
  4. When PP expands 'M' and injects text back in to the LEXER, it simply replaces 'args' with '1 + 2 + 3 + 4'
  5. When PP expands '#' operator it converts the token list of '1 + 2 + 3 + 4' to a string
  6. PP returns to LEXER, which returns to PARSER, which gets the next token 'print' followed by "$""1 + 2 + 3 + 4"""
Nested Macros:

Code: Select all

#define B( arg ) print arg
#define A( arg ) B( arg + " world" )
A( "hello")
Roughly:
  1. LEXER-PP sees 'A' and expands the definition
  2. PP injects B( "hello" + " world" ) back in to the LEXER
  3. LEXER-PP then sees 'B' and expands the definition
  4. PP injects 'print "hello" + " world"'
  5. control finally returns to PARSER to get next token
  6. PARSER sees 'print', followed by "hello"... etc.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Basic-Macros in fbc 1.08

Post by coderJeff »

counting_pine wrote:They should make for some very powerful macros..
Thanks, I hope so...

I've been work on adding the following:
__FB_QUOTE__(arg): It converts the argument to a string, much like '#' stringize operator, except can use anywhere, and will expand the argument before conversion.
__FB_UNQUOTE__(arg): takes a literal string and converts it back to tokens. So it's possible to inject a string to source. Kind of like an 'emit' instruction for source code.

Code: Select all

#define X __FB_QUOTE__( print "hello" )
#macro Y( arg )
  __FB_UNQUOTE__( arg )
#endmacro

print X
Y( X )

/'
OUTPUT:
print "hello"
hello
'/
Except that '__FB_UNQUOTE__' will only take a single literal string.
So something like Y( X + X), is not possible. Because the macro expansion is all in LEXER/PP, there's no opportunity to read the actual expression and evaluate, what could be accomplished with constant-folding in the PARSER/AST.

I was also wanting to have something like __FB_EVAL__( constant-expression ) so that PP could have an opportunity to do the constant-fold. Something like '#if expression' handling directives. But it's been very difficult to work out the handshakes re-entering the PARSER from PP.

Even if I don't get the 'eval' done, I will probably update fbc with the 'quote/unquote' macros. They tests are all written. And seems to work pretty well with unicode sources too. For actual literal strings only though. fbc doesn't support unicode identifiers, etc, so some kinds of quote/unquote combos will lose the unicode.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Basic-Macros in fbc 1.08

Post by fxm »

coderJeff wrote: .....
__FB_UNIQUEID_PUSH__(ID), __FB_UNIQUEID__(ID), __FB_UNIQUEID_POP__(ID)
.....
Honestly, I haven't experimented enough with these new 'UNIQUEID' built-ins, so I will have to work on a simple write-up for what they do.
I absolutely did not understand what these macros do or what they can be used for.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Basic-Macros in fbc 1.08

Post by coderJeff »

The purpose is to produce a unique fb symbol name that can be recalled later.

__FB_UNIQUEID_PUSH__(stack-id) pushes a new unique identifier on to a stack identified by 'stack-id'
__FB_UNIQUEID__(stack-id) gets the identifier at the top of stack identified by 'stack-id'
__FB_UNIQUEID_POP__(stack-id) pops an identifier off of stack identified by 'stack-id'

"stack-id" is the name of the stack to push / pop / access.
the "stack-id" name itself is a separate namespace from all other symbols
"unique identifier" is a name of an fb symbol that is unique(**) to the module, so does not conflict or shadow other symbol names.
(**) "unique" will have the form "LT_xxxx" as a name so it might not be completely unique. fb uses the form "LT_xxxx" internally for labels, symbols, temp variables, etc. Should avoid naming fbc symbols of this form for any fbc program since version 0.0

Just to get a feel for what it's doing:

Code: Select all

__FB_UNIQUEID_PUSH__( stk )
#print __FB_UNIQUEID__( stk )

	__FB_UNIQUEID_PUSH__( stk )
	#print __FB_UNIQUEID__( stk )

		__FB_UNIQUEID_PUSH__( stk )
		#print __FB_UNIQUEID__( stk )
		__FB_UNIQUEID_POP__( stk )

	#print __FB_UNIQUEID__( stk )
	__FB_UNIQUEID_POP__( stk )

#print __FB_UNIQUEID__( stk )
__FB_UNIQUEID_POP__( stk )
Produces:

Code: Select all

Lt_0002
Lt_0003
Lt_0004
Lt_0003
Lt_0002
__FB_UNIQUEID__(stack-id) simply expands to text. So the name, for example Lt_0004, can be used wherever an fb symbol is required. A variable, procedure name, type name, etc.

First, consider this example that doesn't use unique ids.
It works, because '__counter__' variable is defined in a local SCOPE and therefore allows nesting.

Code: Select all

#macro repeat ? ( count )
	scope
		dim __counter__ as uinteger = count
		while( __counter__)
#endmacro

#macro end_repeat
			__counter__ -= 1
		wend
	end scope	
#endmacro


repeat 4
	print "outer"

	repeat 3
		print "--- inner"

	end_repeat

end_repeat
It goes all wrong though, if there happens to be some other '__counter__' symbol defined by either the user or another macro that is defined in an outer scope and then used in an inner scope.

To protect against a duplicate name from somewhere else, can use the unique id macros.

Code: Select all

#macro repeat ? ( count )
	__FB_UNIQUEID_PUSH__( ctx )
	scope
		dim __FB_UNIQUEID__( ctx ) as uinteger = count
		while( __FB_UNIQUEID__( ctx ) )
#endmacro

#macro end_repeat
			__FB_UNIQUEID__( ctx ) -= 1
		wend
	end scope	
	__FB_UNIQUEID_POP__( ctx )
#endmacro

repeat 4
	print "outer"

	repeat 3
		print "--- inner"

	end_repeat

end_repeat
That example is just to kind of give the form of the use. [EDIT: fixed the typos in the examples]
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Basic-Macros in fbc 1.08

Post by fxm »

A number of '__FB_UNIQUEID_POP__(stack_id)' greater than the number of '__FB_UNIQUEID_PUSH__(stack_id)' induces a compiler runtime error.
Can this be fixed to rather induce a compiler error message?

Code: Select all

__FB_UNIQUEID_PUSH__(stack_id)
__FB_UNIQUEID_POP__(stack_id)
__FB_UNIQUEID_POP__(stack_id)
Compiler output:
Aborting due to runtime error 12 ("segmentation violation" signal)
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Basic-Macros in fbc 1.08

Post by fxm »

About updating the documentation to include these "basic macros":
- I was thinking of just adding them to the already existing 'Predefined Symbols' list / 'Intrinsic Definitions' list.
- The only page where they would be functionally grouped together (under the 'Basic-Macros' tittle) would be the 'Intrinsic Defines' documentation page.
=> pages to update: PrintToc, CatPgFullIndex, CatPgFunctIndex, CatPgDddefines.
=> pages to create: one for each "basic macro":

Code: Select all

__FB_ARG_COUNT__        KeyPgDdfbargcount
__FB_ARG_LEFTOF__       KeyPgDdfbargleftof
__FB_ARG_RIGHTOF__      KeyPgDdfbargrightof
__FB_JOIN__             KeyPgDdfbjoin
__FB_QUOTE__            KeyPgDdfbquote
__FB_UNIQUEID__         KeyPgDdfbuniqueid
__FB_UNIQUEID_POP__     KeyPgDdfbuniqueidpop
__FB_UNIQUEID_PUSH__    KeyPgDdfbuniqueidpush
__FB_UNQUOTE__          KeyPgDdfbunquote
What do you think?
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: Basic-Macros in fbc 1.08

Post by D.J.Peters »

Cool but one question what does do the question mark "?"
#macro repeat ? ( count )

Joshy
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Basic-Macros in fbc 1.08

Post by fxm »

See the updated #MACRO...#ENDMACRO documentation page.
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: Basic-Macros in fbc 1.08

Post by D.J.Peters »

@fxm ok I got it but i don't like this "feature"
if a macro call without parentheses are allowed we will run in more complex problems in future !

Joshy
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Basic-Macros in fbc 1.08

Post by fxm »

In the first version of change, parentheses when calling a macro had become systematically optional for all macros, but when CoderJeff discovered possible conflicts with other expressions containing the names of the macros as terms, it added a special declaration syntax that alone allows this.
See history: search.php?keywords=parens&terms=all&au ... mit=Search
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Basic-Macros in fbc 1.08

Post by coderJeff »

fxm wrote:About updating the documentation to include these "basic macros":
- I was thinking of just adding them to the already existing 'Predefined Symbols' list / 'Intrinsic Definitions' list.
...
What do you think?
Sounds good to me.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Basic-Macros in fbc 1.08

Post by coderJeff »

D.J.Peters wrote:if a macro call without parentheses are allowed we will run in more complex problems in future !
Yes. As fxm mentions we tried to allow it on all macros & defines and it did cause problems. So, only if '?' is used can the macro be used without parentheses.

Maybe it will be replaced with some other pattern matching, either where the '?' is now, or in the argument list itself. But it's too much for a first try. And maybe will change in future. For now the '?' in "#macro name ? (args...)" means parentheses are optional. Not a high priority now.

Maybe some future concept:

Code: Select all

#macro qb_line(/({expr}, {expr})-({expr}, {expr}), {expr}, [b|bf]/)
#if fb_arg(5) = ""
gfx.line(fb_arg(0), fb_arg(1), fb_arg(2), fb_arg(3), fb_arg(4))
#elseif fb_arg(5) = "B"
gfx.box(fb_arg(0), fb_arg(1), fb_arg(2), fb_arg(3), fb_arg(4))
#elseif fb_arg(5) = "BF"
gfx.boxf(fb_arg(0), fb_arg(1), fb_arg(2), fb_arg(3), fb_arg(4))
#else
#error "Syntax Error: Expected B or BF at parameter 6 of qb_line"
#endif
#endmacro

So you could do:

var x = 0, y = 0, c = 15
qb_line (x+10, y+10)-(x+100, y+100), c, BF
Post Reply