FreeBASIC Namespace Project

[RockTheSchock]

I have been thinking for a while about the quirks and about namespaces or some sort of package system.
I would introduce some sort of auto namespace mode, which reflects the directory structure. Also things like an auto include mode or a compiler project / options file would be nice to have

some compiler switches:

-auto config,namespaces,includes
-roots path1,path2,path3


Lets say our project has folowing structures:

Code: Select all

include
	freebasic
		ext
			algorithms
				detail
			containers
			conversion
			database (database.bi)
				drivers

Code: Select all

src
	freebasic
		ext
			algorithms
			containers
			conversion
			database  (database.bas)
				drivers

either the namespaces and using statments could automically included by the preprocessor or they could be hold in a lookup table at compile time.

Some additional magic would make using ;-) the system even more intuitive. the parent namespace should see all symbols of its children / grandchildren / grand grand ... and vice versa a child should see the parent / grand parent ... but no sibblings or aunts and oncles.
So if you need a common.bi you would put at in ext at the top of your dir structure.

Manually defined namespaces override the auto namespaces with a warning.

Auto include of header files (.bi) with same name same relative path to the .bas files would be also nice.

If have only one root directory where bi and bas files share the same directory tree it's even simpler.

a fbc compiler options file would be also nice because the length of the command line of some operatinmg systems is limited to 2047 chars
it would be like a builtin make file / project file.

If you are trying to compile a foreign project everthing you need is the source. No IDE or batch file / shell file would be needed.

Maybe you could also put macro into the auto generated namespaces.

If you include header files from external libs which are not prepared for auto namespaces, the path of the include files would just not be put into the -roots option, while others which are aware you would put in the roots path so they could even have there own compiler options file which disable either auto includes or namespaces or both.

[marcov]

How would your using differ from include ?

p.s. Personally I hate classpath and its ilk. It is good to have some minimal hierarchy options, but it is better to use it as little as possible.

[coderJeff]

After re-reading this topic from the top, I get the feeling the interested participants are generally in agreement.

For the big picture view, this is my understanding and opinion: for some types of programs, the many symbols (keywords) defined by fbc by default just get in the way. For example, writing a Win API/SDK program, a framework, or a graphics/gui tool kit. Therefore it would be helpful for the author to be able to put all the default fbc symbols somewhere else, like in a namespace. Can still use them if desired, but not forced to work around them.

To activate, would probably be something like:
- a new command line option (or variation of -lang) to enable
- #pragma or variation of #lang directive to enable in source
- therefore, old sources will compile unchanged with new compiler, and new sources that activate the fb namespace (in theory) should simply fail to compile on old compilers

For implementation:
- The existing fbc symbol naming does not conflict with itself (it can't as all symbols are currently in the global namespace), so I am cautious of needlessly categorizing symbols. Personally, I think deeply nested hierarchies should be avoided unless the design (i.e. a framework) requires it.
- For symbols that are currently defined by default that can also be declared in fbc language itself, for example overloaded functions, moving the symbol in to a namespace is trivial
- For symbols that are purely a lexer/parser token (e.g. "quirk" keywords and data types), a new method of symbol name lookup must be implemented. Can't just be simple text comparison with next token.
- Possible new feature for importing individual symbols from a namespace in to current namespace would require new namespace symbol management within the compiler. Currently, fbc can only import entire namespaces.

[RockTheSchock]

Keywords like Name, Width, Color can't be used as properties or membervars in types, that's a bit annoying. That would be no problem if most keywords get out of global namespace.

[fxm]

I do not see where the problem is because member names are in the namespace of their Type, and not in the global namespace.

[Juergen Kuehlwein]

Jeff,

for some types of programs, the many symbols (keywords) defined by fbc by default just get in the way. For example, writing a Win API/SDK program, a framework, or a graphics/gui tool kit.

i don´t want to be contra-productive, but coming from PB, where we don´t have namespaces but (i didn´t count, but i have a strong feeling) even more key words, i never considered this (too many key words) to be a major problem. In general regardless how you do it, including something always bears the risk of naming conflicts, which then must be resolved some way or other. So why put so much effort in a thing, which can only be minimized but not fixed for good anyway?

At least (and this is my second point) the existing key words should remain key words. If i understand it right, your proposal would make it possible to have existing key words like "OPEN" or "KILL" in different versions. Outside fbc namespace your could define your own "OPEN" or "KILL", that is, when reading "OPEN" or "KILL" you now must know it´s context (namespace) too in order to know, what it actually does - i´m not convinced, if this really helps.

I agree, that it can be annoying sometimes, if words like "name", "width", "color" and the like are forbidden for variable and procedure names. On the other hand it keeps the code more readable, if you just cannot mix certain words with variable and procedure names. It makes their meaning clear because these are unique by default. And it´s easy to find acceptable alternatives: "clr", "back_color", "filename", etc.

What i could imagine is putting parts (topic related statements) of the language like graphics, file, maybe string and others in it´s own namespace in order to make some key words re-usable in a different context e.g: file open ..., audio open ..., video open ..., audio set ... This would reduce the number of key words available by default in the main namespace and it would allow the syntax for new things to follow existing rules by implementing existing key words in a similar manner. In terms of work to be done - maybe not much better than your approach.

[Xusinboy Bekchanov]

In my opinion, when adding new keywords, you need to choose the names from the existing dialects of Basic, first of all VisualBasiс.net, then VisualBasic 6, PowerBasiс, DarkBasiс and etc. (PureBasic does not count, because it does not look like basic).

[paul doe]

...
i don´t want to be contra-productive, but coming from PB, where we don´t have namespaces but (i didn´t count, but i have a strong feeling) even more key words, i never considered this (too many key words) to be a major problem. In general regardless how you do it, including something always bears the risk of naming conflicts, which then must be resolved some way or other. So why put so much effort in a thing, which can only be minimized but not fixed for good anyway?
...

That's the problem: thinking that namespaces are just 'places where to dump stuff'. They can be used, in fact, as complete abstraction layers on their own:

Code: Select all

namespace Graphics
  namespace FBGFX
    type _
      Display _
      extends Object
      
      public:
        declare constructor()
        declare destructor()
        
        declare sub _
          render()
    end type
    
    constructor _
      Display()
    end constructor
    
    destructor _
      Display()
    end destructor
    
    sub _
      Display.render()
      
      ? "Rendering using the FBGFX backend"
    end sub
  end namespace
end namespace

namespace Graphics
  namespace SDL2
    type _
      Display _
      extends Object
      
      public:
        declare constructor()
        declare destructor()
        
        declare sub _
          render()
    end type
    
    constructor _
      Display()
    end constructor
    
    destructor _
      Display()
    end destructor
    
    sub _
      Display.render()
      
      ? "Rendering using the SDL2 backend"
    end sub
  end namespace
end namespace

'' Change this to 'Graphics.SDL2' to use the SDL2 'backend'
using Graphics.FBGFX

var _
  d => Display()

d.render()

sleep()

So if you're crafty, all you really need to do to switch an entire abstraction layer is changing exactly one line of code. That's the real power of namespaces (and of the using keyword). I can't even begin to comprehend why folks would refuse namespacing. Sometimes (as in the example above) you actually want symbols to collide (to avoid mixing things that shouldn't be mixed). Not that that will stop you if you want to, alas.

In my opinion, when adding new keywords, you need to choose the names from the existing dialects of Basic, first of all VisualBasiс.net, then VisualBasic 6, PowerBasiс, DarkBasiс and etc. (PureBasic does not count, because it does not look like basic).

Indeed.

[marcov]

Jeff,

for some types of programs, the many symbols (keywords) defined by fbc by default just get in the way. For example, writing a Win API/SDK program, a framework, or a graphics/gui tool kit.

i don´t want to be contra-productive, but coming from PB, where we don´t have namespaces but (i didn´t count, but i have a strong feeling) even more key words, i never considered this (too many key words) to be a major problem. In general regardless how you do it, including something always bears the risk of naming conflicts, which then must be resolved some way or other. So why put so much effort in a thing, which can only be minimized but not fixed for good anyway?

Because those conflicts are then local (in the scope), not global (for all your sourcecode) anymore.

Granted, if you don't make larger programs or even only make single file sources that doesn't matter. But for the language the scale it is crucial.

Personally I think the first thing is to clarify the difference between namespace and scope. Note that these definition are fairly common, but maybe not wholly universal.

0. keyword is typically something related to language constructs and are crucial to the structure of the program. These are usually absolute forbidden to be reused. Things like WHILE and END etc.
1. reserved identifiers. These are library or pseudo language constructs that are generally visible. They might be implemented in the library, or in the compiler, but they have typically a pseudo-function syntax and are overridable. Typical example: the function for length of a string
If you override them, you can often still access them by prefixing the name of the system namespace. So if you redefine len, you can still access the old len (even if only in the implementation of the new len) with e.g. System.Len or RTS.Len or FB.Len or whatever you use.
2. a namespace is something where duplicate identifiers can't exist and is generally fairly static. There are some problems with this definition, specially if namespaces can span modules.
3. Scope is the active visibility of identifiers. Scope can have duplicates and in that case the newer one usually overrides the older one. It is also dynamic (changes while compiling the file) It is more or less a stacking of namespaces. First the buildins of the compiler, then
the headers at the top of the source, e.g. a function or a struct scope with its local variables. If the function ends, the struct/function's namespace is popped off the scope stack again.

The default scope is whatever is hardcoded into the compiler (but can still change due to parameters), and then the compiler imports some module's external (exported) namespace, usually this is called system. This system declares default visible libraries identifiers (functions,types,constants etc)

Some more advanced systems load several (e.g. a minimal module, a language dialect additional declarations module(s). Modules in this stage are not just loaded for their namespace, but sometimes also to just bring them into the ctors/dtors system, to intiialize/finalize them.

Some compilers even hide builtins (compiler built-in reserved identifiers) till they are explicitely declared. This helps keeping the default visible identifiers very low, and you can then use various headers for various language modes and options.

Then the actual source is read and if you have #langs (that changes compiler parsing state and imports some additional modules) or some module USING/USES/IMPORT then identifiers that are imported from the namespace of that module might be added to be used unqualified with namespace. Then you have own definitions that add to it etc etc.

#Include don't really change the source, but are processed as input, just as if you had typed it on the spot.


The gap in my experience however is that i know only two kind of namespaces, and that is specially a problem with finding all files for a namespace

- Delphi, where namespace equals module, if you have the name, you have the module and with that all symbols
- Java/C# where a namespace is a partial path, and where files are class "modules" implementing a namespace (import x.y.z.*) are found by iterating the classpath and find all files in each root+/x/y/z/

[Juergen Kuehlwein]

Well, to clarify a few things, i´m not against using namespaces - not at all. Namespaces are a great thing and i know the difference between "scope" and "namespace". And i´m used to deal with projects of several hundred files and more (even without the possibilty of using namespaces).

If i interpret Jeff´s post right (correct me, if i´m wrong), his goal is to make the global namespace initially empty, that is, to put all existing statements in a namespace, which must be activated (using ..., or explicit prefix "fbc." or the like).

My point was not to remove all statements, but to leave at least the core language (FOR ... NEXT, IF, SELECT, ...) always in the global namespace. And maybe reduce the number of default statements in the global namespace by putting topic related statements like graphic, file, console, etc. in separate namespaces. Not everyone wants to use graphics or the console in a program, or wants to read or write to files.

Jeff´s proposal would make heavy use of the USING statement or explicit prefixing a requirement, which, and this is my personal opinion, would make code look ugly - at least not more readable.

Restricting the need for USING or explicit prefixing to certain distinct categories of statements would make it more acceptable in my view. In procedural and in object oriented code i would like to handle different things in different procedures. It would be of help to be able to handle certain related things inside a procedure or method without the need of switching namespaces to and fro all the time. In other words in most cases one single USING statement per procedure would be sufficient, which seems acceptable and reasonable.

[marcov]

If i interpret Jeff´s post right (correct me, if i´m wrong), his goal is to make the global namespace initially empty, that is, to put all existing statements in a namespace, which must be activated (using ..., or explicit prefix "fbc." or the like).

Where (I think) you are wrong is that you assume that the minimal situation is something that an average user will ever see. Usually the extreme minimal configurations are for things like bootloaders, to tie into the windows device driver SDK, or compiling the RTS (once it is 100% FB). The minimal state is not necessary the default.

IOW, the default startup scope will be that minimal state, augmented with some default options that enable additional compiler state + the default header/namespaces. And that can be whatever you want it to be, and both are subject to parameters. It is just that the very minimal compiler principle will make other (more minimal, but also more elaborate) alternate startup states possible because every end situation is the result of a minimal state and certain additions, never subtractions. Also move as many of the predefined identifiers (keywords, reserved or whatever) to headers rather than builtins. It is simply more flexible that way.

Forgive me, if I compare to FPC again, its default state would be QB (well, Turbo Pascal in our case, but same era) minus some very dos and 16-bit oriented stuff (like port handling, segment and offset stuff etc). Yes. That is the FPC default startup state, and it was that way in 1998, and it still is, even though the bulk of developers have shifted to the VB(read: Delphi) dialect or variants.

IOW the default adds all clean QB keywords and namespace over the default scope, but if parameter -XX is passed to the compiler it won't add anything. And if parameter --ImAnOldFart is passed, the non-clean QB stuff will be added too.

My point was not to remove all statements, but to leave at least the core language (FOR ... NEXT, IF, SELECT, ...) always in the global namespace.

I think those would indeed remain. Rule of thumb is to keep all keywords that are common to most dialectsand require elaborate helpers.

And maybe reduce the number of default statements in the global namespace by putting topic related statements like graphic, file, console, etc. in separate namespaces. Not everyone wants to use graphics or the console in a program, or wants to read or write to files.

True. If only because by default proper Windows GUI programs don't even open an console. It is wise to not fall for the *nix pitfall and assume a console is always there.

Jeff´s proposal would make heavy use of the USING statement or explicit prefixing a requirement, which, and this is my personal opinion, would make code look ugly - at least not more readable.

As said not necessarily. Usually for the default startup state some popular dialect is chosen, and the compiler enables additional keywords and adds implicit USINGs (IOW that the user doesn't have to type). But keeping the core free of such concerns means that other parameters can NOT enable those.

Additive parametrization is easier to test in isolation than subtractive. If you choose a more complex base state and then try to hide based on parameters, then every time the base state changes, you have to validate all parameters.

In the additive situation, the compiler base state rarely changes, since even if the default compiler startup scope changes, it is more likely to be in a library. Or implemented in the parameter layer by default setting some mode unless a parameter is added.

Restricting the need for USING or explicit prefixing to certain distinct categories of statements would make it more acceptable in my view. In procedural and in object oriented code i would like to handle different things in different procedures. It would be of help to be able to handle certain related things inside a procedure or method without the need of switching namespaces to and fro all the time. In other words in most cases one single USING statement per procedure would be sufficient, which seems acceptable and reasonable.

The default startup state is political and a matter of taste. Progressive and conservative forces will always war about that. But this is not about that, it is about having the internal architecture to easily make and manage several such modes, regardless of which one is default or parameter.

As a personal point, I would however advise to clean the worst and most dated things from the default state. People that want them then must add additional dialect options (e.g. --QB-FULL) or so. And keep an eye open for multi-dialect programs (selected dialect a per module basis)

[coderJeff]

The default startup state is political and a matter of taste.

lol, yes. The most difficult issue to mitigate. ;)

marcov, paul doe, fxm, thank-you. Your contributions to the on-going discussion are invaluable.

In the early days, the common rule was if it was in QB then should be in fbc too, unless was too hard or had serious performance penalties. For example, QB's FIELD statment and the ON TIMER|COM|etc events statements have such serious performance penalties, their implimentation was abandoned.

The namespace project is in line with the expected "advanced" features of fbc: where we give more control over to the user/programmer how they use the language and express their vision.

[marcov]

Note that after I wrote the above text I realized there might be a fundamental difference between FB and FPC.

This relates to the module system and the fact that FPC loads module-interfaces from binary, not textual (header) representations. This binary representation is dialect independent.

So if FB imports headers and parses them, that might complicate things dialectwise (needing to specify dialect in every header, and switch dialect while parsing a header, restoring the old dialect going back) The parsing would have to create a dialect independent AST.