MOCKUP of FreeBASIC 1.0 Goals

General discussion for topics related to the FreeBASIC project or its community.
marcov
Posts: 3462
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Post by marcov »

Personally, I don't see that anything has changed in the past week except for the fact that cha0s is now officially "on hiatus",
Well, let's bury it, but it simply seems that there is confusion over the course. I fully agree with the fact that the devels, and the devels alone set the course, but I would like to hear what it was as an user. (well, if I actually were one)
For example, I've set up Natural Docs in my working copy of the compiler, and have begun documenting parts that I can understand; currently I'm working on the lexer, as it seems very straightforward to me. If it seems to be working out OK, I will submit a patch. I've looked into modifying doxygen for FB support in the past, but I'm not very familiar with flex, and its language module is one huge monolithic mess.
Note that I said doxygen-LIKE. An own initiative might be better. It was just an example that the compiler is not the only place where a language parser (lexer/parsergenerator based or not) is needed.

At FPC we have an own doxygen-like documenting system, tailored to our use, but more importantly also package the parser classes of the doc tool as library, so it can be used for other things.

One of the interesting things would be to try to convert all FPC headers to FB.
v1ctor
Site Admin
Posts: 3804
Joined: May 27, 2005 8:08
Location: SP / Bra[s]il
Contact:

Post by v1ctor »

My two dead cents:

I think that rewriting the compiler in anything but FreeBASIC wouldn't be a good idea for two reasons:

1) The time it would take to rewrite those 130k lines of code in other languages could be used to implement many of the complex items in the TODO list;

2) A compiler should eat its own dog food; it should compile itself, period. That's where all the fun is.

I don't think that rewriting it in C++ would help to attract more contributors. If someone is really up to the task to help in the development of a (not so simple) BASIC compiler, the minimum that is expected is him to code in BASIC (and like the language a bit).

Now the lack of internal docs, yeah, that was my fault. I wanted to release FB the faster i could, and the documentation was neglected. That shows how the dev team is doing an incredible job. Adding the more complex features (fully implementing the OOP features, for example) without any clues would be heroic.
Mentat
Posts: 332
Joined: Oct 27, 2007 15:23
Location: NC, US
Contact:

Post by Mentat »

v1ctor wrote:My two dead cents:

I think that rewriting the compiler in anything but FreeBASIC wouldn't be a good idea for two reasons:

1) The time it would take to rewrite those 130k lines of code in other languages could be used to implement many of the complex items in the TODO list;

2) A compiler should eat its own dog food; it should compile itself, period. That's where all the fun is.

I don't think that rewriting it in C++ would help to attract more contributors. If someone is really up to the task to help in the development of a (not so simple) BASIC compiler, the minimum that is expected is him to code in BASIC (and like the language a bit).

Now the lack of internal docs, yeah, that was my fault. I wanted to release FB the faster i could, and the documentation was neglected. That shows how the dev team is doing an incredible job. Adding the more complex features (fully implementing the OOP features, for example) without any clues would be heroic.
Not to mention where the starting file is :P.
marcov
Posts: 3462
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Post by marcov »

v1ctor wrote:My two dead cents:

I think that rewriting the compiler in anything but FreeBASIC wouldn't be a good idea for two reasons:

1) The time it would take to rewrite those 130k lines of code in other languages could be used to implement many of the complex items in the TODO list;

2) A compiler should eat its own dog food; it should compile itself, period. That's where all the fun is.
While I agree with 2 for many reasons including the ones you name (attracting contributors, compiler is its own best testsuite, at least users run that testsuite :-) ), and I don't think using C++ will improve much, but reason 1 is not so simple.

Nearly always, full versions are close to complete rewrites or very wide restructures, so if the conversion is timed together with some very wide restructure (new codegenerator, new parser tool, new sym table design), the work resulting from the actual change of language could be a minor detail
notthecheatr
Posts: 1759
Joined: May 23, 2007 21:52
Location: Cut Bank, MT
Contact:

Post by notthecheatr »

After reading cha0s' post as well, I've revised my ideas. I think for the most part what anonymous1337 says is a good idea. I have arguments for and against various points, but I guess ultimately, not being a developer, I have to concede that this might really be the best way to proceed - if possible.
marcov
Posts: 3462
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Post by marcov »

notthecheatr wrote:After reading cha0s' post as well, I've revised my ideas. I think for the most part what anonymous1337 says is a good idea. I have arguments for and against various points, but I guess ultimately, not being a developer, I have to concede that this might really be the best way to proceed - if possible.
What would be your reasons to recode in C++ ? What would be the benefit trade off for the FB project in your opinion?
John Spikowski
Posts: 453
Joined: Dec 24, 2005 2:32
Location: WA - USA
Contact:

Post by John Spikowski »

What would be your reasons to recode in C++ ? What would be the benefit trade off for the FB project in your opinion?
Isn't the 'selling' point of FreeBASIC that it's written in itself with the idea that developing in Basic would entice users to contribute?

If you feel the need to use a C compiler then check out BCX Basic to C translator. Euphoria is another way to use a Basic like language and generate C source. Why recreate the wheel?

The problem seems to be that there is no road map and the initial developers left everyone in the dark.


John
marcov
Posts: 3462
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Post by marcov »

John Spikowski wrote:
What would be your reasons to recode in C++ ? What would be the benefit trade off for the FB project in your opinion?
Isn't the 'selling' point of FreeBASIC that it's written in itself with the idea that developing in Basic would entice users to contribute?
Hmm, I would put more bullets in the VB compat bullet.
If you feel the need to use a C compiler then check out BCX Basic to C translator. Euphoria is another way to use a Basic like language and generate C source. Why recreate the wheel?
The universe is not one dimensional. Not any "compiles to C" (or actually more "may use a to-c converter instead of the interpreter") is equal to FB in C or C++. And compile to C, and being written in C are totally different things again.
The problem seems to be that there is no road map and the initial developers left everyone in the dark.
That is a feeling I have too though.
yetifoot
Posts: 1710
Joined: Sep 11, 2005 7:08
Location: England
Contact:

Post by yetifoot »

For me, the roadmap is those things in the TODO list, and the visions for FB that v1ctor has spoken about in his forum postings. Although perhaps it would be nice to have a more detailed plan, we do have plenty to go on in many areas.

http://fbc.svn.sourceforge.net/viewvc/f ... iew=markup
John Spikowski
Posts: 453
Joined: Dec 24, 2005 2:32
Location: WA - USA
Contact:

Post by John Spikowski »

Maybe you could find sponsors for items on this list and give some of the FreeBASIC programmers looking for work a job. Google sponsors projects like FreeBASIC in the 'Summer of Code' program. This is only one example of how the project can refocus it's direction.

FreeBASIC has a large enough presence that taking it the next level with sponsors makes sense. Loosing key developers is another good reason to regroup and come up with with a plan most will go along with. Open source is about community participation for the benefit of all. Everyone has the ability to contribute in one form or another. It just takes commitment and a willingness to invest some your free time for free software.

John
VirusScanner
Posts: 775
Joined: Jul 01, 2005 18:45

Post by VirusScanner »

After reading all this talk of rewriting FB from the ground up, something comes to mind:
I've started many (probably >100) projects that never got finished because I decided it would be better to write it in another language, or I couldn't understand the code because I'd been away from it too long, or I wanted to change some part of it but thought it'd be easier to rewrite it than restructure the existing code. I think if FB will ever reach v1.0, it will be with everyone's persistence to learn to work with the existing code and the existing goals. And that's another one: if you have too big of a feature list, sure it looks nice on paper, but is it really obtainable? Maybe to a big company that can pay a team of people with programming degrees to work all day, Mon-Fri. I don't know about everyone else, but I would be thrilled to see all the todo-list items completed and v1.0 reached, but who knows how long that will take. It will be even longer if we add too much extra stuff.

FB is a compiler, and since it compiles itself, if you feel a feature needs to be added to make the code easier to work with, then add it! There's really no need to rewrite it in C++. But that's not to say you can't - just don't change the existing FB. Start a new project. And when your rewrite is done, if it's better than the current one, maybe we'll adopt it. But I don't count on that happening any time soon.

You may think it'll make things better to do a complete rewrite, but it really won't - it'll just stall the project for a while, and then we'll be right back here where we started. However, if you stick around long enough, there will eventually be a need to write a v2.0. That's the time to think about complete rewrites.
Quadrescence
Posts: 10
Joined: Jul 30, 2008 17:28
Location: Minnesota, USA
Contact:

Post by Quadrescence »

And for my two cents.

I am not at all active on this forum, where I suppose the majority of FreeBASIC users show themselves. Though, I am active on IRC, so, I'm not oblivious to the FreeBASIC community and to, of course, FreeBASIC itself.

I usually program in other languages (e.g., Lisp, C, C++, Fortran), but I use FreeBASIC often to prototype things. Anyway, I've always been especially in FreeBASIC's development, and especially helping out with it (though, I have not developed for FreeBASIC, even though I have had fun looking though the sources).

Anyway, the way I have been liking to see FreeBASIC go is summarized by the following points.

Standardize the Current Language. I think it has been talked about by anonymous1337, and I know I have brought it up a few times in IRC, but I think there should be a written standard for the language, containing the necessary details for a compiler implementation.

Rational. The wiki could almost be thought of a standard, but it's really not. It defines it, more or less, but many things are left open. Things like defining what exactly should happen if one divides a 64-bit integer by a 8-bit unsigned integer, and the reverse.

Things like this could be tested, but this still does not ensure correctness, and it does not guarantee anything. This leads me to my next point.

The FreeBASIC Compiler in Standard C89. After a specification is written, FreeBASIC (as in, a compiler) should be written strictly in ANSI C89, and code should adhere precisely to the standard, and the implementation should adhere exactly to the written specification.

Rational. What benefit does C89 give us? Correctness. If we know what C should do, we will know what FreeBASIC should do if it's written in C. Why not rewrite FreeBASIC in FreeBASIC right away? FreeBASIC isn't standardized nor well-documented. How can we be absolutely sure FreeBASIC runs exactly the way we expect it if it's compiling itself? I argue we can't without checking very extensively.

The other large reason for writing it in C is because C is very, very portable. We all know that very well. Rewriting in strict C89 would allow FreeBASIC to be ported to other platforms very easily. I think this is very beneficial, especially for up-and-coming 64-bit processors. It's well known, or at least it is to me, that FreeBASIC is rather tedious to install and get working (I don't know if it has been done) on a 64-bit operating system.

C++ would definitely make things easier, especially for FreeBASIC's head to more object-oriented nature. I argue that, for a compiler, C++ will likely lead to less efficient and sloppier implementations of things (no, I am not saying good code can't be written in C++, or bad code can't be written in C). Also, the C++ standard is very, very large. I think it would be inconvenient to follow the standard closely. I see C++ a lot better for application-programming, and not as much for compiler implementation. Lastly, not only is the C++ standard large, so is the run-time library, and its availability on other platforms in lesser than that of C's.

Write a FreeBASIC Compiler in "Standard FreeBASIC". "Um, Quadrescence? We just wrote the entire compiler in C. You want it in FreeBASIC now?" Yes.

Rational. I do think, in light of v1ctor's and others' views, that FreeBASIC should be a self-hosting compiler. It's kind of a . . . part of FreeBASIC's legacy (?). But why didn't we start with just writing it in FreeBASIC? We didn't have any insurance in correctness of the generated code for every case. Some have argued on IRC that we wouldn't have insurance with C-generated code either. I've refuted that with the fact that, though they might be correct for some probability, that probability would be a hell of a lot less than writing it in a language for compiler that didn't follow any clear rules. C is old, and very, very well tested both as a language, and as a good language implementation for systems-programming.

Those are my major points, and some of my other points are things that have to do with the language itself, whether it be the addition or subtraction of new or current features, respectively. These I don't find very important relative to these aforementioned considerations.

To summarize, my ideal road-map for FreeBASIC is this:
  1. Write a FreeBASIC specification.
  2. Find community approval.
  3. Write a conforming compiler in ANSI C89 strictly.
  4. Test and test and test, iron out all the buggies.
  5. Write a conforming compiler in "Standard FreeBASIC"
  6. Test and test and test, iron out all the buggies.
  7. Continue with the development of FreeBASIC and its respective standard.
I do want to thank the FreeBASIC developers and contributors. They have done much hard work for everyone, and have developed a language that's good for both novices and experts for programming, developing, and prototyping in. And so, thank you all.

-Quadrescence
TheMG
Posts: 376
Joined: Feb 08, 2006 16:58

Post by TheMG »

Quadrescence, what you are proposing would be incredibly difficult. It would take years to complete both rewrites, and what would we have at the end of it? What we have now, but we would be sure of what that is. I think we should get as far as we can on the current code base, release that as v1.0, then rewrite in FreeBASIC. I do not think we should ever move away from self hosting, as it lets everyone take a look, and understand the source well, in turn leading to more developers. Writing it in C would require knowing both languages, and possibly some of the current devs aren't profficient in C.
marcov
Posts: 3462
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Post by marcov »

Quadrescence wrote: The FreeBASIC Compiler in Standard C89. After a specification is written, FreeBASIC (as in, a compiler) should be written strictly in ANSI C89, and code should adhere precisely to the standard, and the implementation should adhere exactly to the written specification.

Rational. What benefit does C89 give us? Correctness. If we know what C should do, we will know what FreeBASIC should do if it's written in C.
Nobody proves programs of that size mathematically. So while this sounds like reasonable, it doesn't add up IMHO.

A implementation of Basic in C that satisfies C89 rules perfectly doesn't automatically implement the correct FB rules.

I do think there is some intuitive point in that (avoid that problems with unstable compiler leave you without a running compiler in the FB-FB case) , but people have done that before (C is an example here), and there are standard ways to deal with that.

A similar fundamental thought mistake is in the portability suggestion further below. Being made in C89 makes only the C89 code implemented in it portable. It doesn't make the x86 codegenerator in it suddenly start emitting powerpc code.
Why not rewrite FreeBASIC in FreeBASIC right away? FreeBASIC isn't standardized nor well-documented. How can we be absolutely sure FreeBASIC runs exactly the way we expect it if it's compiling itself? I argue we can't without checking very extensively.
You run a testsuite. How else can you be sure that what you write down in C does exactly what you MEAN it to do?
The other large reason for writing it in C is because C is very, very portable. We all know that very well.
This is a dangerous (and incorrect) mind-jump too. While this goes for a normal application, it doesn't go for the compiler because it contains a large CPU dependant part (the codegenerator, assembler parsers etc), that must be explicitely ported.

Worse, _those_ non automatically ported parts are the parts that are actually the hard parts. Why? Large parts of the rest of the compiler are not that terribly OS dependant, at least not to the part that is already abstracted by the standard runtime. Files opening/reading - stringprocessing - writing are simply not the biggest porting problems. It _IS_ the OS and CPU dependant part that is the problem. And doing that part in C89 won't make it work on another sstme.
Rewriting in strict C89 would allow FreeBASIC to be ported to other platforms very easily. I think this is very beneficial, especially for up-and-coming 64-bit processors. It's well known, or at least it is to me, that FreeBASIC is rather tedious to install and get working (I don't know if it has been done) on a 64-bit operating system.
This has to do with a missing FB x86_64 codegenerator, not portability of the language in which it was implemented.

Rewriting to C89 won't help one bit with that, since it will leave you with a 64-bit compiler emitting 32-bit code.

SO you will need to add a x86_64 backend no matter what, and to do it in FB you might not directly have to rewrite, or make it very efficient.
C++ would definitely make things easier, especially for FreeBASIC's head to more object-oriented nature.
I think OOP-with-inheritance will improve maintenance. But that doesn't automatically makes a case for C++, one could also make a case to start adding this feature to the current compiler so that it can act as bootstrap.l
I argue that, for a compiler, C++ will likely lead to less efficient and sloppier implementations of things (no, I am not saying good code can't be written in C++, or bad code can't be written in C).
Agree, also I'd argue that C++ has a tendecy to overengineer, be overly complex and has horribly bad characteristics of the compiler (understanding the standard alone requires a lot of skills)
Lastly, not only is the C++ standard large, so is the run-time library, and its availability on other platforms in lesser than that of C's.
And its stronger version dependancy. C++ can't even link with different C++.
Write a FreeBASIC Compiler in "Standard FreeBASIC". "Um, Quadrescence? We just wrote the entire compiler in C. You want it in FreeBASIC now?" Yes.

Rational. I do think, in light of v1ctor's and others' views, that FreeBASIC should be a self-hosting compiler. It's kind of a . . . part of FreeBASIC's legacy (?). But why didn't we start with just writing it in FreeBASIC? We didn't have any insurance in correctness of the generated code for every case. Some have argued on IRC that we wouldn't have insurance with C-generated code either.
You need a stable starting point. But maybe the old FB stuff would have been more standard now if the effort had been put into it.

Note that a problem with FB->FB translation is also that you are creating a not exactly trivially sized codebase in FB. Long term dialect compatibility becomes way more important then.
I've refuted that with the fact that, though they might be correct for some probability, that probability would be a hell of a lot less than writing it in a language for compiler that didn't follow any clear rules. C is old, and very, very well tested both as a language, and as a good language implementation for systems-programming.
But lousy for compiler work, since basic structures are hard to get typesafe, and must be emulated manually.
To summarize, my ideal road-map for FreeBASIC is this:
  1. Write a FreeBASIC specification.
  2. Find community approval.
  3. Write a conforming compiler in ANSI C89 strictly.
  4. Test and test and test, iron out all the buggies.
  5. Write a conforming compiler in "Standard FreeBASIC"
  6. Test and test and test, iron out all the buggies.
  7. Continue with the development of FreeBASIC and its respective standard.
My big problems of that are
* "double work", and (I want one compiler, I have to write two to get there, including one I'm not interested in)
* a long time of not investing in FB code (but investing in C, doing that
work in FB would lead to high quality experience in using/improving FB)
* finding the ideal "program FB in FB" and "program FB in C" skills might not be in the same person, frustrating many of the core, alienating potential new ones.
* I don't buy the portability argument. Often the tighter coupling with a C compiler makes it harder rather than easier (e.g. having complete free C toolchain, on windows)
* the C part would be in a non OOP language, which is unnecessarily hard. If throw away anyway, then even C++ would be better (though more of a management problem)

My own list would be
  1. finalize the core FB dialect, preferably with OOP with inheritance. (good for symboltables and complex heterogenic trees)
  2. continue/start developing testsuite of typical test programs (e.g. old bugreports)
  3. branch
  4. Use the old "stable" branch to develop the new branch. Work around problems in the old branch by either relucatantly updating it in extreme cases and with ifdef otherwise.
  5. When the new branch autocompilate is systematically better than the old, branch a release branch and stabilize that branch to release. That is then the new stable compiler to develop the "unstable" branch.
  6. repeat ad infinitum. Keeping enhancing dialect.
Last edited by marcov on Dec 20, 2008 12:48, edited 1 time in total.
jevans4949
Posts: 1186
Joined: May 08, 2006 21:58
Location: Crewe, England

Post by jevans4949 »

I agree with the other guys - Quadrescence's proposal makes no sense.

Apart from anything else, FB is a far better language for compiler development than C - it is richer in datatypes.

And re-coding - in C or anything else - doesn't necessarily prove the correctness of the solution. Witness the constant security flaws with Internet Explorer.

Talk of "standardising" FB is also a bit of a non-starter. The "problem" with the whole array of implementations of Basic is that they are all a load of ad-hoc bolt-ons to the original Dartmouth version. Redefine the language and all you get is yet another version.

As mentioned, the portability problem is not in the compiler itself, but the fact that the code-generator is only x86 32-bit. The "gcc backend" solution was touted for a while - I'm not sure what happened to that. However, I think the real problem here is that nobody has the equipment, or more importantly the motivation, to do it. Somebody was trying to port to the Apple x86 platform (which should be easy), but he hasn't been heard from for a while either.

I could go on, but I'll refrain ...
Post Reply