FB dll for PowerBASIC

New to FreeBASIC? Post your questions here.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: FB dll for PowerBASIC

Post by marcov »

TeeEmCee wrote: Writing the rtlib in anything other than C seems like a pain, needing to translate the system headers for every flavour of Unix.
Unix is less of a problem, you only need about 100 calls, and some more for threading support (pthreads). The windows headers are a bigger problem, but users need access to them anyway, and large parts can be semi automatically translated.
So FP is written purely in FP? How does it do that? Does it call into libc for Unix system calls, or use syscalls?
Well, there can be assembler in FPC RTL sources, this is of course minimized to some base functions (think floating points and primitives like copying bytes, searching in memory (ptr+distance) for a byte, word, dword etc, primitives for exception handling)

Some platforms have a small fragment of assembler for the startup code (equivalent to crt0.o on *nix). Usually this is modified decompiled GCC code or assembler, whatever the target provides. Linux has an all pascal startup on most targets. FreeBSD not, partially because weak symbols on FPC level were not yet working when I lasted tried.

Other than assembler and FPC native code there is no other code in the project for native code, other than Makefiles. For some weirder targets (like javascript and Java JVM), parts of the runtime there are in js/java.

The Unix RTL can be built to interface to libc (from Pascal code) and to syscalls. Both FreeBSD and Linux default to syscalls, Mac OS X defaults to libc. Basically maintainers preference, but the Linux and BSD ports were done much earlier (1995 and 1998) than Mac OS X (2004-6), the targets were much less coherent then and syscall binaries are better copyable between Linux distributions.

On *nix targets ld (though not necessary GNU) is used, and GAS on the more odd ball targets. On Windows and non windows (Amiga, Dos etc) it is native. The number of non binutils targets has steadily increased since 2005.
Yes, FB is very very heavily inspired by C and C++. I always describe FB as a superset of C and subset of C++ with BASIC syntax, better strings, and a graphics library. Features of C/C++ have been methodically copied verbatim into FB over the years. I do question whether this was a wise design.
I also think the copying was too easy. That said it is difficult, features copying happens everywhere, also in FB. (and not just C/C++, but also VB(COM!), C#/Java etc, even if only because Delphi does).

Also, some people want to do high level app development, some want to basically use it as a system compiler, and there is constant friction between all those requirements.

It has gotten to the strange level where most new Delphi functions are by default wrapped in a struct to make it look a namespace, because customers demand it because qualifying and grouping the functions is more "high level" and "OOP" like (even if there is no state to instantiate). I'm not against OOP, on the contrary, but it must be functional, this is madness. It sometimes makes you doubt keeping compatibility.

However I think FB losing an own philosophy is worse than some feature creep at the edges. If you don't set boundaries, all new up and coming devels will do is copy-and-paste the quickest and dirtiest way.
However, personally I'm quite happy with it, because I link lots of C, C++ and FB code together, and it makes interoperability easy. So I'm reliant on a C++ compiler anyway.
So do I with FPC, and objective C too if needed. (There is a native interface to objective C, and you can define objective C compatible classes in Pascal to some degree).

You don't need a C backend for that, and unless FB suddenly magically started to understand C/C++ headers(which would be frontend matter), the backend doesn't really add much there, except that it is mandatory to be GCC compatible instead of voluntary.
But you're mistaken in arguing that FB supporting features of C is because FB compiles to C or uses library written in C. FB has all these features of C because they've been reimplemented from scratch, not because it relies on a C compiler.
Or just because the mindset turned C, nobody bothered to look any further or to a bigger picture anymore.
Of course, if you dig you will find semantics of the C runtime or ncurses libraries that bubble through to FB (e.g. the effect of LANG environmental variable on 8bit -> wstring conversions)
Yup. Basically there is no windows port. There is only a port to the *nix emulation engine on Windows. Of course that is gradual difference (from cygwin to one end to very simple things that could be said out of multiplatform concerns). But if you place *nix and gcc and its libs central, you might as well use cygwin.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: FB dll for PowerBASIC

Post by caseih »

TeeEmCee wrote:Writing the rtlib in anything other than C seems like a pain, needing to translate the system headers for every flavour of Unix. So FP is written purely in FP? How does it do that? Does it call into libc for Unix system calls, or use syscalls?
Well it's really no different than the C runtime (libc) being written in C, as is the C compiler itself. There's no technical reason, other than the OS-specific header file issue that FB's runtime library couldn't be written in pure FB. As long as the compiler can make calls in other DLLs, and make syscalls (that may require assembly, or support from the compiler), there's no requirement for C to be involved. But for portability, I support the continued use of C for the runtime library. This in no way makes FB more or less C-like. If C abstractions leak into FB, they must also leak into Pascal, which is structured very similarly to FB and C, with looping constructs, functions and subs, the same scoping rules, etc.
Yes, FB is very very heavily inspired by C and C++. I always describe FB as a superset of C and subset of C++ with BASIC syntax, better strings, and a graphics library. Features of C/C++ have been methodically copied verbatim into FB over the years. I do question whether this was a wise design. However, personally I'm quite happy with it, because I link lots of C, C++ and FB code together, and it makes interoperability easy. So I'm reliant on a C++ compiler anyway.
I kind of wish FB was more compatible with C++. Semantically it looks a lot like the C++ object model, but under the hood it's not compatible with C++ in terms of name mangling and C++ inheritance. FB has no idea how to interact with C++'s vtables, so anything other than a simple C++ object is inaccessible from FB. I wish I could use Qt from FB directly. That would be very nice indeed. As it stands now, a shim layer would have to be generated for FB that would map FB objects to Qt and vice versa. Binding C++ libraries to any other language including popular ones like Python is surprisingly difficult.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: FB dll for PowerBASIC

Post by caseih »

marcov wrote:Yup. Basically there is no windows port. There is only a port to the *nix emulation engine on Windows. Of course that is gradual difference (from cygwin to one end to very simple things that could be said out of multiplatform concerns). But if you place *nix and gcc and its libs central, you might as well use cygwin.
Except that FBC is built as a native win32 executable by Mingw GCC. As I understand it mingw-generated executables have no dependencies on any dll, besides ones explicitly linked. In fact if there was any additional dependency the original poster would not be able to ship a DLL that works in PB without an additional emulation layer dll.

Thus I am very surprised that FBC is using any part an emulation layer. Which parts of this layer is FBC relying on? It this just a decision on the part of the FB developers until such time as the emulated bits are replaced?
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: FB dll for PowerBASIC

Post by TeeEmCee »

Thanks for the detailed reply. I don't really have much too much say in response. Really interesting that you can use either syscalls or libc. Certainly, avoiding all glibc versioning problems is a nice feature to have.

You do have a point about copying features: even if you have to implement them from scratch yourself, designing a language is a lot of work (I've been working on designing my own for years and I'm still no where close to happy with it), so just copying an existing language is the quick and easy route.
marcov wrote:Yup. Basically there is no windows port. There is only a port to the *nix emulation engine on Windows.
That's far too harsh (although maybe you were talking about the mingw build chain rather than the libraries?) The setlocale/mbstowcs etc example I gave is C99, not posix, the rtlib does seem to respect the Windows codepage, and I was wrong to mention curses. Although I know that the whole C runtime is treated like a compatibility layer on windows, what with msvcrt.dll being a part of Visual C++ rather than a part of the OS. But the reality is that almost everyone uses msvcrt on Windows. The rtlib has 6kLOC of code written against the win32 api and 27kLOC platform agnostic and I've always felt FB's support for Windows was better than Unix. For example the FB winapi headers are great, while the Linux/Unix system ones are an incomplete mess.
caseih wrote:I kind of wish FB was more compatible with C++. Semantically it looks a lot like the C++ object model, but under the hood it's not compatible with C++ in terms of name mangling and C++ inheritance. FB has no idea how to interact with C++'s vtables
You're asking for C++ ABI compatibility but there's no such thing, as different C++ compilers use different ABIs. Probably what you meant to write is that you want compatibility with g++ (but not with Visual C++)? That is intention of fbc, and it does mangle names identically to g++. I have never seen any case where it does incompatible mangling. It supports g++ vtables too, at least partially (obviously multiple inheritance isn't supported).

However, I've never tried linking FB and g++/C++ virtual classes before, so I just tried it. I found that fbc doesn't export the D2 "base object destructor" for classes with virtual destructors which g++ wants. Classes without virtual destructors work fine, in my limited testing. Also, virtual operator member functions aren't implemented. These are just partially unfinished features, which I'm interested in implementing myself.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: FB dll for PowerBASIC

Post by caseih »

TeeEmCee wrote:You're asking for C++ ABI compatibility but there's no such thing, as different C++ compilers use different ABIs. Probably what you meant to write is that you want compatibility with g++ (but not with Visual C++)? That is intention of fbc, and it does mangle names identically to g++. I have never seen any case where it does incompatible mangling. It supports g++ vtables too, at least partially (obviously multiple inheritance isn't supported).
FB does mangle names like GCC, but when you're trying to interact with methods from objects from classes with an inheritance hiearchy, FB mangles the names wrong. For example, if I'm calling a method defined in an ancestor class, FB mangles it as if it's a member of the current class. Basically FB can only handle very simple C++ (well, g++ ;) classes with no inheritance, and certainly not with virtual methods. This is not a bug in FB at all. Rather a limitation. There's no way to get around this problem except to pretty much be C++.

You're right about the ABI thing. I'm not sure but maybe the ABI across compilers will be the same in future revisions of the C++ spec? This is the single biggest problem with C++ in my opinion.
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: FB dll for PowerBASIC

Post by TeeEmCee »

g++ name mangling does actually follow a standard: the Itanium C++ ABI (on x86 gnu/linux & mingw, not sure about other platforms), with some extensions.
The C++ spec says nothing about the ABI. Compilers are free to do whatever they want. Parts of the ABI, in particular libstdc++/libc++, are evolving all the time.
And FB already is pretty much C++ with BASIC syntax, as I've said, so this can definitely be accomplished. Did you know that dkl even had a branch where he started work on allowing including .h files?


I actually only tried defining derived classes with virtual methods on the C++ side, not the FB side, and that seemed to work.
Can you point to some example code that doesn't work, so that I can look into the problem?
I don't have time to do so now - not least because I'm leaving for Australia soon. But I'd like to look into it some time next year :)
St_W
Posts: 1619
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: FB dll for PowerBASIC

Post by St_W »

I don't think that making FB compatible with g++ is a good idea. This ties it even more to the GNU toolchain instead of becoming more independent. Additionally the C++ ABI is not standardized and not stable. You cannot mix Visual C++ with g++ for example and often even can't mix code compiled with different versions of those compilers. So if you want to use a C++ library use C++ and the same compiler to compile everything. If you want to create some interface to be able to use the library from another programming language choose a well-defined and stable ABI like the one implemented by C.

Neither would I allow to include .h or even .c files. If you want to do that and depend on certain C headers which are not guaranteed to be compatible among different versions and systems (e.g. the Linux CRT library headers would be such an example) then again use C code and a C compiler to create a library and an according interface to access it from FB.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: FB dll for PowerBASIC

Post by marcov »

caseih wrote:
marcov wrote:Yup. Basically there is no windows port. There is only a port to the *nix emulation engine on Windows. Of course that is gradual difference (from cygwin to one end to very simple things that could be said out of multiplatform concerns). But if you place *nix and gcc and its libs central, you might as well use cygwin.
Except that FBC is built as a native win32 executable by Mingw GCC.
As said it is is not black-white, though it is nice to avoid msvcrt due to versioning. But it is not that code is suddenly not an emulation just because it is linked into the EXE instead of a DLL. I most reacted to the points mentioned by TeeEmCee (LANG environment variable etc)

So I meant it as behaviour and structure of the EXE in its totality, not just dependencies.
Thus I am very surprised that FBC is using any part an emulation layer. Which parts of this layer is FBC relying on? It this just a decision on the part of the FB developers until such time as the emulated bits are replaced?
A portable abstraction layer always favours one target, and that feels like an emulation layer on others. Mingw in itself is not the biggest problem, it is more starting to layering *nix centric libraries (like intl etc) on top.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: FB dll for PowerBASIC

Post by caseih »

marcov wrote:A portable abstraction layer always favours one target, and that feels like an emulation layer on others. Mingw in itself is not the biggest problem, it is more starting to layering *nix centric libraries (like intl etc) on top.
I see, so your beef is with the choices of libraries, which you feel are not native feeling or operate in a native fashion on Windows. I've done a bit of googling into how Win32 apps are supposed to internationalize strings. It's... very complicated as many things are in Windows. So sticking with the "unixy" libraries seems the best bet for me. Especially for a command-line app with not that many calls to wide win32 functions.

But I don't agree that the use of such libraries (which are native win32 ports by the way, not running through posix emulation) makes FBC not a port. FBC for windows absolutely is a native port. Complete with #ifdef's and win32 code paths I am quite sure. So I think you're splitting hairs with your definitions.

I guess there's also the issue of command-line flags. Unix being - and Windows mostly being /. But on the other hand there are several command line tools that ship with Windows that use the unix-style flags, including ping. And Windows does actually support both flag types officially. Sure one make an effort to use windows-style flags on Windows, and unix-style flags on Unix.
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: FB dll for PowerBASIC

Post by TeeEmCee »

The 'favoured' platform argument rings true.
But any old OS is a pile of emulations layers; Windows maybe even more so than Unix due to the incredible backcompatibility. And Microsoft love coming up with new ways of doing things every couple of years so it gets hard to say which is the "canonical" way to do things on Windows... of course on Unix someone invents a new way every week.
Neither would I allow to include .h or even .c files. If you want to do that and depend on certain C headers which are not guaranteed to be compatible among different versions and systems (e.g. the Linux CRT library headers would be such an example) then again use C code and a C compiler to create a library and an according interface to access it from FB.
(Oh, we're having one of those "in-the-open development direction" discussions suggested in the other thread about advancing FB)
I agree 100% that if you want to write portable Unix code you should use C instead of FB and link it into your program, because the FB headers aren't portable. You didn't give any argument against not allowing including .h files though.
However, it's a huge project to integrate fbfrog, so I guess it's not going to happen in the conceivable future.

Yes, being compatible with g++ C++ is further lock-in both to the toolchain and to C++. Though it is possible to support more than one C++ ABI.
As I mentioned, parts of g++'s ABI are actually standardised and slow changing. I would agree that trying to be support more of the ABI, such as the C++ standard library, is definitely the route to madness (and completely impractical). But if the couple bugs/missing pieces of FB's currently aimed level of compatibility are fixed, it will be useful.
But honestly, my motivation is largely that fixing up compatibility sounds like fun. And I would actually make use of it.
St_W
Posts: 1619
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: FB dll for PowerBASIC

Post by St_W »

TeeEmCee wrote:(Oh, we're having one of those "in-the-open development direction" discussions suggested in the other thread about advancing FB)
[...] You didn't give any argument against not allowing including .h files though.
(I don't understand your first sentence)
One reason is that it would tie FreeBasic (both the compiler and the language) even more to C. That argument has been already mentioned by others too. IMHO FB should be independent of a specific operating system and tools as a lock-in a bad thing in general (there may be exceptions, though). For example how would you handle case-sensivity or other features that are currently only available in C? You would have to either add them to the FreeBasic language or find some workarounds. Neither solution being a really good one.
TeeEmCee wrote:Yes, being compatible with g++ C++ is further lock-in both to the toolchain and to C++. Though it is possible to support more than one C++ ABI. [...] But if the couple bugs/missing pieces of FB's currently aimed level of compatibility are fixed, it will be useful.
The thing is that as soon as FB becomes somewhat compatible with any specific C++ implementation people will start using that instead of doing it the proper way by writing a C wrapper to get portability. And as soon as people are using the feature you can't go back - at least not without breaking compatibility and making people angry when things won't work anymore that have previously worked. What to do if the g++ ABI changes one day? Add a compiler option to allow selecting a suitable C++ ABI? Define a certain (range of) version(s) that are compatible and dropping the support for other ones?
Mixing some C++ toolchain with FreeBasic or even another C++ toolchain will cause troubles sooner or later. And IMHO FreeBasic should be easy to use and not cause troubles.
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: FB dll for PowerBASIC

Post by TeeEmCee »

Yes, importing .h files would be a huge challenge; it could only work for a subset of headers, such ones with no naming clashes due to case sensitivity and no C code in macros. fbfrog can rename symbols to avoid clashes, but you can't just do that when the user doesn't get a header to inspect to see what it got renamed to!
I'm already very happy with using fbfrog to generate FB headers.
What to do if the g++ ABI changes one day? Add a compiler option to allow selecting a suitable C++ ABI? Define a certain (range of) version(s) that are compatible and dropping the support for other ones?
Yes that is the logical conclusion. Yes, it creates occasional additional work on an on-going basis, and people will complain if it breaks. Can't refute that. It depends on someone actually wanting it and keeping it working, just like everything else that needs to be kept up to date. Stuff is constantly breaking as the build tools, OS, libraries, etc are updated.
I don't know how many other languages are compatible with C++. D and Obj-C++ come to mind.

I do actually agree with your complaints about FB being so heavily married to C and C++ instead of continuing in the BASIC philosophy. But that's the reality of the language that it already is. FB has some great features that C doesn't, such as strings. It's not at all constrained by C against continuing to add new features.
For example there are deficiencies of the language, like not being able to initialise var-len strings in object literals or array literals, because that's something you can't do in C either and FB does the same. But there's no reason at all that that can't be implemented. And it annoys me so much that I will probably implement it.

Sorry, I can't respond any more since I'm leaving for Australia.
Post Reply