OpenBSD porting work

[TeeEmCee]

This thread is split off of FreeBASIC OpenBSD package to discuss work to properly support OpenBSD.

I downloaded an OpenBSD install disc but don't have time to install and play around for a few days.

On OpenBSD, this compiler happens to be named egcc, as to not conflict with some architectures that have not or cannot make the switch to the LLVM toolchain (the Motorola 88000 architecture comes to mind as one such cannot), which then have an in-base compiler named gcc.

So based on this, while we could invoke gcc if it happens to be present, it's unlikely and we should just use egcc instead of gcc. And if both gcc and egcc are present, we should prefer egcc anyway even if we do check for both names.
(fbc also uses the GCC envvar if present to find gcc).

Regarding forcing att syntax to be used: I assume this is because fbc by default uses the system as, which doesn't support intel syntax. On FreeBSD, there was exactly the same problem. Must be the case in all other BSDs. My solution for FreeBSD is this patch (not merged in yet, since I wandered off without finishing the FreeBSD crt headers).

On the subject of the missing CRT headers, dkl has already created a system for automatically generating the headers for all OSes recognised by FB, including OpenBSD: fbbindings. I don't think any of those inc/crt/ headers are actually used yet, though. FB's CRT/system headers are disorganised. And because each OS organises its headers differently (and especially the ways in which they include each other), it doesn't appear that their organisation is something that can be done fully automatically.
Missing crt headers has been a pain everywhere - headers for even basic C or Unix functions aren't actually complete on any OS, though they're by far the most complete for Windows, and completely absent for all Unices aside from Linux. So far, as a temporary fix, I've just been manually vreating a few critical headers by hand for Linux, Android, Mac, and FreeBSD, with help from fbfrog. I never got that merged in, because I wasn't happy with my way of doing it manually.

[marcov]

With FPC we also did everything manual, but used that to grow a common, parameterized set(*). (define a bunch of types OS specific, then do general headers). We use a

But that is only for the base set. Termio, pthreads etc are still pretty much per OS, often with architecture ifdefs.

The FreeBSD cons header was particularly annoying with all those _IOR() macros and no parametrised macros.

I did the original port to OpenBSD too, but in the early 2000s (then still a.out), I didn't do much for the current port.

(*) visible here: https://svn.freepascal.org/cgi-bin/view ... iew=markup This is the libc linking list though, in reality the *BSD and Linux. The syscall list is slightly smaller and different (e.g. getdents/getdirentries instead of readdir)

[TeeEmCee]

Thanks for that link.

dkl's fbfrog does automatic 'parameterisation': it combines multiple versions of include files from different OSes into one, spliced with ifdef's.
I think it can merge multiple FB headers in a separate step after they've been translated individually, but I'm not sure. If so, that could help, after manual cleaning up of headers for individual OSes. There are a number of its features don't understand how to use.

On looking more closely at the crt headers in fbbindings, I see they're more limited than I expected, only have time.bi, types.bi, locale.bi and signal.bi for each OS, plus some glibc headers such as pthread.bi... which are of course actually GNU/linux-specific. I generated fcntl.bi for linux, but it's a mess due to dependencies on other files.

I really don't understand how system headers are organised on Unix OSes. Which files are libc, which come from the kernel (no distinction on BSDs, I assume), what's the meaning of /usr/include/bits/ and other subdirectories. I know a lot/most of it is a glibc-specific mess since I looked at the headers in the glibc source, but I can't see reason behind it, which makes this hard for me, whether manually or automatically.

Why are the functions names prefixed with Fp? Is that just to avoid clashes?

[marcov]

The trouble is with using converters, the next person just will run some new converter and declare the result is the best. We have (had) such problems with e.g. the gtk headers, and the old Kylix libc headers.

Keeping it tight and only allowing patches that fix/extend things (rather than just new conversions with different formatting) helps keeping somewhat track of the state of the port using the version system, and avoids reintroducing old bugs with newer conversions. (e.g. because they didn't pass a define to the converter).

Also often one goes more the "configure" way, compiling small programs and dumping the results. Dump the size of most *nix types (off_t etc) using a C and do a few structs (stat,statfs) and if there is not too much weirdness, you already have enough to compile the compiler (which doesn't use much terminal stuff other than one simple function to see if the result is piped and disables buffering)

[TeeEmCee]

Yes, as an example, many of the existing include files distributed with FB were generated with SWIG. dkl has been replacing them with new headers generated with fbfrog, which has resulted in some breaking changes, but the results do seem better. The crt/system headers are still SWIG+hand written.

The way dkl has the fbbindings project set up (which generates most of the bindings included with FB), the headers are generated from a makefile which downloads and runs a certain version of fbfrog on a certain version of the original headers. Manual edits are supported by having two git branches, with the edits made only to one, and then merging regenerated headers (eg due to a version change) into that branch. So that avoids some of the problems you warned of: results are totally reproduceable.

But it doesn't prevent large changes from occurring, which could break someone's code. I found that adding another linux system include file to be processed resulted in huge changes to unrelated headers -- the makefile rule for the crt/system headers reads a whole lot of original .h files, and then produces all the .bi files at once. As you said, breaking it up into smaller pieces would bring more stability. I guess I'll look into that.