fbfrog header translator

[TeeEmCee]

OK, thanks. I see now that size_t doesn't appear very much in the headers shipped with FB. However, it is used everywhere in the existing crt/ headers which were translated with SWIG, so I wanted to be consistent to make it easier to compare.

However, adding crt/ headers for FreeBSD to match all the ones available for Linux and merging them into the existing headers is too much work, so I don't think I'll be completing that project. Header files is also why I never finished my work on the Darwin port.
There are various existing FB crt/ headers that have a common part plus some OS-specific parts in separate files. But it turns out that a lot of those common parts aren't actually entirely the same between different unixes, because not all of the functions are standardised. And these common parts shouldn't be duplicated in the OS-specific headers, which can only possibly be maintainable if a script is written to remove the duplication automatically. So it would be a lot easier and safer to avoid common sections entirely.

Worse, the locations of declarations also differs between the OSes (and the standard headers always cross-include each other in a tangled web). Should the locations of the declarations in the FB headers match the original ones (different on every OS), or should they be always the same? The latter is more or less the current situation.

Is it possible to make fbfrog skip anything that's already been defined in another file?

[dkl]

It's possible to tell fbfrog to only emit parts of the input (filtered by input file name), or to drop symbol declarations by name pattern. That's how I used it - parse the whole API in one go, then emit the needed parts, check if anything is missing afterwards. For example: freeglut or libcairo

A while ago I tried to do a CRT/POSIX/system header update, including FreeBSD. fbfrog's merging algorithm didn't produce nice results with the vastly different headers of the various systems; it seemed easier to keep header files separated by system (for example crt/time.bi would redirect to crt/freebsd/time.bi or crt/linux/time.bi).

Much like with Windows API headers I encountered a lot of problems though. Due to preprocessing/parsing all the headers in one run, all #ifs and macros are solved out, and the headers may now depend on the order they were preprocessed in. Thus, when using -emit to produce multiple .bi files, you have to test whether they actually work when used alone, or in different order than the one they were generated from.

[bfg]

A big thanks to you dkl!!!

With fbfrog now I can play with shapefile (without reinventing the wheel) using shapefile C library (http://shapelib.maptools.org).

I've translated the header and, except for a macro, it seems is working well...

Code: Select all

'' TODO: # define SHP_CVSID(string) static const char cpl_cvsid[] __attribute__((used)) = string;

Shapefiles, and the associated attribute file (.dbf) are a de-facto standard and an easy way to work with geospatial vector data. And actually I'm trying to convert and analyze gtfs files (https://developers.google.com/transit/gtfs/), a format for public transportation schedules and associated geographic information.

[speedfixer]

help?

No output from fbfrog -- all files. Clearly, I am doing something wrong, or missed an instruction. Or is the include guard stopping everything? All the files have one.

source:

Code: Select all

/*
    Copyright (c) 2012 Martin Sustrik  All rights reserved.

    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"),
    ...
    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
    IN THE SOFTWARE.
*/

#ifndef TCP_H_INCLUDED
#define TCP_H_INCLUDED

#ifdef __cplusplus
extern "C" {
#endif

#define NN_TCP -3

#define NN_TCP_NODELAY 1

#ifdef __cplusplus
}
#endif

#endif

output:

Code: Select all

'' FreeBASIC binding for nanomsg 1.1.2 -- xlate: fbfrog, speedfixer, 11/2017
''
'' based on the C header files:
''
''   /*
''       Copyright (c) 2013 Martin Sustrik  All rights reserved.
''
''       Permission is hereby granted, free of charge, to any person obtaining a copy
''       of this software and associated documentation files (the "Software"),
...
''       FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
''       IN THE SOFTWARE.
''   */
''
'' translated to FreeBASIC by:
''
''   Translator: fbfrog by dkl
''       FreeBASIC c header *.h to FB *.bi program
''       with help from speedfixer (David Barnes) - 11/2017

commandline:

.../FB/fbfrog/fbfrog nano.fbfrog

nano.fbfrog:

Code: Select all

# nano.fbfrog

# fbfrog specification file to create headers for nanomsg

-title 'nanomsg 1.1.2 -- xlate: fbfrog, speedfixer, 11/2017' original-license.txt translators.txt

-v -v -target linux-x86_64
-emit 'nn.h' ./target/nn.bi
-emit 'bus.h' ./target/bus.bi
-emit 'inproc.h' ./target/inproc.bi
-emit 'ipc.h' ./target/ipc.bi
-emit 'pair.h' ./target/pair.bi
-emit 'piepline.h' ./target/pipeline.bi
-emit 'pubsub.h' ./target/pubsub.bi
-emit 'reqrep.h' ./target/reqrep.bi
-emit 'survey.h' ./target/survey.bi
-emit 'tcp.h' ./target/tcp.bi
-emit 'ws.h' ./target/ws.bi

The console out is a lot of #defines, the emitting: messages - nothing else. All files processed. All just as empty.
Any suggestions?

David

[Munair]

It looks like there's nothing to translate from the source (0 declarations).

[speedfixer]

Example files were tcp.h and tcp.bi

Are you saying that:


#define NN_TCP -3


should not be in a resultant tcp.bi file?

[Munair]

Example files were tcp.h and tcp.bi

Are you saying that:


#define NN_TCP -3


should not be in a resultant tcp.bi file?

You can copy that as it is.

[dkl]

It could be a problem with the -emit options, maybe it will work if you add a * wildcard:

Code: Select all

-emit 'nn.h' ./target/nn.bi
-emit '*/nn.h' ./target/nn.bi

Also it can help to try without any -emit.

[speedfixer]

No change.

Removing the emit just gave me a single 'unknown.bi' in the source folder - 1 emit only.

I notice that my original nano.fbfrog file had misspelled a filename - pipeline.h

fbfrog did not complain that it could not find that file.

[speedfixer]

Tried the absolute path also - yields one file per entry - still empty.

Its after midnight here, I'll try more in the morning.
On re-examining these files, I'll just do it by hand. I was going to wrap the whole thing into a single FB static library set anyway. Only one has any real code in it. The others are just feed parameters it looks like.

I DO want to get fbfrog to work, though, so I will keep playing with that.

Thanks. I will post more if I learn anything. Any ideas would help.

David

[dkl]

I saved your source as nn.h and ran fbfrog like this:

Code: Select all

$ fbfrog nn.h -emit '*/nn.h' nn.bi
[ 1/23] linux-x86
[ 2/23] linux-x86_64
[ 3/23] linux-arm
[ 4/23] linux-aarch64
[ 5/23] freebsd-x86
[ 6/23] freebsd-x86_64
[ 7/23] freebsd-arm
[ 8/23] freebsd-aarch64
[ 9/23] openbsd-x86
[10/23] openbsd-x86_64
[11/23] openbsd-arm
[12/23] openbsd-aarch64
[13/23] netbsd-x86
[14/23] netbsd-x86_64
[15/23] netbsd-arm
[16/23] netbsd-aarch64
[17/23] darwin-x86
[18/23] darwin-x86_64
[19/23] win32
[20/23] win64
[21/23] cygwin-x86
[22/23] cygwin-x86_64
[23/23] dos
	nn.h
emitting: nn.bi (3 declarations, 0 TODOs)
$ cat nn.bi
#pragma once

#define TCP_H_INCLUDED
const NN_TCP = -3
const NN_TCP_NODELAY = 1

[Munair]

I ran the same thing this morning, but without parameters. Why does it ignore the declarations with a default run like fbfrog nn.h ?

[speedfixer]

@dkl:

Under options in your doc:

* `-emit '*.h' foo.bi`: Emit code from matching .h into specified .bi


Your use of it here was:

nn.h -emit '*/nn.h' nn.bi

Used this way, it works.
Why reference the source 2 times?


Are these important?

Code: Select all

/usr/include/errno.h(46): expected identifier for the symbol declared in this declaration but found 'extern'
    __BEGIN_DECLS extern int errno ;
                  ^~~~~~
/usr/include/stdint.h(36): expected identifier for the symbol declared in this declaration but found 'typedef'
    __END_DECLS typedef signed char int8_t ;
                ^~~~~~~
nn.h(46): expected expression but found 'extern'
    # define NN_EXPORT extern 

David

[speedfixer]

regarding 'Are these important' above.

I changed the -incdir from the system compiler location </usr/include/> to the FB crt include location</usr/local/include/freebasic/crt>.
The messages went away - only left with one 'extern' reference as TODO.

So, all is good, though I would be interested to understand what the intended syntax of the -emit option is supposed to be.
-----------------------------------------------------------
I will be using nanomsg as my interprocess, inter-program, inter-system protocol. (I currently have 14 systems at home for the same program set.)
I will wrap the nanomsg API into one FB include and share that when it is all polished and shiny. I am using Linux, only. Perhaps the generous D.J.Peters will pick it up and make it available for the other platforms.

https://github.com/nanomsg/nanomsg
If you research nanomsg, you will eventually find that it is abandoned (2+ years), even though minor fixes are applied sometimes. Basically, a FOSS cultural thing, NOT technical. It works. It isn't bloated like ZMQ (also, abandoned -- same reasons). C source only so easy to interface. It still looks like the best for my modest purpose.

Thank you so much, dkl, for your contributions to FB, and your personal efforts to help me with my bugs now and in the past.

David

[speedfixer]

The include for crt for FB defines standard errors.

Many packages also define several of these errors using pretty much the same convention, and nanomsg is no different.

Example:
FB:
#define EINVAL 22

nanomsg:
#define EINVAL = NN_HAUSNUMERO + 14
(NN_HAUSNUMERO == some large random number)

The original compiled nanomsg provided the values for the translated headers - I can't change that without some effort.

A simple compilation gives:
/rdkl/FB_net/nanomsg/target/nn.bi(101) error 4: Duplicated definition, EINVAL in '#define EINVAL = NN_HAUSNUMERO + 14'


What to do?
Should the FB <#include once "crt/errno.bi"> just be remarked out?
Put a compile guard on the duplicates (about 6)?

Also, those were originally translated as <const EINVAL = NN_HAUSNUMERO + 14> but the compiler gave:

/rdkl/FB_net/nanomsg/target/nn.bi(62) error 14: Expected identifier in 'const EINVAL = NN_HAUSNUMERO + 14'

so I changed them to #defines
Is that correct?

David