Pure FB Runtime Library (in progress)

User projects written in or related to FreeBASIC.
Post Reply
fatman2021
Posts: 215
Joined: Dec 14, 2013 0:43

Re: Pure FB Runtime Library (in progress)

Post by fatman2021 »

Here is the memcpy in C. Maybe someone else may have better luck.

Code: Select all

void memcpy(void *dest, void *src, size_t n)
{
   // Typecast src and dest addresses to (char *)
   char *csrc = (char *)src;
   char *cdest = (char *)dest;
 
   // Copy contents of src[] to dest[]
   for (int i=0; i<n; i++)
       cdest[i] = csrc[i];
}
 
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: Pure FB Runtime Library (in progress)

Post by D.J.Peters »

(only for info)
Some C-runtime methods like memcpy are a kernel syscall or a call inside the OS.
If you boot your OS Linux or Windows it research the target CPU
and examined some features of the CPU (if available like MMX/2,3DNOW, SSE/2,3,4)
After this and the shared C-runtime lib is loaded the first time in memory
the pointer of memcpy or the kernel syscall will point to the fastes copy method found by the kernel/OS.

For the good old DOS (real mode) memcopy are hard coded in the C-runtime.

Since I study some parts of the kernel source codes I use only memcpy not my own MMX,SSE versions of copy anymore.

Joshy
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: Pure FB Runtime Library (in progress)

Post by TeeEmCee »

What, really? I've never heard of delegating something like that to the kernel on Unix, though I would expect memcpy on Windows to just call the equivalent routine in user32.dll (which is still user-mode, not part of the kernel). It seems crazy to do a syscall for an optimised memcpy, since that adds a lot of overhead, so it wouldn't make sense except for large copies. And libc can and does pick an optimised variant for the CPU itself (although without actually doing some timings, like the kernel does for some things, it wouldn't necessarily pick correctly), which is probably what you meant. Of course, the kernel will do the same thing, but they are completely independent.
If it's done on Linux, could you name the syscall used?
Josep Roca
Posts: 564
Joined: Sep 27, 2016 18:20
Location: Valencia, Spain

Re: Pure FB Runtime Library (in progress)

Post by Josep Roca »

This is what both memcpy and memmove do:

Code: Select all

void* __cdecl memcpy (void *dest,
      const void * src,
      size_t count 
   )

{
     char *char_dest = (char *)dest;
     char *char_src = (char *)src;
 
     if ((char_dest <= char_src) || (char_dest >= (char_src+count)))
     {
         /*  non-overlapping buffers */
         while(count > 0)
     {
             *char_dest = *char_src;
             char_dest++;
             char_src++;
             count--;
     }
     }
     else
     {
         /* overlaping buffers */
         char_dest = (char *)dest + count - 1;
         char_src = (char *)src + count - 1;
 
         while(count > 0)
     {
            *char_dest = *char_src;
            char_dest--;
            char_src--;
            count--;
     }
     }
 
     return dest;
 }
and wmemmove:

Code: Select all

__CRT_INLINE wchar_t *__cdecl wmemmove (_Out_writes_all_opt_(_N) wchar_t * _S1,
      _In_reads_opt_(_N) const wchar_t * _S2,
      _In_ size_t _N 
   )

  {
     return (wchar_t *)memmove(_S1,_S2,_N*sizeof(wchar_t));
   }
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Pure FB Runtime Library (in progress)

Post by fxm »

I thought that the C function was more optimized, for example using Integer datatype copy where that is possible.
Josep Roca
Posts: 564
Joined: Sep 27, 2016 18:20
Location: Valencia, Spain

Re: Pure FB Runtime Library (in progress)

Post by Josep Roca »

It is a generic funcion to copy bytes. For wmemcpy and wmemmove they could move words instead of bytes. Don't know if currently they are doing it this way, since I don't have the C source code.
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Pure FB Runtime Library (in progress)

Post by dodicat »

Plenty of hypothesising.
Do a run test gas/gcc 32/64 bit with a simulator.

try -O3 optimisations.
(Maybe the simulator could be improved, of course)

Code: Select all

#include "crt.bi"

#macro memcopy(dest,src,size)
    for n as long=0 to (size)/sizeof(*(src))-1
        (dest)[n]=(src)[n]
    next
#endmacro


#macro GetSize(array,d)
    d=ubound(array,0)
            for n as integer=1 to d
                if n=1 then d=1
              d=d*(ubound(array,n)-lbound(array,n)+1)
          next
          d=d*sizeof(array)
    #endmacro
    
    dim as integer size
    
    dim as zstring ptr a(8,1 to 7,-3 to 5,12)
    dim as zstring ptr b(8,1 to 7,-3 to 5,12)
    
    getsize(a,size)
    print "Size of array ",size
    
    a(6,3,-1,2)=@"Custom"
    
    dim as double t
    
    t=timer
    for n as long=1 to 5000
        memcopy(@b(0,1,-3,0),@a(0,1,-3,0),size)
    next
    print timer-t,*b(6,3,-1,2)
    
    sleep 100
    
    
     a(6,3,-1,2)=@"Runtime"
    
    t=timer
    for n as long=1 to 5000
        memcpy(@b(0,1,-3,0),@a(0,1,-3,0),size)
    next
    print timer-t,*b(6,3,-1,2)
    print
    sleep
    
    
    
     
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Pure FB Runtime Library (in progress)

Post by coderJeff »

Wow. Cool. I had dreams of this, never thought anyone would attempt. Imortis, kudos. Thank-you for putting your self out there, asking questions, willing to learn. Awesome.

Allow me to offer a few comments:

MEMCPY: an optimization for sure. See memcpy.c. Implemented trivially as a for(), char * copy loop. Hand-rolled ASM existed in DJGPP since 1995.

Dependency on C runtime: to completely replace common C runtime on all platforms will be a monumental task. Even if major parts of libfb rewritten in FB, plan for a mix of C and FB. At some point you have to interface with the execution environment. Leverage the work of others and target what is typically available on most platforms - a working standard common C runtime - initially at the very least. Especially file I/O, pipes, shell & process execution.

I had a hand in writing serial and printer drivers on dos/win/*nix, something that does not exist in standard C runtimes. Every platform is different. And each had to be written specifically for each platform. Even within windows environment, different code paths for win98, winxp, etc.

FBRT0.bas: to write this as pure FB, then would have to add an __attribute__() type feature in to FB. I was responsible for the C hacks you see in fbrt0.c. Purpose is to initialize C runtime, then FB runtime, in that order. Ultimately __attribute__() controls the linker and where the global ctors/dtors get added. We tried other solutions like custom linker scripts, but ultimately the C hacks were the most portable (within gcc world). But even with C this is non-standard and not completely portable as DJ handles global ctors/dtors in DOS differently than on win/*nix. Quite possible that unknown future targets will be different from anything seen so far.

What will be most challenging is implementing the run-time in FB without using any feature you would normally get from the FB runtime. For example, can't use STRING in the run time, because from the run-time point of view, it doesn't exist yet.

Question: what is the varied use of Extern "C", cdecl, FBCALL? Not saying it is wrong, I just don't understand.
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: Pure FB Runtime Library (in progress)

Post by TeeEmCee »

coderJeff wrote:FBRT0.bas: to write this as pure FB, then would have to add an __attribute__() type feature in to FB.
...
But even with C this is non-standard and not completely portable as DJ handles global ctors/dtors in DOS differently than on win/*nix. Quite possible that unknown future targets will be different from anything seen so far.
Oh yeah, aside from ctor/dtor priorities not being supported on Darwin, there are problems on Android too. Because Android has its own custom ld instead of using ld-linux-*.so, which does things differently.
Maybe we should just use ctor/dtor priority 101 for the FB runtime, and disallow users from using priority 101. Then we can avoid nonportable __attribute__((section)) hacks which in general can't be done on every OS.
Constructors/destructors are already implemented in FB; see here. With -gen gcc it compiles to __attribute__(( constructor( <priority> ) )) as you would expect.
What will be most challenging is implementing the run-time in FB without using any feature you would normally get from the FB runtime. For example, can't use STRING in the run time, because from the run-time point of view, it doesn't exist yet.
That's not true. Any features of FB can be used in the rtlib, except to implement those same features. The rtlib can be simplified greatly by making using of strings.
Question: what is the varied use of Extern "C", cdecl, FBCALL? Not saying it is wrong, I just don't understand.
Imortis was using cdecl before I told him to use Extern "C" instead. Now all the cdecl's do nothing and can be removed.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Pure FB Runtime Library (in progress)

Post by marcov »

Another solution is to hack main to let it first init fb runtime and FB level module initialization and then call the FB main, and deinit fb runtime afterwards.
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: Pure FB Runtime Library (in progress)

Post by TeeEmCee »

But that requires actually having a list of module initialisers. The gas backend could do that by embedding metadata in object files which is then read at the linking stage. But to do that in the gcc backend without inspecting the object files (eg for constructor functions which are recognisable by a special name) is more difficult.

Actually, I see a way: in each module with one or more FB constructors or destructors, those ctors/dtors become SUBs, plus a single constructor which calls some rtlib function to register all the module ctors/dtors plus their priorities. The list is then sorted and run when entering main().
Tje problem is that this isn't compatible with constructor priorities defined in C or other languages using the linker (eg __attribute__((constructor))) mechanism: all the FB constructors will happen after all the real constructors.

Also, shouldn't the destructors be called via atexit() rather than when leaving main()? Otherwise exit() will skip them.
Imortis
Moderator
Posts: 1923
Joined: Jun 02, 2005 15:10
Location: USA
Contact:

Re: Pure FB Runtime Library (in progress)

Post by Imortis »

TeeEmCee wrote:Damnit, I forgot to actually add the makefile to git! Sorry about about; I made another pull req for it.
There's a lot of stuff in FB's makefile that I left out, in particular most OS-specific stuff, including support for building for DOS. It can be added back later. But I guess actually targetting DOS is something that fbrtLib could excel at. Anyway, at 300+ lines, almost nothing in this makefile is actually necessary, but I thought it would be good to stay close to FB's own build system.
Thanks. I have merged it.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Pure FB Runtime Library (in progress)

Post by marcov »

TeeEmCee wrote:But that requires actually having a list of module initialisers. The gas backend could do that by embedding metadata in object files which is then read at the linking stage. But to do that in the gcc backend without inspecting the object files (eg for constructor functions which are recognisable by a special name) is more difficult.
I mentioned it, because it is how Free Pascal does it. But if you can't store metadata per compilation unit, you have a problem yes.

FPC has a strong module system, and regardless of backend (can be JVM, can link to objective C etc) or object format, there is always a precompiled header per module with such info (.ppu), and all modules must be compiled (and their info loaded) before the main module is compiled (the compiler takes care of that automatically), so then the info is there.
Tje problem is that this isn't compatible with constructor priorities defined in C or other languages using the linker (eg __attribute__((constructor))) mechanism: all the FB constructors will happen after all the real constructors.
Afaik those attributes those are not standard C but gccisms
Also, shouldn't the destructors be called via atexit() rather than when leaving main()? Otherwise exit() will skip them.
Calling the raw Exit(2) call will always skip them. Any higher level exit will wrap the exit of the level below and add its own finalization. exit(3) works so also.

Note that you can work with ctors/dtors where it makes sense (e.g. *nix). It is just that you don't *have* to, and force yourself in unnatural patterns on non *nix.
Imortis
Moderator
Posts: 1923
Joined: Jun 02, 2005 15:10
Location: USA
Contact:

Re: Pure FB Runtime Library (in progress)

Post by Imortis »

Here is a question:

file_dir.bas on windows calls this file when not compiled via CYGWIN:

Code: Select all

#include <direct.h>
Now, looking that up, it seems to be something that Microsoft distributes as part of their WinAPI headers but we don't seem to have it in FB.

Do we just make one? That should work, right?
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: Pure FB Runtime Library (in progress)

Post by TeeEmCee »

Those functions are in crt/io.bi rather than direct.h
marcov wrote:FPC has a strong module system, and regardless of backend (can be JVM, can link to objective C etc) or object format, there is always a precompiled header per module with such info (.ppu), and all modules must be compiled (and their info loaded) before the main module is compiled (the compiler takes care of that automatically), so then the info is there.
FB doesn't work like that. The main module can be compiled separately from the others. Or there may be no main module at all; main() could be written in a different language. The only initialisation done in main() is to grab argc and argv.
Afaik those attributes those are not standard C but gccisms
They are features of the OS, not the language, which GCC and FB and other compilers expose.
So a possible solution is to simply throw an error if you use them and attempt to compile for a platform where they aren't supported. GCC probably does the same? But it's the FB way to attempt to paper over OS differences.
Calling the raw Exit(2) call will always skip them. Any higher level exit will wrap the exit of the level below and add its own finalization. exit(3) works so also.
Well yes, that's the point. You call exit() if you want finalisers to run, and _exit() if you don't.
Post Reply