Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

General FreeBASIC programming questions.
D.J.Peters
Posts: 8145
Joined: May 28, 2005 3:28
Contact:

Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby D.J.Peters » Jul 21, 2019 19:39

Only for fun as a short challenge I cracked a trial version of Milkshape 3D so it's a full version now.

How ever Milkshape 3D is written in Visual Studio C++ and more important for this topic it's plugin interface also.

This is the third time I use Microsoft Visual Studio C++ classes with FreeBASIC.
One was the real time audio interface ASIO from the company Steinberg I wrote in FreeBASIC and published years ago.
The others I wrote many plugins for Basic4GL also written in VS C++ in FreeBASIC.

Here are the simple Milkshape 3D plugin C++ class with virtual members.
file "msPlugin.h":

Code: Select all

struct msModel;
class cMsPlugIn {
public:
  cMsPlugIn () {};
  virtual ~cMsPlugIn () {};

public:
  virtual int             GetType () = 0;
  virtual const char *    GetTitle () = 0;
  virtual int             Execute (msModel* pModel) = 0;
};

typedef cMsPlugIn* (*FN_CREATE_PLUGIN)();
typedef void (*FN_DESTROY_PLUGIN)(cMsPlugIn*);

cMsPlugIn *CreatePlugIn ();
void DestroyPlugIn (cMsPlugIn *plugin);
A Milkshape 3D plugin must extend this class and implement the 3 virtual functions.
GetType() GetTitle() and Execute().

Only __stdcall CreatePlugIn() must be visible exported from a C++ plugin dll.

As a first test I used a well tested Milkshape plugin "msTextExporter.dll" (32-bit) and call it's virtual class members from a FreeBASIC *.exe successful.

file "interfacetest.bas"

Code: Select all

#inclib "msTextExporter"

type msModel as any

type MsPlugIn
  declare constructor (byval pCPPClass as any ptr)
  declare function GetType as long
  declare function GetTitle as const zstring ptr
  declare function Execute(byval pModel as msModel ptr) as long
  private:
  as any ptr m_pCPPClass
end type
constructor MsPlugIn (byval pCPPClass as any ptr)
  m_pCPPClass=pCPPClass
end constructor

function MsPlugIn.GetType as long
  if m_pCPPClass=0 then beep:return -1
  var pThis=m_pCPPClass
  dim as long result
  asm
  mov  ecx,[pThis] ' put the hidden this pointer in ecx
  mov  eax,[ecx]   ' get address from virtual table
  call [eax + 4]   ' call __stdcall vtable[1]
  mov [result],eax
  end asm
  return result
end function

function MsPlugIn.GetTitle as const zstring ptr
  if m_pCPPClass=0 then beep:return 0
  var pThis=m_pCPPClass
  dim as const zstring ptr result
  asm
  mov  ecx,[pThis] ' put the hidden this pointer in ecx
  mov  eax,[ecx]   ' get address from vtable
  call [eax + 8]   ' call __stdcall vtable[2]
  mov [result],eax
  end asm
  return result
end function

function MsPlugIn.Execute(byval pMilkShapeModel as msModel ptr) as long
  if m_pCPPClass=0 then beep:return -1
  if msModel=0 then beep:return -1
  var pThis  = m_pCPPClass
  var pModel = pMilkShapeModel
  dim as long result
  asm
  push dword [pModel]
  mov  ecx,  [pThis] ' put the hidden this pointer in ecx
  mov  eax,  [ecx]   ' get address from vtable
  call [eax + 12]    ' 12 call __stdcall vtable[3]
  mov [result],eax
  end asm
  return result
end function

extern "Windows-MS"
  ' !!! all stdcall without @# decoration !!!
  declare function CreatePlugIn() as any ptr
  declare sub      DestroyPlugIn(byval plugin as any ptr)
end extern

var pPlugin = CreatePlugIn()
dim as MsPlugIn plugin = MsPlugIn(pPlugin)
print "plugin.GetType() : " & plugin.GetType()
print "plugin.GetTitle(): " & *plugin.GetTitle()
print "plugin.Execute(0): " & plugin.Execute(123)
print "..."
sleep
First to note are Microsoft C++ don't put the hidden THIS pointer on the stack it stores it in the ECX register before any class members call.

mov ecx, [pThis] ' put the hidden this pointer in ecx

Another important fact are the pointer returned by the C++ pseudo code :
auto pClasspoiter = new myPlugin()

Is in real a pointer on a table of other pointers. (I name it VTABLE but I don't mean the interface table from MS COM interfaces)

mov eax, [ecx] ' get address from vtable

call [eax + function_number * sizeof(pointer)]

the memory layout are:

call vtable[1] = GetType()
call vtable[2] = GetTitle()
call vtable[3] = Execute()

after the call I never restore the stack frame so it must stdcall (the functions self restore the stack pointer)

I'm not sure but for me looks like:
call vtable[0] is the address of the class () {} constructor.

Normally I would think if I can call VS C++ classes from FreeBASIC it should be possible to implement such (simple) classes in FreeBASIC also.

With other words I will try to write a Milkshape 3D plugin as a FreeBASIC dll.

Before I give it a try I recall what we have so far:

Code: Select all

class msPlugin {
  a(){};
  virtual ~a() = 0;
  virtual int  GetType () = 0;
  virtual const char * GetTitle () = 0;
  virtual int Execute (msModel* pModel) = 0;
}
class myPlugin public msPlugin {
  myPlugin();
  int  GetType () ;
  const char * GetTitle ();
  int Execute (msModel* pModel);
}
myPlugin::myPlugin() { /* code removed */}
myPlugin::GetType () { /* code removed */}
myPlugin::GetTitle () { /* code removed */}
myPlugin::Execute (msModel* pModel) { /* code removed */}
// export it from dll
msPlugIn *CreatePlugIn () {
  return new myPlugin;
}
One problem are if the plugin functions are called by Milkshape the THIS pointer are in the ECX register that differs from gnu C++ or FreeBASIC class model.
May be naked class members in FreeBASIC are a kind of solution.

Does any fbc guru can explain the memory layout of FreeBASIC classes please ?

How do you would create a faked VS C++ class in FreeBASIC ?

Joshy

edit:A bad solution looks simpler as excepted
getType() as long
getTitle() as const zstring ptr
Execute(pModel as any ptr) as long
Does not need the THIS pointer so it can be ignored

pseudo code:

Code: Select all

var vtable new any ptr [5]
vtable[0] = @myDummyContructor
vtable[1] = @myGetType
vtable[2] = @myGetTitle
vtable[3] = @myExecute
vtable[4] = NULL
' export only the entry point of the plugin
function CreatePlugIn () as any ptr export
  return @vtable
end function
marcov
Posts: 3010
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby marcov » Jul 21, 2019 20:42

I don't see the constructor declared as virtual, so why would it be in the vtable? Are constructors always virtual by default?
D.J.Peters
Posts: 8145
Joined: May 28, 2005 3:28
Contact:

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby D.J.Peters » Jul 21, 2019 23:10

marcov wrote:I don't see the constructor declared as virtual, so why would it be in the vtable?
You are right I have done a beep() in myDummyConsturctor()
and if the faked plugin are loaded by Milkshape 3D
I can hear the beep and Milkshape 3D exit (or crashed without a crash report)

So I think vtable[0] points to something that will be called
or the vtable is longer and it crashed after the call to myDummyConstructor

imagine (pseudo code)

Code: Select all

class A {
  A(){} ; // none virtual A constructor
  virtual int F1() = 0;
  virtual int F2() = 0;
  virtual int F3() = 0;
};
class B public A {
  B(); // none virtual B constructor
  int F1(); // implemented virtual methods
  int F2();
  int F3();
};
B::B() { }
int B::F1() { }
int B::F2() { }
int B::F3() { }

may be this memory layout is the right
jumpTable[0] = is B::B()
jumpTable[1] = is B::F1()
jumpTable[2] = is B::F2()
jumpTable[3] = is B::F3()
// new
jumpTable[4] = is A::A()

or this
jumpTable[0] = A::A first all A class members (it's only one constructor)
jumpTable[1] = is B::F1() followed by the virtual members of class B
jumpTable[2] = is B::F2()
jumpTable[3] = is B::F3()
jumpTable[4] = is B::B() followed by the all none virtual members of class B

I will test it also :-)

Joshy
marcov
Posts: 3010
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby marcov » Jul 22, 2019 9:24

I've no clue of C++ vtable layout, but if vtable[0] is not a method, it might also be a pointer to some other descriptor table (like for multiple inheritance or RTTI).
dkl
Site Admin
Posts: 3212
Joined: Jul 28, 2005 14:45
Location: Germany

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby dkl » Jul 24, 2019 19:30

Maybe I can help out with this...

fbc uses the same basic layout for structs and vtables as g++ (and msvc and COM), but some parts are skipped since FB doesn't support the full C++ features.

Regarding fbc's vtable layout: https://github.com/freebasic/fbc/blob/1 ... p.bas#L150
For reference, the full version: https://itanium-cxx-abi.github.io/cxx-a ... components
fbc does not have the first two offsets (related to virtual base classes), but it has the "offset to top" (always 0 though since FB only supports single inheritance) and the "typeinfo pointer" (for RTTI), followed by the array of function pointers for the virtual methods in declaration order.

Regarding struct layout: The only thing special about classes with virtuals is that they have the vptr (pointer to vtable) added at the top. In FB that's done by "extends object". One interesting detail is that the vptr does not point to begin of the whole vtable, but to the begin of the function pointer array in the vtable (so typeinfo pointer is at vptr[-1]). Concerning struct layout in general, fbc follows the rules of g++ and msvc, I think only bitfields cause differences between all 3 compilers.

It's true that calling convention also matters. For class methods, g++/msvc often uses __thiscall (or __fastcall? I don't even know), which fbc does not support. So to be compatible to FB, you have to use __cdecl or __stdcall on the C++ side (and put the same on the FB side). This matters mostly for 32bit x86, since for 64bit there is a unified calling convention per platform, which is used by fbc too. So other than that, fbc follows the g++/msvc calling conventions pretty well. I don't remember any problems, except for an incompatibility with functions returning structs (aggregate results), where even g++/msvc disagree: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64384
D.J.Peters
Posts: 8145
Joined: May 28, 2005 3:28
Contact:

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby D.J.Peters » Sep 16, 2020 17:01

Dam I got it it's was to simple for me :lol:

First Visual Studio C++ class members are by default "stdcall" same as in FreeBASIC (not cdecl) !

Of course the Microsoft C++ member name mangling is not GNU ABI g++ or FreeBASIC compatible !

How ever in memory it's an array of pointers to the C++ class member methods or functions.

pInterface[0] = @Constructor
pInterface[1] = @GetType
pInterface[2] = @GetTitle
pInterface[3] = @Execute
pInterface[4] = NULL


So far so good I have done it before but it crashes !

The point are I declare and implemented the constructor wrong :-(
sub _constructor stdcall : end sub

I forgot the constructor get a pointer on the hidden THIS pointer on the stack also !
sub _constructor stdcall (pInterface as any ptr ptr) : end sub

I hear you saying I_d_i_o_t there is always the hidden this pointer on the stack !

But look how the other class members GetType, GetTitle and Execute
are implemented (without a hidden this pointer) only the constructor used the stack with the THIS pointer !

In real all other C++ class members are called with the this pointer in CPU register ECX !

Not on the stack frame (and not declared as argument in FreeBASIC).

No more crash and all kinds of Milkshape3D plugins works great with FreeBASIC :-)

Joshy

Code: Select all

#if defined(__FB_64BIT__) or (__FB_OUT_DLL__ = 0)
 #error 666: this source code must be compiled as 32-bit Windows DLL !
#endif

#include "inc/msPlugin.bi"

dim shared as any ptr ptr pInterface
dim shared as zstring ptr zsTitle = @"My Plugin :-)"

sub _constructor stdcall (pInterface as any ptr ptr)
end sub

' return the type of your plugin
' eTypeImport, eTypeExport, eTypeTool, eTypeEdit, eTypeVertex, eTypeFace, eTypeAnimate
function GetType stdcall () as PluginType
  return eTypeImport
end function

' return a const pointer of the string showed in the Milkshape3D menu (import/export/tool,edit ...)
' be shure the string exists the whole runtime of Milkshape3D
' if not and milkshape will show the string in any of the menus it will crash !°
function GetTitle stdcall () as const zstring ptr
  return zsTitle
end function

' run your plugin on the one and only Milkshape 3D model (scene)
' on end return eOK or eError !
function Execute stdcall (model as any ptr) as PluginError
  ' should never happen !
  if model=0 then return eError
 
  ' if your plugin is an model exporter
  ' you got a copy of the model to export
  ' that means you have to destroy it with "msModel_Destroy(model)"
  ' in all other cases importer, tool, edit ...
  ' you modify the model (add meshes, manipulate meshes, triangles, material ...)
 
  return eOk
end function

sub _plugin_load_ constructor
  pInterface = allocate(5*sizeof(any ptr))
  pInterface[0] = @_Constructor
  pInterface[1] = @GetType
  pInterface[2] = @GetTitle
  pInterface[3] = @Execute
  pInterface[4] = 0
end sub

sub _plugin_unload_ destructor
  if pInterface then deallocate pInterface
end sub

extern "Windows-MS" ' no name decoration !
' the one and only exported function
function CreatePlugIn as any ptr export
  return @pInterface
end function

end extern
Last edited by D.J.Peters on Sep 16, 2020 18:25, edited 1 time in total.
marcov
Posts: 3010
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby marcov » Sep 16, 2020 18:21

Actually they are afaik "thiscall" which is a method version of stdcall.
cwolf
Posts: 4
Joined: Sep 18, 2020 0:49

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby cwolf » Sep 18, 2020 22:24

I dont suppose your crack for milkshape is available anywhere?
Tourist Trap
Posts: 2904
Joined: Jun 02, 2015 16:24

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby Tourist Trap » Sep 19, 2020 7:20

cwolf wrote:I dont suppose your crack for milkshape is available anywhere?

Hi cwolf.

Edit: I was not aware of the work on a SDK for that stuff ;)

If you need a character modeling software, I use Makehuman, it's free and opensource and pretty cool :)
http://www.makehumancommunity.org/
Last edited by Tourist Trap on Sep 19, 2020 11:50, edited 1 time in total.
D.J.Peters
Posts: 8145
Joined: May 28, 2005 3:28
Contact:

Re: Let us talk about memory layout of classes Microsost C++ vs gnu g++ vs FreeBASIC !

Postby D.J.Peters » Sep 19, 2020 11:18

cwolf wrote:I dont suppose your crack for milkshape is available anywhere?
I added the clean code here: viewtopic.php?f=14&t=27732

It works well I made many plugins last two days :-)

Joshy

Return to “General”

Who is online

Users browsing this forum: fxm, jevans4949, marcov and 5 guests