Wiki improvements

Forum for discussion about the documentation project.
Post Reply
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

fxm wrote:Maybe this article is a bit too technical...
Yes, I've been following your posts. I'm not sure what the technical interests are of most users. It varies often; byref function returns were popular for a while, then it was ImageCreate/ImageDestroy, then function pointers, etc.

Most users probably just want to know how to use it, but more importantly, why to use it. For example, I think primary purpose of new operator in C++ is to allocate memory and call the constructor. delete, obviously destructor and deallocate. But in C++ it is generalized to every kind of type, including arrays. In fbc we have fixed & resizable array container types, so why would we want to use new[]/delete[]?

The articles you have been writing have the content that could maybe one day fill in some of the gaps of the wiki/manual. And there's some wiki pages that have very technical information that only makes sense after reading some other pages first. So the order of understanding plays some role in how technical a reader might perceive the information.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

coderJeff wrote:In fbc we have fixed & resizable array container types, so why would we want to use new[]/delete[]?
It's a shame, I gave dkl some work on the 'New[]'/'Delete[]' subject (case without destructor, case with destructor), and also especially for the debug of 'Placement New[]' !
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

Maybe my tone of voice comes across strange. To clarify, if we have it as a valid option to the user, then we do want it to work correctly. fbc has quite a few features that probably hardly ever get used, until you get to that one scenario, and it's really convenient that the feature was added.

I've looked at the bugs a little, and I think I've seen the cases you are talking about, but I'm losing track of all the places. It's good information, and not a loss, fxm.

For me, I recently merged a number of changes, and just need to work with the compiler to try it out. Like, not break too many things too quickly.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

Confusion in the documentation and perhaps in the minds

In the documentation (and often in the literature), there is confusion between the 'New keyword' (unfortunately also called 'New operator') and the 'operator New'. Similarly, there is a confusion between the 'Delete keyword' (unfortunately also called 'Delete operator') and the 'operator Delete':
  • The 'operator New' ('Operator New(size As UInteger) As Any Ptr') is a static function that only allocates memory. It is not very different from Allocate().
    Only it can be overloaded for user-defined types as a member operator (static function).
  • The 'operator Delete' ('Operator Delete(buf As Any Ptr)') is a static sub that only frees the memory. It is not very different from Deallocate().
    Only it can be overloaded for user-defined types as a member operator (static sub).
  • The 'New keyword' starts by using the 'operator New' (the implicit or the overload) to allocate memory.
    Then it invokes the constructor for the right type of object. If that object contains any other objects (either embedded or as base types) those constructors as invoked as well.
    So the final result is memory allocated and object constructed.
  • The 'Delete keyword' starts by invoking the destructor for the right type of object. If that object contains any other objects (either embedded or as base types) those destructors as invoked as well.
    Then it uses the 'operator Delete' (the implicit or the overload) to deallocate memory.
    So the final result is object destroyed and memory freed.
Similar remark for 'New[]' and 'Delete[]'.

In each documentation page, there is a confusing mix between the keyword and the operator:
  • The 'Title' refers to the operator only.
  • The 'Syntax' paragraph corresponds to the operator.
  • The 'Usage', 'Parameters', 'Return Value' paragraphs correspond to the keyword.
  • The 'Description' paragraph processes the two but with confusion of roles.
I do not think it's necessary or even good to have two separate pages, one for the keyword and the other for the operator, since the two are very related: the keyword calling the operator but doing something else.
I think that some simple modifications should allow to clarify the text body.

For example for the 'New' page:
  • Title: 'Operator NEW', then some sentences to present the two separate functions induced by the keyword (*).
  • 'Operator Syntax' paragraph.
  • 'Keyword usage', 'Keyword Parameters', 'Keyword Return Value' paragraphs.
  • 'Description' paragraph: highlighting that is done by the operator or the keyword.
  • For the keyword, use the wording: 'NEW' or 'NEW (keyword)' or 'New keyword'.
  • For the operator, use the wording: 'operator NEW'
For the 'Delete' page, similar text improvement.

For 'Placement New', the problem is simpler because there is no associated operator (no memory allocation).
For homogeneity with other pages, perhaps suppress the operator term and call it: 'Placement New' or 'Placement New (keyword)' or 'Placement New keyword'.

Perhaps also suppress 'Op' in the 3 links:
- KeyPgOpNew
- KeyPgOpDelete
- KeyPgOpPlacementNew
but anyway I do not know how to do it at the level of the 3 pages, and is it very important?

What do you all think?


(*) Example of page top for New:
New

Dynamically allocates memory and constructs data of a specified type.

New is a keyword as well as an operator (a function).
Operator New just allocates raw memory, nothing else.
New (keyword) calls operator New to allocate the memory, then makes base construction + calls user constructor if there is.
So final result is memory allocated and data constructed.

Operator Syntax:
.....

Keyword Parameters:
.....

Keyword Return Value:
.....

Description:
.....
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

fxm wrote:Confusion in the documentation and perhaps in the minds
...
What do you all think?
Yes, I agree, confusing. And easy to see how it could be confusing with the information you have presented in your post.

A little history:
An earlier version of the wiki only had the "Syntax" section and there was no "Usage" section. The old previous "Syntax" section is now the "Usage" section. We went from having a very simple description of the keyword to both a specific description and a simple description. I think most of this work was done by Laanan Fisher (stylin) to make the description of each FreeBASIC language element more specific & explicit. The result is that, when you read the wiki, you may notice it is written in two different "voices". One voice has a general tone. And the other voice more specific, exact.

I do think the language terms should be cleaned up so it is less confusing.

I can rename pages, sort of. The wiki itself cannot rename a page, only clone a page to a new name, and delete the old. The revision history will be lost when renaming a page.

But, before we go too far with this, consider:

- yes, new/delete is both a keyword and an operator. And when overloaded, the implementation looks very much like a function.

Compare with:
- Multiply operator is invoked with symbol '*'
- Dereference operator is invoked with symbol '*'
- New operator is invoked with keyword 'new'
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

Operator New and construction process are invoked sequentially with the only keyword 'new'.
For an UDT, only the operator New can be overloaded by user (to define its own memory allocation process).
The construction process cannot be modified.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

fxm wrote:The construction process cannot be modified.
Except with constructor()?

I get the feeling like we both understand this topic very well, but don't know how to explain it to each other. :)
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

coderJeff wrote:
fxm wrote:The construction process cannot be modified.
Except with constructor()?
I was principally thinking about the base construction of objects that can not be shortened (like the initialization of the vptr to point to the right vtable/RTTI block).
The user can only add his own construction in second position. There is a hole in the user racket (or he loses his grip) between the Return (of pointer) of his overload code of New and the first instruction of his constructor!

Symmetric behavior for Delete overloading.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

coderJeff wrote:I get the feeling like we both understand this topic very well, but don't know how to explain it to each other. :)

Code: Select all

        NEW KEYWORD
             V
             |
     call if there is >-----------------.
           else                         :
             v                          v
             :               (overload operator New)
             :                          :
  BASIC MEMORY ALLOCATION           USER BODY
             :                          :
             :<-------------------------'
             |
DATA FIELDS INITIALIZATION
OBJECT FIELDS CONSTRUCTION
   (VPTR INITIALIZATION)
             |
     call if there is >------------.
           else                    :
             v                     v
             :            (user constructor)
             :                     :
             :                 USER BODY
             :                     :
             :<--------------------'
             |
             V

Code: Select all

      DELETE KEYWORD
             V
             |
  (VPTR REINITIALIZATION)
             |
     call if there is >-----------.
           else                   :
             v                    v
             :            (user destructor)
             :                    v
             :                    :
             :                USER BODY
             :                    :
             :<-------------------'
             |
 OBJECT FIELDS DESTRUCTION
             |
     call if there is >---------------------.
           else                             :
             v                              v
             :                 (overload operator Delete)
             :                              :
  BASIC MEMORY DEALLOCATION              USER BODY
             :                              :
             :<-----------------------------'
             |
             V
The problem is also that if we want to synthesize all this in the documentation, for example for New, is it better to do it in 2 pages (New keyword and operator New) or in a single page (New)?
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

A literal picture, that's awesome. Illustrates the mechanism well.
fxm wrote:The problem is also that if we want to synthesize all this in the documentation, for example for New, is it better to do it in 2 pages (New keyword and operator New) or in a single page (New)?
I tried to reason it out... like a human language, fb language can be self referential, so I examined something similar on which to draw from experience.

I started with: DIM allocates memory and constructs an object.
DIM variable as UDT [ = intitializer ]
variable = expression
With one keyword "DIM" and one symbol '=', there are many mechanisms that could be in use depending on what overloads are defined in "UDT".

Conclusions:
1) allocation and construction are inseparable. Even though we can alter behaviour of each, for example with STATIC, SHARED, CONSTRUCTOR, both allocation (for example, from the stack) and construction will happen
2) We have special terms & definitions to help us describe use of '=' in each context:
- '= initializer' is overloaded using CONSTRUCTOR
- '= expression' is overloaded using OPERATOR LET (assignment)
- '(a = b)' overloaded with OPERATOR = (equality), not shown
3) Meaning of '=' symbol on its own without a context is ambiguous.


Then I compared with usage of "NEW":

Context 1: Invocation
expression = NEW UDT( initializers, ... )
- allocation (this time from the system heap) and construction of an object.
- the 2 actions are inseparable, in fact, if the allocation fails, we get a segmentation violation due the null pointer access in construction.
- the actual mechanism used depends on what overloads are defined in "UDT".

Context 2: Allocation overload definition
operator UDT.NEW( size as integer ) as any ptr
operator = ...
end operator
Conclusion:
- "NEW" is a keyword (reserved token) in both contexts, so to call it a keyword is ambiguous
- Similar arguments for "DELETE"

That's about as far as I got.
In my mind, I am thinking of it as:
- Context 1: OPERATOR NEW (allocate & construct)
- Context 2: OPERATOR NEW (allocate overload)
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

coderJeff wrote: That's about as far as I got.
In my mind, I am thinking of it as:
- Context 1: OPERATOR NEW (allocate & construct)
- Context 2: OPERATOR NEW (allocate overload)
I agree, and that's why I looked for a shortcut (compared to above) to identify the true 'New' in each of the 2 contexts.
Most often in the literature, 'New' is called 'New operator' in context 1 and 'operator New' in context 2.
I was looking for more discriminating appellations while remaining quite short.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

I can't think of anything shorter. Anyway, here's the best I can come up with that still follows the pattern of other "OPERATOR <symbol>" usage in the documentation. I think having consistent terms, and 2 pages, will help the reader disambiguate.

General Term: New Operator (KeyPgNew)

Specific:
Context 1: New Operator Expression (KeyPgNew)
Context 2: Operator New Overload (KeyPgOpNew)
Context 3: Placement New Operator (KeyPgOpPlacementNew => KeyPgPlacementNew, maybe)

Short-cut:
Context 1: New Expression (KeyPgNew)
Context 2: New Overload (KeyPgOpNew)
Context 3: Placement New (KeyPgOpPlacementNew => KeyPgPlacementNew, maybe)

General Term: Delete Operator (KeyPgDelete)

Specific:
Context 1: Delete Operator Statement (KeyPgDelete)
Context 2: Operator Delete Overload (KeyPgOpDelete)

Short-cut:
Context 1: Delete Statement (KeyPgDelete)
Context 2: Delete Overload (KeyPgOpDelete)
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

I think the definition for the concerned pages in the documentation is progressing well.
(I am ready to update the documentation as soon as we will reach a consensus on the different open points)

By the way, I am wondering about the representativeness of my literal pictures compared to the really implemented code:
Does the basic allocation/deallocation part is directly in-lined in the main code of New/Delete (context 1) as drawn, or is there a true call to a basic "operator New/Delete"?

Other formulation:
Have we:
Context 2: OPERATOR NEW (allocate overload) / OPERATOR DELETE (deallocate overload)
or:
Context 2: OPERATOR NEW (allocate basic/overload) / OPERATOR DELETE (deallocate basic/overload)
?
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

Yes, the allocation/deallocation is in-lined with the main code.

I created this Pseudo code to understand how fbc builds the NEW & DELETE expressions. I think it is fairly accurate, though I just realized I didn't include Placement New.

Technically,
- the initialization and termination code gets emitted to the ctor/dtor procedures
- if the UDT has strings or arrays, and the user has not provided a ctor/dtor, fbc will create an implicit ctor/dtor
- initializers are inlined with the main code is when there is no user defined ctor, and the implicit ctor not needed (I only tried a few test cases though)
- (ctor/dtor = constructor/destructor)

Code: Select all

Pseudo code for NEW/DELETE operator code generation:

expr = NEW DATATYPE
expr = NEW DATATYPE(ANY)
expr = NEW DATATYPE(INITIALIZER)

DELETE expr

function NEW( DATATYPE, INITIALIZER ) as DATATYPE ptr

	dim tmp as DATATYPE ptr = any

	#if defined( DATATYPE.NEW ) then
		tmp = DATATYPE.NEW( sizeof( DATATYPE ) )
	#else
		tmp = allocate( sizeof( DATATYPE ) )
	#end if

	#if defined( DATATYPE.constructor ) then
		#if INITIALIZER = NOTHING then
			call DATATYPE.constructor( *tmp )
		#else
			call DATATYPE.constructor( *tmp, INITIALIZER )
		#end if
	#else
		#if INITIALIZER = NOTHING then
			call memclear( *tmp, sizeof( DATATYPE ) )
		#elseif INITIALIZER <> ANY then
			*tmp = INITIALIZER
		#end if
	#endif
	
	return tmp

end function

sub DELETE( p as DATATYPE ptr )
	if( p <> NULL ) then
		#if defined( DATATYPE.[VIRTUAL].destructor )
			call DATATYPE.[VIRTUAL].destructor( *p )
		#endif
		#if defined( DATATYPE.delete )
			call DATATYPE.delete p
		#else
			call deallocate p
		#endif
	end if
end sub
new[]/delete[] is similar except there is a loop to iterate the elements. I've been looking at those operators also to fix #699 bug.

Context #2, what do we have?
- I think, from the pseudo code, it looks like to me like Allocate Overload.
- Only UDT's can have an operator new/delete overload
- fbc does not allow for any kind of global new/delete overloads
Post Reply