Wiki improvements

Forum for discussion about the documentation project.
Post Reply
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

Thank you very much.

The pseudo code also confirms what I guessed by doing failed memory allocation tests (real or simulated) with or without the operator New overload coded:
- There is no test on the nullity of the returned pointer ('tmp') and it seems that the code always tries to build even from zero memory address and obviously crashes in that case.
- Is there a reason for this choice?
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

Regarding the update of the documentation (for 'New' and 'Delete' that I prefer to do first), I think the work is similar to what I did in 2016 to cut the old single page 'Cast' in 2 separate pages 'Cast' and 'Operator Cast' (after saving the old text edit and the page links, by using page cloning, then cleaning/rewording on each new page).

I think to treat the page 'Placement New' after, in a second step.

About the 4 pages for 'New' and 'Delete', I propose that:
- each page has as title the long version of the name that you specified (viewtopic.php?p=251133#p251133),
- as usually, a first short sentence introduces roughly the function, but also defines the short-cut for the name (only this short-cut will be used after in the text to be cleaned/reworded),
- one thing bugs me a little, because for example by cloning the page 'KeyPgOpNew' to 'KeyPgNew', the revision history will remain attached to the page 'New Overload' (the page a priori the shortest) and not to the page 'New Expression',
- finally, previous links to be readjusted according to the new cutting.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

fxm wrote:... the code always tries to build even from zero memory address and obviously crashes in that case.
- Is there a reason for this choice?
I don't think it was a choice; it just happened. Older fbc called stdc++ and new/delete operators could throw a bas_alloc error. When stdc++ dependency was removed, nothing was added to handle the the error condition. We should probably check for the null pointer. And throw our own error when compiling with '-e'
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

fxm wrote:- one thing bugs me a little, because for example by cloning the page 'KeyPgOpNew' to 'KeyPgNew', the revision history will remain attached to the page 'New Overload' (the page a priori the shortest) and not to the page 'New Expression',
I was thinking about this after, but for a different reason:
- new/delete is an operator. It is implemented that way in the compiler. And that is what we are calling it.
- "KeyPgOp" wiki page name prefix would be correct for all contexts
- KeyPgOpNew, KeyPgOpPlacementNew, KeyPgOpNewOverload
- Side benefit is this naming keeps most revision history. And KeyPgNew could still be added later if desired as disambiguation page.
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

If I well understood, you propose this page mapping.
Consistent with the name of the links, I propose to change the order of the terms for the full names of the page titles (being also the full names of the operators).

Code: Select all

| Page link             | Title/Operator full name  | Short-cut name   | Comment        |
|=======================|===========================|==================|================|
| KeyPgOpNew            | Operator New Expression   | New Expression   | to be modified |
| KeyPgOpNewOverload    | Operator New Overload     | New Overload     | to be created  |
| (KeyPgNew)            | (Operator New)            | (New)            | reserved       |
|-----------------------|---------------------------|------------------|----------------|
| KeyPgOpDelete         | Operator Delete Statement | Delete Statement | to be modified |
| KeyPgOpDeleteOverload | Operator Delete Overload  | Delete Overload  | to be created  |
| (KeyPgDelete)         | (Operator Delete)         | (Delete)         | reserved       |
|-----------------------|---------------------------|------------------|----------------|
| KeyPgOpPlacementNew   | Operator Placement New    | Placement New    | to be modified |
|=======================|===========================|==================|================|
This seems good for all reasons mentioned.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

yea, that looks good, fxm. Many consistent patterns. Go for it.
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

fxm wrote:(I am ready to update the documentation as soon as we will reach a consensus on the different open points)
coderJeff wrote:yea, that looks good, fxm. Many consistent patterns. Go for it.
First update pass done:
- PrintToc → fxm [Page updated after 'Operator New Overload' and 'Operator Delete Overload' page creation]
- CatPgOpIndex → fxm [Page updated after 'Operator New Overload' and 'Operator Delete Overload' page creation]
- CatPgOpMemory → fxm [Page updated after 'Operator New Overload' and 'Operator Delete Overload' page creation]
- CatPgFullIndex → fxm ['New' -> 'New Expression', 'Delete' -> 'Delete Statement']
- KeyPgOperator → fxm [Page updated after 'Operator New Overload' and 'Operator Delete Overload' page creation]
- KeyPgOpPlacementNew → fxm ['New' -> 'New Expression', 'Delete' -> 'Delete Statement']
- KeyPgOpDeleteOverload → fxm [New page created]
- KeyPgOpDelete → fxm [Page updated while 'Operator Delete Overload' page creation]
- KeyPgOpNewOverload → fxm [New page created]
- KeyPgOpNew → fxm [Page updated while 'Operator New Overload' page creation]
- KeyPgDestructor → fxm ['Delete' -> 'Delete Statement']
- GlossaryIndex → fxm ['Delete' -> 'Delete Statement']
- ProPgCtorsDtors → fxm ['New' -> 'New Expression', 'Delete' -> 'Delete Statement']
- KeyPgConstructor → fxm ['New' -> 'New Expression']
- KeyPgAny → fxm ['New' -> 'New Expression']
- KeyPgAllocate → fxm ['New' -> 'New Expression']

Second update pass:
I must now check if there is no miss of information or links.
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

fxm wrote:Second update pass:
I must now check if there is no miss of information or links.
Done.
I think I have not forgotten anything!
You are free to improve my English.
(https://www.freebasic.net/wiki/wikka.ph ... entChanges)
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

Examples of New/Delete operator overloading for documentation

In the literature, one can find examples of these operators overloading, but most are obvious and without any interest (almost like 'Return Allocate(size)' for New and 'Deallocate buf' for Delete), irrelevant or even bugged. :-(

In addition to my example 1 "Aligned memory allocator" at Operator New Overload page, previously at Operator page (it overloads New and Delete), I propose here my second example (overloading New[] and Delete[] now)

"Dynamic allocation manager for UDT, by using the member operators "New[] Overload" and "Delete[] Overload":
- Monitoring of memory allocations/deallocations: addresses, sizes and total memory used.
- Detection of abnormal deallocation requests.
- Detection of a failed allocation (Allocate() returning null pointer).
- Detection of total allocated memory size exceeding a threshold.
- The last two detection cases induces an automatic memory freeing before forcing the program to end.

The principle is to manage a dynamic list of successful allocations, but not yet freed, containing the allocated addresses with their requested sizes:

Code: Select all

Type UDTmanager
  '' user UDT fields:
    Dim As Byte b(1 To 1024*1024)
  '' manager fields:
    Public:
      Declare Operator New[] (ByVal size As UInteger) As Any Ptr
      Declare Operator Delete[] (ByVal buf As Any Ptr)
      Static As UInteger maxmemory
    Private:
      Static As Any Ptr address()
      Static As UInteger bytes()
      Static upbound As UInteger
      Declare Static Function printLine (ByRef text As String, ByVal index As UInteger, ByVal sign As Integer) As UInteger
      Declare Static Sub endProgram ()
End Type

Dim As UInteger UDTmanager.maxmemory = 3 * 1024 * 1024 * 1024
ReDim UDTmanager.address(0)
ReDim UDTmanager.bytes(0)
Dim UDTmanager.upbound As UInteger = 0

Function UDTmanager.printLine (ByRef text As String, ByVal index As UInteger, ByVal sign As Integer) As UInteger
  Dim As UInteger total = 0
  For I As UInteger = 1 To UDTmanager.upbound
    If I <> index Orelse Sgn(sign) > 0 Then
      total += UDTmanager.bytes(I)
    End If
  Next I
  Print text, "&h" & Hex(UDTmanager.address(index), SizeOf(Any Ptr) * 2),
  If sign <> 0 Then
    Print Using " +####.## MB"; Sgn(sign) * Cast(Integer, UDTmanager.bytes(index) / 1024) / 1024;
  Else
    Print Using "( ####.## MB)"; UDTmanager.bytes(index) / 1024 / 1024;
  End If
  Print,
  Print Using "###.## GB"; total / 1024 / 1024 / 1024
  Return total
End Function

Sub UDTmanager.endProgram ()
  Do While UDTmanager.upbound > 0
    Deallocate UDTmanager.address(UDTmanager.upbound)
    UDTmanager.printLine("memory deallocation forced", UDTmanager.upbound, -1)
    UDTmanager.upbound -= 1
    ReDim Preserve UDTmanager.address(UDTmanager.upbound)
    ReDim Preserve UDTmanager.bytes(UDTmanager.upbound)
  Loop
  Print "end program forced"
  Print
  Sleep
  End
End Sub

Operator UDTmanager.New[] (ByVal size As UInteger) As Any Ptr
  Dim As Any Ptr p = Allocate(size)
  If p > 0 Then
    UDTmanager.upbound += 1
    ReDim Preserve UDTmanager.address(UDTmanager.upbound)
    ReDim Preserve UDTmanager.bytes(UDTmanager.upbound)
    UDTmanager.address(UDTmanager.upbound) = p
    UDTmanager.bytes(UDTmanager.upbound) = size
    If UDTmanager.printLine("memory allocation", UDTmanager.upbound, +1) > UDTmanager.maxmemory Then
      UDTmanager.address(0) = p
      UDTmanager.bytes(0) = size
      Print
      UDTmanager.printLine("memory allocation exceeded", 0, 0)
      UDTmanager.endProgram()
    End If
    Return p
  Else
    UDTmanager.address(0) = p
    UDTmanager.bytes(0) = size
    Print
    UDTmanager.printLine("memory allocation failed", 0, 0)
    UDTmanager.endProgram()
  End If
End Operator

Operator UDTmanager.Delete[] (ByVal buf As Any Ptr)
  Dim As UInteger found = 0
  For I As UInteger = 1 To UDTmanager.upbound
    If UDTmanager.address(I) = buf Then
      Deallocate buf
      UDTmanager.printLine("memory deallocation", I, -1)
      For J As UInteger = I + 1 To UDTmanager.upbound
        UDTmanager.address(J - 1) = UDTmanager.address(J)
        UDTmanager.bytes(J - 1) = UDTmanager.bytes(J)
      Next J
      UDTmanager.upbound -= 1
      ReDim Preserve UDTmanager.address(UDTmanager.upbound)
      ReDim Preserve UDTmanager.bytes(UDTmanager.upbound)
      found = 1
      Exit For
    End If
  Next I
  If found = 0 Then
    UDTmanager.address(0) = buf
    UDTmanager.bytes(0) = 0
    UDTmanager.printLine("deallocation not matching", 0, 0)
  End If
End Operator


Print "Message",, "Address" & Space(SizeOf(Any Ptr)), "Size", "Total"
Print
Randomize
Dim As UDTmanager Ptr pu1 = New UDTmanager[Rnd() * 256 + 1]
Dim As UDTmanager Ptr pu2 = New UDTmanager[Rnd() * 256 + 1]
Dim As UDTmanager Ptr pu3 = Cast(UDTmanager Ptr, 1)
Delete[] pu2
Delete[] pu3
Delete[] pu2
Delete[] pu1
Do
  Dim As UDTmanager Ptr pu = New UDTmanager[Rnd() * 512 + 1]
Loop
Output for fbc 32-bit (maximum dynamic data < 2 GB).
Here, program is stopped because of memory allocation failed:

Code: Select all

Message                     Address       Size          Total

memory allocation           &h020E0020       +99.00 MB    0.10 GB
memory allocation           &h083F3020        +3.00 MB    0.10 GB
memory deallocation         &h083F3020        -3.00 MB    0.10 GB
deallocation not matching   &h00000001    (    0.00 MB)   0.10 GB
deallocation not matching   &h083F3020    (    0.00 MB)   0.10 GB
memory deallocation         &h020E0020       -99.00 MB    0.00 GB
memory allocation           &h020ED020      +103.00 MB    0.10 GB
memory allocation           &h087F2020      +106.00 MB    0.20 GB
memory allocation           &h0F20D020      +230.00 MB    0.43 GB
memory allocation           &h1D812020      +137.00 MB    0.56 GB
memory allocation           &h2612C020      +377.00 MB    0.93 GB
memory allocation           &h3DA30020      +275.00 MB    1.20 GB
memory allocation           &h4ED40020      +220.00 MB    1.41 GB
memory allocation           &h5C958020      +229.00 MB    1.64 GB

memory allocation failed    &h00000000    (  142.00 MB)   1.64 GB
memory deallocation forced  &h5C958020      -229.00 MB    1.41 GB
memory deallocation forced  &h4ED40020      -220.00 MB    1.20 GB
memory deallocation forced  &h3DA30020      -275.00 MB    0.93 GB
memory deallocation forced  &h2612C020      -377.00 MB    0.56 GB
memory deallocation forced  &h1D812020      -137.00 MB    0.43 GB
memory deallocation forced  &h0F20D020      -230.00 MB    0.20 GB
memory deallocation forced  &h087F2020      -106.00 MB    0.10 GB
memory deallocation forced  &h020ED020      -103.00 MB    0.00 GB
end program forced
Output for fbc 64-bit (maximum dynamic data < virtual memory).
Here, program is stopped because of total allocated memory size > 3 GB:

Code: Select all

Message                     Address                     Size          Total

memory allocation           &h0000000001EA5040            +105.00 MB    0.10 GB
memory allocation           &h00000000087BC040             +93.00 MB    0.19 GB
memory deallocation         &h00000000087BC040             -93.00 MB    0.10 GB
deallocation not matching   &h0000000000000001          (    0.00 MB)   0.10 GB
deallocation not matching   &h00000000087BC040          (    0.00 MB)   0.10 GB
memory deallocation         &h0000000001EA5040            -105.00 MB    0.00 GB
memory allocation           &h0000000001EA1040            +155.00 MB    0.15 GB
memory allocation           &h000000000B9BF040            +165.00 MB    0.31 GB
memory allocation           &h0000000015ED8040            +382.00 MB    0.69 GB
memory allocation           &h000000002DCE7040            +458.00 MB    1.13 GB
memory allocation           &h000000004A6FB040            +255.00 MB    1.38 GB
memory allocation           &h000000005A607040             +96.00 MB    1.48 GB
memory allocation           &h000000006061B040            +426.00 MB    1.89 GB
memory allocation           &h000000007FFF9040            +221.00 MB    2.11 GB
memory allocation           &h000000008DD03040            +119.00 MB    2.22 GB
memory allocation           &h0000000095413040            +147.00 MB    2.37 GB
memory allocation           &h000000009E727040            +217.00 MB    2.58 GB
memory allocation           &h00000000AC03C040            +334.00 MB    2.91 GB
memory allocation           &h00000000C0E4B040            +280.00 MB    3.18 GB

memory allocation exceeded  &h00000000C0E4B040          (  280.00 MB)   3.18 GB
memory deallocation forced  &h00000000C0E4B040            -280.00 MB    2.91 GB
memory deallocation forced  &h00000000AC03C040            -334.00 MB    2.58 GB
memory deallocation forced  &h000000009E727040            -217.00 MB    2.37 GB
memory deallocation forced  &h0000000095413040            -147.00 MB    2.22 GB
memory deallocation forced  &h000000008DD03040            -119.00 MB    2.11 GB
memory deallocation forced  &h000000007FFF9040            -221.00 MB    1.89 GB
memory deallocation forced  &h000000006061B040            -426.00 MB    1.48 GB
memory deallocation forced  &h000000005A607040             -96.00 MB    1.38 GB
memory deallocation forced  &h000000004A6FB040            -255.00 MB    1.13 GB
memory deallocation forced  &h000000002DCE7040            -458.00 MB    0.69 GB
memory deallocation forced  &h0000000015ED8040            -382.00 MB    0.31 GB
memory deallocation forced  &h000000000B9BF040            -165.00 MB    0.15 GB
memory deallocation forced  &h0000000001EA1040            -155.00 MB    0.00 GB
end program forced
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

fxm wrote:Examples of New/Delete operator overloading for documentation

In the literature, one can find examples of these operators overloading, but most are obvious and without any interest (almost like 'Return Allocate(size)' for New and 'Deallocate buf' for Delete)
I think that this type of ultra-simple examples that do not show the interest in overloading such operators (showing only how to use the parameters of the operators), although (and even because) we see a lot of them in the literature, are useless in our documentation, but that's my personal opinion.

What do you all think?
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

fxm wrote:Done.
I think I have not forgotten anything!
I can not keep up with you. :)
I have made some wiki snapshots every once and a while, but I keep forgetting to push them to fbc repo. I will do another snapshot soon and push it out this time.
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

coderJeff wrote:I can not keep up with you. :)
Now that I'm retired, my reaction time is even shorter !
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Wiki improvements

Post by coderJeff »

fxm wrote:
fxm wrote:What do you all think?
Yes, for something like overloading new[]/delete[], we can probably assume that the reader can make sense of the syntax and usage, and they will benefit from a more complex usage. In comparison, for some simple examples, I think we are only trying to help the reader to see how it would be used. Like after the definition of a word, showing how it is used in a sentence.
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

I added a few days ago a very simple first example to my 2 previous "more interesting" examples, mainly to simply highlight the user interfacing with the parameters of the 2 overload operators:
KeyPgOpNewOverload → fxm [Added much simpler example before others (only for syntax usage)]
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Wiki improvements

Post by fxm »

In order to secure the usage of overload New([])/Delete([]), the compiler could return a warning if the operators are not overload by valid couples as New/Delete or/and New[]/Delete[] (not New([]) or Delete([]) alone)?
Post Reply