Freebasic 1.20.0 Development

[Lost Zergling]

I am interested and surprised by this behavior where REALLOCATE fails, but another separate ALLOCATE (with the same size) succeeds ?

This was an old test, i'll try to repeat it. Did not remember exactly fb version.
Context : I needed a pre allocated zstring available for being used very fast (no need to reallocate)
The size shall be known, logically fixed, but with possible resize to bigger size (never smaller)
The zstring is in a class.
The zstring resize request is inside a property of this class.
As the property was used many and many times, random crashes occured.
Replacing reallocate by allocate + swap did solve the problem. (Smell like a sticky hack)
=> I supposed the new allocated could find available memory outside the class wheras the reallocate was trying finding space into the same area inside class definition ??
( Or reallocate did break class desc integrity)

Usage : you handle a list of key-values, but.., at one moment, because an error or anything else, the key is becoming huge just for a few keys, and this 'exception' shall be handled as a standard key, for robustness.
( ou sinon, c'est la clef qui pète la classe, .. ;- )

[fxm]

I supposed the new allocated could find available memory outside the class wheras the reallocate was trying finding space into the same area inside class definition ??
(Or reallocate did break class desc integrity)

REALLOCATE first attempts to resize the memory already allocated without moving its starting address in order to avoid recopying the data to be kept (returned pointer value unchanged).
If this is not possible, a new separate memory area is allocated with recopy of the data to be kept before deallocating the old memory area (returned pointer value modified).

So I assume you are using in the property body something like:
This.myPtr = Reallocate(This.myPtr, ...)
and not only:
Reallocate(This.myPtr, ...)

[Lost Zergling]

Well I don't remember exactly but probably was this.myptr=reallocate..

[shadow008]

Well I don't remember exactly but probably was this.myptr=reallocate..

Use of reallocate in situations where memory integrity is guaranteed requires a bit more than just that. Specifically you need to assign the reallocate result to a temporary pointer and check the result. If it failed, you do not assign your this.myptr (or whatever) to the temp value or you will have a dangling pointer which is a memory leak.
This is how to properly use reallocate.

Code: Select all

dim tmp as any ptr = reallocate(...)
if tmp then
    this.myptr = tmp
endif

[Lost Zergling]

That's the issue...but not a leak issue.
No fail due to intrinsic scopes or memory fragmentation issues shall be admitted.
Admitted fail shall only be running out of memory. Would require new test...allocate+swap work.
Replacing allocate+swap by reallocate is not a problem, very small benefit expected.

[fxm]

Example of exhaustive testing after REALLOCATE:

Code: Select all

Sub resize(Byval p As Any Ptr, Byval size As Longint)
    Dim As Any Ptr tp = Reallocate(p, size)
    If tp > 0 Then
        Print "REALLOCATE succeeded for " & size & " bytes"
        If tp <> p Then
            Print "memory moved"
'            Deallocate(myPtr)  '' No because previous memory already deallocated under the hood
            p = tp
        Else
            Print "memory not moved"
        End If
    Else
        Print "REALLOCATE failed for " & size & " bytes"
        Deallocate(p)  '' Yes otherwise memory leak
        End
    End If
End Sub

Dim As Any Ptr myPtr = Allocate(10)

resize(myPtr, 9)
Print
resize(myPtr, 1000000)
Print
resize(myPtr, 1000000000000)

Deallocate(myPtr)  '' Yes otherwise memory leak
Sleep

REALLOCATE succeeded for 9 bytes
memory not moved

REALLOCATE succeeded for 1000000 bytes
memory moved

REALLOCATE failed for 1000000000000 bytes

[fxm]

Context : I needed a pre allocated zstring available for being used very fast (no need to reallocate)
The size shall be known, logically fixed, but with possible resize to bigger size (never smaller)
The zstring is in a class.
The zstring resize request is inside a property of this class.
As the property was used many and many times, random crashes occured.
Replacing reallocate by allocate + swap did solve the problem. (Smell like a sticky hack)
=> I supposed .....
( Or reallocate did break class desc integrity)

I have just reworded (in more detail as for REDIM) in the documentation the special case where REALLOCATE use inside a member procedure can induce malfunction if the pointer argument (of REALLOCATE) points to the instance itself of the object (on which the member procedure is called):

Reallocating a pointer inside an object function, when that pointer contains the parent object of the function, is undefined, and will likely result in horrible crashes.
Reallocate cannot be used inside a member procedure if pointer points to the instance itself of the object, because that could cause horrible crashes:
- If pointer is modified by Reallocate (object data moved into memory), the passed This reference becomes inconsistent, in the same way as a dangling pointer.
- In that case, all subsequent accesses to any non-static member field from this member procedure will be erroneous, except if the passed This reference would be readjusted (by means of @This = new pointer value) immediately after executing Reallocate in the body of this member procedure.

Very simple example for REALLOCATE (with @THIS readjustment if necessary):

Code: Select all

Type UDT
    number As Uinteger
    Declare Sub resize(Byval size As Integer)
End Type

Dim Shared pU As UDT Ptr = 0

Sub UDT.resize(Byval size As Integer)
    pU = Reallocate(pU, size * Len(UDT))
    If pU <> @This Then
        Print "memory moved for " & size & " instances"
        @This = pU  '' mandatory when memory is moved, for future use of non-static member field in procedure body
    Else
        Print "memory not moved for " & size & " instances"
    End If
    This.number = size
End Sub


pU = Allocate(5 * Len(UDT))
pU->number = 5

pU->resize(4)
Print pU->number
Print
pU->resize(1000000)
Print pU->number

Deallocate(pU)
Sleep

memory not moved for 4 instances
4

memory moved for 1000000 instances
1000000

Similar example for REDIM [PRESERVE] (with @THIS readjustment if necessary):

Code: Select all

Type UDT
    number As Uinteger
    Declare Sub resize(Byval size As Integer)
End Type

Dim Shared U() As UDT

Sub UDT.resize(Byval size As Integer)
    ReDim  U(size - 1)
    If @U(0) <> @This Then
        Print "memory moved for " & size & " elements"
        @This = @U(0)  '' mandatory when memory is moved, for future use of member field in procedure body
    Else
        Print "memory not moved for " & size & " elements"
    End If
    This.number = size
End Sub


ReDim U(4)
U(0).number = 5

U(0).resize(4)
Print U(0).number
Print
U(0).resize(1000000)
Print U(0).number

Erase U
Sleep

memory not moved for 4 elements
4

memory moved for 1000000 elements
1000000

[fxm]

Replacing allocate+swap by reallocate is not a problem, very small benefit expected.

But, concept consequence in a member procedural body:
- something like (1 line):
This.myPtr = Reallocate(This.myPtr, ...)
- instead of (4 lines):
Dim As Any Ptr p = Allocate(...)
Swap This.myPtr, p
FB_memcopy(Byval This.myPtr, Byval p, ...)
Deallocate(p)

[Lost Zergling]

Noted. Thank you.
Nb : in my use case, I do not need to keep previous value. 'Reallocation' is just for size.

[fxm]

Nb : in my use case, I do not need to keep previous value. 'Reallocation' is just for size.

In your specific case (no data to keep), the 3 lines:
Dim As Any Ptr p = Allocate(...)
Swap This.myPtr, p
Deallocate(p)
are at execution faster or equal to the single line:
This.myPtr = Reallocate(This.myPtr, ...)

[Lost Zergling]

@fxm. As you have understand it, this section using swap is critical, handling scalability while trying to keep safe from memory fragmentation.
Ps added :
In retrospect, my post may seem quite pretentious. At the time and in the context in which I wrote it, I mainly had in mind the fact that my code was particularly difficult to read and a concern about the sustainability of the 'swap' kwd, there was not an intention to against you from me.

[Lost Zergling]

I would like to apologize to fxm (and Jeff) for a particularly clumsy and unbearable formulation, which was however not guided by malice but by anxiety.

[fxm]

Nb : in my use case, I do not need to keep previous value. 'Reallocation' is just for size.

In your specific case (no data to keep), the 3 lines:
Dim As Any Ptr p = Allocate(...)
Swap This.myPtr, p
Deallocate(p)
are at execution faster or equal to the single line:
This.myPtr = Reallocate(This.myPtr, ...)

@Jeff,

Concerning the above exchange, I am surprised that 'ReAllocate' does not have a qualifier like 'ReDim' to keep or not the original data.
This requires for large data sizes, in order to optimize the execution time, to decompose this functionality into at least 3 instructions as above.

To maintain backward compatibility, we cannot add to 'ReAllocate' the 'Preserve' qualifier as for 'ReDim', but why not adding a last optional parameter 'preserved' such that the operating is equivalent to this:

Code: Select all

Function _Reallocate(ByVal p As Any Ptr, ByVal count As UInteger, Byval preserved As Boolean = True) As Any Ptr
    If preserved = True Then Return Reallocate(p, count)
    Dim As Any Ptr pnew = Allocate(count)
    If pnew > 0 Then
        Deallocate(p)
    End If
    Return pnew
End Function

[Jattenalle]

Nb : in my use case, I do not need to keep previous value. 'Reallocation' is just for size.

In your specific case (no data to keep), the 3 lines:
Dim As Any Ptr p = Allocate(...)
Swap This.myPtr, p
Deallocate(p)
are at execution faster or equal to the single line:
This.myPtr = Reallocate(This.myPtr, ...)

@Jeff,

Concerning the above exchange, I am surprised that 'ReAllocate' does not have a qualifier like 'ReDim' to keep or not the original data.
This requires for large data sizes, in order to optimize the execution time, to decompose this functionality into at least 3 instructions as above.

To maintain backward compatibility, we cannot add to 'ReAllocate' the 'Preserve' qualifier as for 'ReDim', but why not adding a last optional parameter 'preserved' such that the operating is equivalent to this:

Code: Select all

Function _Reallocate(ByVal p As Any Ptr, ByVal count As UInteger, Byval preserved As Boolean = True) As Any Ptr
    If preserved = True Then Return Reallocate(p, count)
    Dim As Any Ptr pnew = Allocate(count)
    If pnew > 0 Then
        Deallocate(p)
    End If
    Return pnew
End Function

Adding convolution and conditionals to core methods is not a good idea. Especially if one is concerned about performance, as you said.
There is absolutely no reason to add an optional parameter, and slowing everything down with multiple branching conditionals, just to save a single line of code:

Code: Select all

deallocate(p)
p = allocate(size)

vs

Code: Select all

p = reallocate(p, size, false)

[fan2006]

my mistake.there has been changes with string since fb 1.20.