Question on behind the scenes storage management

[wallyg]

I am trying to trace down a memory leak and had a question on String memory usage. Is there a definitive discussion somewhere. A simplified code sample of my question is

Dim s as Zstring Ptr

s = function returning Zstring pointer
...
s = strptr( string expression)

At this point does the system return the allocated space to the string that was pointed to by the first statement? If yes, can I prevent this.

Now we have a third statement

s = function returning ZString pointer

Now does the temp string created by the second statement and now pointed to by s, get returned when s is changed in the third statement? If not then is the space lost? and if it is lost how can I prevent that from happening.

Thanks

[1000101]

This is partly a question of garbage collection of which it only applies to implicit strings.

The other part is variable assignments which is absolute. If you explicitly allocate string space you must explicitly deallocate it (like any memory pool). Implicit strings are strings which are in intermediate datatype. These strings are created by the usage of the String, ZString, WString datatypes will be released when they go out of scope. String pointers will be valid so long as the string is in scope but as stated above, explicit allocation requires explicit deallocation.

Examples of what I mean by implicit and explicit strings.

Code: Select all

Dim As String foo
Dim As String * 10 bar

/'
   The above strings are "implicit" strings.  What is created are actually freebasic
   descriptors (an internal user defined type).  A string descriptor contains an
   integer length and a pointer to the 0-terminated string data.  When the string
   goes out of scope the memory for the string data is deallocated and string
   descriptor is deallocated (removed from the stack).
'/

Code: Select all

Dim As ZString Ptr foo = New ZString[ 128 ]
Dim As WString Ptr bar = New WString[ 128 ]

/'
   The above strings are "explicit" strings.  The only thing that is created is the
   required memory for the size of the string desired - in this case 128 characters.
   Note that this is characters and not bytes as the amount of memory is dependant
   on the character width.  For a ZString this will be one byte per character but
   for wide strings (WString) it is system dependant.
'/


foo = New ZString[ 32 ]

/'
   At this point the old ZString data is now orphaned and lost.
'/


Delete foo
Delete bar

/'
   Since we explicitly allocated these strings we need to explicitly deallocate them,
   however the 128 character ZString initially allocate is lost because of the new
   allocation.  There is no "scope" in regards to explicit memory allocation and the
   memory will remain allocated until it is deallocated.  This means that is you
   assign the pointer a different value then the memory will be "lost" and remain
   allocated until the entire process (program) terminates.  This is OS level
   garbage collection and should not be relied on (bad programming practice).
'/

[dodicat]

These dont work:
Dim As ZString Ptr foo = New ZString[ 128 ]
Dim As WString Ptr bar = New WString[ 128 ]

compiler error fb24.
Here's a workaround, it throws no errors.
BUT
Using -gen gas -exx -- OK
Using -gen gas -- OK
Using -gen gcc -exx -- GARBAGE
Using -gen gcc -- GARBAGE
Strange!
Also using the 128 th. index throws garbage all round now and then!

Code: Select all

type _zstring as zstring pointer


Dim As _ZString Ptr foo = New _ZString[128]

for n as integer=0 to 127
     var s = cast(zstring pointer,@n)
    foo[n]=s
next n

for n as integer=0 to 127
    print *foo[n];
next n

delete[] foo
sleep
  

[fxm]

Dim As String foo
creates only a descriptor (12 bytes) of var-len string.

Dim As String * 10 bar
creates a fixed-len string by allocating 10+1 bytes of memory (no descriptor).

See documentation STRING.

[fxm]

..... Here's a workaround, it throws no errors.
BUT
Using -gen gas -exx -- OK
Using -gen gas -- OK
Using -gen gcc -exx -- GARBAGE
Using -gen gcc -- GARBAGE
Strange!
.....

Illusionist dodicat,

In the first For...Next loop, the program fills entirely the memory allocated for the 128 bytes zstring ptr with the same value which corresponds to the address of the local variable "n".
In the second For...Next loop:
- if the address of the local variable "n" is unchanged, then the instruction 'print *foo[n]' is equivalent to 'print *cast(zstring pointer,@n)' or 'print chr(n)' with the local variable "n",
- else the program points on the address of the previous local variable which remains to the value 127+1 (if not overwritten) corresponding to the ASCII character 128.

I modified the program to display the address of the local variable "n" in each For...Next loop:

Code: Select all

type _zstring as zstring pointer


Dim As _ZString Ptr foo = New _ZString[128]

for n as integer=0 to 127
    if n = 0 then print @n
    var s = cast(zstring pointer,@n)
    foo[n]=s
next n

for n as integer=0 to 127
    if n = 0 then print @n
    print *foo[n];
next n

delete[] foo
sleep

- With -gen gas, the address of the local variable "n" is unchanged between the two For...Next loops:

Code: Select all

1244992
1244992
☺☻♥♦♣
♫☼►◄↕‼¶§▬↨↑↓→←∟↔▲▼ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]
^_`abcdefghijklmnopqrstuvwxyz{|}~⌂

- With -gen gcc, the address of the local variable "n" is changed between the two For...Next loops:

Code: Select all

1244972
1244960
ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ
ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ

On my PC, "Ç" corresponds to ASCII(128).

Anyway, when it seems to work, the program is actually a great illusion at first sight!


With two separated iterator variables, the program never works:

Code: Select all

type _zstring as zstring pointer


Dim As _ZString Ptr foo = New _ZString[128]
Dim As Integer i, j

for i = 0 to 127
    var s = cast(zstring pointer,@i)
    foo[i]=s
next i

for j = 0 to 127
    print *foo[j];
next j

delete[] foo
sleep

Code: Select all

ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ
ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ

Also using the 128 th. index throws garbage all round now and then!

Allocating memory of 128 bytes corresponds to pointer index from 0 to 127. The 128 th. pointer index overflows the allocated memory previously, and this byte may be used for other thing depending on the context.

[1000101]

hrm, after looking at my reply there are a number of factual errors (not the least of which as fxm pointed out being the string descriptor size). I may or may not clarify in the future. I think despite some of the inaccuracies I get the point across.

Errors people/I have noticed:

dodicat - It's not compilable code on the gcc backend. I didn't really expect anyone to try tbh. Just to note - it's pseudo-code.

fxm - String descriptor

For reference, this is what the internal data type looks like:

Code: Select all

typedef struct _FBSTRING {
     char           *data;    /**< pointer to the real string data */
     int             len;     /**< String length. */
     int             size;    /**< Size of allocated memory block. */
} FBSTRING;

fxm - The first example would only create a descriptor for "foo." "bar" was given an explicit size (10) so it would have directly allocated 11 bytes (10+zero termination). Despite this it is still an implicit string and will follow scoping rules.

myself - Tons of grammatical errors. Not worth fixing.