Question on behind the scenes storage management

General FreeBASIC programming questions.
wallyg
Posts: 89
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Question on behind the scenes storage management

Postby wallyg » Nov 16, 2012 23:02

I am trying to trace down a memory leak and had a question on String memory usage. Is there a definitive discussion somewhere. A simplified code sample of my question is

Dim s as Zstring Ptr

s = function returning Zstring pointer
...
s = strptr( string expression)

At this point does the system return the allocated space to the string that was pointed to by the first statement? If yes, can I prevent this.

Now we have a third statement

s = function returning ZString pointer

Now does the temp string created by the second statement and now pointed to by s, get returned when s is changed in the third statement? If not then is the space lost? and if it is lost how can I prevent that from happening.

Thanks
1000101
Posts: 2556
Joined: Jun 13, 2005 23:14
Location: SK, Canada

Re: Question on behind the scenes storage management

Postby 1000101 » Nov 16, 2012 23:55

This is partly a question of garbage collection of which it only applies to implicit strings.

The other part is variable assignments which is absolute. If you explicitly allocate string space you must explicitly deallocate it (like any memory pool). Implicit strings are strings which are in intermediate datatype. These strings are created by the usage of the String, ZString, WString datatypes will be released when they go out of scope. String pointers will be valid so long as the string is in scope but as stated above, explicit allocation requires explicit deallocation.

Examples of what I mean by implicit and explicit strings.

Code: Select all

Dim As String foo
Dim As String * 10 bar

/'
   The above strings are "implicit" strings.  What is created are actually freebasic
   descriptors (an internal user defined type).  A string descriptor contains an
   integer length and a pointer to the 0-terminated string data.  When the string
   goes out of scope the memory for the string data is deallocated and string
   descriptor is deallocated (removed from the stack).
'/


Code: Select all

Dim As ZString Ptr foo = New ZString[ 128 ]
Dim As WString Ptr bar = New WString[ 128 ]

/'
   The above strings are "explicit" strings.  The only thing that is created is the
   required memory for the size of the string desired - in this case 128 characters.
   Note that this is characters and not bytes as the amount of memory is dependant
   on the character width.  For a ZString this will be one byte per character but
   for wide strings (WString) it is system dependant.
'/


foo = New ZString[ 32 ]

/'
   At this point the old ZString data is now orphaned and lost.
'/


Delete foo
Delete bar

/'
   Since we explicitly allocated these strings we need to explicitly deallocate them,
   however the 128 character ZString initially allocate is lost because of the new
   allocation.  There is no "scope" in regards to explicit memory allocation and the
   memory will remain allocated until it is deallocated.  This means that is you
   assign the pointer a different value then the memory will be "lost" and remain
   allocated until the entire process (program) terminates.  This is OS level
   garbage collection and should not be relied on (bad programming practice).
'/
dodicat
Posts: 6029
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Question on behind the scenes storage management

Postby dodicat » Nov 17, 2012 1:46

These dont work:
Dim As ZString Ptr foo = New ZString[ 128 ]
Dim As WString Ptr bar = New WString[ 128 ]

compiler error fb24.
Here's a workaround, it throws no errors.
BUT
Using -gen gas -exx -- OK
Using -gen gas -- OK
Using -gen gcc -exx -- GARBAGE
Using -gen gcc -- GARBAGE
Strange!
Also using the 128 th. index throws garbage all round now and then!

Code: Select all

type _zstring as zstring pointer


Dim As _ZString Ptr foo = New _ZString[128]

for n as integer=0 to 127
     var s = cast(zstring pointer,@n)
    foo[n]=s
next n

for n as integer=0 to 127
    print *foo[n];
next n

delete[] foo
sleep
 
fxm
Posts: 9315
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Question on behind the scenes storage management

Postby fxm » Nov 17, 2012 6:28

Dim As String foo
creates only a descriptor (12 bytes) of var-len string.

Dim As String * 10 bar
creates a fixed-len string by allocating 10+1 bytes of memory (no descriptor).

See documentation STRING.
fxm
Posts: 9315
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Question on behind the scenes storage management

Postby fxm » Nov 17, 2012 8:40

dodicat wrote:..... Here's a workaround, it throws no errors.
BUT
Using -gen gas -exx -- OK
Using -gen gas -- OK
Using -gen gcc -exx -- GARBAGE
Using -gen gcc -- GARBAGE
Strange!
.....

Illusionist dodicat,

In the first For...Next loop, the program fills entirely the memory allocated for the 128 bytes zstring ptr with the same value which corresponds to the address of the local variable "n".
In the second For...Next loop:
- if the address of the local variable "n" is unchanged, then the instruction 'print *foo[n]' is equivalent to 'print *cast(zstring pointer,@n)' or 'print chr(n)' with the local variable "n",
- else the program points on the address of the previous local variable which remains to the value 127+1 (if not overwritten) corresponding to the ASCII character 128.

I modified the program to display the address of the local variable "n" in each For...Next loop:

Code: Select all

type _zstring as zstring pointer


Dim As _ZString Ptr foo = New _ZString[128]

for n as integer=0 to 127
    if n = 0 then print @n
    var s = cast(zstring pointer,@n)
    foo[n]=s
next n

for n as integer=0 to 127
    if n = 0 then print @n
    print *foo[n];
next n

delete[] foo
sleep

- With -gen gas, the address of the local variable "n" is unchanged between the two For...Next loops:

Code: Select all

1244992
1244992
☺☻♥♦♣
♫☼►◄↕‼¶§▬↨↑↓→←∟↔▲▼ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]
^_`abcdefghijklmnopqrstuvwxyz{|}~⌂

- With -gen gcc, the address of the local variable "n" is changed between the two For...Next loops:

Code: Select all

1244972
1244960
ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ
ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ
On my PC, "Ç" corresponds to ASCII(128).

Anyway, when it seems to work, the program is actually a great illusion at first sight!


With two separated iterator variables, the program never works:

Code: Select all

type _zstring as zstring pointer


Dim As _ZString Ptr foo = New _ZString[128]
Dim As Integer i, j

for i = 0 to 127
    var s = cast(zstring pointer,@i)
    foo[i]=s
next i

for j = 0 to 127
    print *foo[j];
next j

delete[] foo
sleep

Code: Select all

ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ
ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ


dodicat wrote:Also using the 128 th. index throws garbage all round now and then!

Allocating memory of 128 bytes corresponds to pointer index from 0 to 127. The 128 th. pointer index overflows the allocated memory previously, and this byte may be used for other thing depending on the context.
1000101
Posts: 2556
Joined: Jun 13, 2005 23:14
Location: SK, Canada

Re: Question on behind the scenes storage management

Postby 1000101 » Nov 17, 2012 17:39

hrm, after looking at my reply there are a number of factual errors (not the least of which as fxm pointed out being the string descriptor size). I may or may not clarify in the future. I think despite some of the inaccuracies I get the point across.

Errors people/I have noticed:

dodicat - It's not compilable code on the gcc backend. I didn't really expect anyone to try tbh. Just to note - it's pseudo-code.

fxm - String descriptor

For reference, this is what the internal data type looks like:

Code: Select all

typedef struct _FBSTRING {
     char           *data;    /**< pointer to the real string data */
     int             len;     /**< String length. */
     int             size;    /**< Size of allocated memory block. */
} FBSTRING;


fxm - The first example would only create a descriptor for "foo." "bar" was given an explicit size (10) so it would have directly allocated 11 bytes (10+zero termination). Despite this it is still an implicit string and will follow scoping rules.

myself - Tons of grammatical errors. Not worth fixing.

Return to “General”

Who is online

Users browsing this forum: No registered users and 1 guest