Losing memory because of String usage

General FreeBASIC programming questions.
wallyg
Posts: 88
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Losing memory because of String usage

Postby wallyg » May 31, 2018 17:54

As my program was running, I noticed the amount of memory used kept going up. After a long time tracing down all "Allocate" memory usage and fixing the holes, it still kept going up. I think I have tracked it down to the following situation.

Code: Select all

type xyz1 extends sysObject
    ...
    As String   name
    ...
    end type
   
    ...  about 57 other simular types
   
    Union World
       as xyz1 ptr      x1
       as xyz2 ptr      x2
       ...
     End Union
     
     or
     
     Dim as any ptr      Machine
     
     If ... Then
         Machine = New xyz1
     Else
         Machine = New xyz2
     End if
     ...
     Delete Machine
     
    '


When I destroy an object containing the union World, FreeBasic obviously does not know what type is stored in the variable pointing to World. Therefore it cannot possibly call the destructor for the correct xyz object and certainly does not know to garbage collect the string variable.

I have my own way to garbage collect everything in the object stored in World, but how do I cause FreeBasic to garbage collect the area holding the value of the string type.
fxm
Posts: 9123
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Losing memory because of String usage

Postby fxm » May 31, 2018 18:20

One solution is by using polymorphism:
- with a base typed pointer (Dim as sysObject ptr Machine),
- and an overriding destructor in each "xyzN" type, overriding its base type destructor (the one of "sysObject" type).

Principle:

Code: Select all

type sysobject extends object
  ...
  declare virtual destructor ()
end type
destructor sysobject ()
end destructor

type xyz1 extends sysObject
   ...
  As String   name
  ...
  declare destructor () override
end type
destructor xyz1 ()
end destructor

type xyz2 extends sysObject
  ...
  As String   name
  ...
  declare destructor () override
end type
destructor xyz2 ()
end destructor

  ...  about 57 other simular types

Dim as sysObject ptr Machine

If ... Then
  Machine = New xyz1
Else
  Machine = New xyz2
End if
  ...
Delete Machine
wallyg
Posts: 88
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Re: Losing memory because of String usage

Postby wallyg » May 31, 2018 20:51

Thank you for your response.

I have never looked into polymorphism or virtual functions. And overriding the destructor is something I need to think about. This was just a simple example. I have some structures that are many extend levels deep and trying to override and take care of the consequences of overriding all the intermediate levels seems overwhelming.

I was just looking for some simple way to tell Freebasic that I wanted the space allocated to a string variable to be garbage collected. A simple request verses your more complex solution.

It seems that just saying stringvar = "" doesn't accomplish this. I wa looking for something simple like this.
sancho3
Posts: 358
Joined: Sep 30, 2017 3:22

Re: Losing memory because of String usage

Postby sancho3 » Jun 01, 2018 1:23

Are you sure that using a union is the best method?
You can always store a flag to determine which type world is.

Code: Select all

Enum WorldType
   wtNone
   wtXYZ1
   wtXYZ2
End Enum

Dim As WorldType world_type   ' flag
coderJeff
Site Admin
Posts: 3018
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Losing memory because of String usage

Postby coderJeff » Jun 01, 2018 1:33

wallyg wrote:It seems that just saying stringvar = "" doesn't accomplish this. I wa looking for something simple like this.

I think your memory leak is not where you think. If you assign a pointer with NEW TYPE, then you should have a matching DELETE TYPE PTR to clean up. DELETE will automatically deallocate string members. Or if using CALLOCATE/DEALLOCATE, stringvar = "" will also deallocate.

This example shows a number of ways to create/destroy the type.

Code: Select all

type T
   n as string
   union
      p as T ptr
      a as any ptr
   end union
end type

scope
   dim x as T
   x.n = "scoped"

   x.p = new T
   x.p->n = "typed pointer, new/delete"
   
   '' use callocate will ensure we are initializing zero length string
   x.p->p = callocate( sizeof(T) )
   x.p->p->n = "typed pointer,  allocate/deallocate"

   '' can use new with an any ptr, need to CAST to make use of it
   x.p->p->a = new T
   cast( T ptr, x.p->p->a )->n = "any pointer, new/delete"
   
   '' use callocate will ensure we are initializing zero length string
   '' need to CAST any ptr to something we can use
   cast( T ptr, x.p->p->a )->a = callocate( sizeof(T) )
   cast( T ptr, cast( T ptr, x.p->p->a )->a )->n = "any pointer, allocate/deallocate"

   '' Dump List
   dim k as T ptr = @x, i as integer = 0
   while( k )
      print space(i) & "@ = &h" & hex(k,sizeof(any ptr)*2)
      print space(i) & "n = """ & k->n & """"
      print space(i) & "p = &h" & hex(k->p,sizeof(any ptr)*2)
      k = k->p
      i += 4
   wend

   '' fbc doesn't know what type x.p->p->a->a is, need to CAST
   '' and deallocate string and memory explicitly
   cast( T ptr, cast( T ptr, x.p->p->a )->a )->n = ""
   deallocate cast( T ptr, x.p->p->a )->a

   '' fbc doesn't know what type x.p->p->a is, need to CAST
   '' before using DELETE
   delete cast( T ptr, x.p->p->a )

   '' deallocate doesn't call destructors, need to
   '' deallocate string and memory explicitly
   x.p->p->n = ""
   deallocate x.p->p
   
   '' when type is known, DELETE will call destructors
   '' (for the string) and deallocate memory
   delete x.p
   
   '' no need to do anything with x
   '' x.n and x are deallocated when x goes out of scope   
end scope


How do you know what you have stored in your UNION? It could be any of the types you specify.
wallyg
Posts: 88
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Re: Losing memory because of String usage

Postby wallyg » Jun 01, 2018 6:02

Thank you all for trying to help. However given that this is a very large program almost 200,000 lines long and the best part is that it is working correctly. I do not want to make wholesale changes to my nearly 50 year programming style. I asked a simple question ( I am sorry I gave an overly simplistic example of what I think the code in error is ). I was not looking to re-code a major portion of my code. I simply asked if there was a way to tell FreeBasic that I wanted a string variable garbage collected (ie have no heap storage assigned).

I expected to get a 2 character response ie. NO. or a 1 line example on how to do this.

I appreciate the advice on perhaps a better way of doing the example (which is no way as complex and interconnected as the real program is). I have some really bad programming habits and ideas and if you looked at my code you would probably cringe (I love GoTos) but my programs work and get the job done, but are actually quite easy to debug due to the large number of detailed comments and a strict adherence to a ridgid set of rules. My first programming language was Assembler and then this really modern concept came along called Fortran II.

And I will consider this advice for future projects. But could I just get a way to return the storage assigned to a String variable.

Thank you
fxm
Posts: 9123
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Losing memory because of String usage

Postby fxm » Jun 01, 2018 9:13

wallyg wrote:I expected to get a 2 character response ie. NO. or a 1 line example on how to do this...
But could I just get a way to return the storage assigned to a String variable.

I nevertheless take the right to answer in more than one line!

Another principle (initialize in the Base a pointer to the string to clean before call Delete on a Base pointer):

Code: Select all

type sysobject
  ...
  dim as string ptr ps
end type

type xyz1 extends sysObject
  ...
  As String   name
  ...
end type

type xyz2 extends sysObject
  ...
  As String   name
  ...
end type

...  about 57 other simular types

Dim as sysObject ptr Machine

If ... Then
  Machine = New xyz1
  Machine->ps = @cptr(xyz1 ptr, Machine)->name
Else
  Machine = New xyz2
  Machine->ps = @cptr(xyz2 ptr, Machine)->name
End if

...

*Machine->ps = ""
Delete Machine
Last edited by fxm on Jun 01, 2018 12:25, edited 1 time in total.
dodicat
Posts: 5913
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Losing memory because of String usage

Postby dodicat » Jun 01, 2018 11:01

If I simply delete the any pointer, I get a slow but sure leak
But If I go through object, it seems to stop the slow leak.
Test with resource monitor.

Code: Select all

 Type sysobject extends object
   
    As String Name
   
End Type



Dim As Const String pad=String(1000,"z")

type xyz1 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz2 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz3 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz4 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz5 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz6 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz7 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz8 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz9 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz10 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz11 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz12 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz13 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz14 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz15 extends sysObject
 As String name
 as string a(10000)
end type
 type xyz16 extends sysObject
 As String name
 as string a(10000)
end type


sub destroy(byref p as any ptr)
    var t=cast(object ptr,p)
    delete t
    end sub



#define Intrange(f,l) Int(Rnd*((l+1)-(f))+(f)) 

Dim As Any Ptr p
dim as long i
Do
    i=intrange(1,16)
    Select Case as const i
    Case 1:p=New xyz1:Cast(xyz1 Ptr,p)->Name=pad
    Case 2:p=New xyz2:Cast(xyz2 Ptr,p)->Name=pad
    Case 3:p=New xyz3:Cast(xyz3 Ptr,p)->Name=pad
    Case 4:p=New xyz4:Cast(xyz4 Ptr,p)->Name=pad
    Case 5:p=New xyz5:Cast(xyz5 Ptr,p)->Name=pad
    Case 6:p=New xyz6:Cast(xyz6 Ptr,p)->Name=pad
    Case 7:p=New xyz7:Cast(xyz7 Ptr,p)->Name=pad
    Case 8:p=New xyz8:Cast(xyz8 Ptr,p)->Name=pad
    Case 9:p=New xyz9:Cast(xyz9 Ptr,p)->Name=pad
    Case 10:p=New xyz10:Cast(xyz10 Ptr,p)->Name=pad
    Case 11:p=New xyz11:Cast(xyz11 Ptr,p)->Name=pad
    Case 12:p=New xyz12:Cast(xyz12 Ptr,p)->Name=pad
    Case 13:p=New xyz13:Cast(xyz13 Ptr,p)->Name=pad
    Case 14:p=New xyz14:Cast(xyz14 Ptr,p)->Name=pad
    Case 15:p=New xyz15:Cast(xyz15 Ptr,p)->Name=pad
    Case 16:p=New xyz16:Cast(xyz16 Ptr,p)->Name=pad
    End Select
   
    destroy(p)
    Sleep 1
Loop Until Inkey=Chr(27)

Sleep

 

(Note it is a trial, not a method)
But it only
1) uses object
2) one sub for all
fxm
Posts: 9123
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Losing memory because of String usage

Postby fxm » Jun 01, 2018 11:09

Variant of my previous code:
- "This.ps = @This.name" is added in each xyzN constructor.
- "If This.ps <> 0 Then *This.ps = "" : This.ps = 0" is added in the sysobject destructor.
The remaining code is not modified.

Code: Select all

type sysobject
  ...
  dim as string ptr ps
  declare destructor ()
end type
destructor sysobject ()
  ...
  If This.ps <> 0 Then *This.ps = "" : This.ps = 0
End destructor

type xyz1 extends sysObject
  ...
  As String   name
  ...
  declare constructor ()
end type
constructor xyz1 ()
  ...
  This.ps = @This.name
End Constructor

type xyz2 extends sysObject
  ...
  As String   name
  ...
  declare constructor ()
end type
constructor xyz2 ()
  ...
  This.ps = @This.name
End Constructor

...  about 57 other simular types

Dim as sysObject ptr Machine

If ... Then
  Machine = New xyz1
Else
  Machine = New xyz2
End if

...

Delete Machine
Last edited by fxm on Jun 01, 2018 12:09, edited 1 time in total.
fxm
Posts: 9123
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Losing memory because of String usage

Postby fxm » Jun 01, 2018 11:26

dodicat wrote:But If I go through object, it seems to stop the slow leak.
Test with resource monitor.

Me too, but does not work and must not work on principle.
(the amount of memory used increases indefinitely)
coderJeff
Site Admin
Posts: 3018
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Losing memory because of String usage

Postby coderJeff » Jun 01, 2018 12:01

wallyg wrote:I simply asked if there was a way to tell FreeBasic that I wanted a string variable garbage collected (ie have no heap storage assigned). I expected ... 1 line example on how to do this.

Yes
STRING_variable = ""
dodicat
Posts: 5913
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Losing memory because of String usage

Postby dodicat » Jun 01, 2018 12:44

OK
One sub to destroy the any ptr to udt
One sub to nullify the name string of any ptr to udt.
(Ran for 30 mins, no leaks)
win 10

Code: Select all

Type sysobject Extends Object
   
    As String Name
   
End Type



Dim As Const String pad=String(1000,"z")

Type xyz1 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz2 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz3 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz4 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz5 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz6 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz7 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz8 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz9 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz10 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz11 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz12 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz13 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz14 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz15 Extends sysObject
    As String Name
    As String a(10000)
End Type
Type xyz16 Extends sysObject
    As String Name
    As String a(10000)
End Type


Sub destroy(Byref p As Any Ptr)
    Var t=Cast(Object Ptr,p)
    If *t Is xyz1 Then Delete Cast(xyz1 Ptr,p)
    If *t Is xyz2 Then Delete Cast(xyz2 Ptr,p)
    If *t Is xyz3 Then Delete Cast(xyz3 Ptr,p)
    If *t Is xyz4 Then Delete Cast(xyz4 Ptr,p)
    If *t Is xyz5 Then Delete Cast(xyz5 Ptr,p)
    If *t Is xyz6 Then Delete Cast(xyz6 Ptr,p)
    If *t Is xyz7 Then Delete Cast(xyz7 Ptr,p)
    If *t Is xyz8 Then Delete Cast(xyz8 Ptr,p)
    If *t Is xyz9 Then Delete Cast(xyz9 Ptr,p)
    If *t Is xyz10 Then Delete Cast(xyz10 Ptr,p)
    If *t Is xyz11 Then Delete Cast(xyz11 Ptr,p)
    If *t Is xyz12 Then Delete Cast(xyz12 Ptr,p)
    If *t Is xyz13 Then Delete Cast(xyz13 Ptr,p)
    If *t Is xyz14 Then Delete Cast(xyz14 Ptr,p)
    If *t Is xyz15 Then Delete Cast(xyz15 Ptr,p)
    If *t Is xyz16 Then Delete Cast(xyz16 Ptr,p)
End Sub

sub killstring(byref p as any ptr)
    Var t=Cast(Object Ptr,p)
    If *t Is xyz1 Then Cast(xyz1 Ptr,p)->name=""
    If *t Is xyz2 Then  Cast(xyz2 Ptr,p)->name=""
    If *t Is xyz3 Then  Cast(xyz3 Ptr,p)->name=""
    If *t Is xyz4 Then  Cast(xyz4 Ptr,p)->name=""
    If *t Is xyz5 Then  Cast(xyz5 Ptr,p)->name=""
    If *t Is xyz6 Then  Cast(xyz6 Ptr,p)->name=""
    If *t Is xyz7 Then  Cast(xyz7 Ptr,p)->name=""
    If *t Is xyz8 Then  Cast(xyz8 Ptr,p)->name=""
    If *t Is xyz9 Then  Cast(xyz9 Ptr,p)->name=""
    If *t Is xyz10 Then  Cast(xyz10 Ptr,p)->name=""
    If *t Is xyz11 Then  Cast(xyz11 Ptr,p)->name=""
    If *t Is xyz12 Then  Cast(xyz12 Ptr,p)->name=""
    If *t Is xyz13 Then  Cast(xyz13 Ptr,p)->name=""
    If *t Is xyz14 Then  Cast(xyz14 Ptr,p)->name=""
    If *t Is xyz15 Then  Cast(xyz15 Ptr,p)->name=""
    If *t Is xyz16 Then  Cast(xyz16 Ptr,p)->name=""
    end sub



#define Intrange(f,l) Int(Rnd*((l+1)-(f))+(f)) 

Dim As Any Ptr p
Dim As Long i
Do
    i=intrange(1,16)
    Select Case As Const i
    Case 1:p=New xyz1:Cast(xyz1 Ptr,p)->Name=pad
    Case 2:p=New xyz2:Cast(xyz2 Ptr,p)->Name=pad
    Case 3:p=New xyz3:Cast(xyz3 Ptr,p)->Name=pad
    Case 4:p=New xyz4:Cast(xyz4 Ptr,p)->Name=pad
    Case 5:p=New xyz5:Cast(xyz5 Ptr,p)->Name=pad
    Case 6:p=New xyz6:Cast(xyz6 Ptr,p)->Name=pad
    Case 7:p=New xyz7:Cast(xyz7 Ptr,p)->Name=pad
    Case 8:p=New xyz8:Cast(xyz8 Ptr,p)->Name=pad
    Case 9:p=New xyz9:Cast(xyz9 Ptr,p)->Name=pad
    Case 10:p=New xyz10:Cast(xyz10 Ptr,p)->Name=pad
    Case 11:p=New xyz11:Cast(xyz11 Ptr,p)->Name=pad
    Case 12:p=New xyz12:Cast(xyz12 Ptr,p)->Name=pad
    Case 13:p=New xyz13:Cast(xyz13 Ptr,p)->Name=pad
    Case 14:p=New xyz14:Cast(xyz14 Ptr,p)->Name=pad
    Case 15:p=New xyz15:Cast(xyz15 Ptr,p)->Name=pad
    Case 16:p=New xyz16:Cast(xyz16 Ptr,p)->Name=pad
    End Select
    killstring(p) 'optionbal
    destroy(p)
    Sleep 1
Loop Until Inkey=Chr(27)

Sleep

 
MrSwiss
Posts: 3219
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Losing memory because of String usage

Postby MrSwiss » Jun 01, 2018 13:32

I've just extended fxm's code somewhat, to show the *String decstuction*, as proposed
by coderJeff (in xyz2) while leaving xyz1 mostly "as-is", but with a "commented out"
destructor (with explanation, why it isn't needed ...):

Code: Select all

Type sysobject
    Dim As String Ptr   ps
    Declare Destructor()
End Type
Destructor sysobject()
    If This.ps <> 0 Then *This.ps = "" : This.ps = 0
End Destructor

Type xyz1 extends sysobject
    As String   name_
    Declare Constructor()
    'Declare Destructor()
End Type
constructor xyz1()
    This.ps = @This.name_
End Constructor
'Destructor xyz1()
'    ' nothing to do: since, .name_ doesn't ever have any dada!
'End Destructor

Type xyz2 extends sysobject
    As String   name_
    Declare Constructor(ByRef n As Const String)
    Declare Destructor()
End Type
Constructor xyz2(ByRef n As Const String)
    This.name_ = n
    This.ps = @This.name_
End Constructor
Destructor xyz2()
    This.name_ = ""
End Destructor

''
Dim As sysobject Ptr    Machine(1 To 10)
' initioalize Machine (array of ptr)
For i As UInteger = LBound(Machine) To UBound(Machine)
    Select Case As Const i
        Case 1, 2, 3, 4, 5
            Machine(i) = New xyz1()
        Case 6, 7, 8, 9, 10
            Machine(i) = New xyz2("xyz2 " + Str(i))
        Case Else
            Print "Machine() out of range!"
    End Select
Next
' show Machine()
For i As UInteger = LBound(Machine) To UBound(Machine)
    Print *Machine(i)->ps, Machine(i)->ps
Next
Print
' destroy Machine()
For i As UInteger = LBound(Machine) To UBound(Machine)
    Delete Machine(i)
Next

Sleep
Btw: this code can be compiled/run ...
PS: I've also renamed 'Name' to 'name_' (to avoid confusion, with FB-Function 'Name()').
dodicat
Posts: 5913
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Losing memory because of String usage

Postby dodicat » Jun 01, 2018 13:39

But wallyg is using any ptr, not sysobject Ptr.

sysobject Ptr makes it easy, any ptr makes it easy enough, but more work to type out explicitly each case.
MrSwiss
Posts: 3219
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Losing memory because of String usage

Postby MrSwiss » Jun 01, 2018 13:53

I don't quite understand, because "what then is the use, of all the OOP boilerplate?"

aka: Why not making use of it, after the effort, of writing it?

You can of course, almost always find a way, to make live more complex ...

Return to “General”

Who is online

Users browsing this forum: No registered users and 5 guests