How to Manage Dynamic Memory (Allocation / Deallocation) in FB

[Tourist Trap]

because:
Aborting due to runtime error 12 ("segmentation violation" signal) in ...\FBIde0.4.6r4_fbc1.06.0\FBIDETEMP.bas::()

Here no crash, but I understand thank you. So when can a New fail (as mentioned in your last remark in the 1st post) if not when assigned to a fake address?

(on 'var x = New(-1) UDT')[/list]

This crashes 100% of time. But this not (for me):

Code: Select all

var x = (New(-1) UDT)->Foo

This doesn't work either. No crash, no work...

***
I don't want to go too much off topic, but I'm trying various syntaxes of the placement new page, and some doesn't work. For instance this one works:

Code: Select all

var x = (New(0) UDT[-1] {any})->Foo

When:

Code: Select all

 var x = (New(0) UDT (any))->Foo

doesn't.
Of course I had an already initialised field (ii = 9). But this was really difficult to figure out if the -1 is allowed or not (what meaning of it?), if this was a problem of the type of braces (there are many types of braces even for the 'any')... And the compiler thinks that we want to use New[] or the dereferencing operator, depending on the brackets in use for any.
I think humbly that in some future there will be the need to review this page. At least a second example with a list of all possible syntax would help.
End of the off topic... About the -1 in the count, it leaves me very puzzled.

[fxm]

Concerning the behavior of the current fbc rev 1.06 with the NEW operator, have you entirely read the (now fixed) bug report '#699 Bug on the "New[]" operator when attempting to allocate a negative number of elements' and all its discussions?

[fxm]

(on 'var x = New(-1) UDT')

This crashes 100% of time. But this not (for me):

Code: Select all

var x = (New(-1) UDT)->Foo

This doesn't work either. No crash, no work...

Similarly, this also aborts by run-time error, for me.

[coderJeff]

Not in the case of the Placement New?

Placement New doesn't allocate anything. Memory address is already given with New(address).

[Tourist Trap]

Not in the case of the Placement New?

Placement New doesn't allocate anything. Memory address is already given with New(address).

Thanks, so the failure is about the allocation only, not the other actions of New like placement, initialisation or whatever.

quote="fxm"]

(on 'var x = New(-1) UDT')

This crashes 100% of time. But this not (for me):

Code: Select all

var x = (New(-1) UDT)->Foo

This doesn't work either. No crash, no work...

Similarly, this also aborts by run-time error, for me.

I know that sometimes bad allocations dont crash, so maybe it's just luck. My idea was to make the New fail to test if it resulted in a null pointer. But Coder Jeff answered, and what is meant by "fail" is about allocation only if I understand correctly.

[coderJeff]

My idea was to make the New fail ...

It's pretty tough to get allocate to fail on an OS with virtual memory, unless you are really stressing it.

The only easy way I know is to simulate it. You could do this with a UDT that overloads new operator, and return a null pointer.

Or you can do like this, and hi-jack the allocate/deallocate built-in functions:

Code: Select all

dim shared simulate_fail as boolean = false

#undef allocate
#undef deallocate

'' hi-jack allocate/deallocate function
extern "c"
	declare function malloc alias "malloc" ( byval size as const uinteger ) as any ptr
	declare sub free alias "free" ( byval p as const any ptr )

	function allocate ( byval size as const uinteger ) as any ptr
		print "  allocate " & size & " bytes: ";
		dim p as any ptr = iif( simulate_fail, 0, malloc(size) )
		print iif( p, "OK", "failed")
		function = p
	end function
	
	sub deallocate ( byval p as const any ptr )
		print "  deallocate @" & cint(p)
		free( p )
	end sub
end extern

scope
	print "var p = new integer"
	var p = new integer
	print "  @p = " & cint(p)
	delete p
end scope

scope
	simulate_fail = true
	print "var p = new integer"
	var p = new integer
	print "  @p = " & cint(p)
end scope

var p = new integer
allocate 4 bytes: OK
@p = 4794904
deallocate @4794904
var p = new integer
allocate 4 bytes: failed
@p = 0

This works because fbc compiler generates the code (which is fairly simple) for the New/Delete expressions doing a function lookup for allocate/deallocate. Hi-jacking allocate/deallocate won't work for other data like strings and arrays because fbc calls other run-time library functions, which then call malloc/free directly.

[fxm]

It's pretty tough to get allocate to fail on an OS with virtual memory, unless you are really stressing it.

Not so much in 32-bit:

Code: Select all

Const As Uinteger n = 128*1024*1024
Dim As Integer i = 0

Do
  Dim As Byte Ptr p = New Byte[n]
  i += 1
  Print Using "##.### MB"; i * n / 1024 / 1024 / 1024;
  Print , p
  If p = 0 Then Exit Do
Loop

Print "heap overflow"
Sleep

Code: Select all

 0.125 MB     32022560
 0.250 MB     166297632
 0.375 MB     300580896
 0.500 MB     434872352
 0.625 MB     569159712
 0.750 MB     703402016
 0.875 MB     837726240
 1.000 MB     971964448
 1.125 MB     1106284576
 1.250 MB     1240571936
 1.375 MB     1374863392
 1.500 MB     1509138464
 1.625 MB     1643429920
 1.750 MB     2000338976
 1.875 MB     0
heap overflow

[coderJeff]

Interesting. That makes sense that allocation would start to go wrong around the 2 GB range on 32-bit with 2^31 bytes allocating half the address space:

Code: Select all

type T
	a(0 to 1024*1024*1024-1) as ubyte
	b(0 to 1024*1024*1024-2) as ubyte
end type
var x = new T
print "@x = "; cint( x )

EDIT:

Related to this topic, I checked the compiler source for limits on the "Array Too Big" and "UDT Too Big" errors:

compiler limits on all targets:
- max array() elements: 2^31-1
- max array() byte size: 2^31-1
- max UDT size: 2^31-1

compiler limit on 32 bit target:
- max NEW size: sizeof(data) <= 2^32-1
- max NEW[] size: elements * sizeof(data) + sizeof(integer) <= 2^32-1

compiler limit on 64-bit target:
- max NEW size: sizeof(data) <= 2^64-1
- max NEW[] size: elements * sizeof(data) + sizeof(integer) <= 2^64-1

operating system limits on all targets
- max NEW size: depends on allocate
- max NEW[] size: depends on allocate
- allocate(uinteger): depends on system's malloc

operating system limits:
- 32-bit max allocate size parameter: 2^32-1
- 64-bit max allocate size parameter: 2^64-1
- actual max size that can be allocated depends on operating system memory manager (would have to look at limitations for each OS and/or C runtime)