[solved] Is the FreeBASIC array alignment safe ?

[dodicat]

What do you mean "nothing justifies the principle of your algorithm."?
Are we not allowed to take part in this discussion about 16 bit alignment?
I did say it was not really usable at the start.
If a moderator puts me down like this, then it is hardly worth me posting any more.
I used 32 bits -gen gas here and it is OK.

[fxm]

What do you mean "nothing justifies the principle of your algorithm."?

My sentence has absolutely nothing contemptuous.
I'm just saying that there is no evidence that the algorithm you are using can achieve this goal of array alignment.

[jj2007]

Code: Select all

#macro align(s,n)
scope
redim s(0)
    dim as long count
    do
        count+=1
        var i=cast(uinteger,@s(0))
     if  (i mod 32<>0) and (i mod 16=0) then exit do
     redim as byte b(count)
    redim s(0 to n+count)
loop
redim preserve s(n)
end scope
#endmacro

Do I understand your code correctly (still a beginner in FB) that you reallocate that array in a loop until it is properly aligned? That works in principle, but Windows' HeapAlloc can force you to do more loops that you might think necessary.

A dedicated Alloc16(nbytes) follows a different logic: You allocate 8 bytes more than you need. If your HeapAlloc pointer is aligned 16, fine, you have wasted 8 bytes. If it's align 8, you add 8 to the pointer and mark somewhere that your HeapFree pointer is 8 bytes lower than the align16 pointer. No looping required.

Btw does dim as long count zero the counter? I remember we had a thread on zeroing local variables some time ago. I see no compiler warning about ininitialised variables.

[dodicat]

Thanks jj2007.
That was my idea, but redim preserve back to the requested size sometimes gets a different pointer.
Maybe you should try out your idea in fb.
fxm's supercilious remarks are a put off for me.
I was going to try and repair my algorithm (I usually get a job done one way or another), but now I'll rain check that one.

[fxm]

I, too, have had enough of my remarks being misinterpreted.

[jj2007]

Maybe you should try out your idea in fb.

When I come back from my holidays in Venice, I'll give it a try ;-)

[dodicat]

Very nice jj2007.
I look forward to your code.
I think I have got a working method.
I am using good old fashioned goto, it saves fluffing around with whiles e.t.c.

Tested 32/64 bits and 32 gas.
Maybe it won't work everywhere (like the rules of physics), but it works here.

Code: Select all

#include "fbc-int/array.bi"

function flagsString(flags as uinteger) as string
  return str(flags and &h0000000f) & " allocated dims " & _
  iif(flags and &h00000010,"dimensions are fixed " ," ") & _
  iif(flags and &h00000020,"memory is fixed lenght" ,"")
end function

function alignedString(adr as uinteger) as string
  if (adr and 31) = 0 then return " aligned(32)"
  if (adr and 15) = 0 then return " aligned(16)"
  if (adr and  7) = 0 then return " aligned(8)"
  return " not SSE aligned !"
end function

sub printArrayInfo(array() as const single)
  var p=FBC.ArrayConstDescriptorPtr(array())
  print "index_ptr         " & str(p->index_ptr) & alignedString(cast(uinteger,p->index_ptr))
  print "base_ptr          " & p->base_ptr & alignedString(cast(uinteger,p->base_ptr))
  print "size              " & p->size
  print "element_len       " & p->element_len
  print "dimensions        " & p->dimensions
  print "flags             " & p->flags & " " & flagsString(p->flags)
  if p->dimensions<1 then return
  for i as uinteger = 0 to p->dimensions-1
    print "dimtb(" & i & ").elements " & p->dimtb(i).elements
    print "dimtb(" & i & ").lbound   " & p->dimtb(i).lbound
    print "dimtb(" & i & ").ubound   " & p->dimtb(i).ubound
    print
  next
end sub


 sub align(s() as single,n as long,f as long=0)
 lbl:
 redim as byte b(f) 'get some granularity
 redim s(n+f)
 if cast(uinteger,@s(0)) mod 32 =0  then f+=1:goto lbl
 if cast(uinteger,@s(0)) mod 16 <>0 then f+=1:goto lbl
 var i=cast(uinteger,@s(0))
 redim preserve s(n)
 if cast(uinteger,@s(0))<> i then f+=1:goto lbl
end sub





'screen 20

redim as single vec3(),vec4(),mat4(),tst()
align(vec3(),2)
align(vec4(),3)
align(mat4(),15)
align(tst(),5)

locate 1
print "vec3()"
printArrayInfo(vec3())
print
print "vec4()"
printArrayInfo(vec4())
print
print "mat4()"
printArrayInfo(mat4())
print
print "test()"
printArrayInfo(tst())


sleep
 

I have included a screen 20 if you like to un comment it.
This adds a purge of extra memory to start with to test the robustness of the code.
Anyway, foottie on the telly now, Scotland versus England.

[jj2007]

Very nice jj2007.
I look forward to your code.

Here it is (I'm still on holidays but found an hour...)

Code: Select all

#include "Windows.bi"	' HeapAlloc + HeapFree
#include "crt.bi"	    ' MasmBasic's Alloc16 ported to FreeBasic (https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1391)
#define maxalloc16 1000 ' #allocations

Dim shared a16table(maxalloc16) As Any ptr  ' all align 16 pointers must be registered in this array

Function Alloc16(bytes as long) As integer
Dim As integer a16=HeapAlloc(GetProcessHeap(), HEAP_GENERATE_EXCEPTIONS or HEAP_ZERO_MEMORY, bytes+8)
Dim ct as long=0
  Do
	if a16table(ct)=0 Then Exit Do	' get a free slot
	ct+=1
  Loop until ct>=maxalloc16
  if a16 and 8 Then
	a16table(ct)=a16+9	' mark as align 8
  else
	a16table(ct)=a16	' don't mark, take as is
  endif
  function=(a16+8) and -16	' return the aligned 16 pointer
end function

Sub Free16(oldptr as integer)
Dim ct as long=0
Dim px as any ptr=@a16table(0)
  Do
	if a16table(ct)=oldptr Then HeapFree(GetProcessHeap(), 0, oldptr):Exit Do
	if a16table(ct)=oldptr+1 Then HeapFree(GetProcessHeap(), 0, oldptr-8):Exit Do
	ct+=1
  Loop until ct>=maxalloc16
  a16table(ct)=0	' mark as freed
End Sub

Dim as long somearray(100)
For outerloop as long=1 to 20
  For x as long=0 To 7
	somearray(x)=Alloc16(Rnd()*10000)
	printf("%x ", somearray(x))
  Next 
  print
  For x as long=0 To 7
	Free16(somearray(x))
  Next
Next
sleep

Typical output:

Code: Select all

6b3330 6b4030 6b4d20 6b6200 6b7fc0 6b98f0 6bb280 6bbe20
6b3330 6b5920 6b7e50 6b9350 6b27a0 6bae00 6bc810 6bc920
6b27a0 6b3330 6b4580 6b65e0 6b85f0 6ba910 6bb3b0 6bc050
6b3330 6b4460 6b5a00 6b7c90 6b8670 6b8d90 6ba320 6bb220
6b3330 6b4390 6b4e80 6b6fb0 6b8230 6b8d80 6ba800 6bc600
6b3330 6b4080 6b5950 6b7f30 6b8f80 6ba460 6bbea0 6bdae0
6b3330 6b57c0 6b77a0 6b96b0 6bba70 6bdba0 6be6a0 6c0c80
6b3330 6b4b20 6ab440 6b5be0 6b27a0 6b8210 6b9880 6ba5b0
6b3330 6b5270 6b6680 6b82a0 6b8bd0 6ba8f0 6bb8b0 6bd7b0
6b3330 6b37d0 6b4840 6b53d0 6b7700 6b9570 6b9a80 6b27a0
6b3330 6b3e10 6b4c00 6b27a0 6b6030 6b70a0 6ab440 6b7d30
6b3330 6b58d0 6b6000 6b7550 6b7c70 6b94c0 6bb9f0 6bcfd0
6b3330 6b5220 6b7750 6b8850 6bacb0 6bce00 6bd340 6bdc00
6b3330 6b4630 6b5a90 6b7f60 6b9ca0 6ba7e0 6bccc0 6be8b0
6b3330 6b5800 6b78c0 6b9100 6b9ed0 6bb0e0 6bd5f0 6beea0
6b3330 6b4b90 6b6ec0 6b27a0 6b7d00 6b9440 6b9f30 6bb620
6b3330 6b27a0 6b4a60 6b6d70 6b8860 6ba6c0 6bc880 6bd280
6b3330 6b51b0 6b6080 6b8200 6b9bf0 6baf70 6bc120 6be550
6b3330 6b3b10 6b5f50 6b7240 6b8840 6b27a0 6b9650 6bae90
6b3330 6b5120 6b5650 6b74f0 6b7fe0 6b9eb0 6bbc90 6bc450

It will throw some absolutely useless warnings - just ignore them ;-)

[dodicat]

I ran a parallel method to get addresses of pointers aligned to 16.

Code: Select all

#include "Windows.bi"   ' HeapAlloc + HeapFree
#include "crt.bi"       ' MasmBasic's Alloc16 ported to FreeBasic (https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1391)
#define maxalloc16 1000 ' #allocations

Dim shared a16table(maxalloc16) As Any ptr  ' all align 16 pointers must be registered in this array

Function Alloc16(bytes as long) As integer
Dim As integer a16=HeapAlloc(GetProcessHeap(), HEAP_GENERATE_EXCEPTIONS or HEAP_ZERO_MEMORY, bytes+8)
Dim ct as long=0
  Do
   if a16table(ct)=0 Then Exit Do   ' get a free slot
   ct+=1
  Loop until ct>=maxalloc16
  if a16 and 8 Then
   a16table(ct)=a16+9   ' mark as align 8
  else
   a16table(ct)=a16   ' don't mark, take as is
  endif
  function=(a16+8) and -16   ' return the aligned 16 pointer
end function

Sub Free16(oldptr as integer)
Dim ct as long=0
Dim px as any ptr=@a16table(0)
  Do
   if a16table(ct)=oldptr Then HeapFree(GetProcessHeap(), 0, oldptr):Exit Do
   if a16table(ct)=oldptr+1 Then HeapFree(GetProcessHeap(), 0, oldptr-8):Exit Do
   ct+=1
  Loop until ct>=maxalloc16
  a16table(ct)=0   ' mark as freed
End Sub


'==============================
function alignedString(adr as uinteger) as string
  if (adr and 31) = 0 then return " aligned(32)"
  if (adr and 15) = 0 then return " aligned(16)"
  if (adr and  7) = 0 then return " aligned(8)"
  return " not SSE aligned !"
end function

sub Get16Align(spaces() as any ptr,num as long)
static as uinteger ref(num*40)
dim as long count=-1,n
do
    var i=cast(uinteger,@ref(n))
     if  (i mod 32<>0) and (i mod 16=0) then
         count+=1
         redim preserve spaces(count)
         spaces(count)=cptr(any ptr,@ref(n))
   end if
   n+=1
loop until count>=num
end sub

redim as any ptr slot16()
 Get16Align(slot16(),10)
'===================================


Dim as long somearray(100)
print "jj2007",,"dodicat"
For outerloop as long=1 to 2
  For x as long=0 To 7
   somearray(x)=Alloc16(Rnd()*10000)
   print somearray(x),,slot16(x)
   print alignedString(somearray(x)),,alignedString(cast(uinteger,slot16(x)))
   print "-----------------------------------"
  ' printf("%x ", somearray(x))
  Next
  print
  For x as long=0 To 7
   Free16(somearray(x))
  Next
  print
  print
Next
sleep 

But it would be impossible to set manually the array base descriptor to any of these pointers in fb?
Also I think in assembler it would not be easy.
I think you'll need a few beers in Venice to contemplate, but wait till you get back.

[D.J.Peters]

A C/C++ fixed sized array, aligned or not, are simple a pointer as parameter,
so all my aligned data types return automatically per "operator cast as data type ptr"
the right pointer when used as parameter to the SIMD (Single Instruction Multiple Data) math library.

Only to show you an example here are vec4 aligned 16 bytes.

Joshy

Code: Select all

#include "crt.bi"


const as uinteger ALIGN = 15, MASK = NOT ALIGN

type _VEC4
  as single x=any,y=any,z=any,w=any
end type  


type vec4 ' is aligned(16)
  declare constructor
  declare constructor(byref v as vec4)
  declare constructor(byref x as const single, _
                      byref y as const single, _
                      byref z as const single, _
                      byref w as const single)
  declare operator cast as _VEC4 ptr ' <--- NOTE this data type are declared as parameter as replacement for C/C++ aligned array !
  declare operator cast as string
  declare operator let(byref v as vec4) 
  declare operator [] (byref index as const uinteger) byref as single
  declare property x as single
  declare property x (byref v as const single)
  declare property y as single
  declare property y (byref v as const single)
  declare property z as single
  declare property z (byref v as const single)
  declare property w as single
  declare property w (byref v as const single)  
  private:
  memory as zstring * 31 ' sizeof(single)*4=16 + 15 more bytes for alignment
end type
constructor vec4
end constructor
constructor vec4(byref v as vec4) 
  memcpy(this,v,sizeof(_VEC4))
end constructor
constructor vec4(byref a as const single, _
                 byref b as const single, _
                 byref c as const single, _
                 byref d as const single)
  this[0]=a : this[1]=b : this[2]=c : this[3]=d 
end constructor
operator vec4 . cast as _VEC4 ptr
  return cptr(_VEC4 ptr,cast(uinteger,@memory+15) and MASK)
end operator
operator vec4 . let(byref v as vec4)
  memcpy(this,v,sizeof(_VEC4))
end operator
' the primary key are here, all member access end in aligned adresses !
operator vec4 . [] (byref index as const uinteger) byref as single
  return cptr(single ptr,cast(uinteger,@memory+15) and MASK)[index]
end operator
operator vec4 . cast as string : return "vec4[" & this[0] & "," & this[1] & "," & this[2] & "," & this[3] & "]":end operator
property vec4 . x as single : return this[0] : end property
property vec4 . x (byref v as const single) : this[0]=v  : end property  
property vec4 . y as single : return this[1] : end property
property vec4 . y (byref v as const single) : this[1]=v  : end property  
property vec4 . z as single : return this[2] : end property
property vec4 . z (byref v as const single) : this[2]=v  : end property  
property vec4 . w as single : return this[3] : end property
property vec4 . w (byref v as const single) : this[3]=v  : end property  

dim as Vec4 a=vec4(1,2,3,4)
dim as _VEC4 ptr p=a
print a
print a.x,a.y,a.z,a.w
print p->x,p->y,p->z,p->w

print hex(mask)
sleep

[jj2007]

I ran a parallel method to get addresses of pointers aligned to 16.

Looks ok!

[adeyblue]

#include "crt.bi"

If you're going to include crt.bi, you can just use _aligned_malloc /_aligned_realloc / _aligned_free, no hoop jumping required

[jj2007]

#include "crt.bi"

If you're going to include crt.bi, you can just use _aligned_malloc /_aligned_realloc / _aligned_free, no hoop jumping required

Sounds interesting, but I always get undefined reference...

Code: Select all

#include "Windows.bi"	' HeapAlloc + HeapFree
#include "crt.bi"	' MasmBasic's Alloc16 ported to FreeBasic (https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1391)
#define maxalloc16 1000	' #allocations

declare function _aligned_malloc(byval as size_t,byval as size_t) as any ptr
declare sub _aligned_free(byval as any ptr)

Dim shared a16table(maxalloc16) As Any ptr

function Alloc16(bytes as long) As integer
Dim As integer a16=HeapAlloc(GetProcessHeap(), HEAP_GENERATE_EXCEPTIONS or HEAP_ZERO_MEMORY, bytes+8)
Dim ct as long=0
  Do
	if a16table(ct)=0 Then Exit Do	' get a free slot
	ct+=1
  Loop until ct>=maxalloc16
  if a16 and 8 Then
	a16table(ct)=a16+9	' mark as align 8
  else
	a16table(ct)=a16	' don't mark, take as is
  endif
  function=(a16+8) and -16	' return the aligned 16 pointer
end function

Sub Free16(oldptr as integer)
Dim ct as long=0
Dim px as any ptr=@a16table(0)
  Do
	if a16table(ct)=oldptr Then HeapFree(GetProcessHeap(), 0, oldptr):Exit Do
	if a16table(ct)=oldptr+1 Then HeapFree(GetProcessHeap(), 0, oldptr-8):Exit Do
	ct+=1
  Loop until ct>=maxalloc16
  a16table(ct)=0	' mark as freed
End Sub
Dim as long somearray(1000)
Dim As double t
Dim As long useC
for tloop as Long=0 to 9
	t=timer
	For outerloop as long=1 to 200
	For x as long=0 To 999
		if useC and 1 then
			somearray(x)=Alloc16(Rnd()*10000)
		else
			somearray(x)=_aligned_malloc(Rnd()*10000, 16)
		endif
		' printf("%x ", somearray(x))
	Next 
	' print
	For x as long=0 To 999
		if useC and 1 then
			Free16(somearray(x))
		else
			_aligned_free(somearray(x))
		endif
	Next
	Next
	t=timer-t
	print int(t*1000);" ms, C=";useC
	useC=useC xor 1
next
sleep

[fxm]

Only to show you an example here are vec4 aligned 16 bytes.

Code: Select all

.....
operator vec4 . cast as _VEC4 ptr
  return cptr(_VEC4 ptr,cast(uinteger,@memory) and MASK)
end operator
.....
operator vec4 . [] (byref index as const uinteger) byref as single
  return cptr(single ptr,cast(uinteger,@memory) and MASK)[index]
end operator
...

• (viewtopic.php?p=283752#p283752)

MASK ?
Anyway, I don't see how you can align this pointer with a simple AND (with the condition: pointer >= @memory) ?
I would rather seen a pointer value like this:
((cast(uinteger, @memory) + 15) and (-16))

[dodicat]

How on earth does this work?
All arrays redim to start.
mat4() remains dynamic while all the rest become fixed.(because I have omitted (0)
(I have looped everything to check 16 bit alignment)
The memory allocations change from time to time, I suppose depending what the OS is doing in the background.

Code: Select all

#include "fbc-int/array.bi"

function flagsString(flags as uinteger) as string
  return str(flags and &h0000000f) & " allocated dims " & _
  iif(flags and &h00000010,"dimensions are fixed " ," ") & _
  iif(flags and &h00000020,"memory is fixed lenght" ,"")
end function

function alignedString(adr as uinteger) as string
  if (adr and 31) = 0 then return " aligned(32)"
  if (adr and 15) = 0 then return " aligned(16)"
  if (adr and  7) = 0 then return " aligned(8)"
  return " not SSE aligned !"
end function

sub printArrayInfo(array() as const single)
  var p=FBC.ArrayConstDescriptorPtr(array())
  print "index_ptr         " & str(p->index_ptr) & alignedString(cast(uinteger,p->index_ptr))
  print "base_ptr          " & p->base_ptr & alignedString(cast(uinteger,p->base_ptr))
  print "size              " & p->size
  print "element_len       " & p->element_len
  print "dimensions        " & p->dimensions
  print "flags             " & p->flags & " " & flagsString(p->flags)
  if p->dimensions<1 then return
  for i as uinteger = 0 to p->dimensions-1
    print "dimtb(" & i & ").elements " & p->dimtb(i).elements
    print "dimtb(" & i & ").lbound   " & p->dimtb(i).lbound
    print "dimtb(" & i & ").ubound   " & p->dimtb(i).ubound
    print
  next
end sub


 sub align(s() as single,n as long,f as long=0)
 lbl:
 redim as integer b(f) 'get some granularity
 redim s(n+f)
 if cast(uinteger,@s(0)) mod 32 =0  then f+=1:goto lbl
 if cast(uinteger,@s(0)) mod 16 <>0 then f+=1:goto lbl
 var i=cast(uinteger,@s(0))
 redim preserve s(n)
 if cast(uinteger,@s(0))<> i then f+=1:goto lbl
end sub


dim as long counter
screenres 1024,800
width 1024\8,800\14
do
      counter+=1
    
redim as single vec3(0),vec4(0),mat4(),tst(0)

align(vec3(),2)
align(vec4(),3)
align(mat4(),15)
align(tst(),5)
screenlock
cls
print "vec3()"
printArrayInfo(vec3())
print
print "vec4()"
printArrayInfo(vec4())
print
print "mat4()"
printArrayInfo(mat4())
print
print "test()"
printArrayInfo(tst())
print counter;" loops"
screenunlock
sleep 1
loop until len(inkey)

sleep