remove the values duplicated in the array not efficient

General FreeBASIC programming questions.
Post Reply
aloberoger
Posts: 507
Joined: Jan 13, 2009 19:23

remove the values duplicated in the array not efficient

Post by aloberoger »

the result is not what I espected

Code: Select all

Const MAX_ELEMENTS = 30
Dim Shared STACK (MAX_ELEMENTS) As String
Dim Shared tos As Integer

sub RemoveDuplicates()
Dim I as integer                                    
dim J as integer  
		
 for I = 0 To tos-1
  for J = I+1 to tos-1
   if STACK(I) = STACK(J) Then
      STACK(J) = STACK(tos-1)
      tos=tos-1:J=J-1
      If J=1 Then Exit For
   end if
  Next J
 Next I
 Exit Sub
 
End Sub

 
     tos=12
     STACK(0) ="A" :
     STACK(1) ="A" : STACK(2) ="B"  : STACK(3) ="A"  : STACK(4) ="A"  : STACK(5) ="C"  : STACK(6) ="A"
     STACK(7) ="D" : STACK(8) ="E"  : STACK(9) ="A"  : STACK(10) ="E"  : STACK(11) ="D"  : STACK(12) ="D"
     
     RemoveDuplicates()
     
     For i As Integer=0 To tos
     	Print "i= " & i  & " " & STACK(i)
     Next
     
 Sleep
fxm
Moderator
Posts: 12107
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: remove the values duplicated in the array not efficient

Post by fxm »

Perhaps like that:

Code: Select all

sub RemoveDuplicates()
Dim I as integer                                   
dim J as integer 
      
 for I = 0 To tos-1
  for J = I+1 to tos''-1
   if STACK(I) = STACK(J) Then
      STACK(J) = STACK(tos)''-1)
      tos=tos-1:J=J-1
''      If J=1 Then Exit For
      if J=tos-1 Then Exit For
   end if
  Next J
 Next I
'' Exit Sub
 
End Sub
petan
Posts: 683
Joined: Feb 16, 2010 15:34
Location: Europe
Contact:

Re: remove the values duplicated in the array not efficient

Post by petan »

Mhm, and what do you expect exactly ?
What do you want to do in 'removeDuplicates()' exactly ?
How big is your array ?
Why not reducing it at array's first creation immediately ?

BTW, seems as second loop has wrong limit, if you want generate/test all pairs

Code: Select all

for j=i+1 to tos
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: remove the values duplicated in the array not efficient

Post by MrSwiss »

Duplicated entries are simply deleted (String = "").
Try below code:

Code: Select all

Dim As Integer tos = 12
Dim As String  STACK (tos)

sub RemoveDuplicates  ( ByVal t As Integer, _
						STK() As String )
	For I As Integer = 0 To t - 1
		For J As Integer = I + 1 to t
			If STK(I) = STK(J) Then STK(J) = ""
		Next J
	Next I
End Sub

' === MAIN ===
STACK(0) ="A"
STACK(1) ="A" : STACK(2) ="B"  : STACK(3) ="A"  : STACK(4) ="A"  : STACK(5) ="C"  : STACK(6) ="A"
STACK(7) ="D" : STACK(8) ="E"  : STACK(9) ="A"  : STACK(10) ="E"  : STACK(11) ="D"  : STACK(12) ="D"

RemoveDuplicates( tos, STACK() )

For i As Integer = 0 To tos
	Print "i="; i, STACK(i)
Next

Sleep
aloberoger
Posts: 507
Joined: Jan 13, 2009 19:23

Re: remove the values duplicated in the array not efficient

Post by aloberoger »

I expected a thing like This:

tos=4 ' new limit for entries
' the order of entries is preserved
STACK(0) ="A"
STACK(1) ="B"
STACK(2) ="C"
STACK(3) ="D"
STACK(4) ="E"

STACK(5) ="" To STACK(MAX_ELEMENTS) =""
fxm
Moderator
Posts: 12107
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: remove the values duplicated in the array not efficient

Post by fxm »

You are never as well served as when you serve yourself.
Zippy
Posts: 1295
Joined: Feb 10, 2006 18:05

Re: remove the values duplicated in the array not efficient

Post by Zippy »

fxm wrote:You are never as well served as when you serve yourself.
Even a stopped clock is right twice a day.
fxm
Moderator
Posts: 12107
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: remove the values duplicated in the array not efficient

Post by fxm »

Code: Select all

Const MAX_ELEMENTS = 30
Dim Shared STACK (MAX_ELEMENTS) As String
Dim Shared tos As Integer

Sub RemoveDuplicates()
   For I As Integer = 0 To tos-1
      For J As Integer = I+1 To tos
         If STACK(I) = STACK(J) Then
            STACK(J) = ""
            For K As Integer = J To tos-1
               Swap STACK(K), STACK(K+1)
            Next K
            tos=tos-1:J=J-1
            If J=tos Then Exit For
         End If
      Next J
   Next I
End Sub

 
tos=12
STACK(0) ="A" :
STACK(1) ="A" : STACK(2) ="B"  : STACK(3) ="A"  : STACK(4) ="A"  : STACK(5) ="C"  : STACK(6) ="A"
STACK(7) ="D" : STACK(8) ="E"  : STACK(9) ="A"  : STACK(10) ="E"  : STACK(11) ="D"  : STACK(12) ="D"

RemoveDuplicates()

For I As Integer = 0 To MAX_ELEMENTS
   Print "i= " & i  & " " & "'" & STACK(i) & "'"
Next I

Sleep

Code: Select all

i= 0 'A'
i= 1 'B'
i= 2 'C'
i= 3 'D'
i= 4 'E'
i= 5 ''
i= 6 ''
i= 7 ''
i= 8 ''
i= 9 ''
i= 10 ''
i= 11 ''
i= 12 ''
i= 13 ''
i= 14 ''
i= 15 ''
i= 16 ''
i= 17 ''
i= 18 ''
i= 19 ''
i= 20 ''
i= 21 ''
i= 22 ''
i= 23 ''
i= 24 ''
i= 25 ''
i= 26 ''
i= 27 ''
i= 28 ''
i= 29 ''
i= 30 ''
dodicat
Posts: 7983
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: remove the values duplicated in the array not efficient

Post by dodicat »

How's about a complete clean up.

Code: Select all



#macro cleanup(a,b)
redim as typeof(a) b(0)
scope
dim as long flag
for n1 as long=lbound(a) to ubound(a)
    flag=0
for n2 as long=n1+1 to ubound(a)
    if a(n1)=a(n2) then flag=1
 next n2
 if flag=0 then
        redim preserve b(1 to ubound(b)+1)
        b(ubound(b))=a(n1)
        end if
next n1
end scope
#endmacro



#macro show(a)
    For n As Integer=Lbound(a) To Ubound(a)
        Print a(n);
    Next
    Print
#endmacro

'======================================================
Const MAX_ELEMENTS = 30
Dim Shared STACK (MAX_ELEMENTS) As String

STACK(0) ="A" :
STACK(1) ="A" : STACK(2) ="B"  : STACK(3) ="A"  : STACK(4) ="A"  : STACK(5) ="C"  : STACK(6) ="A"
STACK(7) ="D" : STACK(8) ="E"  : STACK(9) ="A"  : STACK(10) ="E"  : STACK(11) ="D"  : STACK(12) ="D"
print "original"
show(stack)
print
Redim As String result()
cleanup(stack,result)
print
print "Remains"
show(result)
'other
dim as long L(...)={1,2,3,1,5,5,8,8,9,4,1,6,0,0,0,9,2,2,3,6,7,7}
print
print "original"
show(L)
redim as long res()
cleanup(L,res)

Print
print "Remains"
show(res)
sleep


sancho2
Posts: 547
Joined: May 17, 2015 6:41

Re: remove the values duplicated in the array not efficient

Post by sancho2 »

This method uses instr to build a string of unique characters. It will only work with single characters.

Code: Select all

Const MAX_ELEMENTS = 30
Dim Shared STACK (MAX_ELEMENTS) As String
Dim Shared tos As Integer

Declare Function RemoveDupes() As Integer
Function RemoveDupes() As Integer
	Dim As Integer n
	Dim As String s = ""
	
	For x As Integer = 0 To tos
		n = InStr(s,stack(x))
		If n = 0 Then
			s += stack(x)
		EndIf
		stack(x) = ""
	Next
	
	For x As Integer = 1 To Len(s)
		'Print Chr(s[x-1])
		stack(x - 1) = Chr(s[x-1])
	Next

	Return Len(s)

End Function

 
     tos=12
     STACK(0) ="A" :
     STACK(1) ="A" : STACK(2) ="B"  : STACK(3) ="A"  : STACK(4) ="A"  : STACK(5) ="C"  : STACK(6) ="A"
     STACK(7) ="D" : STACK(8) ="E"  : STACK(9) ="A"  : STACK(10) ="E"  : STACK(11) ="D"  : STACK(12) ="D"
     
     'RemoveDuplicates()
     removedupes()
     
     For i As Integer=0 To tos
        Print "i= " & i  & " " & stack(i)
     Next
     
 Sleep
dodicat
Posts: 7983
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: remove the values duplicated in the array not efficient

Post by dodicat »

Does this prove that a bubblesort remove method is probably correct?

Code: Select all

type vct
    as long x,y
    declare operator cast() as string
end type

operator vct.cast() as string
return str(x) +" , " +str(y)
end operator

operator =(a as vct,b as vct) as long
return a.x=b.x and a.y=b.y
end operator

#macro cleanup(a,b)
redim  b(0)
scope
dim as long flag
for n1 as long=lbound(a) to ubound(a)
    flag=0
for n2 as long=n1+1 to ubound(a)
    if a(n1)=a(n2) then flag=1:exit for
 next n2
 if flag=0 then
        redim preserve b(1 to ubound(b)+1)
        b(ubound(b))=a(n1)
        end if
next n1
end scope
#endmacro



#macro show(a)
    For n As Integer=Lbound(a) To Ubound(a)
        Print n, a(n)
    Next
    Print
#endmacro

#define Intrange(f,l) int(Rnd*((l+1)-(f))+(f))

'===============================================
redim as vct V(20000),result()

for n as long=lbound(V) to ubound(V)
    V(n).x=IntRange(300,500)
    V(n).y=IntRange(200,400)
next

cleanup(V,result)
print "remain"
show(result)
print "Number discarded = ";(ubound(V)-lbound(V)+1)-ubound(result)
print "Press a key"
sleep

screen 19,32
dim as ulong W=rgb(255,255,255),counter
for n as long=1 to ubound(result)
   if point(result(n).x,result(n).y)=W then counter+=1
    pset (result(n).x,result(n).y),W
next
print counter
sleep

 
sancho2
Posts: 547
Joined: May 17, 2015 6:41

Re: remove the values duplicated in the array not efficient

Post by sancho2 »

@dodicat:
There are a couple of things I don't understand in this code.
1. What are you showing us? This is what I am reading in the code:
A: Vct stands for vector.
B: You are building 'V', an array of up to 20000 random vectors within the range x = 300 to 500 y = 200 to 400
C: You then send this array to macro 'cleanup' which iterates all 20000 items in v and begins build 'result' by way of checking each item against the new result array, adding them if they are not already existing in the result array (flag = 1).
Is it the speed at which this is accomplished that you are showing us? Or is there some other reason that you say this is "probably correct"?
2. Is this really a sort? Sorted on the criteria of uniqueness I guess. I would call it a filter. A bubble filter.
3. The overloaded Cast operator is called when you print a vct type variable, correct? Is this is called a default cast?

I'm not denying that you are correct, just trying to figure it out.
dodicat
Posts: 7983
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: remove the values duplicated in the array not efficient

Post by dodicat »

Hi Sancho2.
Just that the thread is about getting rid of duplicates.
Strings are the subject of course, but with 2d vectors (vct) you can check graphically if a method does remove duplicates efficiently.
The bubble loops are used, they are a bit slow.
I've been trying out other sort types (insertion/comb).
The idea behind this type of removal is that in every sorting algorithm, every array element directly faces every other array element sometime, and (removal) can be implemented at that instance, via building up another array.
That's all really.
Cheers.
fxm
Moderator
Posts: 12107
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: remove the values duplicated in the array not efficient

Post by fxm »

sancho2 wrote: 3. The overloaded Cast operator is called when you print a vct type variable, correct? Is this is called a default cast?
- This year, I added to documentation a specific page about the Cast operator (KeyPgOpCast) in addition to the one about the Cast keyword (KeyPgCast).
- You can also look at the "Coercion and Conversion" page (ProPgDataConversion) updated at the same time.

So, there is already material for thinking in documentation.
Post Reply