LZLE List Engine with user friendly powerfull syntax - BETA 0.997a
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
sizeof(string) corresponds to the size of var-len string descriptor (3 uinteger variables), no the size of one character !
In all rigor (if you prefer):
Allocate( ( iLen+1 ) * sizeof( zstring ) )
In all rigor (if you prefer):
Allocate( ( iLen+1 ) * sizeof( zstring ) )
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
I'll post a fix later, whole retest highly recommended, .., speed issues also.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Some remarks:
You were advised before to use a more recent (ie 1.05.0) build of fbc. Many bugs and issues were addressed since the 0.23 version, and the first 'stable' release was considered to be 0.24. As MrSwiss told you, coding new things for old versions of fbc is asking for trouble. The code doesn't even work, much less has an already established user base, for you to worry about making it 'compatible' with older versions of the compiler.Lost Zergling wrote:On my side : W7 32 bits, fbc 0.23 : compile & run : no crash. Accelerate execution : crash. What's important is compile & run.
There's no 'cache issues' with pointers, you have them dangling all over the place. You have like 120 functions or so, and pretty much all of them perform allocations and deallocations of memory, so screwing something up is not as difficult as you may imagine.Lost Zergling wrote:When it was running OK one time, then it becomes ok even though with "Accelerate execution" => Looks like a cache issue on pointers at compile time or somewhat else ? What a puzzle! I don't know what I could do more to fix this issue.
Retested, crashes (both on 32 and 64 bits, although it did get a little better). Also, don't worry about 'speed issues' yet, focus on getting it right, you can always optimize it later. As it stands, there's no use, you'll spend more time debugging the thing than recoding it from scratch...Lost Zergling wrote:I'll post a fix later, whole retest highly recommended, .., speed issues also.
Last edited by paul doe on Apr 27, 2018 13:35, edited 1 time in total.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Remark:
I am not sure that use zstring for returning a substring is faster than use string + mid(), because all allocations and deallocations must be taken into account.
On the other hand, it is a more delicate coding because it is necessary to avoid the memory overflows.
(same remark for left() and right())
I am not sure that use zstring for returning a substring is faster than use string + mid(), because all allocations and deallocations must be taken into account.
On the other hand, it is a more delicate coding because it is necessary to avoid the memory overflows.
(same remark for left() and right())
Code: Select all
Dim As String s1 = "Viva FreeBASIC version 1.06"
Dim As Double t
Sleep 1000
t = Timer
For I As Integer = 1 To 10000000
Scope
Dim As String s2 = Mid(s1, 6, 9)
'result is in s2
End Scope
Next I
Print Cast(Single, Timer - t)
Sleep 1000
t = Timer
For I As Integer = 1 To 10000000
Scope
Dim As Zstring Ptr pz1 = Callocate(Len(s1) + 1)
*pz1 = s1
Dim As Zstring Ptr pz2 = pZ1 + 5
pz2[9] = 0 '' or (*pz2)[9] = 0
'result is in *pz2
Deallocate(pz1)
End Scope
Next I
Print Cast(Single, Timer - t)
Sleep
Last edited by fxm on Aug 07, 2018 6:08, edited 3 times in total.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Yea fxm.
I allocated sizeof(string) after I tried sizeof(zstring) which crashed of course.
Deep down I knew really that sizeof(string) was too much allocation, but it stopped the crash.
Here is an attempt at speeding up mid
I allocated sizeof(string) after I tried sizeof(zstring) which crashed of course.
Deep down I knew really that sizeof(string) was too much allocation, but it stopped the crash.
Here is an attempt at speeding up mid
Code: Select all
#include "crt.bi"
function middle(byref _in as string, byval x as long,byval y as long) byref as const zstring
static as zstring * 10000 g
dim as ubyte ptr p=@g
memcpy(p,Cast(Ubyte Ptr,Strptr(_in)) + x-1, y)
return g
end function
dim as string s1,s2
dim as double t
dim as string freebasic="Freebasic"
dim as long limit=1000000
for z as long=1 to 10
t=timer
for n as long=1 to limit
's1=""
s1=middle(Freebasic,3,4)
next
print s1,timer-t,"middle",freebasic
sleep 50
t=timer
for n as long=1 to limit
's2=""
s2=mid(Freebasic,3,4)
next
print s2,timer-t,"mid",freebasic
print
next z
sleep
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Yes but with a static allocation, this fails:
s1=middle(Freebasic,3,2) & middle(Freebasic,5,2)
s1=middle(Freebasic,3,2) & middle(Freebasic,5,2)
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
@Paul Doe : I retested it 32 bits just copy-past LZLE and your exemple from web site : no crash on my side, even with uInteger.
I could replicate the random bug, one time, then it vanished. I don't know why, perhaps because I'm on a 0.23 (on this computer).
I do not believe that the software is not usable, I do not share at all your point of view on this subject. I also tested the code on a version 1.05 in 64 bits and I did not note any major differences of behavior of the code (except those related to the use of uInteger and a better speed in 64 bits).
I respect your opinion, but arguments you mentionned to this point did not change my mind.
I will continue to support this project, trying to fix bugs and answering questions. I respect your opinion about it.
Everyone shall make its own opinion about the usefulness of it.
I'm open minded and I accept the critics when they make me progresss. I will be very attentive to any other remarks.
@fxm :
"I am not sure that use zstring for returning a substring is faster than use string + mid(), because all allocations and deallocations must be taken into account."
I already tried to allocate a fixed size in constructor and deallocate in destructor : this was just a thick faster, drawbacks : fixed max size and really unstable(perhaps not only because the missing 0), so I thought it was not worth and I decided not to go further in this way. Nevertheless, my tests on "long keys" and lots of keys show a 20%-30% speed increase in this specific context. The code is just a string split, not a replace for mid, that's why it might be faster. About the bugs you mentionned : first bug : +1 car and end string 0 can be fixed (0 shall be fill when reach end string otherwise it is slow), the second one is more problematic because it is linked to speed. I tried a solution for theses 2 issues, but I need to validate if it still is intersting compared to Mid or not. ps : left & right are "free stuff" because pointers are already allocated/deallocated.
I could replicate the random bug, one time, then it vanished. I don't know why, perhaps because I'm on a 0.23 (on this computer).
I do not believe that the software is not usable, I do not share at all your point of view on this subject. I also tested the code on a version 1.05 in 64 bits and I did not note any major differences of behavior of the code (except those related to the use of uInteger and a better speed in 64 bits).
I respect your opinion, but arguments you mentionned to this point did not change my mind.
I will continue to support this project, trying to fix bugs and answering questions. I respect your opinion about it.
Everyone shall make its own opinion about the usefulness of it.
I'm open minded and I accept the critics when they make me progresss. I will be very attentive to any other remarks.
@fxm :
"I am not sure that use zstring for returning a substring is faster than use string + mid(), because all allocations and deallocations must be taken into account."
I already tried to allocate a fixed size in constructor and deallocate in destructor : this was just a thick faster, drawbacks : fixed max size and really unstable(perhaps not only because the missing 0), so I thought it was not worth and I decided not to go further in this way. Nevertheless, my tests on "long keys" and lots of keys show a 20%-30% speed increase in this specific context. The code is just a string split, not a replace for mid, that's why it might be faster. About the bugs you mentionned : first bug : +1 car and end string 0 can be fixed (0 shall be fill when reach end string otherwise it is slow), the second one is more problematic because it is linked to speed. I tried a solution for theses 2 issues, but I need to validate if it still is intersting compared to Mid or not. ps : left & right are "free stuff" because pointers are already allocated/deallocated.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
That's ok, of course. Do whatever you want, it's your code, not mine. I'm sure it's very interesting and does something very cool, if you can get it off the ground (it still crashes at the end with the latest fix, in case you were wondering). So, I'm off for now. Best of luck.Lost Zergling wrote:@Paul Doe : I retested it 32 bits just copy-past LZLE and your exemple from web site : no crash on my side, even with uInteger.
I could replicate the random bug, one time, then it vanished. I don't know why, perhaps because I'm on a 0.23 (on this computer).
I do not believe that the software is not usable, I do not share at all your point of view on this subject. I also tested the code on a version 1.05 in 64 bits and I did not note any major differences of behavior of the code (except those related to the use of uInteger and a better speed in 64 bits).
I respect your opinion, but arguments you mentionned to this point did not change my mind.
I will continue this project, trying to fix bugs and answering questions. I respect your opinion about it.
Everyone shall make its own opinion about the usefulness of it.
I'm open minded and I accept the critics when they make me progresss. I will be very attentive to any other remarks.
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
@Paul Doe. I'll try to do my best even thought it is not perfect or bugged. Your post was interesting for me because it forces me to improve the level of requirement. Welcome back with suggestions as your choice.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
- Just to check my hypotheses of bug, I modified as following two code lines:fxm wrote:- First bug:
when allocating memory for a zstring, you must allocate an additional ubyte for the terminal character (0).
.....
Dim As zString Ptr Listptemp=Allocate(iLen+1), Listptemp2=Allocate(iLen+1), zp1, zp2, zp3=Allocate(1) ' * Alternative to Mid, left & right ==>+20-25% speed tested
.....
Dim As zString Ptr Listptemp=Allocate(Len(str_Tag)+1), zp1, zp2, zp3=Allocate(1)
- Second bug in 'Property List.HashTag(str_Tag As String) As Byte':
'len(this.HashTag)' may be greater than 'ilen' (up to 8 characters greater), so allocated memory overflows when executing '*Listptemp=Str_tmp'
- Eventual some next bugs because I stopped here my code review.
Code: Select all
..........
''''Dim As zString Ptr Listptemp=Allocate(iLen), Listptemp2=Allocate(iLen), zp1, zp2, zp3=Allocate(1) ' * Alternative to Mid, left & right ==>+20-25% speed tested
Dim As zString Ptr Listptemp=Allocate(iLen+8+1), Listptemp2=Allocate(iLen+1), zp1, zp2, zp3=Allocate(1) ' * Alternative to Mid, left & right ==>+20-25% speed tested
..........
''''Dim As zString Ptr Listptemp=Allocate(Len(str_Tag)), zp1, zp2, zp3=Allocate(1)
Dim As zString Ptr Listptemp=Allocate(Len(str_Tag)+1), zp1, zp2, zp3=Allocate(1)
..........
- Remain all the other problems seen by paul doe.
Last edited by fxm on Apr 28, 2018 8:53, edited 4 times in total.
-
- Posts: 42
- Joined: Apr 09, 2015 20:49
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Example #1 leaks memory.
Just the leaks valgrind attributed to LZListsEngine.bi:
Edit2: pValTmp and pValTmp->ListData aren't deallocated properly in the destructor.
Just the leaks valgrind attributed to LZListsEngine.bi:
Edit: Your code is *really* hard to read. Why is so much code cramped into a single line with ":"?==15494== 104 bytes in 1 blocks are definitely lost in loss record 237 of 246
==15494== at 0x4C2EEF5: calloc (vg_replace_malloc.c:711)
==15494== by 0x4024AA: LIST::ALLOWPANCAKE__get__() (LZListsEngine.bas:168)
==15494== by 0x403D12: LIST::VALPRIVATE__get__(FBSTRING&) (LZListsEngine.bas:292)
==15494== by 0x4064E1: LIST::VAL__get__(FBSTRING&) (LZListsEngine.bas:443)
==15494== by 0x414F70: main (test__.bas:243)
==15494==
==15494== 141 (104 direct, 37 indirect) bytes in 1 blocks are definitely lost in loss record 238 of 246
==15494== at 0x4C2EEF5: calloc (vg_replace_malloc.c:711)
==15494== by 0x4024AA: LIST::ALLOWPANCAKE__get__() (LZListsEngine.bas:168)
==15494== by 0x403D12: LIST::VALPRIVATE__get__(FBSTRING&) (LZListsEngine.bas:292)
==15494== by 0x4064E1: LIST::VAL__get__(FBSTRING&) (LZListsEngine.bas:443)
==15494== by 0x413074: main (test__.bas:119)
==15494==
==15494== 141 (104 direct, 37 indirect) bytes in 1 blocks are definitely lost in loss record 239 of 246
==15494== at 0x4C2EEF5: calloc (vg_replace_malloc.c:711)
==15494== by 0x4024AA: LIST::ALLOWPANCAKE__get__() (LZListsEngine.bas:168)
==15494== by 0x403D12: LIST::VALPRIVATE__get__(FBSTRING&) (LZListsEngine.bas:292)
==15494== by 0x4064E1: LIST::VAL__get__(FBSTRING&) (LZListsEngine.bas:443)
==15494== by 0x413792: main (test__.bas:140)
==15494==
==15494== 141 (104 direct, 37 indirect) bytes in 1 blocks are definitely lost in loss record 240 of 246
==15494== at 0x4C2EEF5: calloc (vg_replace_malloc.c:711)
==15494== by 0x4024AA: LIST::ALLOWPANCAKE__get__() (LZListsEngine.bas:168)
==15494== by 0x403D12: LIST::VALPRIVATE__get__(FBSTRING&) (LZListsEngine.bas:292)
==15494== by 0x4064E1: LIST::VAL__get__(FBSTRING&) (LZListsEngine.bas:443)
==15494== by 0x402C12: LIST::GARBAGECOLLECTOR__get__() (LZListsEngine.bas:199)
==15494== by 0x40EEAD: LIST::DROPALL__get__() (LZListsEngine.bas:928)
==15494== by 0x40FBDF: LIST::DESTROY__get__() (LZListsEngine.bas:979)
==15494== by 0x416F92: main (test__.bas:312)
Edit2: pValTmp and pValTmp->ListData aren't deallocated properly in the destructor.
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Hello String Epsilon.
I am not very used to the logs of the grid of values. About memory leaks, do you know if it is a fixed leak, proportionnal with the size of the lists or both ? A fixed leak of a few kb is not very good and it's even pretty ugly I grant you, however it would not be unacceptable (for not too demanding users) what would not be the case of a significant memory loss proportional to the load. (ps pValTmp are misses)
I'm using ":" to save space on the screen so as to see a maximum code in the same time. Code is hard to read because on my programming style which is not very conventional. Moreover, this project is the result of a progressive evolution and therefore everything is not at the same level because of the evolutionary maintenance. I focused on time constraints and bug fixes. Thus, the software is a bit complex by some aspects.
@fxm : bug 2 is up to my point of view a variable number of caracters depending on the keys structure. to be fixed (I got a solution wich cost is 2-5% speed). Just need few time, working on it but now on holidays.
I am not very used to the logs of the grid of values. About memory leaks, do you know if it is a fixed leak, proportionnal with the size of the lists or both ? A fixed leak of a few kb is not very good and it's even pretty ugly I grant you, however it would not be unacceptable (for not too demanding users) what would not be the case of a significant memory loss proportional to the load. (ps pValTmp are misses)
I'm using ":" to save space on the screen so as to see a maximum code in the same time. Code is hard to read because on my programming style which is not very conventional. Moreover, this project is the result of a progressive evolution and therefore everything is not at the same level because of the evolutionary maintenance. I focused on time constraints and bug fixes. Thus, the software is a bit complex by some aspects.
@fxm : bug 2 is up to my point of view a variable number of caracters depending on the keys structure. to be fixed (I got a solution wich cost is 2-5% speed). Just need few time, working on it but now on holidays.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Quelle zone pour les vacances scolaires (quelle région de France) ?Lost Zergling wrote:Just need few time, working on it but now on holidays.
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Merci.
(end of personal message)
(end of personal message)