LZLE Tutorial - In progress

Forum for discussion about the documentation project.
Post Reply
Lost Zergling
Posts: 534
Joined: Dec 02, 2011 22:51
Location: France

LZLE Tutorial - In progress

Post by Lost Zergling »

The tutorial is mainly provided as commented code
You are welcome using this topic for questions about how to use or possible bugs, posting code examples and suggestions about new features

LZLE can be found here : viewtopic.php?f=8&t=26533
What is it ? LZLE is an intuitive list implementation for FreeBasic :
It offers an instruction set for manipulating lists using some memory virtualization and map/reduce features.

-------------------- Preamble --------------------
One of the difficulties of using a list engine is the technical heterogeneity of the developers. It is difficult to meet the expectations of beginner or intermediate programmers while at the same time some developers will have developed their own tools and their own way of working with them.
This tool (LZLE) comes from my own experience and my own tools. Initially, I saw it only as a simple toolbox, that is to say that the homogeneity of the overall logic was only seen from an end user point of view. This global logic has been the subject of many technical and conceptual changes, such as the abandonment of the idea of ​​the 'BranchConsider' on a body to the benefit of the 'Snatch' multi-instances at the same time richer, more flexible and more explicit. And so on.
As the project progressed, difficulties became clear:
First, at the technical level, the project was an opportunity to develop and acquire skills, even at a level sometimes dilettante or self-taught.
But the real challenge lies in the richness of the problem: from the moment you start working on the memory and that the program itself is a project to help the user to program on the memory, the Pandora's box the computer is open: there are always ideas for improvements and it affects both the performance and functionality or even the intrinsic logic of the tool's work.
Important functional and technical achievements are still to be considered but would require more time and especially skills from my point of view very specific or too difficult to acquire.
Accumulation of simple things makes it a complex result, and this complex result is itself simplistic with regard to the technical perspectives imaginable in the medium term or sometimes effective in another way in a professional context.
Working with the tool therefore involves learning. This is inevitable because the tool itself wants to be easy to approach must also be able to allow the achievement of complexity and because this complexity affects the way of working with RAM.
It is nevertheless possible to use the tool in a simple and effective way, but the deepening will always be exciting, if only by the ideas and perspectives of functional orientations.
The documentation thus becomes a site in its own right unless it is first done with a series of commented examples, which is the case here.
LZLE and its associated 'documentation' is not intended to deliver a truth about how to work with memory but to propose methods of analysis and associated practical solutions around the use of lists.
The way of using the tool is multiple leaving a space of expression to the creativity.

-------------------- Essential features --------------------
The tool can manage lists in memory indexed or unindexed, meaning : behave like a classic list or as a hashtable, depending on usage.
The developer thus specifies the most optimal memory management for his algorithms : the tool is used to better automate the management of the memory while keeping for the developer the possibility of a certain fineness in the specifications.

The basic syntax is similar to a ForEach:
ForEach ref in MyVar .. End ForEach
Is translated:
For i = 1 to MyList.AllOf: MyList.BlindStep
...
Next i
Or
MyList.AllOf:
For i = 1 to MyList.Count: MyList.BlindStep (-1)
...
Next i
Or
MyList.AllOf:
While MyList.fStep
..
Wend
But iterators are also available for indexed entries:
While MyList.HashStep
...
Wend
Iterators on indexed entries also work on unindexed entries.
To create an unindexed entry, use the keyword Tag (Key) or BlindTag (Key) (equivalent to Add (Key or Value)). Indeed, when we know in advance that the key is unique, and that we will not need an index, it is possible to fill fast an unindexed list using BlindTag.
To create an indexed entry, use the keyword HashTag (Key).
Same spirit for not indexed ('flat') lists :
mylist.Add ($ Key, $ Value) is translated:
mylist.BlindTag ($ Key): mylist.Val ($ Value) or mylist.BlindTag ($ Key): mylist.RwTag1 ($ Value) (depending on the value size)
mylist.Add ($ Key, $ Value1, $ Value2, etc) is translated:
mylist.BlindTag ($ Key): mylist.Val ($ Value1): mylist.BlindTag ($ Key): mylist.Val ($ Value2): etc
Or you can also support multi-values ​​with string concatenation using .val property (long strings) :
mylist.BlindTag ($ Key): mylist.Val ($ Value1+stringSep+$ Value2+etc..)
Also available :
* Multi-values ​​features using HashTag ($ Key) with HashKeyUnique property previously set to 0
* Reverse or partial iterarors on non indexed and indexed lists using fStepRev, BlindStep (-1), HashSpetpRev, KeyStepRev.
* You can choose how and when a list, a branch of a tree or even a listnode can be send to garbage or to protected flat list or to another list.
meaning : list data exchanges at any level supported, recycling on demand or on any subset or on whole list (using Snatch, NodeFlat, Recycle, GarbageFlat and GarbageSnatch).
* Data can be sent from indexed to non indexed and vice versa(NodeFlat and RestoreHash).
* Creation & management index supported (CopyCat and Follow).
* Syntax behavior is optimized for detecting new entries while setting (If MyList.HashTag ($ Key)).
* Soft deletion is possible on indexed lists (RwTag1 ("") and parsing with KeyStep).
* Reverse seek supported, other advanced optimizations options.
* Hierarchical count down. Consistent speed. Consistent deallocations.
* Tracking enable features that would require pointers just by instruction set extension and without having to manipulate pointers (HoldBack, Track, TrackStep)
A "companion extension" is in development and is dedicated to integration with Arrays https://freebasic.net/forum/viewtopic.php?f=8&t=27695
---------------------------------------------------------

001 : Basic knowledeges working with "flat" list
By 'flat list' we mean a list in which the elements are connected to each other in a sequential order: to find a given element we must then go through the elements that precede it. This type of list is the easiest to understand.

Code: Select all

' TUTO 001 : Basic knowledeges working with "flat" list

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MyList As List
Dim i As Integer

'Flat list : each flat list entry can be set with MyList.BlindTag("Key") : in a "flat" list elements are linked in a sequential order
'Be carefull : BlindTag just Add an element at the end of the list without checking if the Key already exists or not
'This is usefull fast filling a list with elements you know they are not redondant
For i=1 to 20
    MyList.BlindTag(str(i))             ' str(i) is going to be the key of the element - this instruction create a new element in list
    MyList.RwTag1(str(i) & "*")     ' each element of the list can store a small value in Tag1 to Tag4 : for one small values just use Tag1 may be best choice
    MyList.Val(str(i) & "+")            ' a bigger string can be stored in MyList.Val
Next i

'Let's see the sequential parsing : this is equivalent a "Foreach"
For i=1 To MyList.AllOf : MyList.BlindStep
    Print "Key= " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
Next i
Sleep

'There is another way to parse a flat list
Print "Second Parsing method (Flat) :"
MyList.AllOf
While MyList.fStep
    Print "Key= " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
Wend
Sleep

' You can as well parse backward
Print "Third Parsing method (Flat) :"
MyList.AllOf
While MyList.fStepRev
    Print "Key= " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
Wend
Sleep

'Now we know how to fill and parse a flat list
'Important feature is to know wether an element is alraedy in list or not
If MyList.HasTag("15") Then
    Print "Found : " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
End If
'You can notice the current element is now the one was found
If MyList.HasTag("73") Then
    'Never happend
Else   
    Print "Not Found : 73"
End If
Print "Current : " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
MyList.fStep
Print "Current is now : " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
Sleep
MyList.First
Print "Current : " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val ' this is the first "hidden" element : it is the starting point of the flat list
MyList.fStep
Print "Current is now : " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
Sleep
Print
Print "---- Accessing elements while creating them ----"

'MyList.Tag("Key") is equivalent to BlindTag if an element does not exist and to HasTag if an element exist.
If MyList.Tag("15") Then  'Code for an Update
    Print "Found : " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
    MyList.Val( MyList.Val + " ##")
Else
    'Never happend
End If

If MyList.Tag("73") Then ' => "Creating 73" 
    'Never happend
Else    'Code for a new element
    Print "New = 73"
    MyList.RwTag1("73" & "*")
    MyList.Val("73" & "+")
End If
MyList.Root

sleep
Print "Result :"
For i=1 To MyList.AllOf : MyList.BlindStep
    Print "Key= " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
Next i

MyList.First : MyList.fStep ' stepping hidden node
Print "First= " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
MyList.Last : MyList.fStepRev ' stepping back hidden node
Print "Last= " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val

' Let's see some other feature
Print "Count= " & MyList.Count
Print "Value of Tag 12 = " & MyList.ValTag("12")

' Aside & Recover
For i=1 To MyList.AllOf : MyList.BlindStep
    If MyList.Tag="17" Then : MyList.Aside : End If
Next i
Print "Current= " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
MyList.Recover
Print "Recover= " & MyList.Tag & " " & MyList.Tag(1) & " " & MyList.Val
MyList.Destroy
gCollector.Destroy
Print "%%AllocateDeallocateCounter =" & AllocateDeallocateCounter
Sleep
System
fStep, bStep and BlindStep do almost the same thing: advance the current element's cursor in the list.
fStep and fStepRev do this more securely and returns true if the next or previous element could be reached.
bStep is faster but does not check: a loop of type For i = 1 to MyList.AllOf: MyList.bStep: ..: Next i will be faster than a loop of type While MyList.fStep: .. : Wend both because of the For next and the gain of a test per node traveled. BlindStep is the backward-compatible version of fStep and also lets you navigate +n or -n items in the list at a time : MyList.BlindStep(3) jump 3 nodes forward, MyList.BlindStep(-2) jump 3 nodes backward with no check like bStep.
When using bStep or BlindStep(n) you must ensure you do not jump out of the list (before first node or after last node) otherwise you may crash or get unexpected results. fStep and fStepRev shall be secured.
fmove is changing the position of a node in a non indexed (flat) list

Code: Select all

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MyList As List
Dim i As Integer

For i=10 to 20
    Print "Already exists ? " & MyList.Tag(str(i))
Next i

For i=1 to MyList.AllOf : MyList.bStep
    Print MyList.Tag
Next i
Print "--------------"
Print "Current=" &  MyList.Tag
sleep
Print "Current is send 4 nodes backward :"
MyList.fMove(-4)
For i=1 to MyList.AllOf : MyList.bStep
    Print MyList.Tag
Next i
sleep
Print "Already exists ? " & MyList.Tag("11") ' Tag "11" already exists so, it just moving cursor to it.
' MyList.Tag("key") always set cursor to Tag and return 1 if key already exists in list otherwise return 0
' MyList.HasTag(key) Return 1 if key already exists in list otherwise return 0 and you can choose wether you want to set cursor on key on success or not (AutoCursor 0/1 property)

Print "Tag 11 is send 6 nodes forward :"
MyList.fMove(6) 'Tag 11 is send 6 nodes forward
For i=1 to MyList.AllOf : MyList.bStep
    Print MyList.Tag
Next i
MyList.Destroy
gCollector.Destroy
Print "%%AllocateDeallocateCounter =" & AllocateDeallocateCounter
sleep
system
Other 'flat list' manipulating :

Code: Select all


# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MyList As List
Dim i As Integer

For i=20 to 35
    MyList.Tag(str(i))
    MyList.RwTag1(Chr(100-i))
    MyList.Val(" ==>" & str(i))
Next i

For i=1 to MyList.AllOf : MyList.bStep
    Print "Tag=" & MyList.Tag & " - Tag1=" & MyList.Tag(1) & " value=" & MyList.Val
Next i
sleep
' Bug report (Beta 0.99) please replace the following line in LZLE ValTag Property:
'       If bSearchRes=1 Then : If str_value=this.Tag(0) Then : pValTmp=pSearchNode : Return this.ValPrivate : End If
'By : If bSearchRes=1 And str_value=this.Tag(0) Then : pValTmp=pSearchNode : Return this.ValPrivate
'The list contains CONST MAX_COLS columns for values
Print "List contains 25 ? " & MyList.HasTag("25") & "  ListContains K ? " &  MyList.HasTag("K")
Print MyList.ValTag("30") & " is the value of key '30'"
MyList.ColTags(1) 'Setting working column to 1 (Tag1 is considered as if it were keys column
Print "List contains 25 ? " & MyList.HasTag("25") & "  ListContains K ? " &  MyList.HasTag("K")
Print MyList.ValTag("30") & " is the value of key '30'" 'Return empty string when key is not found
sleep

sleep
Print "CAREFULL : list till is on ColTag(1) mode : " & MyList.ColTags
For i=1 to MyList.AllOf : MyList.bStep
    Print "Tag=" & MyList.Tag & " - Tag1=" & MyList.Tag(1)
Next i
sleep
Print "Just indicates working key tag is back to index 0"
MyList.ColTags(0)
For i=1 to MyList.AllOf : MyList.bStep
    Print "Tag=" & MyList.Tag & " - Tag1=" & MyList.Tag(1)
Next i
sleep
Print "Doing a 'flat' sort Ascending but on index column 1 :"
MyList.ColTags(1)
' MyList.ColSort(0)
MyList.fSort
For i=1 to MyList.AllOf : MyList.bStep
    Print "Tag=" & MyList.Tag & " - Tag1=" & MyList.Tag(1)
Next i
sleep
'Please note that First and Last are pointing nodes that are previous first key and after last key
'Because these instructions are mostly used for preparing iteration
MyList.First : MyList.BlindStep
Print "First=" & MyList.Tag
MyList.Last : MyList.BlindStep(-1)
Print "Last=" & MyList.Tag
Print "Count=" & MyList.Count
MyList.Destroy
gCollector.Destroy
Print "%%AllocateDeallocateCounter =" & AllocateDeallocateCounter
sleep
system
AllKeywords allowing the manipulation of flat lists (not indexed) were discussed.


002 : Understanding list implementation
What we call here a hash list is a cascading tree of keys, the concatenation of the cascaded keys forming the key of an element of the list. This name of "hash list" does not correspond to the technical or scientific terminology but to an intuitive interpretation used in the common language (by reference to #HashTag which is worldwide known by anybody as a keyword allowing to access the data related). Each branch of the tree is concretely a "flat list" and each flat list is made of sequential cascaded keys (Tags). Thus, a hash list is a hierarchical organization of flat lists with n levels sharing in principle the same length of cascaded keys. (the key is split into a hierarchy of sub-keys called Tags which are assigned to the hierarchical tree). So far, you can see a hash list as an indexed list. The main interest is the search speed which is much faster on indexed values.
=> LZLE helps user handling the complexity of indexed lists as they can be manipulated combining advantages of non indexed lists.
One example is if the list contains the keys "12" and "123" : the list element containing the Tag "2" will be at the same time the last element of the key "12", but also the cascaded tag allowing access to "3", last element forming the key "123". By key, we mean the complete key representing a hashtag. Each hashtagged value represent a key item accessing to an element of the list thrue a hierarchy of Tags.
=> Working with list is becoming very easy just having a mental representation of that tree of indexed values, and imagine you can just pick values in the tree to send them to a garbage collector, to a protected flat list or to another list and vice-versa.

To resume this a very simple way :
MyList.HashTag("1234") => "1", "12", "123" and "1234" are 'HashTags' but only "1234" is 'HashKey' as well
MyList.HashTag("1234") and MyList.HashTag("12") => => "1", "12", "123" and "1234" are 'HashTags', "12" and "1234" are 'HashKeys'
A value will be (automatically) considered as a key from the moment it has been explicitely 'hashtagged'

The exemple below illustrate the difference between keys and tag (cascaded parts of a key).

Code: Select all

' TUTO 002 : Understanding list implementation

' The list object is a combination of 3 lists (basically)
'   1 : a TREE list (elements are linked as a tree structure, so they are indexed) and this Tree can be manipulated as well as a "hash list" (tree) or as a flat list into any branch of the tree
'   2 : a Garbage Collector which is a sequential (flat) list of element identified by Chr(3)
'   3 : a Protected flat list which is a sequential (flat) list of element that is a children of the First visible node Chr(4)

'   It is important to understand some of the power of the engine is the hability to handle Tree items as if they were flat and the user
'   can goes from flat to tree or from tree to flat in a list depending on the optimization strategy he or she wants to implement in algorithms.
'   Working with list is becoming trivial just having a mental representation of that tree of indexed values, and imagine you can just pick values in the tree to send them to garbage collector or to protected flat list
'   You don't have to manage Garbage Collector, it is automatic.
'   Protected flat list is so called, because you can "Recycle" a whole tree, sending all items to garbage collector, the protected flat list is not destroyed and can be restored as an index (tree)
'   A lot of other features are also provided : for exemple, it is possible to use several instances to perform some specific manipulations such as working just on a part of a tree or sending datas to another list


# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MyList As List
Dim i As Integer

For i=220 to 240
    MyList.HashTag(Str(i))
Next i

Print "TREE Representation --"
MyList.RootNode
While MyList.HashStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "----------------------"
Print "1st element Key=" & Chr(4) & " Tag=" & Chr(4) & " = Protected flat list entry"
Print "2nd element Key=" & Chr(4) & " Tag= : Protected flat list always contain an empty element"
Print "3rd element Key=" & Chr(3) & " Tag=" & Chr(3) & " Garbage Collector always contain at least 1 element"
Print "Tree structure : by default, the keys are indexed character by character"
Print "----------------------"
Sleep
Print
Print "KEYS Representation --"
MyList.Root
While MyList.KeyStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "----------------------"
sleep

' Notice a element is marked as a key as soon as it has been "HashTagged"
' This is indicated by Tag1 wich is not empty (if no value, the key is indicated by a blank)
Print "New HashTag 23"
MyList.HashTag("23")
MyList.Root
While MyList.KeyStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "----------------------"
Sleep
MyList.HashTag("23") ' Set cursor to key "23"
MyList.RwTag1("") ' unmark the element as key
MyList.Root
While MyList.KeyStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "----------------------"
Print
Sleep
' Same as Tag with flat list, HashTag is returning 1 if element exist, otherwise it is created
If MyList.HashTag("24") Then
    Print "24 found : code for updating values associated with that key (Tag1-4 and Val)"
Else
    'Never happend
End If
If MyList.HashTag("21") Then
    'Never happend
Else   
    Print "21 not found but now created : code for new values"
End If
' This is very practical when adding new values to an indexed list

' Same as HasTag with flat list, HasHashTag is testing wether an element is present or not but without creating it
If MyList.HasHashTag("21") Then
    Print "21  found"
End If
If MyList.HasHashTag("19") Then
    Print "19 not found, not created"
End If
Sleep
MyList.Root
While MyList.KeyStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "----------------------"

' But now, do not forget we are parsing a tree wich some elements are keys and other not
' So, HasKey is equivalent to HasHashTag returning 1 only on Key elements, 0 if HashTag exists, otherwise -1
If MyList.HasKey("220") Then
    Print "Key 220 found"
End If
If MyList.HasKey("22") Then
Else   
    Print "Key 22 NOT found"
End If

'Like the flat list, you can also parse backward
Print "Parse back showing whole Tree structure"
MyList.Root
While MyList.HashStepRev
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "*----------------------"
sleep
Print "Parse back showing keys"
MyList.Root
While MyList.KeyStepRev
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "%----------------------"
MyList.Destroy
gCollector.Destroy
Print "%%AllocateDeallocateCounter =" & AllocateDeallocateCounter
sleep
system
003 : Navigating

Code: Select all

' TUTO 003 : Navigating

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MyList As List
Dim i As Integer

For i=220 to 240
    MyList.HashTag(Str(i))
Next i

Print "TREE Representation --"
MyList.RootNode
While MyList.HashStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "0----------------------"
Sleep

'The list can be parse as a series of flat lists
MyList.Root
While MyList.fStep
    Print "HashTag=" & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "1----------------------"
' Note the root list just contains "2"
Print "Tag2 exists ? " & MyList.Tag("2")
Print "Going to sublist (children) of 2"
MyList.Down
'Print "*HashTag=" & MyList.HashTag & " current Tag=" & MyList.Tag
While MyList.fStep
    Print "HashTag=" & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "2----------------------"
'sleep : system

MyList.Tag("3")
Print "Going to sublist (children) of 3"
MyList.Down
While MyList.fStep
    Print "HashTag=" & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "Returning to level up"
MyList.Up  ' Up and Down will return 1 on success and 0 on failure (if you are top or bottom of the list)
MyList.AllOf
While MyList.fStep
    Print "HashTag=" & MyList.HashTag & " current Tag=" & MyList.Tag
Wend

' It is possible to navigate up and down, but also to direct access with HashTag
Print "Direct access to 22"
MyList.HashTag("22")
MyList.AllOf
While MyList.fStep
    Print "HashTag=" & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "-----"
' Aside and Recover also operational with indexed elements
MyList.Aside
MyList.Root : MyList.fStep
Print "HashTag=" & MyList.HashTag & " current Tag=" & MyList.Tag
MyList.Recover
Print "HashTag=" & MyList.HashTag & " current Tag=" & MyList.Tag

MyList.Destroy
gCollector.Destroy
Print "%%AllocateDeallocateCounter =" & AllocateDeallocateCounter
sleep
system
About the optimization logic:
The lzle engine is inspired by the following paradigms:
The work of the processor on the RAM is approached from the angle of a logistic flow optimization problem: each memory address or element of the list is a little like a small packet that must be sent to the destination. What matters is not the routing speed of a package but the overall fluidity. Take a metaphor: the Wintergaten Marble Machine (see YouTube): each ball vector is a node of the list, the re-routing of the balls must be continuous with minimal cost. The music is your program, the notes represent the progress of your algorithm and the fall of the metal balls as you reuse the memory continuously.
The list engine knows that each ball is identical and therefore substitutable, whereas at the level of the VM of the OS this information can not be presupposed. As soon as the memory substitution takes place in the program, it is not necessary for the OS or the machine to request the cache to defragment and organize the reallocation.
Intrinsic optimization is more about flow efficiency than pure speed, trying to keep cache and UAL as safe as possible.
The cache can be used, for example, for quick tables to perform calculations from reading the list's nodes, such as musical notes in your algorithms.
Similarly, any optimization calculation has a cost and the UAL that is not requested to accelerate the calculation of memory access remains available for other processes, which can be useful if you plan to launch several programs using simultaneously the engine lzle.
Two other points are also to consider:
Parsing optimization: complex because variable and dependent on the structure of the dataset. This problem is approached in two ways: by the presence of a basic optimization algorithm (will be the object of future evolutions, depending on the results obtained) but effective for the sequential keys (transparent to the user) of a on the one hand, by the flexibility allowed by the instruction set to allow finesse in the coding on the other hand, the latter point being very important to consider.
The optimization of the memory load: it can be technical (length of hashLen intermediate keys to optimize rather than the speed or the memory), or conceptual and algorithmic of the developer by the way the keys are organized (more the left part is common to a large number of keys, the less the tree will take up space in memory).
In the musical field, there is no better solution. It is thought that the instrument serves the musician whereas it is sometimes the musician should be at the service of his instrument.


004 : Using meters and exemple with HashTag/NodeFlat/RestoreHash (?=>"Map/Reduce/Rollbackreduce")

Like the notion of 'HashTag' which is not a hash key, the notion of 'Map Reduce' is an analogy. It is possible to browse the algoritmic tree in a manner that appears recursive to the user and implement key association functions. An example of feedback from the lower nodes is the calculation function of the count by branch. But the analogy relates to the notion of transformation from a tree structure to a sequential structure in ram. There is no parallelization, but we could see the tool as an aid to the preparation of input data, or as a tool for experimenting data processing.

Code: Select all

' TUTO 004 : Using meters and exemple with HashTag/NodeFlat/RestoreHash (?=>"Map/Reduce/Rollbackreduce")

' The list object implements the following meters :
'   MyList.Count : the number of elements in current flat list
'   MyList.NodeCount : the number of elements that were created :
'       this number is bigger than the number of elements you can normally parse because some of them are hidden when parsing
'       each flat list is preceeded by an hidden entry element
'   MyList.BranchCount : the number of chidrens in a branch : this meter is off by default because it is consuming a few ressources to update meters of each parents nodes on each move
'   MyList.FlatCount : the number of elements in protected flat list
'   MyList.GarbageCount : the number of elements in GarbageCollector
'   MyList.ContainerCount : the number of MyList.Val in hidden GarbageCollector : usually, you do not need to care with this

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MyList As List
Dim i As Integer
MyList.BranchCountDown(1) 'Activate

For i=220 to 240
    MyList.HashTag(Str(i))
Next i

Print "TREE Representation --"
MyList.Root
While MyList.HashStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag & " branch " & str(MyList.BranchCount) & " count=" & str(MyList.Count)
Wend
Print "----------------------"
Sleep

'Let's Garbage the tree
MyList.Recycle
MyList.RootNode
While MyList.HashStep
    Print "Key= " & MyList.HashTag & " Tag=" & MyList.Tag
Wend
Print "----------------------"
Print "De-referenced elements=" & MyList.GarbageCount & " Total elements=" & MyList.NodeCount
' About NodeCount : Garbage=29 elements + 1 Last + 1 First (not visible) + 2 for flat list + 3 (internal)=36
Sleep

For i=120 to 130
    MyList.HashTag(Str(i))
Next i
Print "TREE Representation --"
MyList.RootNode
While MyList.HashStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag & " branch " & str(MyList.BranchCount) & " count=" & str(MyList.Count)
Wend
Print "----------------------"
Print "De-referenced elements = " & MyList.GarbageCount & " Total elements=" & MyList.NodeCount
Sleep
' Total of elements is same because the new tree construction used de-refenced nodes

Print "Now sending 2 elements of the Tree to the protected flat list"
MyList.NFrecursive(1)
MyList.HashTag("121") : MyList.NodeFlat
MyList.HashTag("122") : MyList.NodeFlat
MyList.NFrecursive(0)
' This for the exemple, NodeFlat is multimode and is designed by default to be used into a Loop using HashStep or KeyStep
' Outside the Loop scope, we may specified NFrecursive(1) thrue otherwise parents nodes will be ignored by NodeFlat with consequence Tree index not properly maitained & memory usage not optimal.

Print "TREE Representation --"
MyList.RootNode
While MyList.HashStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag & " branch " & str(MyList.BranchCount) & " count=" & str(MyList.Count)
Wend
Print "----------------------"
' 121 and 122 have been moved to flat list
Print "Flat list=" & MyList.FlatCount
Print "De-referenced elements=" & MyList.GarbageCount & " Total elements=" & MyList.NodeCount
Print
Print "Now accessing and parsing the flat list"
MyList.FlatStack
While MyList.fStep
     Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print
Print "Or alternative parsing for protected flat list"
For i=1 To MyList.AllOf : MyList.fStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Next i
Print
' To leave the protected flat list just do (and not MyList.Up)
Print "KeyStep : "
MyList.Root
While MyList.KeyStep
     Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "----------------------"
Sleep
Print "Sending 124 and 125 to derefenced elements"
MyList.NFMethod(-1) 'Default is 1
MyList.HashTag("124") : MyList.NodeFlat
MyList.HashTag("125") : MyList.NodeFlat
MyList.RootNode
While MyList.HashStep
     Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "----------------------"
Print "De-referenced elements=" & MyList.GarbageCount & " Total elements=" & MyList.NodeCount
Sleep

' Let's Garbage the tree
MyList.Recycle
MyList.RootNode
While MyList.HashStep
     Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag
Wend
Print "Tree garbaged ----------------------"

' Accessing flat list
Print "Accessing and parsing the flat list"
MyList.FlatStack
While MyList.fStep
     MyList.RestoreHash
Wend
MyList.RootNode
While MyList.HashStep
    Print "Key= " & MyList.HashTag & " current Tag=" & MyList.Tag  & " branch " & str(MyList.BranchCount) & " count=" & str(MyList.Count)
Wend
Print "121 and 122 are now restored as indexed elements"

MyList.Destroy
gCollector.Destroy
Print "%%AllocateDeallocateCounter =" & AllocateDeallocateCounter
sleep
system
005 : Managing several instances
The list object is a combination of 3 lists (basically)
1 : a Tree list (elements are linked as a tree structure, so they are indexed) and this Tree can be manipulated as well as a "hash list" (tree) or as a flat list into any branch of the tree
2 : a Garbage Collector which is a sequential (flat) list of element identified by Chr(3)
3 : a Protected flat list which is a sequential (flat) list of element that is a children of the First visible node Chr(4)
In combination with Map/Reduce/RollBackReduce, data exchanges and possibilities of navigation, a wide panel of operations can be performed on indexed lists in a very intuitive and versatile way.

Code: Select all

' TUTO 005 : Managing several instances

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MyList_1 As List
Dim MyList_2 As List

Dim i As Integer

For i=220 to 240
    MyList_1.HashTag(Str(i))
Next i

Print "TREE Representation --"
MyList_1.Root
While MyList_1.HashStep
    Print "Key= " &  MyList_1.HashTag & " current Tag=" &  MyList_1.Tag
Wend
Print "----------------------"
Sleep
Print "Going 22 on list 1"
MyList_1.HasHashTag("22") 'Set cursor to 22 without making "22" considered a key
Print "Key= " & MyList_1.HashTag
Print "List 2 Snatching List 1"
MyList_2.Snatch(MyList_1)
Sleep
Print "Result on List 1 :"
MyList_1.RootNode
While MyList_1.HashStep
    Print "Key= " &  MyList_1.HashTag & " current Tag=" &  MyList_1.Tag
Wend
Print "----------------------"
Print
Sleep
Print "Result on List 2 :"
MyList_2.RootNode
While MyList_2.HashStep
    Print "Key= " &  MyList_2.HashTag & " current Tag=" &  MyList_2.Tag
Wend
Print "----------------------"
Sleep

'Let's Garbage the tree
MyList_2.Recycle
Print "Tree garbaged on list 2"
Print "Result on List 2 :"
MyList_2.RootNode
While MyList_2.HashStep
    Print "Key= " &  MyList_2.HashTag & " current Tag=" &  MyList_2.Tag
Wend
Print "----------------------"
Sleep

'Now List 1 is snatching Lit 2 garbage
Print "List 1 garbage count=" & MyList_1.GarbageCount
Print "List 2 garbage count=" & MyList_2.GarbageCount
MyList_1.GarbageSnatch(MyList_2)
Print "List 1 new garbage count=" & MyList_1.GarbageCount
Print "List 2 new garbage count=" & MyList_2.GarbageCount
Print "Result on List 1 :"
MyList_1.RootNode
While MyList_1.HashStep
    Print "Key= " &  MyList_1.HashTag & " current Tag=" &  MyList_1.Tag
Wend
Print "----------------------"

MyList_1.Destroy : MyList_2.Destroy
gCollector.Destroy
Print "%%AllocateDeallocateCounter =" & AllocateDeallocateCounter
Sleep

' Snatch can be performed from a list to another on anywhere : a branch of the Tree,
' the dereferenced elements list, or even the protected flat list
' In previous exemple we used Snatch to garbage a specific branch in List 1, transfering work to a second List instance
' We can imagine List2 can be used as filter to transfert choosen element to flat list 2, and so on
' In combination with Map/Reduce/RollBackReduce and possibilities of navigation,  a so wide panel of
' operations can be performed on indexed lists in a very intuitive and versatile way

System
"TestSorts" exemples - Updated 13/08/2018 : New Feature : NFMethod(3) fSort, fSortMethod, ColSort, and Sort new Properties
Here some new exemple and one FIX :
Illustrate how to sort & track

Code: Select all

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MaListe1 As List
Dim i as integer
'MaListe1.Root
Print "Start"
MaListe1.HashTag("D4") : MaListe1.HoldBack : MaListe1.RwTag1("D4-1")
MaListe1.HashTag("C") : MaListe1.HoldBack : MaListe1.RwTag1("C")
MaListe1.HashTag("A") : MaListe1.HoldBack : MaListe1.RwTag1("A")
MaListe1.HashTag("B") : MaListe1.HoldBack : MaListe1.RwTag1("B")

MaListe1.HashTag("C5") : MaListe1.HoldBack :  MaListe1.RwTag1("C5")
MaListe1.HashTag("C4") : MaListe1.HoldBack :  MaListe1.RwTag1("C4")
MaListe1.HashTag("C3") : MaListe1.HoldBack  : MaListe1.RwTag1("C3")
MaListe1.HashTag("D2") : MaListe1.HoldBack : MaListe1.RwTag1("D4-2")
MaListe1.HashTag("D1") : MaListe1.HoldBack : MaListe1.RwTag1("D4-3")


MaListe1.Root
While MaListe1.KeyStep
    Print MaListe1.HashTag
Wend
Print ' "OK -------------"
Sleep

Print "Liste 1 (...Partial sort on D branch) :"
MaListe1.Root : MaListe1.HashTag("D") : MaListe1.Down
MaListe1.fSort

MaListe1.Root
While MaListe1.HashStep
    Print MaListe1.HashTag
Wend
Print "OK -------------"
sleep


Print "Snatching C branch from List 1 to List2 : "
Dim MaListe2 As List
MaListe1.HashTag("C")
MaListe2.Snatch(MaListe1)

Print "Liste 1 :"
MaListe1.Root
While MaListe1.KeyStep
    Print MaListe1.HashTag
Wend
Print ' "OK -------------"
Print "Liste 2 :"
MaListe2.Root
While MaListe2.KeyStep
    Print MaListe2.HashTag
Wend
sleep
Print
Print "Sorting Liste 2 in 2 steps :"
MaListe2.NFMethod(1)
MaListe2.Root
While MaListe2.HashStep
    MaListe2.NodeFlat
Wend
MaListe2.HashSort(1)
MaListe2.FlatStack
While MaListe2.fStep
    MaListe2.RestoreHash
Wend
'Print "OK -------------"
Print "Liste 2 sorted after Reduce+Restore :"
MaListe2.Root
While MaListe2.KeyStep
    Print MaListe2.HashTag
Wend

'MaListe2.Root : MaListe2.fStep
MaListe2.HashTag("C") ' Set cursor to C on liste 2 : snatch entry point
MaListe1.Root
MaListe1.Snatch(MaListe2)
Print "OK -------------"
Print "Liste 1 (partially sorted) :"
MaListe1.Root
While MaListe1.HashStep
    Print MaListe1.HashTag
Wend
Print : sleep

MaListe1.fSortMethod(-1)
MaListe1.Root : MaListe1.fSort
Print "Liste 1 : root flat list REV sorted !! ) :"
MaListe1.Root
While MaListe1.HashStep
    Print MaListe1.HashTag
Wend
Print "OK -------------"
sleep



Print "Reduce+Restore on Liste1's root flat list"
MaListe1.Root : MaListe1.NFMethod(3)
While MaListe1.fStep
    MaListe1.NodeFlat
Wend
MaListe1.FlatStack
MaListe1.HashSort(1)
While MaListe1.fStep
    Print "Restoring sorted : " & MaListe1.Tag
    MaListe1.RestoreHash
Wend

Print "Liste 1 (...sorted !! ) :"
MaListe1.Root
While MaListe1.HashStep
    Print MaListe1.HashTag
Wend
Print "OK -------------"
sleep

MaListe1.fSortMethod(-1)
MaListe1.Sort
Print "WHOLE Liste 1 (...sorted Rev !! ) :"
MaListe1.Root
While MaListe1.HashStep
    Print MaListe1.HashTag
Wend
Print "OK -------------"
sleep

Print "Chronological parsing on List1 :"
MaListe1.Root
MaListe1.Track
While MaListe1.TrackStep
    Print MaListe1.HashTag
Wend
Print "Reduce+Restore + HoldBack Tracking - fixed"
Print
sleep

Print "Sorting on Tag1 :"
MaListe1.fSortMethod(1)
MaListe1.ColSort(1)
MaListe1.Sort

MaListe1.Root : MaListe1.RootNode
While MaListe1.HashStep
    Print MaListe1.HashTag & " Tag1= " & MaListe1.Tag(1)
Wend
Print "###############"

Print "Fin"
Sleep
Print "%?=" & AllocateDeallocateCounter & " nodecount=" & MaListe1.NodeCount
MaListe1.Destroy :MaListe2.Destroy : gCollector.Destroy
Print "%%=" & AllocateDeallocateCounter & " nodecount=" & MaListe1.NodeCount
sleep
system


Sort speed improvments (compared to previous sorting method)

Code: Select all

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MaListe As List ': MaListe.Root
MaListe.HashLen(1)
MaListe.SeekMethod(1)
MaListe.HashSort(1)
Dim str_tmp As String="ABCDEFGHIJKLMNOPKRSTUVWXYZ"
Dim i as integer : Dim t as integer : Dim k as string
Print time
For t=1 To 12
    Print "Clef ++"
    For i=2000000 To 1000000 Step -1
        MaListe.HashTag( str_tmp & str(i)) : MaListe.RwTag1("#" & str(i))  : MaListe.RwTag2("123456" & str(i)) : MaListe.RwTag3("123456" & str(i)) : MaListe.RwTag4("123456" & str(i))
    Next i           
    If str_tmp<>"" Then : Print "Oui" : Else : Print "Non" : End If
    Print "Sorting"
    If t=1 or t=2 Then
       Print "Sorting using whole rebuild "
        MaListe.Root
        MaListe.NFMethod(1)
        While MaListe.KeyStep
            MaListe.NodeFlat
        Wend
        MaListe.FlatStack
        While MaListe.fStep
     '       ? "* " & MaListe.HashTag
            MaListe.RestoreHash
        Wend
     '   MaListe.Up
    Else
        'MaListe.Root
        Print "Sorting using Sort property "
        MaListe.Sort       
    End If
    Print "Sorting done"
   
    Print "Garbage=" & MaListe.GarbageCount
    Print "Recycle.." & MaListe.Recycle
   ' Print  "Dealloues=" & MaListe.DropAll  & " nodecount=" & MaListe.NodeCount
  '  Print "Recycle : garbage=" & MaListe.GarbageCount & " container=" & MaListe.ContainerCount
    If str_tmp="" Then : str_tmp="ABCDEFGHIJKLMNOPKRSTUVWXYZ" : Else : str_tmp="" : End If
Next t
Print time
Print "Fin"
'MaListe.DropAll
Print "%?=" & AllocateDeallocateCounter & " nodecount=" & MaListe.NodeCount
MaListe.Destroy : gCollector.Destroy
Print "%%=" & AllocateDeallocateCounter & " nodecount=" & MaListe.NodeCount
sleep
system

Reindexing on other column

Code: Select all

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MaListe1 As List
Dim i As Integer

For i=110 to 122
    MaListe1.HashTag(str(i)) : MaListe1.RwTag1(str(i-100))
Next i

MaListe1.NFMethod(1)
MaListe1.Root
While MaListe1.HashStep
    MaListe1.NodeFlat
Wend
MaListe1.Recycle

Print "#"
MaListe1.RootNode
While MaListe1.HashStep
    Print MaListe1.HashTag & " - " & MaListe1.Tag(1)
Wend
Print "##"
sleep

MaListe1.ColTags(1)
MaListe1.RHmethod(1)
MaListe1.FlatStack
While MaListe1.fStep
    If MaListe1.Tag<>"" Then
        ? "Restoring to tree structure=>" & MaListe1.Tag(1)
        MaListe1.RestoreHash
    End If   
Wend

Print "*"
MaListe1.ColTags(0)
MaListe1.Root
While MaListe1.HashStep
    Print MaListe1.HashTag & " - " & MaListe1.Tag(0)  & " -" & MaListe1.Tag(1) & "-"
Wend
Print "fin"
MaListe1.Destroy : gCollector.Destroy
Print "%%=" & AllocateDeallocateCounter & " nodecount=" & MaListe1.NodeCount
sleep
system
Create an index with CopyCat (Copy Category), managing link between index and source with Follow, using tracking

Code: Select all

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"    
# ELSE
    Print "NEW release"' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MaListe As List
Dim MaListe2 As List
Dim i as integer

For i=11 to 28
    MaListe.HasHTag(Str(i))
    MaListe.RwTag1("ABC"+Str(i))
    If i=14 Or i=15 Or i=20 Or i=21 Then : MaListe.RwTag1("ABC13") : End If
    MaListe.Val("Long string accepted here.. "+Str(i))
Next i

MaListe.HasHTag("1") : MaListe.RwTag1("") ' 1 is not a key, so RwTag1("") neutralize HashTag implicit key setting. This way HashTag is used as a if it were a setcursor before copying to List2
'***********************
MaListe.ColTags(1) ' The index on list2 will be build from list's tag1 (not from tree hierarchy cascaded keys
MaListe.CopyCatMethod(0)
MaListe2.TrackMultiKeys(1) ' Is Doing a HolBack on each indexed key
MaListe.CopyCat(MaListe2) 'CopyCat is a special feature wich is copying a branch from a list to another while setting a tracking from List2 to source list : this tracking is used by SourceList.Follow(TargetList)
' It is possible to use destination list (List2) as an index on source list. It is also possible to build several indexes on a single source list.
' Optimize mem usage to requirements increasing or decreasing Constants RUP_COLS=1 and  MAX_COLS=5 depending on the number of indexes needed
MaListe.ColTags(0)

Print "MV auto tracking"
MaListe2.Track(1)

While MaListe2.TrackStep
  '   Print MaListe2.HashTag
     MaListe.Follow(MaListe2)
     Print "hashtag=" & MaListe.HashTag & " tag1=" & MaListe.Tag(1) & " value=" & MaListe.Val & " List2=" & MaListe2.HashTag
Wend
Print "ok?"

sleep

MaListe.HasHTag("2") : MaListe.RwTag1("TITI")
MaListe.ColTags(1)
MaListe.CopyCat(MaListe2)
MaListe.ColTags(0) ' Back to 'Normal' context

Print "List2 : "
MaListe2.Root
While MaListe2.HashStep
    Print "hashtag=" & MaListe2.HashTag
Wend
Print "---------------"
Sleep
Print
Print "List :"
MaListe.Root
While MaListe.HashStep
    Print "hashtag=" & MaListe.HashTag & " tag1=" & MaListe.Tag(1) & " value=" & MaListe.Val
Wend
Print "---------------"
sleep


MaListe2.HasHTag("ABC12")
MaListe.Follow(MaListe2)
Print "Following exemple :"
Print "Key=" & MaListe.HasHTag & " val=" & MaListe.Val
sleep
Print
Print "Follow in a loop"
MaListe2.fSortMethod(-1)
MaListe2.Sort
MaListe2.Root
While MaListe2.HashStep   
'While MaListe2.KeyStep     
    If MaListe.Follow(MaListe2) Then
        Print " Maliste hashTag= " & MaListe.HashTag  & " - Maliste2 hashTag= " & MaListe2.HashTag ' & " val=" & MaListe.Val
    Else
        Print "* " & MaListe2.HashTag
    End If
Wend
Print "---------------"

Print "MV auto tracking"
MaListe2.Track(1)
While MaListe2.TrackStep
  '   Print MaListe2.HashTag
     MaListe.Follow(MaListe2)
     Print "hashtag=" & MaListe.HashTag & " tag1=" & MaListe.Tag(1) & " value=" & MaListe.Val & " List2=" & MaListe2.HashTag
Wend
Print "?? ok?"
sleep
MaListe2.Root
MaListe.Destroy : MaListe2.Destroy : gCollector.Destroy
Print "??=" & AllocateDeallocateCounter & " nodecount=" & MaListe.NodeCount & " container=" & MaListe.ContainerCount  & " nodecount=" & MaListe2.NodeCount & " container=" & MaListe2.ContainerCount
Print "Fin"
sleep
system
Last edited by Lost Zergling on Apr 03, 2020 18:09, edited 34 times in total.
Lost Zergling
Posts: 534
Joined: Dec 02, 2011 22:51
Location: France

Re: LZLE Tutorial

Post by Lost Zergling »

-------------------- Advanced features --------------------

006 : Understanding the methods to reduce a Tree indexed list

Code: Select all

' TUTO 006 : Understanding the methods to reduce a Tree indexed list

' The NodeFlat property (MyList.NodeFlat) can be used stand alone but is designed to be used into a HashStep, KeyStep, HashStepRev, KeyStepRev loop
' There are three methods reducing an index (tree)  into a loop  :
'   - MyList.NFMethod(-1) : MyList.NodeFlat will send all dereferenced elements respecting the condition and having no remaining childs to dereferenced list
'   - MyList.NFMethod(0) :  MyList.NodeFlat will send all dereferenced elements respecting the condition and having no remaining childs to flat list
'   - MyList.NFMethod(0) :  MyList.NodeFlat will send all dereferenced elements respecting the condition to flat list including those wich have remaing childrens that do not respect the condition

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MaListe As List
dim str_tmp as string
Dim i as Integer
MaListe.Root
Print "Flat=" & MaListe.FlatCount
Print "Garbage=" & MaListe.GarbageCount

MaListe.HashLen(1)
MaListe.BranchCountDown(1)

MaListe.HashTag("99")
MaListe.HashTag("72") : Maliste.Val("-12*-")
MaListe.HashTag("78")
For i=720 To 728
    MaListe.HashTag(str(i)) : Maliste.Val("-" & str(i) & "-" )
Next i
MaListe.Root
MaListe.HashTag("7210")
MaListe.HashTag("7211")
MaListe.HashTag("7212")

MaListe.Root
While MaListe.HashStepRev=1
    Print MaListe.HashTag & " = " & MaListe.Val    & " BranchCount=" & MaListe.BranchCount
Wend
Print "List loaded ------------------------"
Print
sleep
MaListe.Root
Print "Does list contains 728 ? => " & MaListe.HasHashTag("728")
MaListe.AllOf
While MaListe.fStepRev=1
    Print "'hashTag'=" & MaListe.HashTag & " current tag=" & MaListe.Tag & " value=" & MaListe.Val   
Wend
Print "Flat reverse parsing on current branch"
Print
sleep

MaListe.Root
While MaListe.KeyStepRev=1
    Print "TestRev => " & MaListe.HashTag & " = " & MaListe.Val & " tag1=" & str_tmp & " BranchCount=" & MaListe.BranchCount  & " count=" & MaListe.Count ''
Wend
Print "Key reverse parsing on all tree"
Print
Sleep

' Test the 3 options and see results
'MaListe.NFmethod(0)
'MaListe.NFmethod(-1)
MaListe.NFmethod(1)  'Default
MaListe.Root
While MaListe.HashStep=1
    If Val( Right(MaListe.HashTag,1) )<6 Then       
        Print "Reducing to protected flat list = " & MaListe.HashTag & " = " & MaListe.Val
        MaListe.NodeFlat
    End If   
Wend
Print "With NFMethod=1, node parents are swapped to protected flat list"
Print "even if unswapped children still present, 72 is reduced" ' If 72 still have childrens, 72 stil exists but is changed to a keypath (not marked as containing values nor being a key), the original 72 & its associated values is send to protected flat list
Print "With NFMethod=0, node parents are swapped to protected flat list"
Print "Till all childrens are not reduced, => 72 is not reduced"
Print "With NFMethod=-1, node parents are swapped to protected dereferenced list"
Print "Till all childrens are not reduced, : => is not garbaged"
sleep

Print "Garbage Count= " & MaListe.GarbageCount
MaListe.RootNode
While MaListe.HashStep=1
    Print MaListe.HashTag & " = " & MaListe.Val & " tag="  & MaListe.Tag & " count=" & MaListe.Count
Wend
Print "Resulting list --------------------------"
Sleep

Print "nb nodes=" & MaListe.NodeCount
Print "*Liste 1 garb count=" & MaListe.GarbageCount
Dim MaListe2 As List
Print "*Liste 2 garb count=" & MaListe2.GarbageCount
MaListe2.GarbageSnatch(MaListe)
Print "Liste 2 garb count=" & MaListe2.GarbageCount
Print "*Liste 1 garb count=" & MaListe.GarbageCount
MaListe2.RootNode
Print  "## count=" & MaListe2.Count
While MaListe2.HashStep=1   
    Print MaListe2.HashTag & " tag=" & MaListe2.Tag & " = " & MaListe2.Val & " count=" & MaListe2.Count
Wend

MaListe.RootNode
Print  "#### count=" & MaListe.Count
While MaListe.HashStep=1   
    Print MaListe.HashTag & " tag=" & MaListe.Tag & " = " & MaListe.Val & " count=" & MaListe.Count
Wend
Print  "Deallocated=" & MaListe.DropAll & " nodeCount=" & MaListe.NodeCount
MaListe.Destroy : MaListe2.Destroy : gCollector.Destroy
Print "%%AllocateDeallocateCounter =" & AllocateDeallocateCounter
Print "Fin"
sleep
system
007 : Using Tracking (sequential parse) inside an indexed list

Code: Select all

' TUTO 007 : Using Tracking (sequential parse) inside an indexed list

' To understand the HoldBack & Track imagine a "virtual reduce" : the possibility parsing sequentially and directly elements in a Tree via TrackStep without having to parse all elements of the Tree.
' It is a flat list in an indexed list : it is an optimization option as well as a special feature usefull for some algorithms.
' Another interesting feature is the possibility using NodeFlat to reduce elements while parsing them via TrackStep
' Caution : do not attempt to reduce an element already reduced while parsing with TrackStep ! (will be corrected)

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"  
# ELSE
    Print "NEW release" ' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim MaListe As List
Dim i As Integer
For i=30 to 50
    MaListe.HashTag(str(i))
    If Right(str(i),1)="0" Or Right(str(i),1)="9" Then
 '   If str(i)<>"50" Then
        Print "* " & MaListe.HashTag ' &  "  = " & MaListe.NextNode
     MaListe.HoldBack
    End If   
Next i

MaListe.RootNode
While MaListe.HashStep=1
    Print "# " & MaListe.HashTag & " - " & MaListe.Val & " tag=" & MaListe.Tag '& " - " & MaListe.NextNode ' & " BranchCount " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count)
Wend

MaListe.NFmethod(1)
MaListe.Root
While MaListe.HashStepRev   
    If Right(MaListe.HashTag,1)="0" Or Right(MaListe.HashTag,1)="9"  Then '
   ' If Right(MaListe.HashTag,1)<>"0" And Right(MaListe.HashTag,1)<>"9"  Then '
  '  If MaListe.HashTag<>"50" Then   
       MaListe.NodeFlat
    End If   
Wend
Print "1------"
sleep
MaListe.RootNode
'While MaListe.fStep 'Rev
While MaListe.HashStep
    Print  "HashTag=" & MaListe.HashTag   & " tag="   & MaListe.Tag
Wend

sleep
MaListe.Track
'Print  "== " & MaListe.HashTag
Print  MaListe.HashTag  & " -- " & MaListe.Tag
Print "2------"
'MaListe.NFmethod(0)
While MaListe.TrackStep
    Print  MaListe.HashTag  & " - "      & MaListe.Tag
   MaListe.NodeFlat
Wend
Print "*" : sleep
MaListe.RootNode
While MaListe.HashStep
    Print MaListe.HashTag
Wend
Print MaListe.DropAll & " / " & MaListe.NodeCount
MaListe.Destroy : gCollector.Destroy
Print "%%=" & AllocateDeallocateCounter 
Print "Fin"
sleep
system
008 : Some aspects of the reduce algorithm problematic

Code: Select all

' TUTO 008 : Some aspects of the reduce algorithm problematic

' When we want to transform an indexed element (tree) into an unindexed element (chained list), there is the question of the deindexation method, which will depend on the result we want to obtain.
' If we are going through the tree, we want the deindexation to be "recursive" or more precisely as IT SEEMS TO us, ie when a branch no longer contains an element,
' the deindexing of the parent element (and so on) can be done automatically. EXCEPT that we also want the DEINDEX CONDITION to be evaluated on the parent node
' in the while wend loop as well  (otherwise we risk deindexing parents elements containing data that we would like to keep). 

' A] Therefore, in that case, the deindexing is not recursive but PSEUDO-recursive :
' it is emulated by the dynamic parse of the tree structure so as to allow the evaluation of the condition on each element traversed.

' B] On the other hand, if we want to deindex a CHOSEN element we may no want having to worry about the fact that parent elements could be keys (and not only parent elements),
' then in this case we can accept the fact of deindexing recursively: as the condition is true on the lowest element of the tree,
' it will be presumed true for every the parent elements in case of recovery of a level following the deindexing of all elements of the level beneath.

' By default, the first method will be applied when browsing the list from a loop, while the second method will be applied in case of TrackStep on elements previously selected by Holback ()

' So far, keep in mind :
'   1* In all cases except for Tracking : Use NodeFlat (reduce) in a loop OTHERWISE it will only impact current element (because of parents could be keys as well as deindex condition could be false or true)
'   2* In a tracking cession (TrackStep) with NodeFlat (reduce) will impact all parents nodes till they no longer have any childrens without taking into account 
'       nor the evaluation of the condition, nor the fact wether it is a key or not (this last point should be changed in future version).

'   1-* A picking reduce with a recursive behaviour stopping on first parent wich is key could be implemented but is not yet in this version.
'   2-* A non-recursive behaviour on tracking would lead to orphan branch. 

#Include once "D:\Basic\LZListsEngine.bi"

009 : Parse string to tree structure

Code: Select all

' Parse string to tree structure (not so trivial, even knowing the engine)
# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"    
# ELSE
    Print "NEW release"' 0.995
    #Include once "F:\Basic\LZLE_.bi"
# ENDIF

Dim Shared ub_MyChoice As uByte=1 ' Let's test 0 or 1 

Declare Function SetMyVirtualTree(Str_tmp As String, L_tmp As List) As Byte
Function SetMyVirtualTree(Str_tmp As String, L_tmp As List) As Byte
    Dim As Integer  t : Dim As String  Str_tmp2
    If Left(Str_tmp,1)="/" Then : Str_tmp=Right(Str_tmp, Len(Str_tmp)-1) : End If    
# IF (IsTEST=1)    
    Str_tmp2=Str_tmp 
    If L_tmp.HasKey(Str_tmp2)=1Then 
        Print "Double on : " & L_tmp.HashTag 
    End If
    L_tmp.HashTag(Str_tmp2)
# ELSE
    If L_tmp.HasKey(Str_tmp)=1 And ub_MyChoice=1 Then 
        Print "+Double on : " & L_tmp.HashTag 
    End If
    If L_tmp.HashTag(Str_tmp)=1 Then ' The new 0.995a version does not impact value passed byref (doesn't need Str_tmp2), thus returning 1 if "key" is found, 2 if hashtga(keypath) is found, otherwise it returns 0, so it can distinguish keys versus keypath
        Print "*Double on : " & L_tmp.HashTag ' Tips : doesn't work when ub_MyChoice=1 simply because MyList is multi-keys (HashKeyUnique=1), so already existing key would not have same meaning ! (because key detected returning true<>current)
    End If    
# ENDIF
    
    t=Instr(Str_tmp, "/")
    While t<>0       
        Str_tmp=Left(Str_tmp, Len(Str_tmp)-t)
        Str_tmp2=Str_tmp 
        If ub_MyChoice<>1 Then
            L_tmp.HashTag(Str_tmp2) ' An already existing key/HashTag could also be tested here ! (whenever needed)
        Else
            If L_tmp.HasHashTag(Str_tmp2)=0 Then : L_tmp.HashTag(Str_tmp2) : beep : sleep : End If ' This line indicates not to set sub tree KEY entries, otherwise we create duplicate branches in tree as soon as branches also are keys ! (just for fun, loop is useless)
        End If        ' (selective key duplication not supported : no "no key duplicate on branches" => a key could have been duplicated before a sub-tree is created) (see (*))
        t=Instr(Str_tmp, "/")
    Wend
    Return 1
End Function

Dim MyList As List 
Dim Str_Path() As String : Redim Str_Path(14)
Dim i As Integer

Str_Path(1)="/path1/path2/itemA"
Str_Path(2)="/path1/path2/itemB"
Str_Path(3)="/path3/itemC"
Str_Path(4)="/itemD"
Str_Path(5)="/path8/path7/path3/itemA"
Str_Path(6)="/path7/path3/itemB"
Str_Path(7)="/path4/path9/path7/itemC"
Str_Path(8)="/path6/itemD"
Str_Path(9)="/itemE"
Str_Path(10)="/path3/path5/path7/itemF"
Str_Path(11)="/path6/path3/itemG"
Str_Path(12)="/path3/itemC"
Str_Path(13)="/itemI"
Str_Path(14)="/path3/path3/itemJ"

 If ub_MyChoice=1 Then
    MyList.HashKeyUnique(0)  'Indicates we want multi-keys on hashlist, but this instruction BUT also autosets SeekMethod to 0, (for default consistency) (see (*))
    MyList.SeekMethod(1)  ' "hacking" => we need MultiKeys for the KEYS of the tree (itemC should appear several times) BUT same time we need to merge branches till they aren't keys also ,.. 
End If

For i=1 To 14 ' Setting Up the TREE structure
    SetMyVirtualTree(Str_Path(i), MyList)
Next i

MyList.Root ' Parsing the TREE structure using ascending/descending recursive-like path
While MyList.KeyStep
    Print MyList.HashTag    
Wend
sleep
?
? "MarkRoot ---------------------------"
MyList.Root ' Parsing the TREE structure using consecutive roots
While MyList.nKeyStep
    Print MyList.HashTag 
Wend
sleep

MyList.Destroy
gCollector.Destroy
Print AllocateDeallocateCounter ' The new 0.995a version has got 0 status
Sleep
Last edited by Lost Zergling on Apr 03, 2020 17:57, edited 4 times in total.
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: LZLE Tutorial

Post by Tourist Trap »

Hello,

Can you add some example of the usage of this nice tool? What is it for? To serve as a basis for a robust database or something like this? It is very elaborate so it's not obvious to know why it is so.

Thanks by advance and for all the effort.
Lost Zergling
Posts: 534
Joined: Dec 02, 2011 22:51
Location: France

Re: LZLE Tutorial

Post by Lost Zergling »

Hello,
I'm using it to check datas integrity in various ways and on variables volumetrics. I am trying to achieve an indexed FIFO buffer doing sliding "Group By" on volumetrics too important to be stored in RAM. Like an elastic GroupBy. Contributions for some ideas for new keywords or instruction set/methods for the language would either be a great hint. Usable state, look likes fast and stable enough for what it is doing, but still need some work on it. I am more comfortable in design than in pure coding. As I wanted to conceive as much as possible as an engine, it should be usable for various programs and algos.
Could be something like this :

Code: Select all

'Not functionnal, just to illustrate
..
MyList.NFMethod(-1)
Do
	Line Input #.. , Filestring
	If MyList.HashTag($Key)=0 Then
		MyList.HoldBack ' every new key of list is tracked
	End If
	
	'Code for controlling buffer size and trigger point
	If MyList.NodeCount>xxxxxxx or Free<xxxxxxx Then
		i=0
		MyList.Aside
		MyList.Track 'set track parsing
		While i<xxxx and MyList.TrackStep
			MyList.NodeFlat : i+=1
		Wend
		MyList.TrackSet ' Reset track start to current element
		MyList.Recover
	End If
Loop Until eof()

Lost Zergling
Posts: 534
Joined: Dec 02, 2011 22:51
Location: France

Re: LZLE Tutorial - In progress

Post by Lost Zergling »

Using Recursion, pseudo-recursion, de-referencing and de-indexing

Code: Select all

 #Include once "F:\Basic\LZLE_.bi"
/'-------------------------------------------------------------------------------------------------------------------------------------------- LZLE howto : de-referencing and/or de-indexing entries and/or keys --------------------------------------------------------------------------------------------------------------------------------------------
English version above
Fonctionnement et usage de la propriété "NFMethod", et de la propriété "NFRecursive"
Ces deux propriétés permettent le paramétrage du comportement de la propriété "NodeFlat" qui lance le dé référencement d'un élément de l'arbre, donc sa DESINDEXATION ainsi que les modalités de recyclage des adresses des clefs d'index en cascade

L'organisation des données est un arbre duquel chaque branche est une liste séquentielle
Chaque élément de cette liste peut à son tour être un un élément racine, point d'entrée vers une autre liste
Lorsque vous souhaitez dé référencer un élément de l'arbre, typiquement, deux cas doivent être distingués :
    1- Soit vous êtes dans une boucle "HashStep" ou "HashStepRev", auquel cas, chacun des éléments de l'arbre est supposé être passé en revue dans un ordre ou agencement connu du fait de l'itération ainsi spécifiée
    2- Soit vous êtes dans un contexte dans lequel l'élément suivant dans votre itération ne permet pas de décrire le parcours de l'arbre pour en assurer la cohérence suite au dé référencement de l'élément courant

1] Dans le premier cas, vous n'avez pas la nécéssité d'une récursion : suite au dé référencement d'un élément, le parcours ("HashStep" ou "HashStepRev") est impacté car "NodeFlat" positionne le prochain élément courant en émulant une récursion.
Ceci est possible car tous les éléments de l'arbre étant par définition parcourus, chaque élément est ainsi passé en revue : il n'y a pas en principe de risque d'élément "oublié" par les règles du dé référencement
De plus, d'autres avantages (ou pas) sont à noter : pas de nécéssité de code récursif, donc pas de sollicitation de la pile, et aussi, vous le développeur peut spécifier des règles de dé référencement spécifiques élément par élément.
Quand vous êtes dans une boucle "HashStep" ou "HashStepRev", le mode le plus fin et le plus modulable en terme de codage est de ne pas utiliser le mode récursif ( ie NFRecursive(1) ) pour le dé-référencement ("NodeFlat")
A cet effet, il suffit de spécifier : MyList.NFRecursive(0) : le moteur va prendre automatiquement en charge le parcours PSEUDO-récursif lors de chaque déréférencement d'un élément de l'arbre

Dans ce mode (pseudo récursion), vous disposez de trois options pour le déréférencement, ces options sont spécifiées par la propriété "NFMethod" :
NFMethod(-1) : indique que l'élément courant est envoyé vers le 'garbage collector' propre à l'instance de la liste dans laquelle l'élément est déréférencé, cet élément n'est plus utilisable et est automatiquement recyclé après le passage de "NodeFlat"
                        Si cet élément contient des enfants, il est marqué pour suppression. 
NFMethod(0) : indique que l'élément courant est envoyé vers une liste séquentielle 'protégée' et propre à l'instance de la liste dans laquelle l'élément indexé est déréférencé, si cet élément ne possède pas lui même des références descendantes (fils)
                        si cet élément possède des enfants, il est 'marqué comme candidat à la suppression logique' : cet élément ne sera désindexé vers la liste séquentielle protégée que lorsque tous les enfants auront étés eux mêmes désindéxés.
                        Le parcours 'pseudo-récursif' de la propriété "NodeFlat" prends en charge le fait que lorsqu'une sous arborescence est entièrement déréférencée, les éléments parents, éventuellements candidats à la suppression, sont réévalués, 
                        et automatiquement désindexés le cas échéant, vers la liste séquentielle 'protégée', le moment venu, c'est à dire, si et seulement si, toutes les dépendances enfants ont été désindexées ou déréférencées.
NFMethod(1) : indique que l'élément courant est envoyé vers une liste séquentielle 'protégée' et propre à l'instance de la liste dans laquelle l'élément indexé est déréférencé, dans TOUS les cas
                        si cet élément possède des dépendances (enfants), alors il est dans TOUS les cas remplacé dans l'arbre par un élément vide (disons un 'hashtag' dans la terminologie utilisée), lequel assure la continuité de la cohérence de la structure de l'index
                        Cet élément de remplacement, n'étant plus une clef, il est susceptible d'être automatiquement envoyé vers le 'garbage collector' propre à l'instance de la liste en cours dès lors qu'il n'aura plus de dépendances enfants
                        Tant qu'il y a des dépendances, le moteur de liste assimile ainsi les éléments qui ne sont pas des clefs aux éléments marqués pour le déréférencement logique en ce sens que ces éléments devront être déréférencés dès lors qu'il n'yaura plus de dépendances enfants
                        Ce mode va donner des résultats différents du précédent (NFMethod(0)) car les éléments qui contiennent encore des dépendances sont bien envoyés vers la liste séquentielle protégée (la 'Flat Stack'), même s'ils contiennent toujours des dépendances dans l'arbre,
                        lesquelles sont toujours accessibles, alors que dans le mode NFMethod(0), ces éléments sont toujours bien présents dans l'arbre et ne sont donc pas désindéxés. 
                        Dans tous les cas, le maintient de la structure de l'index est assuré, dans le mode NFMethod(0) ceci est réalisé par un parcours pseudo récursif décalé, alors que dans le second NFMethod(1), ceci est assuré par la création d'un élément vide venant 
                        assurer dans l'arbre la cohérence de la structure de l'index.

NFMethod(-1), NFMethod(0) et NFMethod(1) sont plus spécifiquement dédiés au mode NFRecursive(0), le mode de parcours en pseudo récursion.
L'avantage de ce mode de parcours de l'arbre est que la condition de désindexation ou de déréférencement peut être évaluée pour chaque élément de l'arbre, ce qui peut ainsi permettre, pour chaque élément, de choisir ou pas de désindexer, ainsi que les modalités.
Lorsque vous utilisez la propriété "NodeFlat" dans le parcours d'une liste indexée, le succès indique :
    - Pour NFMethod(-1) : l'envoi vers le  'garbage collector' propre à l'instance de la liste dans laquelle l'élément est déréférencé et recyclé, ET le parcours pseudo récursif pour évaluation des clefs intermédiaires dans la boucle HashStep
    - Pour NFMethod(0) : Soit l'envoi vers la liste séquentielle 'protégée', soit le marquage de l'élément comme 'virtuellement' désindexé, ET le parcours pseudo récursif pour évaluation des clefs intermédiaires dans la boucle HashStep
    - Pour NFMethod(1) : L'envoi systématique vers la liste séquentielle 'protégée', le remplacement si nécessaire par un élément vide dans l'index, ET le parcours pseudo récursif pour évaluation des clefs intermédiaires dans la boucle HashStep

2] Dans le second cas, il est nécessaire de prévoir un parcours récursif, car la boucle qui encapsule l'instruction "NodeFlat" ne décrit pas tous les éléments de l'arbre
L'inconvénient d'une récursion, c'est que la condition de désindexation ou de déréférencement ainsi que la méthode appropriée (NFMethod) est évaluée ou appliquée à l'identique lors de chacune des récursions
Par exemple, MyList.NFRecursive(1) suivie de MyList.NFMethod(-1) et de MyList.NodeFlat va envoyer vers le Garbage Collector l'élément courant ainsi que tous les parents de cet élément qui ne sont pas eux mêmes des clefs et n'ont plus non plus de liens enfants

'-------------------------------------------------------------------------------------------------------------------------------------------- LZLE howto : de-referencing and/or de-indexing entries and/or keys --------------------------------------------------------------------------------------------------------------------------------------------
Operation and use of the "NFMethod" property, and of the "NFRecursive" property
These two properties allow the configuration of the behavior of the "NodeFlat" property which launches the de-referencing of an element of the tree, therefore its UNINDEXING as well as the methods of recycling the addresses of the index keys in cascade.

The organization of data is a tree of which each branch is a sequential list
Each element of this list can in turn be a root element, entry point to another list
When you want to de-reference an element of the tree, typically, two cases must be distinguished:
    1- Either you are in a "HashStep" or "HashStepRev" loop, in which case, each of the elements of the tree is supposed to be reviewed in a known order or arrangement due to the iteration thus specified
    2- Either you are in a context in which the next element in your iteration does not allow you to describe the path of the tree to ensure its consistency following the de-referencing of the current element

1] In the first case, you do not need a recursion: following the de-referencing of an element, the browse ("HashStep" or "HashStepRev") is impacted because "NodeFlat" positions the next current element by emulating a recursion.
This is possible because all the elements of the tree being by definition scanned, each element is thus reviewed: in principle there is no risk of an element "forgotten" by the de-referencing rules.
In addition, other advantages (or not) are to be noted: no need for recursive code, therefore no solicitation of the stack, and also, you the developer can specify specific de-referencing rules element by element.
When you are in a "HashStep" or "HashStepRev" loop, the finest and most flexible mode in terms of coding is not to use recursive mode (ie NFRecursive (1)) for de-referencing ("NodeFlat ")
To do this, all you have to do is specify: MyList.NFRecursive (0): the engine will automatically take charge of the PSEUDO-recursive path during each dereference of an element in the tree

In this mode (pseudo-recursion), you have three options for dereferencing, these options are specified by the "NFMethod" property:
NFMethod (-1): indicates that the current element is sent to the 'garbage collector' specific to the instance of the list in which the element is dereferenced, this element is no longer usable and is automatically recycled after the passage by "NodeFlat"
                        If this item contains children, it is marked for deletion.
NFMethod (0): indicates that the current element is sent to a 'protected' sequential list specific to the instance of the list in which the indexed element is dereferenced, if this element does not itself have descendant references ( son)
                        if this element has children, it is 'marked as a candidate for logical deletion': this element will only be deindexed to the protected sequential list when all the children have been deindexed themselves.
                        The 'pseudo-recursive' path of the "NodeFlat" property takes care of the fact that when a sub-tree is entirely dereferenced, the parent elements, possibly candidates for deletion, are re-evaluated,
                        and automatically deindexed if necessary, to the 'protected' sequential list, when the time comes, that is to say, if and only if, all the child dependencies have been deindexed or dereferenced.
NFMethod (1): indicates that the current element is sent to a 'protected' sequential list specific to the instance of the list in which the indexed element is dereferenced, in ALL cases
                        if this element has dependencies (children), then it is in ALL cases replaced in the tree by an empty element (say a 'hashtag' in the terminology used), which ensures the continuity of the consistency of the structure of the 'index
                        As this replacement element is no longer a key, it is likely to be automatically sent to the 'garbage collector' specific to the instance of the current list as soon as it no longer has any child dependencies.
                        As long as there are dependencies, the list engine thus assimilates the elements which are not keys to the elements marked for logical dereferencing in the sense that these elements will have to be dereferenced as soon as there will be no more dependencies. children
                        This mode will give different results from the previous one (NFMethod (0)) because the elements which still contain dependencies are sent to the protected sequential list (the 'Flat Stack'), even if they still contain dependencies in the tree,
                        which are always accessible, whereas in the NFMethod (0) mode, these elements are still present in the tree and are therefore not deindexed.
                        In all cases, the maintenance of the structure of the index is ensured, in the NFMethod (0) mode this is achieved by a pseudo-recursive shifted path, while in the second NFMethod (1), this is ensured by the creation of an empty element coming
                        ensure the consistency of the index structure in the tree.

NFMethod (-1), NFMethod (0) and NFMethod (1) are more specifically dedicated to NFRecursive (0) mode, the pseudo-recursion browse mode.
The advantage of this tree browsing mode is that the deindexing or dereferencing condition can be evaluated for each element of the tree, which can thus allow, for each element, to choose or not to deindex, as well. as the terms.
When you use the "NodeFlat" property in browsing an indexed list, success indicates:
    - For NFMethod (-1): sending to the 'garbage collector' specific to the instance of the list in which the element is dereferenced and recycled, AND the pseudo-recursive scan for evaluation of intermediate keys in the HashStep loop
    - For NFMethod (0): Either the sending to the 'protected' sequential list, or the marking of the element as 'virtually' deindexed, AND the pseudo-recursive path for evaluation of the intermediate keys in the HashStep loop
    - For NFMethod (1): The systematic sending to the 'protected' sequential list, the replacement if necessary by an empty element in the index, AND the pseudo-recursive path for evaluation of the intermediate keys in the HashStep loop

2] In the second case, it is necessary to provide a recursive path, because the loop which encapsulates the "NodeFlat" instruction does not describe all the elements of the tree.
The disadvantage of a recursion is that the deindexing or dereferencing condition as well as the appropriate method (NFMethod) is evaluated or applied identically during each of the recursions.
For example, MyList.NFRecursive (1) followed by MyList.NFMethod (-1) and MyList.NodeFlat will send to the Garbage Collector the current element as well as all the parents of this element which are not themselves keys and no longer have any child links

'/

Dim MyList As List
Dim i As Integer

MyList.HashTag("1240") : MyList.Val("Value of 1240")
MyList.HashTag("1241") : MyList.Val("Value of 1241")
MyList.HashTag("1242") : MyList.Val("Value of 1242")
MyList.HashTag("1243") : MyList.Val("Value of 1243")
 MyList.HashTag("11") : MyList.Val("Value of 11")
 MyList.HashTag("124") : MyList.Val("Value of 124")  ' Check behaviour comment:uncomment
 MyList.HashTag("123") : MyList.Val("Value of 123")  ' Check behaviour comment:uncomment
MyList.HashTag("12") : MyList.Val("Value of 12")
MyList.HashTag("1") : MyList.Val("Value of 1")
 MyList.HashTag("110") : MyList.Val("Value of 110")

? " List Keys :"
MyList.Root
While MyList.KeyStep
    ? "Key=" &  MyList.HashTag & " value=" & MyList.Val
Wend
? "-----------------------------------1"


? " List Tree :"
MyList.Root
While MyList.HashStep
    ? "Key=" &  MyList.HashTag & " value=" & MyList.Val & " Is this tree entry a key ? " & MyList.Check
Wend
? "-----------------------------------2"

'/'
'MyList.NFRecursive(0)
MyList.NFRecursive(0)
'MyList.NFRecursive(2)
MyList.NFMethod(-1)  ' When NFRecursive=1, you should use NFMethod outside the hashstep loop, otherwise logical (not technical) behaviour may happend to be subject to inconsistency
MyList.Root
While MyList.HashStep
    If Val(MyList.HashTag)>1000  Then  ' And Val(MyList.HashTag)<>1242
        ? "Dereferencing to garbage : " & MyList.HashTag
        MyList.NodeFlat
    Else
        ? "Triggered : " & MyList.HashTag
    End If
Wend
'/

' MyList.NFMethod(1) : MyList.HashTag("1") : MyList.NodeFlat
?
? " Garbage=" & MyList.GarbageCount
? " New list Tree :"
MyList.Root
While MyList.HashStep
    ? "Key=" &  MyList.HashTag & " value=" & MyList.Val & " Is this tree entry a key ? " & MyList.Check
Wend
? "-----------------------------------3"
? "Please note that with NFRecursive(0) hashtag '124' is still present, whereas NFMethod(-1) will remove it"

/' It is almost possible to get same result (removing 'hashtag'  "124") with NFRecursive(0) using this syntax :
                        MyList.NFRecursive(0)
                        MyList.NFMethod(-1)
                        MyList.Root
                        While MyList.HashStep
                            If Val(MyList.HashTag)>1000 Then
                                ? "Dereferencing to garbage : " & MyList.HashTag
                                MyList.NodeFlat
                            ElseIf MyList.Check=0 Then ' If list entry (current element) is hashtag and not a key
                                ? "Dereferencing to garbage : " & MyList.HashTag
                                MyList.NodeFlat
                            End If
                        Wend
   ' Please note that using code exemple above, the message "Dereferencing to garbage : 124" appears twice (because of the functionning of pseudo recursive parsing), wheras  "Dereferencing to garbage : 124" does not appear at all using NodeFlat true recursive parse
   ' Please note that using pseudo recursive parse (NodeFlat+hashStep), user can manage dereferencing and deindexing at a very fine level, using any condition required. 
   ' Please note that using pseudo recursive parse, NFMethod(-1) is sending 
   ' Pseudo recursive parse ( mode NFRecursive(0) ) is NOT efficient outside a HashStep (hashStepRev) parse, so the code above using MyList.KEYStep instead of MyList.HashStep does not work (orphan useless hashtag "124" not removed
   ' But is working using MyList.NFRecursive(1) :
   ' the following code using KeyStep :
   
    MyList.NFRecursive(1)
    MyList.NFMethod(-1)  ' When NFRecursive=1, you should use NFMethod outside the hashstep loop, otherwise logical (not technical) behaviour may happend to be subject to inconsistency
    MyList.Root
    While MyList.KEYStep
        If Val(MyList.HashTag)>1000 Then
            ? "Dereferencing to garbage : " & MyList.HashTag
            MyList.NodeFlat
        End If
    Wend
   ' is giving same results (in this exemple, regarding dereferencing) than the one using HashStep, because recursive parse have taken precedence 
   ' Please note : in first exemple above (NFRecursive(1)+HashStep), "Dereferencing to garbage : 124" is effective but does not print because recursion is triggered on current node and take precedence on pseudo recursion wich is triggered on desindexation or dereferencing the last element of a branch
   
   'So far, problem is : when you are NOT in a HashStep Parse (KeyStep, TrackStep, or custom), pseudo recursive parsing is not available (or relevant), so you must use true recursion meaning no finess for conditionnal trigger in dereferencing nor deindexing
   'Additionnal NFMethods (2 , 3, 4) are available to cover some missing use cases only when using NFRecursive(1) :
   '     NFMethod(2) : sending current to flat list (deindexing) AND all parents references that are KEYS but NO LONGER parents TO FLAT LIST (deindexing), clearing useless entries (parents no longer parents & not keys), **** BUT leaving parents still having childs references unchanged
   '     NFMethod(3) : sending current to flat list (deindexing) and dereferencing all parents references that are NO LONGER parents NOR KEYS to garbage collector, so it's equivalent the code above (auto-dereferencing useless entries only) (NFRecursive(1), NFMethod(-1)), but saving to flat list (de-indexing) current (first) element instead de referencing it, **** BUT leaving parents still having childs references unchanged
   '     NFMethod(4) : sending current to current instance's garbage collector, sending all parents (keys & hashtags) that are no longer parents to garbage collector, **** BUT leaving parents still having childs references unchanged
   
   ' SO far, problem is : when you are on NFRecusive(1) mode, recursion is stopped on a parent node that still have childrens
   ' NFRecursive(2) is a new mode that forces recursion over parents elements
   ' Please note : prefer using NFRecursive(0) in combination with NFMethod(-1), NFMethod(0) and NFMethod(1) (designed for pseudo recursive parsing)
   '                      prefer using NFRecursive(1) and  NFRecursive(2)  in combination with NFMethod(2), NFMethod(3) or NFMethod(4)
   ' Check behaviour using some tests till NFRecursive>0, NFRecursive(1) and NFRecursive(2) are not designed specifically to be used with NFMethod -1, 0, and 1
   ' Please note that NFRecursive(1)+NFMethod(2) is the recursive equivalence to NFRecursive(0)+NFMethod(1)    
'/
If MyList.FlatCount>0 Then
    ? "Protected flat list status :" & "(" & MyList.FlatCount & ")"
    MyList.FlatStack
    While MyList.fStep
        ? "Key=" &  MyList.HashTag & " value=" & MyList.Val & " was this tree entry a key ? " & MyList.Check
    Wend
End If
Sleep
?
MyList.Destroy : gCollector.Destroy
? "Hello world " & AllocateDeallocateCounter 
sleep

Lost Zergling
Posts: 534
Joined: Dec 02, 2011 22:51
Location: France

Re: LZLE Tutorial - In progress

Post by Lost Zergling »

Using TrackStep & TrackStepRev

Code: Select all


  '  FIFO_LIFO.BAS
 #Include once "F:\Basic\LZLE_.bi"
Dim MyList As List
Dim As Integer i

For i=110 to 90 step -1
    MyList.HashTag(Str(i))
    If i<>107 and i<>108 Then
        MyList.HoldBack'Rev
    End if
Next i
 ' When you HashTag a new value, it will be marked as "key", so recursive NodeFlat on tracking will preserve keys
   MyList.HashTag("10")
 '  MyList.HashTag("11")

MyList.Root
While MyList.KeyStep
    ? MyList.HashTag
Wend
? "------------------------"
sleep ' Tracking starting point is set by default to first "HoldBack"

MyList.Track 'accessing track n°0 (default set by HoldBack) for parsing
While MyList.TrackStep
    ? MyList.HashTag
Wend

? "------------------------@"
sleep

MyList.Track
i=12
While MyList.TrackStep And i>0
    i-=1
    ? MyList.HashTag
Wend
? "------------------------*"
sleep

'MyList.NFRecursive(1)   '=> Implicit on tracking
' Check behaviour
MyList.NFMethod(2) ' NFMethod(1) is sending parent to non indexed, leaving a hashtag if childrens still referenced, NFMethod(2) is the recursive transposition  of pseudo recursive NFMethod(1)
'MyList.NFMethod(0) ' NFMethod(0) is sending parent only when last children is dereferenced from index (TRIE)  
MyList.Track
i=12
While MyList.TrackStep And i>0
    i-=1
    ? MyList.HashTag ' Not define after NodeFlat
    MyList.NodeFlat   'After a NodeFlat, list is ready to iterate depending on context
Wend
? "------------------------?"

? "New track starting point = " & MyList.HashTag 
MyList.TrackSet(3) 'Tracking starting point is now set to current on 'track' n°3
? MyList.HashTag & " ??"
sleep

MyList.HashTag("97")
MyList.NodeFlat 
MyList.HashTag("95")
MyList.NodeFlat 
MyList.HashTag("94")
MyList.NodeFlat 
MyList.HashTag("93")
MyList.NodeFlat 

MyList.Root
While MyList.HashStep
    ? MyList.HashTag  ': sleep
Wend
? "------------------------"
sleep

? "Garbage= " & MyList.GarbageCount
MyList.FlatStack
? "Flat count=" & MyList.FlatCount
While MyList.fStep
    ? MyList.Tag '& "  IsKey=" & MyList.IsKey
Wend
? "------------------------*$"
sleep

MyList.Track'(3)
i=19
While MyList.TrackStep 'And i>0
    i-=1
    ? MyList.HashTag & " *"
Wend
? "------------------------/"

MyList.Track(3) 'accessing track n°3 for parsing
i=19
While MyList.TrackStep And i>0
    i-=1
    ? MyList.HashTag & " *"
Wend
? "------------------------/"
? "TrackCompCounter=" & TrackCompCounter
sleep

Using CopyCat & CopyTrack

Code: Select all

 #Include once "F:\Basic\LZLE_.bi"

Dim MyList1 As List
Dim MyList2 As List
Dim i as Integer

For i=10 to 18 
    MyList1.HashTag(str(i))
    MyList1.RwTag1(" +" & str(i))
    MyList1.Val(" ==>" & str(i))
Next i
MyList1.HashTag("19")
MyList1.RwTag1("*>18")
MyList1.Val(" +=>19")
MyList1.HoldBack

For i=20 to 22 
    MyList1.HashTag(str(i))
    MyList1.RwTag1(" +" & str(i))
    MyList1.Val(" ==>" & str(i))
Next i

For i=12 to 16
    MyList1.HashTag(str(i))
    MyList1.RwTag1("*>" & str(i))
    MyList1.Val(" +=>" & str(i))
    MyList1.HoldBack
Next i
        
For i=30 to 38
    MyList1.HashTag(str(i))
    MyList1.RwTag1("*>" & str(i))
    MyList1.Val(" +=>" & str(i))
    MyList1.HoldBack
Next i

MyList1.Root
While MyList1.KeyStep
    ? MyList1.HashTag & " Tag1= " & MyList1.Tag(1) & " Val=" & MyList1.Val
Wend
? "---------------"
Sleep
MyList1.Track
While MyList1.TrackStep
    ? MyList1.HashTag & " Tag1=" & MyList1.Tag(1)
Wend
? "**---------------"
Sleep
 
MyList1.Root
MyList1.ColTags(1)
MyList1.CopyCatMethod(1)

MyList2.TrackMultiKeys(0)
MyList2.BranchCountDown(1)

sleep
MyList1.Track
MyList1.CopyTrack(MyList2)
MyList1.ColTags(0)

MyList2.Root
While MyList2.HashStep
    ? MyList2.HashTag & " Tag1= " & MyList2.Tag(1) & " Branch count=" & MyList2.BranchCount
Wend
? "---------------?"
Sleep
? "Fin"
Sleep
CopyCat exemple

Code: Select all

# Define IsTEST 0
# IF (IsTEST=1) 
    Print "Regular release" ' 0.994 
    #Include once "F:\Basic\LZLE_TEST.bi"    
# ELSE
    Print "NEW release"' 0.995
    #Include once "F:\Basic\LZLE_.bi"
 '   #Include once "F:\Basic\LZLE_20201228_Ok_.bi"
  '  #Include once "F:\Basic\LZLE_20201226.bi"
    
# ENDIF

Dim MaListe As List
Dim MaListe2 As List
Dim i as integer
 '   MaListe.SeekMethod(2)
 '   MaListe2.SeekMethod(2)

For i=11 to 28
    MaListe.HasHTag(Str(i))
    MaListe.RwTag1("ABC"+Str(i))
    If i=14 Or i=15 Or i=20 Or i=21 Then : MaListe.RwTag1("ABC13") : End If
    MaListe.Val("Long string accepted here.. "+Str(i))
Next i

MaListe.HasHTag("1") : MaListe.RwTag1("") ' 1 is not a key, so RwTag1("") neutralize HashTag implicit key setting. This way HashTag is used as a if it were a setcursor before copying to List2
'***********************
MaListe.ColTags(1) ' The index on list2 will be build from list's tag1 (not from tree hierarchy cascaded keys
MaListe.BranchCountDown(0)
MaListe.CopyCatMethod(0)
MaListe2.TrackMultiKeys(3)
 '   MaListe2.TrackTarget(2)
? "1"
MaListe.CopyCat(MaListe2) 'CopyCat is a special feature wich is copying a branch from a list to another while setting a tracking from List2 to source list : this tracking is used by SourceList.Follow(TargetList)
' It is possible to use destination list (List2) as an index on source list. It is also possible to build several indexes on a single source list.
' Optimize mem usage to requirements increasing or decreasing Constants RUP_COLS=1 and  MAX_COLS=5 depending on the number of indexes needed
? "2"
MaListe.ColTags(0)
? "3"
Print "MV auto tracking"
  '  MaListe2.Track(2)
    MaListe2.Track'(1)
? "%%%"
While MaListe2.TrackStep
  '   Print MaListe2.HashTag
     MaListe.Follow(MaListe2)
     Print "hashtag=" & MaListe.HashTag & " tag1=" & MaListe.Tag(1) & " value=" & MaListe.Val & " List2=" & MaListe2.HashTag
Wend
Print "ok?"

sleep

MaListe.HasHTag("2") : MaListe.RwTag1("TITI")
MaListe.ColTags(1)
MaListe.CopyCat(MaListe2)
MaListe.ColTags(0) ' Back to 'Normal' context

Print "List2 : "
MaListe2.Root
While MaListe2.HashStep
    Print "hashtag=" & MaListe2.HashTag
Wend
Print "---------------"
Sleep
Print
Print "List :"
MaListe.Root
While MaListe.HashStep
    Print "hashtag=" & MaListe.HashTag & " tag1=" & MaListe.Tag(1) & " value=" & MaListe.Val
Wend
Print "---------------"
sleep


MaListe2.HasHTag("ABC12")
MaListe.Follow(MaListe2)
Print "Following exemple :"
Print "Key=" & MaListe.HasHTag & " val=" & MaListe.Val
sleep
Print
Print "Follow in a loop"
MaListe2.fSortMethod(-1)
MaListe2.Sort
MaListe2.Root
While MaListe2.HashStep   
'While MaListe2.KeyStep     
    If MaListe.Follow(MaListe2) Then
        Print " Maliste hashTag= " & MaListe.HashTag  & " - Maliste2 hashTag= " & MaListe2.HashTag ' & " val=" & MaListe.Val
    Else
        Print "* " & MaListe2.HashTag
    End If
Wend
Print "---------------"

Print "MV auto tracking"
MaListe2.Track'(19)
While MaListe2.TrackStep
  '   Print MaListe2.HashTag
     MaListe.Follow(MaListe2)
     Print "hashtag=" & MaListe.HashTag & " tag1=" & MaListe.Tag(1) & " value=" & MaListe.Val & " List2=" & MaListe2.HashTag
Wend
Print "?? ok?"
sleep
MaListe2.Root
MaListe.Destroy : MaListe2.Destroy : gCollector.Destroy
Print "??=" & AllocateDeallocateCounter & " nodecount=" & MaListe.NodeCount & " container=" & MaListe.ContainerCount  & " nodecount=" & MaListe2.NodeCount & " container=" & MaListe2.ContainerCount
? "TrackCompCounter=" & TrackCompCounter
Print "Fin"
sleep
system
Lost Zergling
Posts: 534
Joined: Dec 02, 2011 22:51
Location: France

Re: LZLE Tutorial - In progress

Post by Lost Zergling »

Wip

Code: Select all

/'-------------------------------------------------------------------------------------------------------------------------------------------- LZLE howto : Take advantage of functional architecture --------------------------------------------------------------------------------------------------------------------------------------------

French version above
Each element of the list (key, hashtag, element of the garbage collector or of the 'flat list') contains two 'available' pointers, (therefore hard)
On a series of elements of the list, and connected by an appropriate algotithm, these two pointers form two 'vectors'

The tool offers 5 functionalities which require the use of a vector (a set of elements linked by pointers according to an algorithm:
    1 ° Tracking: it should be noted that HolBack and HolBackRev, like the track numbers (which are only specific points on the same vector) all use the same vector
    2 ° The automated count of the number of elements per branch (BranchCountDown)
    3 ° The link between distinct lists (CopyTrack / CopyCat) will use a vector on the target list, this vector is used by 'Follow' of the source list
    4 ° The 'TrackMultiKeys' function when using CopyTrack / CopyCat will use the second vector available on the target list (it is not possible to simultaneously perform TrackMultiKeys and BranchCountDown on the target list during a CopyTrack / CopyCat)
    5 ° The PVSMethod function aims to optimize the speed according to the structure of the data (produces a very random result in terms of performance gain because it adds weight, but may be suitable for certain datasets)

When you configure the use of a list in your code, you can therefore choose two features (among 5 or 4 depending on the case) (2 out of 5)
You cannot choose the same function twice (ex: two Tracking simultaneously)
You can combine the use of these different vectors as needed, but ensuring consistency (do not change the destination otherwise the pointers will no longer correspond to the rules of the algorithm that is supposed to manage them)
You can use different vector assignments on different lists
These assignments are made by the instructions BranchCountDown, CopyCatMethod, PVSmethod and TrackMultiKeys
The first vector can be used either for tracking or for the link between two lists (copytrack / copycat)
The second vector can be assigned to tracking (if the first is taken by copycat / copytrack / follow), or to counting, or to automated tracking of duplicate keys, or to the PVS algorithm.
This architecture with two (relatively) free vectors with assignment to choose (in addition to the basic operation), must allow most of the combinations commonly used by the programmer (80-20) and allow subsequent evolutions (addition of features of the set of 'instructions).

Taking a comparison with open source science-based C ++ code: https://root.cern/doc/master/classTHashList.html (i.e. a small sub-part of a much larger project)
The philosophy and the ambitions are not at all of the same order, however, and to better understand the logic and locate the targeted use:
    - Lzle offers only one data structure model, which is intended to be generic and not dedicated and optimized for different uses, therefore a single appropriate instruction set, very flexible to use, and no apparent constraints linked to the class hierarchy
    - The Lzle functions are very oriented towards direct use, so the construction of the algorithm does not offer the same access to elementary operations but aims to make it easier to get rid of explicit and implicit pointers
    - As a result, the handling and use of the tool are a priori simpler for a developer, compared to a scientific tool, and the algorithmic construction is closer to the RAD conception of a programmer than to that of a 'a mathematician.
    - Lzle uses its own specific and dedicated virtualization and persistence model, as opposed to the object model in general, which is standard, scalable, generic and conceptually powerful
    - Lzle's philosophy is not to hide the complexity of the memory management issue under a conceptual and functional layer, but to allow direct appropriation in as simplified a way as possible

'------------------------ French version
Chaque élément de la liste (clef, hashtag, élément du garbage collector ou de la 'flat list') contient deux pointeurs 'disponibles', (en dur donc)
Sur une série d'éléments de la liste, et reliés par un algotithme approprié, ces deux pointeurs forment deux 'vecteurs'

L'outil propose 5 fonctionnalités qui nécéssitent l'usage d'un vecteur (un ensemble d'élément reliés par des pointeurs en fonction d'un algorithme :
    1° Le Tracking : il convient de noter que HolBack et HolBackRev, comme les numéros de tracks (qui ne sont que des points ponctuels sur un même vecteur) utilisent tous le même vecteur
    2° Le décompte automatisé du nombre d'éléments par branche (BranchCountDown)
    3° Le lien entre des listes distinctes (CopyTrack/CopyCat) va utiliser un vecteur sur la liste cible, ce vecteur est utilisé par 'Follow' de la liste source
    4° La fonction 'TrackMultiKeys' lors de l'usage de CopyTrack/CopyCat va utiliser le second vecteur disponible sur la liste cible (il n'est pas possible de faire simultanément TrackMultiKeys et BranchCountDown sur la liste cible lors d'une opération CopyTrack/CopyCat)
    5° La fonction PVSMethod vise à optimiser la vitesse en fonction de la structure des données (produit un résultat très aléatoire en terme de gain de performance car ajoute de la lourdeur, mais susceptible de convenir à certains jeux de données)
Lorsque vous paramétrez l'usage d'une liste dans votre code, vous pouvez donc choisir deux fonctionnalités (parmi 5 ou 4 selon les cas) (2 out of 5)
Vous ne pouvez pas choisir deux fois la même fonction (ex : deux Tracking en simultané)
Vous pouvez combiner l'usage de ces différents vecteurs en fonction des besoins, mais en vous assurant de la cohérence (ne pas changer la destination sans quoi les pointeurs ne correspondront plus aux règles de l'algorithme qui est censé les gérer)
Vous pouvez utiliser des affectations de vecteurs différentes sur des listes différentes
Ces affectations s'effectuent par les instructions BranchCountDown, CopyCatMethod, PVSmethod et TrackMultiKeys
Le premier vecteur peut servir soit au tracking, soit au lien entre deux listes (copytrack/copycat)
Le second vecteur peut être affecté au tracking (si le premier est pris par copycat/copytrack/follow), ou bien au décompte, ou bien au tracking automatisé des clefs doublonnées, ou bien à l'algorithme PVS
Cette architecture à deux vecteurs (relativement) libres avec affectation à choisir (en plus du fonctionnement de base), doit permettre la plupart des combinaisons couramment utilisées par le programmeur (80-20) et autoriser des évolutions ultérieures (ajout de fonctionnalités du jeu d'instructions).

Prenant une comparaison avec du code C++ open source à vocation scientifique : https://root.cern/doc/master/classTHashList.html (soit une petite sous-partie d'un projet bien plus vaste)
La philosophie et les ambitions ne sont pas du tout du même ordre, néanmoins, et pour mieux appréhender la logique et situer l'usage ciblé :
    - Lzle ne propose qu'un seul modèle de structure de données, lequel se veut générique et non dédié et optimisé à différents usages, donc un unique jeu d'instruction approprié, très souple d'emploi, et pas de contraintes apparentes liées à la hiérarchie des classes
    - Les fonctions Lzle sont très orientés vers une utilisation directe, donc la construction de l'algorithme n'offre pas le même accès aux opération élémentaires mais vise la facilité à s'affranchir des pointeurs explicite et implicites
    - En résultat, la prise en main et l'utilisation de l'outil sont à priori plus simple pour un développeur, comparées à un outil scientifique, et la construction algorithmique est plus proche de la conception RAD d'un programmeur que de celle d'un mathématicien.
    - Lzle utilise sont propre modèle de virtualisation et de persistence, spécifique et dédié, par opposition au modèle objet en général, lequel est standard, évolutif, générique et puissant d'un point de vue conceptuel
    - La philosophie de Lzle est de ne pas cacher la complexité de la problématique de la gestion mémoire sous une couche conceptuelle et fonctionnelle, mais de permettre une appropriation directe de manière aussi simplifiée que possible
'/
/'
https://alibabatech.medium.com/gcc-vs-clang-llvm-an-in-depth-comparison-of-c-c-compilers-899ede2be378
"Additionally, standards of modern advanced languages constantly abstract the details of underlying hardware and data structures to generate general code that is more logical and mathematical, instead of specific operation instructions and memory access paths. C++ standards are increasingly expressive and abstract. Python is popular because it is more readable and expressive, even at the cost of a lower running speed. Higher expressiveness increases the burden of the compiler to generate good assembly code from the complex structures compiled by programmers. "
In Lzle, the memory management issue is approached in a "logical" way, with dedicated instructions (snatch, restorehash and so on) that can be specified upstream of the compilation process. The memory access path is transposed as an abstraction, which is supported by the programmer who can anticipate and know the substitutability of memory slots, and this substitutable characteristic becomes an integral part of the application design. Two distinct issues:
  * The balance between expressiveness and the level of precision, therefore of difficulty
  * The choice of positioning and distribution of instructions between the language and the compiler
'/
Last edited by Lost Zergling on Jan 05, 2021 11:42, edited 2 times in total.
Lost Zergling
Posts: 534
Joined: Dec 02, 2011 22:51
Location: France

Re: LZLE Tutorial - In progress

Post by Lost Zergling »

Wip
Post Reply