Next generation name is FB++ ?

[marcov]

There were really only two choices. Either compile it into a binary tree once and store the result to a file for fast loading (probably memory-mapped and usable directly), or compile it every time the help file is open.

Yes. Not memory mapped since the tables are stored compressed afaik (with a lz77 derived algorithm also used by .CAB though headers vary). But indeed directly searching without further operations after decompressing was one of the reasons. Speed of on the fly combining the results of multiple CHMs was probably one of the reasons too.
Even that can be cached in a CHW file even (mostly for IDEs with large SDKs installed), but since VS moved on, that is not used much anymore.

Given the hardware constraints of the time, I can understand why they chose the former. Now of course we would choose the latter every time, as that divorces the in-memory format from the disk format, which is desirable for portability and standardization reasons, as well as flexibility. Just like how we now use XML-based formats for MS Office.

The whole reason for XML was a threat to the acceptance of closed formats (old .doc) by governments. It was not a technical decision, or anything that even related to any daily reality of ordinary office users.

[jj2007]

The whole reason for XML was a threat to the acceptance of closed formats (old .doc) by governments. It was not a technical decision

Interesting, thanks. Googling a bit leads to Why OpenDocument Won (and Microsoft Office Open XML Didn’t).

[Lost Zergling]

@caseih you wrote

I just see a propensity for some on this list to roll their own ad-hoc algorithms when a standard algorithm is often simpler and faster.<...> Even if you're not a professional programmer, I still highly recommend this path.

I agree, but I'd like to add precision. Up to my opinion challenge is trying to

create our own syntax, original and adapted to the spirit of FB language. The syntax conditions the working method of the programmer while being the tool he uses: creating a set of instructions is also creating a tool and designing the working method (programming style) that goes with it

.
So far, this a political choice : the technical independence and the will to propose something new rather than tried algo even if reputed better. The risk of reinventing the wheel must be assumed in this case, in particular by limiting the liability of the license to the scope of application in which the need has been identified.

[Imortis]

...I just see a propensity for some on this list to roll their own ad-hoc algorithms when a standard algorithm is often simpler and faster. Even If one is a low-level enthusiast then using existing such things should be appealing because they are so efficient. On the other hand if one just wants to get the job done, such things would be even more appealing because the hard work is already done. Besides all that, it's really fun to see how these things work...

I try to do both. I like to understand what my code is doing and the best way for me to do that is to work it out myself. Finding tried and true algos after the fact makes me more likely to understand what is happening.

For me, the hard work is the fun part. I like working out the puzzle on my own

[marcov]

The first version, which worked well for me for many years, is starting to show scaling problems as directory tree it synchronized has gone 10,000 files to 200,000 files.

A few 100.000 is a typical limit for ordered arrays. If you insert an item, you typically have to move N/2 bytes to make room.

When I was a junior programmer (my first job) I also had this problem, but I also had some reason to be careful with memory (I had to load about 2 000 000 objects into memory).

I solved it with a simple indirection. IOW array of structs, with a dynamic array (a ptr) in every struct. Thus I had a fairly constant number of items to move on insert. Lookup suffered (it is not a hash), but using a visitor to build ordered sub-lists was very fast ( a kind of dataset query system).

As said lately, due to cross compilation unit inline functions and generics there was a boom in FPC container type. A while back sb compared all available container types for Free Pascal, might give you some ideas:

* the site: http://www.benibela.de/fpc-map-benchmark_en.html
* the corresponding forum thread: http://forum.lazarus-ide.org/index.php/ ... #msg225877

[caseih]

The whole reason for XML was a threat to the acceptance of closed formats (old .doc) by governments. It was not a technical decision, or anything that even related to any daily reality of ordinary office users.

Yes that was a factor. But today with MS broadening itself out to other platforms including Android, the old in-memory format (based on the in-memory OLE structures) just don't make sense from a technical point of view. MS Office's file format was originally essentially a hack that later had to, of necessity, become a psuedo specification. Office on Mac never quite interpreted it the same as Office on Windows and in fact between versions of Office on Windows there were often little incompatibilities when it came to loading files.

[marcov]

The whole reason for XML was a threat to the acceptance of closed formats (old .doc) by governments. It was not a technical decision, or anything that even related to any daily reality of ordinary office users.

Yes that was a factor. But today with MS broadening itself out to other platforms including Android, the old in-memory format (based on the in-memory OLE structures) just don't make sense from a technical point of view. MS Office's file format was originally essentially a hack that later had to, of necessity, become a psuedo specification. Office on Mac never quite interpreted it the same as Office on Windows and in fact between versions of Office on Windows there were often little incompatibilities when it came to loading files.

Yes, but that is because it was simply internally an unportable old codebase (the interpretation of the .doc format), so they couldn't run the codebase on mac. Has nothing to do with the format being text or binary.

[coderJeff]

Going back to the first few posts of this topic... and some comments from marcov, I think there are some parallels between FB and FP.

On one end of the spectrum, in comparison to asm/C, the focus is on the machine (CPU), and up to the programmer to express telling the machine what to do. And on the other end of the spectrum, in comparison to C++/Java, the focus is on the problem, and up to the programmer to express telling the compiler what problem to solve.

My opinion, because I am not a strong C++/java programmer, FreeBASIC appears to span the spectrum, allowing both low level machine control and high level expression of problems. Generics/templates, which FB does not have, and which I have no experience with, seem to try and do both, expressing a solution in terms of the problem, but let the underlying generic/template give the best implementation for the machine.

So when working with data structures, it seems we are currently at odds, the two ends of the spectrum, trying to optimize machine instructions, at the expense of clearly expressing the solution. Some of the both worst & best code I have read and worked with was with the Quake sources (from 20 years ago). The tools to build a level were not complicated, and not fast; just brute force to cache data needed to go in the final level data structure. However, the game code was absolutely minimal and fast, though it put all the burden on the programmer to use it correctly.