Speed of FreeBasic

For other topics related to the FreeBASIC project or its community.
marcov
Posts: 2646
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Speed of FreeBasic

Postby marcov » Apr 30, 2018 8:41

Ackerman mostly measures speed of recursion without sideeffects (like walking a tree in memory). How often do you use that in realworld programs?
badidea
Posts: 1016
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: Speed of FreeBasic

Postby badidea » Apr 30, 2018 20:12

Boris the Old wrote:Trying to optimize a program is like hunting for unicorns -- it seems like a good idea but the results are always disappointing.

For my first attempt at a 'pentominoes' puzzle solver, I estimated it to find all solutions in 2 years on a pentium 100 PC. My last version, after various optimizations, took only 15 seconds for this (on an somewhat faster PC as well). https://www.freebasic.net/forum/viewtopic.php?t=15560

Boris the Old wrote:There's no significant difference in programming languages, since all code ultimately exists only as machine code. And since most applications spend 90% or more of their time executing libraries in multitasking operating systems, there's little to be gained from counting machine cycles.

A once converted a PHP program from a friend, that analysed daily firewall-logs, to a C-version. The PHP program needed at a certain moment more 24 hours to process the daily logs, which was a bit of a problem. The C-version took 2 minutes for the same job. The main performance again was the change in file reading and the use of (semi-) binary trees and pointer fun. But it also used only a tenth of the memory and that was mainly because variables seemed take much more memory in PHP then in C (or Freebasic).

Boris the Old wrote:Our energies are best spent writing maintainable, and bug free, code that produces the results that the user wants, and to do it in a way that makes the best use of the language's features.

In general, I agree. But if a program takes more then 3 seconds to start, I start looking for alternatives. If an OS takes more then 30 seconds to start, I install an alternative. I also do various physics simulation (e.g. finite element). There performance optimization can also mean the difference between waiting 10 minutes for each run or only 1 minute.
jj2007
Posts: 897
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Speed of FreeBasic

Postby jj2007 » May 01, 2018 1:48

badidea wrote:The PHP program needed at a certain moment more 24 hours to process the daily logs, which was a bit of a problem. The C-version took 2 minutes for the same job. The main performance again was the change in file reading and the use of (semi-) binary trees and pointer fun. But it also used only a tenth of the memory and that was mainly because variables seemed take much more memory in PHP then in C (or Freebasic).
Speed and memory are often related: If a program is so badly designed that it eats all available physical memory, the OS starts paging out unused memory - and writing to disk is slow (it also kills your harddisk in the long run).
caseih
Posts: 1300
Joined: Feb 26, 2007 5:32

Re: Speed of FreeBasic

Postby caseih » May 01, 2018 2:33

badidea wrote:A once converted a PHP program from a friend, that analysed daily firewall-logs, to a C-version. The PHP program needed at a certain moment more 24 hours to process the daily logs, which was a bit of a problem. The C-version took 2 minutes for the same job. The main performance again was the change in file reading and the use of (semi-) binary trees and pointer fun. But it also used only a tenth of the memory and that was mainly because variables seemed take much more memory in PHP then in C (or Freebasic).

But the thing is, processing logs is not usually CPU intensive; it's I/O-bound. So the language should be nearly irrelevant for this task. Even an interpreted language should be able to parse and analyze logs nearly as fast as C. After all that's what languages like Perl were invented for. I'll admit that many of the text-processing abilities built into Perl (and Python for that matter) are actually written in C. Either way, the cost of reading the data into memory should dwarf the cost of analyzing that data, even in a scripting language.

But my point is the problem wasn't PHP; it was his program. I mean no disrespect for your friend, but if the PHP version took 24 hours to do what your C program took 2 minutes to do, he was doing something wrong. Text processing in PHP is actually quite fast. Might not be 2 minutes fast, but I could believe a proper optimization of the script could get it under 20 minutes, if not under 10, which for a systems environment is totally acceptable, and also portable and maintainable, moreso than a compiled binary. For simple searching, Python can be as fast as grep, which is pretty impressive. I don't think there's anything that could beat grep; it's about as optimized as you can get.

EDIT: I missed the part where you confirmed what I just said: "The main performance gain was the change in file reading and the use of (semi-) binary trees and pointer fun." Proper data structures are just as important in an interpreted language as it is in a compiled language. Even without pointers and memory, you can still implement binary trees and other similar structures. In fact data structures probably ought to be one of the first things you look at when analyzing a slow program. I don't blame you for converting a program from a language you didn't know well and didn't like to one that you did know well, and had great skill in. I see this happen repeatedly through generations of system programmers at various organizations.

Full disclosure: I strongly dislike PHP and don't consider it an appropriate systems language, let alone for web pages(!), but it does work for that purpose and I have used it in the past.
jj2007
Posts: 897
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Speed of FreeBasic

Postby jj2007 » May 01, 2018 2:46

caseih wrote:But the thing is, processing logs is not usually CPU intensive; it's I/O-bound. So the language should be nearly irrelevant for this task.
Well, nearly, yes. But if the language needs more memory for its variables, as badidea writes above, and if that makes the difference between
a) having all data in physical memory or
b) the OS being forced to page out unused memory,

then you can see indeed a dramatic slowdown. It can indeed, as in that php example, reach a factor 700.

Btw I see that every day with my browser: When its memory use goes beyond one GB, it gets very, very slow. I have 4GB, looks like a lot, but the OS and other applications also want their share. So closing some pages or restarting is the only solution.
caseih
Posts: 1300
Joined: Feb 26, 2007 5:32

Re: Speed of FreeBasic

Postby caseih » May 01, 2018 3:40

I'm aware of how virtual memory works, and I know what thrashing is. There can be many reasons why the PHP program was so slow. Consuming virtual memory causing thrashing is but one possibility, but in this case I just don't think it was a factor. badidea said nothing about a general system slowdown when the script was running. He only said it took too long to finish. If it was thrashing, the entire server would be impacted. It would be rather obvious.
marcov
Posts: 2646
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Speed of FreeBasic

Postby marcov » May 01, 2018 8:11

caseih wrote:
But the thing is, processing logs is not usually CPU intensive; it's I/O-bound. .


Actually, when doing normal textfile cooked I/O you touch all data multiple times. Once to scan for lineendings, than a move from buffer to string, and only then your string operations.

The efficiency and avoidance of bufferbloat of the RTS implementation can be quite noticable when processing logs. Specially now that SSDs have sped up I/O expectations.
deltarho[1859]
Posts: 1651
Joined: Jan 02, 2017 0:34
Location: UK

Re: Speed of FreeBasic

Postby deltarho[1859] » May 01, 2018 12:40

jj2007 wrote:When its memory use goes beyond one GB, it gets very, very slow.

I have two tabs in use at the moment - this forum and Google News and am using 219.7MB. I am not a 'power' user so you will not catch me with a lot of tabs open.

If you haven't already, have a look at FlashPeak SlimJet - it started life as Slim Browser, before I started to use it. It has a lot of stuff built in such as an ad blocker so there is no need to have a pile of extensions/plugins. I am not an extensions 'power'user either and only have four including ZenMate which encrypts browsing activity.

Towards the bottom of the home page is a comparison with IE, Chrome and Firefox and it has a lot of green ticks compared with the others. Well, it would wouldn't it but, for the most part, the features compared are useful ones.

It is highly customize-able and there is a memory management feature to “Automatically optimize memory usage”.

I have been using it for, don't know when, but quite a while now.

Added: I am also using the 1.1.1.1 DNS directory which is currently leaving everybody standing.
jj2007
Posts: 897
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Speed of FreeBasic

Postby jj2007 » May 01, 2018 14:41

deltarho[1859] wrote:have a look at FlashPeak SlimJet
A bit OT but yes, it looks good, thanks. I am using Pale Moon nowadays because it needs less memory than Firefox. Still too much, though.
deltarho[1859]
Posts: 1651
Joined: Jan 02, 2017 0:34
Location: UK

Re: Speed of FreeBasic

Postby deltarho[1859] » May 01, 2018 15:34

jj2007 wrote:because it needs less memory than Firefox

We could replace Pale Moon with any browser and the above statement would remain true.<smile>
marcov
Posts: 2646
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Speed of FreeBasic

Postby marcov » May 24, 2018 16:54

Strange. FreePascal doesn't do anything for recursion. I would expect FB's gcc backend to run circles around it.
zxretrosoft
Posts: 21
Joined: Apr 23, 2013 19:12
Contact:

Re: Speed of FreeBasic

Postby zxretrosoft » Jun 15, 2018 8:27

Hi friends,
You might be interested.
I made a simple algorithm, intentionally brute force with CASE.

The task is: The algorithm must find the order on which the LWK3724 code is located. Start numbering AAA0000, AAA0001, AAA0002... etc.
The alphabet is classic. A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z.

The correct answer is: 80183725.

I did exactly the same algorithm in 24 languages. Here are the results, for what time (in seconds) each language found this number on the same PC.

Looks like Freebasic's doing great!

P.S. It's not a complete test of languages. This is a solution to one particular example. Intentionally "example of life", not a textbook example of recursion...

Image

Here is the code in FB:

Code: Select all

dim shared as double p
dim shared as single x=-1
dim shared as single a1,a2,a3
dim shared as string ps1,ps2,ps3
dim shared as string h1,h2,h3
dim shared as single h

h1="L"
h2="W"
h3="K"
h=3724


do

x+=1
p+=1

if x>9999 then
    x=0
    a3+=1
    if a3>25 then
        a3=0
        a2+=1
        if a2>25 then
            a2=0
            a1+=1
        end if
    end if
end if

select case a1
    case 0
        ps1="A"
    case 1
        ps1="B"
    case 2
        ps1="C"
    case 3
        ps1="D"
    case 4
        ps1="E"
    case 5
        ps1="F"
    case 6
        ps1="G"
    case 7
        ps1="H"
    case 8
        ps1="I"
    case 9
        ps1="J"
    case 10
        ps1="K"
    case 11
        ps1="L"
    case 12
        ps1="M"
    case 13
        ps1="N"
    case 14
        ps1="O"
    case 15
        ps1="P"
    case 16
        ps1="Q"
    case 17
        ps1="R"
    case 18
        ps1="S"
    case 19
        ps1="T"
    case 20
        ps1="U"
    case 21
        ps1="V"
    case 22
        ps1="W"
    case 23
        ps1="X"
    case 24
        ps1="Y"
    case 25
        ps1="Z"
end select

select case a2
    case 0
        ps2="A"
    case 1
        ps2="B"
    case 2
        ps2="C"
    case 3
        ps2="D"
    case 4
        ps2="E"
    case 5
        ps2="F"
    case 6
        ps2="G"
    case 7
        ps2="H"
    case 8
        ps2="I"
    case 9
        ps2="J"
    case 10
        ps2="K"
    case 11
        ps2="L"
    case 12
        ps2="M"
    case 13
        ps2="N"
    case 14
        ps2="O"
    case 15
        ps2="P"
    case 16
        ps2="Q"
    case 17
        ps2="R"
    case 18
        ps2="S"
    case 19
        ps2="T"
    case 20
        ps2="U"
    case 21
        ps2="V"
    case 22
        ps2="W"
    case 23
        ps2="X"
    case 24
        ps2="Y"
    case 25
        ps2="Z"
end select

select case a3
    case 0
        ps3="A"
    case 1
        ps3="B"
    case 2
        ps3="C"
    case 3
        ps3="D"
    case 4
        ps3="E"
    case 5
        ps3="F"
    case 6
        ps3="G"
    case 7
        ps3="H"
    case 8
        ps3="I"
    case 9
        ps3="J"
    case 10
        ps3="K"
    case 11
        ps3="L"
    case 12
        ps3="M"
    case 13
        ps3="N"
    case 14
        ps3="O"
    case 15
        ps3="P"
    case 16
        ps3="Q"
    case 17
        ps3="R"
    case 18
        ps3="S"
    case 19
        ps3="T"
    case 20
        ps3="U"
    case 21
        ps3="V"
    case 22
        ps3="W"
    case 23
        ps3="X"
    case 24
        ps3="Y"
    case 25
        ps3="Z"
end select

loop until h1=ps1 and h2=ps2 and h3=ps3 and h=x

locate 3,1:print ps1
locate 3,2:print ps2
locate 3,3:print ps3
locate 3,4:print x
locate 1,1:print "Location: ";p
sleep
end
Last edited by zxretrosoft on Jun 20, 2018 12:16, edited 6 times in total.
marcov
Posts: 2646
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Speed of FreeBasic

Postby marcov » Jun 15, 2018 9:09

caseih wrote:
badidea wrote:A once converted a PHP program from a friend, that analysed daily firewall-logs, to a C-version. The PHP program needed at a certain moment more 24 hours to process the daily logs, which was a bit of a problem. The C-version took 2 minutes for the same job. The main performance again was the change in file reading and the use of (semi-) binary trees and pointer fun. But it also used only a tenth of the memory and that was mainly because variables seemed take much more memory in PHP then in C (or Freebasic).

But the thing is, processing logs is not usually CPU intensive; it's I/O-bound. So the language should be nearly irrelevant for this task. Even an interpreted language should be able to parse and analyze logs nearly as fast as C. After all that's what languages like Perl were invented for. I'll admit that many of the text-processing abilities built into Perl (and Python for that matter) are actually written in C. Either way, the cost of reading the data into memory should dwarf the cost of analyzing that data, even in a scripting language.


This is incorrect. logfile processing is not just moving the data, but also processing it, which tends to be slow, unless the whole operation is nearly entirely expressible in a native routine (like a simple regex). As soon as you process and start datastructures, you find the nature of the beast.

Personally I wonder about Calibre (A python program) slowliness every day. First

order Pascal programs run circles around it when processing epubs, and I then use native Pascal helper routines (zip and xml), which should be slower than the corresponding C libs that Python probably uses.

But my point is the problem wasn't PHP; it was his program. I mean no disrespect for your friend, but if the PHP version took 24 hours to do what your C program took 2 minutes to do, he was doing something wrong. Text processing in PHP is actually quite fast. Might not be 2 minutes fast, but I could believe a proper optimization of the script could get it under 20 minutes, if not under 10, which for a systems environment is totally acceptable, and also portable and maintainable, moreso than a compiled binary.
For simple searching, Python can be as fast as grep, which is pretty impressive. I don't think there's anything that could beat grep; it's about as optimized as you can get.


Hmm, maybe we do agree, that is more the order I expect too. 10 times slow on a good day, 25-50 times on a bad day, 2-5 times in case where scripting is carefully optimized (and native probably not, it rarely is worth the bother) or very simple
jj2007
Posts: 897
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Speed of FreeBasic

Postby jj2007 » Jun 15, 2018 10:48

zxretrosoft wrote:Here is the code in FB:

Code: Select all

Location:  80183725
LWK 3724
30 seconds on a Core i5, compiled with GCC.

You show that SpiderBasic is a factor 4 faster than C++. Suspicious... it probably means that the design of the program is not optimised.
fxm
Posts: 8424
Joined: Apr 22, 2009 12:46
Location: Paris (suburbs), FRANCE

Re: Speed of FreeBasic

Postby fxm » Jun 15, 2018 11:07

Core I7, gas : 15 seconds.
Core i7, gcc (64-bit) : 8 seconds.

Return to “Community Discussion”

Who is online

Users browsing this forum: No registered users and 1 guest