Speed of FreeBasic
Re: Speed of FreeBasic
Ackerman mostly measures speed of recursion without sideeffects (like walking a tree in memory). How often do you use that in realworld programs?
Re: Speed of FreeBasic
For my first attempt at a 'pentominoes' puzzle solver, I estimated it to find all solutions in 2 years on a pentium 100 PC. My last version, after various optimizations, took only 15 seconds for this (on an somewhat faster PC as well). viewtopic.php?t=15560Boris the Old wrote:Trying to optimize a program is like hunting for unicorns -- it seems like a good idea but the results are always disappointing.
A once converted a PHP program from a friend, that analysed daily firewall-logs, to a C-version. The PHP program needed at a certain moment more 24 hours to process the daily logs, which was a bit of a problem. The C-version took 2 minutes for the same job. The main performance again was the change in file reading and the use of (semi-) binary trees and pointer fun. But it also used only a tenth of the memory and that was mainly because variables seemed take much more memory in PHP then in C (or Freebasic).Boris the Old wrote:There's no significant difference in programming languages, since all code ultimately exists only as machine code. And since most applications spend 90% or more of their time executing libraries in multitasking operating systems, there's little to be gained from counting machine cycles.
In general, I agree. But if a program takes more then 3 seconds to start, I start looking for alternatives. If an OS takes more then 30 seconds to start, I install an alternative. I also do various physics simulation (e.g. finite element). There performance optimization can also mean the difference between waiting 10 minutes for each run or only 1 minute.Boris the Old wrote:Our energies are best spent writing maintainable, and bug free, code that produces the results that the user wants, and to do it in a way that makes the best use of the language's features.
Re: Speed of FreeBasic
Speed and memory are often related: If a program is so badly designed that it eats all available physical memory, the OS starts paging out unused memory - and writing to disk is slow (it also kills your harddisk in the long run).badidea wrote:The PHP program needed at a certain moment more 24 hours to process the daily logs, which was a bit of a problem. The C-version took 2 minutes for the same job. The main performance again was the change in file reading and the use of (semi-) binary trees and pointer fun. But it also used only a tenth of the memory and that was mainly because variables seemed take much more memory in PHP then in C (or Freebasic).
Re: Speed of FreeBasic
But the thing is, processing logs is not usually CPU intensive; it's I/O-bound. So the language should be nearly irrelevant for this task. Even an interpreted language should be able to parse and analyze logs nearly as fast as C. After all that's what languages like Perl were invented for. I'll admit that many of the text-processing abilities built into Perl (and Python for that matter) are actually written in C. Either way, the cost of reading the data into memory should dwarf the cost of analyzing that data, even in a scripting language.badidea wrote:A once converted a PHP program from a friend, that analysed daily firewall-logs, to a C-version. The PHP program needed at a certain moment more 24 hours to process the daily logs, which was a bit of a problem. The C-version took 2 minutes for the same job. The main performance again was the change in file reading and the use of (semi-) binary trees and pointer fun. But it also used only a tenth of the memory and that was mainly because variables seemed take much more memory in PHP then in C (or Freebasic).
But my point is the problem wasn't PHP; it was his program. I mean no disrespect for your friend, but if the PHP version took 24 hours to do what your C program took 2 minutes to do, he was doing something wrong. Text processing in PHP is actually quite fast. Might not be 2 minutes fast, but I could believe a proper optimization of the script could get it under 20 minutes, if not under 10, which for a systems environment is totally acceptable, and also portable and maintainable, moreso than a compiled binary. For simple searching, Python can be as fast as grep, which is pretty impressive. I don't think there's anything that could beat grep; it's about as optimized as you can get.
EDIT: I missed the part where you confirmed what I just said: "The main performance gain was the change in file reading and the use of (semi-) binary trees and pointer fun." Proper data structures are just as important in an interpreted language as it is in a compiled language. Even without pointers and memory, you can still implement binary trees and other similar structures. In fact data structures probably ought to be one of the first things you look at when analyzing a slow program. I don't blame you for converting a program from a language you didn't know well and didn't like to one that you did know well, and had great skill in. I see this happen repeatedly through generations of system programmers at various organizations.
Full disclosure: I strongly dislike PHP and don't consider it an appropriate systems language, let alone for web pages(!), but it does work for that purpose and I have used it in the past.
Re: Speed of FreeBasic
Well, nearly, yes. But if the language needs more memory for its variables, as badidea writes above, and if that makes the difference betweencaseih wrote:But the thing is, processing logs is not usually CPU intensive; it's I/O-bound. So the language should be nearly irrelevant for this task.
a) having all data in physical memory or
b) the OS being forced to page out unused memory,
then you can see indeed a dramatic slowdown. It can indeed, as in that php example, reach a factor 700.
Btw I see that every day with my browser: When its memory use goes beyond one GB, it gets very, very slow. I have 4GB, looks like a lot, but the OS and other applications also want their share. So closing some pages or restarting is the only solution.
Re: Speed of FreeBasic
I'm aware of how virtual memory works, and I know what thrashing is. There can be many reasons why the PHP program was so slow. Consuming virtual memory causing thrashing is but one possibility, but in this case I just don't think it was a factor. badidea said nothing about a general system slowdown when the script was running. He only said it took too long to finish. If it was thrashing, the entire server would be impacted. It would be rather obvious.
Re: Speed of FreeBasic
Actually, when doing normal textfile cooked I/O you touch all data multiple times. Once to scan for lineendings, than a move from buffer to string, and only then your string operations.caseih wrote:
But the thing is, processing logs is not usually CPU intensive; it's I/O-bound. .
The efficiency and avoidance of bufferbloat of the RTS implementation can be quite noticable when processing logs. Specially now that SSDs have sped up I/O expectations.
-
- Posts: 4292
- Joined: Jan 02, 2017 0:34
- Location: UK
- Contact:
Re: Speed of FreeBasic
I have two tabs in use at the moment - this forum and Google News and am using 219.7MB. I am not a 'power' user so you will not catch me with a lot of tabs open.jj2007 wrote:When its memory use goes beyond one GB, it gets very, very slow.
If you haven't already, have a look at FlashPeak SlimJet - it started life as Slim Browser, before I started to use it. It has a lot of stuff built in such as an ad blocker so there is no need to have a pile of extensions/plugins. I am not an extensions 'power'user either and only have four including ZenMate which encrypts browsing activity.
Towards the bottom of the home page is a comparison with IE, Chrome and Firefox and it has a lot of green ticks compared with the others. Well, it would wouldn't it but, for the most part, the features compared are useful ones.
It is highly customize-able and there is a memory management feature to “Automatically optimize memory usage”.
I have been using it for, don't know when, but quite a while now.
Added: I am also using the 1.1.1.1 DNS directory which is currently leaving everybody standing.
Re: Speed of FreeBasic
A bit OT but yes, it looks good, thanks. I am using Pale Moon nowadays because it needs less memory than Firefox. Still too much, though.deltarho[1859] wrote:have a look at FlashPeak SlimJet
-
- Posts: 4292
- Joined: Jan 02, 2017 0:34
- Location: UK
- Contact:
Re: Speed of FreeBasic
We could replace Pale Moon with any browser and the above statement would remain true.<smile>jj2007 wrote:because it needs less memory than Firefox
Re: Speed of FreeBasic
Strange. FreePascal doesn't do anything for recursion. I would expect FB's gcc backend to run circles around it.
-
- Posts: 22
- Joined: Apr 23, 2013 19:12
- Contact:
Re: Speed of FreeBasic
Hi friends,
You might be interested.
I made a simple algorithm, intentionally brute force with CASE.
The task is: The algorithm must find the order on which the LWK3724 code is located. Start numbering AAA0000, AAA0001, AAA0002... etc.
The alphabet is classic. A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z.
The correct answer is: 80183725.
I did exactly the same algorithm in 24 languages. Here are the results, for what time (in seconds) each language found this number on the same PC.
Looks like Freebasic's doing great!
P.S. It's not a complete test of languages. This is a solution to one particular example. Intentionally "example of life", not a textbook example of recursion...
Here is the code in FB:
You might be interested.
I made a simple algorithm, intentionally brute force with CASE.
The task is: The algorithm must find the order on which the LWK3724 code is located. Start numbering AAA0000, AAA0001, AAA0002... etc.
The alphabet is classic. A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z.
The correct answer is: 80183725.
I did exactly the same algorithm in 24 languages. Here are the results, for what time (in seconds) each language found this number on the same PC.
Looks like Freebasic's doing great!
P.S. It's not a complete test of languages. This is a solution to one particular example. Intentionally "example of life", not a textbook example of recursion...
Here is the code in FB:
Code: Select all
dim shared as double p
dim shared as single x=-1
dim shared as single a1,a2,a3
dim shared as string ps1,ps2,ps3
dim shared as string h1,h2,h3
dim shared as single h
h1="L"
h2="W"
h3="K"
h=3724
do
x+=1
p+=1
if x>9999 then
x=0
a3+=1
if a3>25 then
a3=0
a2+=1
if a2>25 then
a2=0
a1+=1
end if
end if
end if
select case a1
case 0
ps1="A"
case 1
ps1="B"
case 2
ps1="C"
case 3
ps1="D"
case 4
ps1="E"
case 5
ps1="F"
case 6
ps1="G"
case 7
ps1="H"
case 8
ps1="I"
case 9
ps1="J"
case 10
ps1="K"
case 11
ps1="L"
case 12
ps1="M"
case 13
ps1="N"
case 14
ps1="O"
case 15
ps1="P"
case 16
ps1="Q"
case 17
ps1="R"
case 18
ps1="S"
case 19
ps1="T"
case 20
ps1="U"
case 21
ps1="V"
case 22
ps1="W"
case 23
ps1="X"
case 24
ps1="Y"
case 25
ps1="Z"
end select
select case a2
case 0
ps2="A"
case 1
ps2="B"
case 2
ps2="C"
case 3
ps2="D"
case 4
ps2="E"
case 5
ps2="F"
case 6
ps2="G"
case 7
ps2="H"
case 8
ps2="I"
case 9
ps2="J"
case 10
ps2="K"
case 11
ps2="L"
case 12
ps2="M"
case 13
ps2="N"
case 14
ps2="O"
case 15
ps2="P"
case 16
ps2="Q"
case 17
ps2="R"
case 18
ps2="S"
case 19
ps2="T"
case 20
ps2="U"
case 21
ps2="V"
case 22
ps2="W"
case 23
ps2="X"
case 24
ps2="Y"
case 25
ps2="Z"
end select
select case a3
case 0
ps3="A"
case 1
ps3="B"
case 2
ps3="C"
case 3
ps3="D"
case 4
ps3="E"
case 5
ps3="F"
case 6
ps3="G"
case 7
ps3="H"
case 8
ps3="I"
case 9
ps3="J"
case 10
ps3="K"
case 11
ps3="L"
case 12
ps3="M"
case 13
ps3="N"
case 14
ps3="O"
case 15
ps3="P"
case 16
ps3="Q"
case 17
ps3="R"
case 18
ps3="S"
case 19
ps3="T"
case 20
ps3="U"
case 21
ps3="V"
case 22
ps3="W"
case 23
ps3="X"
case 24
ps3="Y"
case 25
ps3="Z"
end select
loop until h1=ps1 and h2=ps2 and h3=ps3 and h=x
locate 3,1:print ps1
locate 3,2:print ps2
locate 3,3:print ps3
locate 3,4:print x
locate 1,1:print "Location: ";p
sleep
end
Last edited by zxretrosoft on Jun 20, 2018 12:16, edited 6 times in total.
Re: Speed of FreeBasic
This is incorrect. logfile processing is not just moving the data, but also processing it, which tends to be slow, unless the whole operation is nearly entirely expressible in a native routine (like a simple regex). As soon as you process and start datastructures, you find the nature of the beast.caseih wrote:But the thing is, processing logs is not usually CPU intensive; it's I/O-bound. So the language should be nearly irrelevant for this task. Even an interpreted language should be able to parse and analyze logs nearly as fast as C. After all that's what languages like Perl were invented for. I'll admit that many of the text-processing abilities built into Perl (and Python for that matter) are actually written in C. Either way, the cost of reading the data into memory should dwarf the cost of analyzing that data, even in a scripting language.badidea wrote:A once converted a PHP program from a friend, that analysed daily firewall-logs, to a C-version. The PHP program needed at a certain moment more 24 hours to process the daily logs, which was a bit of a problem. The C-version took 2 minutes for the same job. The main performance again was the change in file reading and the use of (semi-) binary trees and pointer fun. But it also used only a tenth of the memory and that was mainly because variables seemed take much more memory in PHP then in C (or Freebasic).
Personally I wonder about Calibre (A python program) slowliness every day. First
order Pascal programs run circles around it when processing epubs, and I then use native Pascal helper routines (zip and xml), which should be slower than the corresponding C libs that Python probably uses.
Hmm, maybe we do agree, that is more the order I expect too. 10 times slow on a good day, 25-50 times on a bad day, 2-5 times in case where scripting is carefully optimized (and native probably not, it rarely is worth the bother) or very simpleBut my point is the problem wasn't PHP; it was his program. I mean no disrespect for your friend, but if the PHP version took 24 hours to do what your C program took 2 minutes to do, he was doing something wrong. Text processing in PHP is actually quite fast. Might not be 2 minutes fast, but I could believe a proper optimization of the script could get it under 20 minutes, if not under 10, which for a systems environment is totally acceptable, and also portable and maintainable, moreso than a compiled binary.
For simple searching, Python can be as fast as grep, which is pretty impressive. I don't think there's anything that could beat grep; it's about as optimized as you can get.
Re: Speed of FreeBasic
zxretrosoft wrote:Here is the code in FB:
Code: Select all
Location: 80183725
LWK 3724
You show that SpiderBasic is a factor 4 faster than C++. Suspicious... it probably means that the design of the program is not optimised.
Re: Speed of FreeBasic
Core I7, gas : 15 seconds.
Core i7, gcc (64-bit) : 8 seconds.
Core i7, gcc (64-bit) : 8 seconds.