FreeBASIC versus Python (speed of execution)

Post your FreeBASIC tips and tricks here. Please don’t post your code without including an explanation.
AGS
Posts: 1284
Joined: Sep 25, 2007 0:26
Location: the Netherlands

FreeBASIC versus Python (speed of execution)

Postby AGS » May 19, 2008 5:32

'Interpreted code runs slower than compiled code'. Everyone knows that, right? As I love to use interpreters I thought I'd put this common knowledge about interpreters to the test.

I wrote a program in two languages. One in FreeBASIC (what else?) and one in Python.

Here is the program in FreeBASIC:

Code: Select all

#include "dir.bi"
#include "file.bi"

Dim filename As String
Dim output_file As String
Dim input_file As String
Dim begin_ As Double
Dim end_ As Double
Dim pos1 As Integer
Dim pos2 As Integer
Dim in_path As String
Dim out_path As String


out_path = "d:\test_output\"
in_path = "c:\"
filename = Dir("c:\wikka.php@wakka*")
begin_ = Timer()
While len(filename) <> 0
  pos1 = Instr(filename,"=") + 1
  pos2 = Instr(filename,"%")
  output_file = out_path + Mid(filename,pos1,pos2-pos1)
  input_file  = in_path + filename
  FileCopy input_file,output_file
  filename = Dir()
Wend
end_ = Timer()
Print "Time spent = ";end_ - begin_
End


And here is the same program in Python (that is, functionally the same):

Code: Select all

import glob
output_path = 'd:\\wikka_files\\'
mylist = glob.glob('c:\\wikka.php@wakka*')
for el in range(len(mylist)):
    f = open(mylist[el],'r')
    str_ = f.read()
    equalsign = mylist[el].split('=')
    percent = equalsign[1].split('%')
    new_name = percent[0]
    g = open(output_path+new_name,'w')
    g.write(str_)
    g.close()
    f.close()


The program copies files from one directory to another while changing the name of the file. A very specific batch file copier/renamer, that is what it is.

Now for the benchmarks. Python comes with it's own benchmarking program. It's output when the above program runs:


Commandline: D:\Python25\python.exe D:\Python25\Lib\profile.py D:\fbpydoc\RENAME~1.PY
Workingdirectory: D:\fbpydoc
Timeout: 0 ms

24359 function calls (24356 primitive calls) in 6.073 CPU seconds


6 seconds to rename and copy 917 files (give or take a few).

And then the FreeBASIC benchmark.

On the first run it takes 6.4 seconds. On the second run it takes 6.7 seconds and after that it takes 5.12 seconds.

How the FreeBASIC benchmark should be interpreted.... I don't know. Perhaps some filecaching is going on making the program run faster on it's third try.

If I take the lowest number of the FreeBASIC benchmark (5.12 seconds) and the Python benchmark (6 seconds) then the difference is 1 second. That's less then 20% difference. Not a whole lot I think?

If someone can write a faster FreeBASIC version of the program (surely it can be done faster?) then please do post it.
cha0s
Site Admin
Posts: 5317
Joined: May 27, 2005 6:42
Location: Illinois
Contact:

Postby cha0s » May 19, 2008 6:21

I don't know too much more you can do to squeeze speed out of that.

I think the thing to keep in mind here is that you're basically benchmarking the filesystem, not the languages. Do something mathematical if you want to really get an idea of how fast the languages are. Do some factoring, calculate pi to n-digits, or something.

I'm not trying to bash python, I actually like what little of it I know, and it's not that slow, either. This isn't a good way to benchmark language speed, is all. For really intensive stuff, the only way python has a chance to beat FB (or any language that compiles natively) is if you use some kind of C emitter from the python, which would then be compiled natively...
KristopherWindsor
Posts: 2428
Joined: Jul 19, 2006 19:17
Location: Sunnyvale, CA
Contact:

Postby KristopherWindsor » May 19, 2008 6:24

Since this uses file I/O, file I/O will be the bottleneck.
CPU usage will be negligible, so the fact that one language is interpreted will not matter.
The speed difference is due to file caching, or the file copying routine you are using. (Maybe it is copying the file one byte at a time instead of one sector at a time.)
BTW, this should not be in the "Tips and Tricks" section. ;-)
ciw1973
Posts: 157
Joined: Jun 12, 2007 15:03
Location: Isle of Man (United Kingdom)

Postby ciw1973 » May 19, 2008 8:35

As has been mentioned, execution speed in the case of your example will be restricted by I/O rather than processor speed.

That said, Python (a language which I have a great deal of love for) essentially compiles to byte-code the first time it's run, and most of the shipped libraries are written in C, so the difference in execution time is often a lot less than people expect.

Of course, if you start doing any amount of actual processing in Python the difference in performance between it and fully compiled languages like FreeBASIC starts to become apparent.

For things like GUI code and communicating with a DBMS which account for 90% of most business applications, as often as not Python and other languages like Java and the ever increasing number which compile the .NET actually perform perfectly well.
yetifoot
Posts: 1710
Joined: Sep 11, 2005 7:08
Location: England
Contact:

Postby yetifoot » May 19, 2008 9:43

Here are some benchmarks from the computer language game, of FreeBASIC vs Python

http://shootout.alioth.debian.org/gp4/b ... ng2=python

In those tests we go up to 86x faster, and use up to 13x less memory. Those are extremes and could be due to a poorly written test on their side, but I would say it's fair to say that we are about 5x faster and use about 3x less memory in the general case, at least as far as benchmark type programs go.

A benchmark of copying files doesn't really stress the language as mentioned, I would expect all languages to come out pretty equal on a test like that.
marcov
Posts: 2806
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Postby marcov » May 19, 2008 9:56

cha0s wrote:I don't know too much more you can do to squeeze speed out of that.

I think the thing to keep in mind here is that you're basically benchmarking the filesystem, not the languages. Do something mathematical if you want to really get an idea of how fast the languages are. Do some factoring, calculate pi to n-digits, or something.


That is not so great either. Such problems are very local (data and code wise) and have relatively low amounts of input data, which benefits dynamic systems (VMs) that can optimize it heavily.

However such problems rarely appear so much in real life, since most programs are either larger (and thus less local, with complex call chains), and even the CPU intensive ones operate on more data.
Last edited by marcov on Oct 19, 2011 14:30, edited 1 time in total.
AGS
Posts: 1284
Joined: Sep 25, 2007 0:26
Location: the Netherlands

Postby AGS » May 19, 2008 20:49

KristopherWindsor wrote:Since this uses file I/O, file I/O will be the bottleneck.
CPU usage will be negligible, so the fact that one language is interpreted will not matter.


90%+ CPU usage on my PC, Kristopher.

KristopherWindsor wrote:The speed difference is due to file caching, or the file copying routine you are using. (Maybe it is copying the file one byte at a time instead of one sector at a time.)
BTW, this should not be in the "Tips and Tricks" section. ;-)


The Trick: convert a program written in Python to FreeBASIC.
The Tip: do not believe all that is being said about interpreted languages.

O yes: when posting to this part of the forum you're supposed to post source code. Which I did ;-)
AGS
Posts: 1284
Joined: Sep 25, 2007 0:26
Location: the Netherlands

Postby AGS » May 19, 2008 21:10

yetifoot wrote:Here are some benchmarks from the computer language game, of FreeBASIC vs Python

http://shootout.alioth.debian.org/gp4/b ... ng2=python


Those are Linux benchmarks. There is no FreeBASIC WIN32 benchmark and that is what I am using (Python Win32, FreeBASIC Win32). That's not to say those benchmarks are of no use when using Win32 versions of FreeBASIC/Python.

yetifoot wrote:In those tests we go up to 86x faster, and use up to 13x less memory.

Those are extremes and could be due to a poorly written test on their side, but I would say it's fair to say that we are about 5x faster and use about 3x less memory in the general case, at least as far as benchmark type programs go.


On Win32 you are right about the 3x less memory (at least when executing the program I've written). 5x faster..... if that would hold true on Win32 I'd be more than happy.

A benchmark of copying files doesn't really stress the language as mentioned, I would expect all languages to come out pretty equal on a test like that.


There are some IO benchmarks where the differences are considerable. But the thing is that I didn't write the programs to show how fast/slow Python/FreeBASIC is, just to show that performance depends on what you are building. And in some cases performance differences can be very small.
AGS
Posts: 1284
Joined: Sep 25, 2007 0:26
Location: the Netherlands

Postby AGS » May 19, 2008 21:27

ciw1973 wrote:As has been mentioned, execution speed in the case of your example will be restricted by I/O rather than processor speed.

That said, Python (a language which I have a great deal of love for) essentially compiles to byte-code the first time it's run, and most of the shipped libraries are written in C, so the difference in execution time is often a lot less than people expect.

Of course, if you start doing any amount of actual processing in Python the difference in performance between it and fully compiled languages like FreeBASIC starts to become apparent.

For things like GUI code and communicating with a DBMS which account for 90% of most business applications, as often as not Python and other languages like Java and the ever increasing number which compile the .NET actually perform perfectly well.


AGS1973 agrees with ciw1973. And I personally like the idea of mixing things up. Python mixed with a bit of FreeBASIC or FreeBASIC with a bit of Python. I feel like Python is THE language to use with FreeBASIC (or vice versa) because they seem to go well together. Much better then Python with C (C syntax? No thank you!). Python is a bit like FB, FB is a bit like Python.

It's a pity those Python header files haven't been swigged yet.
sir_mud
Posts: 1401
Joined: Jul 29, 2006 3:00
Location: US
Contact:

Postby sir_mud » May 20, 2008 1:47

i started to, but there are a crap load and the few i started on would have needed a lot of work
marcov
Posts: 2806
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Postby marcov » May 21, 2008 7:58

AGS wrote:

Those are Linux benchmarks. There is no FreeBASIC WIN32 benchmark and that is what I am using (Python Win32, FreeBASIC Win32).


We benchmarked some of them on win32 too for FPC, and except for threading there was not much different. We did rewrite some tests that use pipes to get their inputs to normal file I/O, but that is to avoid problems because pipes might be less efficient on windows, specially if their exact unix semantics are emulated.

Heavily intwined threading is a difficult subject. The benchmark often has a knack for a certain implementation (which is also noticable between different *nix targets, and also when going between architectures with Linux)

That's not to say those benchmarks are of no use when using Win32 versions of FreeBASIC/Python.


I do not expect significant differences. Unless the Python version on win32 is immature.

There are some IO benchmarks where the differences are considerable. But the thing is that I didn't write the programs to show how fast/slow Python/FreeBASIC is, just to show that performance depends on what you are building. And in some cases performance differences can be very small.


I somehow never seem to have enough work for scripting to make it worthwhile actually dive into a scripting language. It's easier done with the normal programming language.

Return to “Tips and Tricks”

Who is online

Users browsing this forum: MSN [Bot] and 2 guests