When Pavel microbenchmarked every language he came across it turned out as expected, Ruby was the slowest language in all three micro tests. It was really embarrassing.
I was curious if this was about to change in future versions of Ruby so I checked out the current Ruby 1.9 version from their svn repository
svn co http://svn.ruby-lang.org/repos/ruby/trunk ruby
After compiling I ran Pavels microbenchmarks with the current stable Ruby 1.8.6, with the SVN version 1.9, Python 2.4 and Perl 5.8.8. Here are the results. I ran every Benchmark three times:
Ruby 1.8.6
$ ruby ~/Desktop/simple-bench.rb
Pavel's trivial ruby benchmark V.0.1
1000000 x {i+=1} took 0.243 sec. (4.114 mio. per sec)
2000000 x {i = my_add(i, 1)} took 0.940 sec. (2.128 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 2.608 sec. (19172.294 per sec)
$ ruby ~/Desktop/simple-bench.rb
Pavel's trivial ruby benchmark V.0.1
1000000 x {i+=1} took 0.240 sec. (4.175 mio. per sec)
2000000 x {i = my_add(i, 1)} took 0.941 sec. (2.126 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 2.611 sec. (19149.538 per sec)
$ ruby ~/Desktop/simple-bench.rb
Pavel's trivial ruby benchmark V.0.1
1000000 x {i+=1} took 0.242 sec. (4.135 mio. per sec)
2000000 x {i = my_add(i, 1)} took 0.943 sec. (2.122 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 2.626 sec. (19038.328 per sec)
Ruby 1.9
$ /usr/local/bin/ruby ~/Desktop/simple-bench.rb
Pavel's trivial ruby benchmark V.0.1
1000000 x {i+=1} took 0.142 sec. (7.040 mio. per sec)
2000000 x {i = my_add(i, 1)} took 0.409 sec. (4.893 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 0.660 sec. (75701.488 per sec)
$ /usr/local/bin/ruby ~/Desktop/simple-bench.rb
Pavel's trivial ruby benchmark V.0.1
1000000 x {i+=1} took 0.142 sec. (7.023 mio. per sec)
2000000 x {i = my_add(i, 1)} took 0.410 sec. (4.882 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 0.658 sec. (75979.874 per sec)
$ /usr/local/bin/ruby ~/Desktop/simple-bench.rb
Pavel's trivial ruby benchmark V.0.1
1000000 x {i+=1} took 0.143 sec. (7.002 mio. per sec)
2000000 x {i = my_add(i, 1)} took 0.409 sec. (4.889 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 0.660 sec. (75772.730 per sec)
Python 2.4
$ python2.4 ~/Desktop/simple-bench.py
Pavel's trivial python benchmark V.0.1
10000000 x {i = i + 1} took 2.54 sec. ( 3.93700787402 mio. per sec), i= 10000000
5000000 x {i = my_add(i, 2)} took 2.46 sec. ( 2.0325203252 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 1.75 sec. ( 57142.8571429 per sec)
$ python2.4 ~/Desktop/simple-bench.py
Pavel's trivial python benchmark V.0.1
10000000 x {i = i + 1} took 2.54 sec. ( 3.93700787402 mio. per sec), i= 10000000
5000000 x {i = my_add(i, 2)} took 2.46 sec. ( 2.0325203252 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 1.74 sec. ( 57471.2643678 per sec)
$ python2.4 ~/Desktop/simple-bench.py
Pavel's trivial python benchmark V.0.1
10000000 x {i = i + 1} took 2.54 sec. ( 3.93700787402 mio. per sec), i= 10000000
5000000 x {i = my_add(i, 2)} took 2.47 sec. ( 2.02429149798 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 1.72 sec. ( 58139.5348837 per sec)
Python 2.5
$ python2.5 ~/Desktop/simple-bench.py
Pavel's trivial python benchmark V.0.1
10000000 x {i = i + 1} took 2.5 sec. ( 4.0 mio. per sec), i= 10000000
5000000 x {i = my_add(i, 2)} took 2.53 sec. ( 1.97628458498 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 1.78 sec. ( 56179.7752809 per sec)
$ python2.5 ~/Desktop/simple-bench.py
Pavel's trivial python benchmark V.0.1
10000000 x {i = i + 1} took 2.52 sec. ( 3.96825396825 mio. per sec), i= 10000000
5000000 x {i = my_add(i, 2)} took 2.52 sec. ( 1.98412698413 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 1.78 sec. ( 56179.7752809 per sec)
$ python2.5 ~/Desktop/simple-bench.py
Pavel's trivial python benchmark V.0.1
10000000 x {i = i + 1} took 2.5 sec. ( 4.0 mio. per sec), i= 10000000
5000000 x {i = my_add(i, 2)} took 2.53 sec. ( 1.97628458498 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 1.78 sec. ( 56179.7752809 per sec)
Perl 5.8.8
$ perl ~/Desktop/simple-bench.pl
Pavel's trivial perl benchmark V.0.1
20000000 x {i = i + 1} took 2.34329295158386sec. (8.534997720401 mio. per sec), i=20000000
5000000 x {i = my_add(i, 2)} took 3.05162501335144sec. (1.63847129910263 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 3.62118482589722sec. (27615.2709148788 per sec)
$ perl ~/Desktop/simple-bench.pl
Pavel's trivial perl benchmark V.0.1
20000000 x {i = i + 1} took 2.35069394111633sec. (8.50812589856002 mio. per sec), i=20000000
5000000 x {i = my_add(i, 2)} took 3.05917191505432sec. (1.63442923079765 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 3.61791110038757sec. (27640.2590404411 per sec)
$ perl ~/Desktop/simple-bench.pl
Pavel's trivial perl benchmark V.0.1
20000000 x {i = i + 1} took 2.3465690612793sec. (8.52308177501345 mio. per sec), i=20000000
5000000 x {i = my_add(i, 2)} took 3.05434393882751sec. (1.63701275957788 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 3.62034606933594sec. (27621.6687810573 per sec)e178085100
All of Pavels microbenchmarks can be found here: http://acker3.ath.cx/wordpress/archives/11
I ran these tests on a MacBook Pro 2,33 GHz Intel Core 2 Duo with 2GB of RAM and some Apps running.
To be continued. And yes I know microbenchmarks are not saying anything meaningful.
[UPDATE 1]
So for completeness sake I also ran the benchmark with Python 2.5 but that didn’t really improve anything in this particular microbenchmark. The conclusion of this little exercise is that the new ruby interpreter really seems to be a lot faster than the old one. Now even ahead of some other scripting langunages in terms of speed. That is so good news!
Your running an old version of Python.
Try Python 2.5.1 from http://www.python.org/download/
– Paddy.
Congratulations on the speed increase. I look forward to the improved results on the Computer language shootout
– Paddy.
You couldn’t publish it in a nice little text table? Even try?
Urk, sorry, I shouldn’t be, and partly didn’t mean to sound, so nasty – please re-translate as: ‘could you try to post the results as a test table, even if it must be a crude, text one because of the blog software?’. Thanks. And, wow, it’s hard to believe.
This information is useless unless put into a graph. Otherwise blah blah blah.
Ummm…. Why did you test Ruby with less data than you tested Python and Perl? You cheated.
ruby:
1000000 x {i+=1} took 0.142 sec. (7.023 mio. per sec)
2000000 x {i = my_add(i, 1)} took 0.410 sec. (4.882 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 0.658 sec. (75979.874 per sec)
python:
10000000 x {i = i + 1} took 2.5 sec. ( 4.0 mio. per sec), i= 10000000
5000000 x {i = my_add(i, 2)} took 2.53 sec. ( 1.97628458498 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 1.78 sec. ( 56179.7752809 per sec
perl:
20000000 x {i = i + 1} took 2.34329295158386sec. (8.534997720401 mio. per sec), i=20000000
5000000 x {i = my_add(i, 2)} took 3.05162501335144sec. (1.63847129910263 mio. per sec)
100000 x {i = fullMatMult(a, b)} took 3.62118482589722sec. (27615.2709148788 per sec)
You, DOUBLED the amount of data for ALL of your tests to perl/python. “not saying anything meaningful” is an understatement. Flatout lying for the sake of hype would be a better statement.
I think the thread[1] over at Reddit nicely sums up how misleading this is.
[1] http://programming.reddit.com/info/2fnwl/comments
Ok, these benchmarks are bit sketchy, but the real news is the average 3x speedup of ruby as reported here: http://www.rubychan.de/share/yarv_speedups.html
What a crock! Liar liar pants on fire!
nulld4y:/tmp# python2.5 simple-bench.py
Pavel’s trivial python benchmark V.0.1
1000000 x {i = i + 1} took 0.24 sec. ( 4.16666666667 mio. per sec), i= 1000000
500000 x {i = my_add(i, 2)} took 0.22 sec. ( 2.27272727273 mio. per sec)
10000 x {i = fullMatMult(a, b)} took 0.16 sec. ( 62500.0 per sec)
nulld4y:/tmp# python2.5 simple-bench.py
Pavel’s trivial python benchmark V.0.1
1000000 x {i = i + 1} took 0.24 sec. ( 4.16666666667 mio. per sec), i= 1000000
500000 x {i = my_add(i, 2)} took 0.22 sec. ( 2.27272727273 mio. per sec)
10000 x {i = fullMatMult(a, b)} took 0.16 sec. ( 62500.0 per sec)
nulld4y:/tmp# python2.4 simple-bench.py
Pavel’s trivial python benchmark V.0.1
1000000 x {i = i + 1} took 0.24 sec. ( 4.16666666667 mio. per sec), i= 1000000
500000 x {i = my_add(i, 2)} took 0.22 sec. ( 2.27272727273 mio. per sec)
10000 x {i = fullMatMult(a, b)} took 0.17 sec. ( 58823.5294118 per sec)
nulld4y:/tmp# python2.5 -V
Python 2.5.1
nulld4y:/tmp# python2.4 -V
Python 2.4.4
VPS CPU Xeon 2394.064 x2, memory – 256Mb
What do I do not so? =))
with n == as in ruby tests:
nulld4y:/tmp# python2.4 simple-bench.py
Pavel’s trivial python benchmark V.0.1
1000000 x {i = i + 1} took 0.24 sec. ( 4.16666666667 mio. per sec), i= 1000000
2000000 x {i = my_add(i, 2)} took 0.87 sec. ( 2.29885057471 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 0.86 sec. ( 58139.5348837 per sec)
nulld4y:/tmp# python2.5 simple-bench.py
Pavel’s trivial python benchmark V.0.1
1000000 x {i = i + 1} took 0.24 sec. ( 4.16666666667 mio. per sec), i= 1000000
2000000 x {i = my_add(i, 2)} took 0.89 sec. ( 2.24719101124 mio. per sec)
50000 x {i = fullMatMult(a, b)} took 0.81 sec. ( 61728.3950617 per sec)
ps: ruby sux B)
Ruby != Ruby, esp. when it comes to speed! See the snippets on http://snippets.dzone.com/posts/show/4148 to find all duplicates in an array. Really interesting! (For performance-critical parts you may also resort to rocaml and similar Ruby libs).
I’m the one who did http://rubychan.de/share/yarv_speedups.html – Ruby 1.9 isn’t 3x faster than Ruby 1.8 in real applications. these are synthetic benchmarks. expected speedups for real pplications are much lower – about 15-33%. Ruby will stay slower than Perl and Python in benchmarks, but some cases will really get a boost from YARV (the VM of Ruby 1.9).
Please take more care when doing benchmarks.