Another one of these benchmarks:
http://www.tommti-systems.de/main-Dateien/reviews/languages/benchmarks.html
If someone would point you to this one and say “See, C++ is still many times faster than Java” - what’s the correct answer please? (I’m no benchmark expert, just a Java fan.)
Except for the trig scores (again) most of the tests seem to be only 10-15% faster in the C versions, which is what most people say anyway.
Im dubious about some of their tests - their matrix multiplies were way half the speed of the C ones, which doesn’t sound right to me, and the hashmap ‘s’ test has Java as double the speed of C++, which suggests they are using completely different algorythms.
It looks about as useful as any other benchmark Ive seen - i.e. not very at all.
The interesting quote is:
[quote]In my opinion it goes much faster to develop complex apps with Java or C#, than with plain Cpp and the STL. Type safety and buffer overrun issues are also much weaker in Cpp, so you have to do more work, which takes more time and will cost some speed! So every language has its strong areas, but as always, nothing comes for free…
[/quote]
There you go. Profiling & specific algorithm optimisation is far more important for execution speed, Language features for development speed, and Benchmarking for filling up empty web pages.
Thats a great quote CS, I may quote you sometime 
Thsi is the biggest problem with Microbenchamrks and why ocassionally you hear me react somewhat frustrated.
They are easy to write, most often wrong in terms of conclusions, and take a lot more work to explain WHY they are wrong then they did to write.
If I was going to try to debunk all the worthless "benchmarks’ out there it would be a full time job. Look at all the comments on past microbenhcmarks that are already here and see how many apply, probably a lot of them do.
Strange when other people write benchmarks, Java is slower than C. When I write benchmark Java is comparable with ASM. Perhaps it’s just way how do you aproach this thing.
[quote]Original Source Code is from Christopher W. Cowell-Shah http://www.cowell-shah.com/research/benchmark/code and from http://dada.perl.it/shootout/ and Doug Bagley http://www.bagley.org/~doug/shootout . I took portions and whole parts from both and put them into a single file and did some changes + bugfixes.
[/quote]
Well he admits this. So all that current benchmarks were based on that old Doug code.
It’s funny if C++ is faster than Java by around 15/100, then that’s acidentaly the same value as is ASM faster than C++.
Such benchmarks could be more interesting if they wouldn’t use FileReader/Writer.
As to B question. This bechmark measured different things so it’s just partialy valid. Also he is unexperienced with Java, and possibly in C++ (I hate when I’d have to repair programs screwed by someone else). Actually if we would ignore sin/cos tests, Java was faster in four cases.
C++ has unprecise sin, Java uses more perfect way. So he compared two method with two different results. We could be nasty and say C++ method doesn’t have high enough precision so C++ failed this test. JET failed as well.
Remmember that old joke about Intel and Motorola? How much is…
On my computer trig math completed in (scaled towards his test) 6000 ms. True this is twice as much as C++, but it’s uninteresting for real game.
Interesting for real aplication is that correct double results of sin and cos are avilable.
0.990665 vs 0.9906646477361263
it might be interesting if typing sin() without Math or StrictMath would activate native instruction “where avilable” with possibility of as bad results as could be on Intel processors.
Good thoughts on that benchmark topic, again. Thanks.
So, if somebody with spare time knew how to tune Java it should be possible to do a similar shallow benchmark (for all those “C++ benchmark fans” out there) which shows that Java “is faster than C++”…
I don’t need such one, because I’m interested in real applications, but regularly I meet people liking shallow benchmarks. It would be nice to be able to present them such one.
So you really want to participate in such a religous cake fight, huh? ;D
Maybe some useful tips:
You could also write a cool game and say you could never have pulled it off in the amount of time spent using C++ and brag about being 100% sure you don’t have any memory leaks in your code.
[quote]So you really want to participate in such a religous cake fight, huh? ;D
[/quote]
No. I just would find it handy to be able to show friends (C++ programmers) and others, that Java is ready for fast applications / games. All they find on the Net are these vague benchmarks.
Unfortunately Doom99 written in Java isn’t ready yet, so I can’t convince them with “hard facts” (=macro benchmarks) so far…
[quote]* Use the slowest C++ compiler you can find and make sure you don’t use any optimization options.
[/quote]
Well, these mentioned guys would take care of the C++ optimization themselves.
[quote]* Use an algorithm to benchmark which java is particulary good at, such as FFT.
[/quote]
FFT is too much for me. Pathfinding would be suitable, too, I read. I’ll have to check the few pathfinding threads to see which algorithm fits best to Java.
[quote]You could also write a cool game and say you could never have pulled it off in the amount of time spent using C++ and brag about being 100% sure you don’t have any memory leaks in your code.
[/quote]
Yes, but then I’d have to wait some years…
So then, why not show off some games to demonstrate how capable java is for fast games without creating benchmarks?
Erik
and the old old old Shaven Puppy terrain demo, which is probably the original eye-popper for Java performance critics.
Cas
Commercial game (C++) programmers are more interested in macro benchmarks and such. If they see “trig benchmarks” with Java taking 200% of the C++ one they’re thinking “OK, let’s wait for another Java generation”…
Of course I explain them that it’s been a silly benchmark and compared apples (Intel precision C++ trig routines) with … chocolate (Javas higher precision trig routines), but still…
Maybe I could take Cas’ Java implementation of A* and see how it compares to a C++ implementation… Did anybody do this already?
Maybe this, http://www.jroller.com/resources/rickard/FFT.java, could be an interesting start to a FFT implementation to compare with a “regular” (non-optimized;) c++ version?
An old but well done set of benchmarks doing real tasks:
(You really need to read the details to understand it, dont just look at the pictures.)
http://www.aceshardware.com/Spades/read.php?article_id=153
An updated run of these benchmarks, though not as complete:
http://members.lycos.co.uk/wjgoh/Java%20vs%20C%20benchmarks.doc