Java vs. c#, c++ and D benchmark

Performance Tuning
#1

Just wanted to let you know that a very thorough benchmark is available on http://www.windevnet.com/wdn/webextra/2003/0313/ (You have to register for free to gain access).

It compares language features and a good deal of library functions. Despite the author’s c++ / windows background it seems as if he tried to do a fair benchmark (if possible at all).
In the outcome it doesn’t look too good for java. The author shows that c++ can be fastest if you use intel c++ and are willing to choose whatever library is fastest for the task at hand (or write your own function) - which is what we all knew before. But the comparison between c# and java shows some deficies in java’s runtime regarding recursive method calls, float to int conversion, memory allocation and removing of bounds checks for array access.
Nevertheless there are some good points for java like exception handling and string comparisons.

Eagerly waiting for your opinions?

#2

[quote]Just wanted to let you know that a very thorough benchmark is available on http://www.windevnet.com/wdn/webextra/2003/0313/ (You have to register for free to gain access).
…
Eagerly waiting for your opinions?
[/quote]
Eagerly awaiting a version to be published somewhere public ;)…

#3

I would hope that MS made some improvements to java when the ported it and called it c# :slight_smile: They’re not stupid. I use java, for the most part, because there is so much good help out there and it is mostly cross platform… write once, test everywhere. I know the idea behind .net is that MS opened the runtime so that anyone can implement it, but you know that it will never run as well as it does on any platform other than windows. Also, machines are getting faster and faster everyday. I’ve seen some pretty impressive programs and games written in java. If a few cycles are being lost to c, I guess I just dont really care.

Just my 2 cents.

#4

Quite a poor article, not because of how it was done, but because of what it was measuring. It’s a microbenchmark article and has about as much value as discussions on the relative power-to-weight ratios of motorcycles and cars over beer in the pub.

It also doesn’t use the server VM which can be up to 100% faster for many operations, and it doesn’t use 1.4.2 which is 100% faster for floating point - so it’s already way out of date.

Cas :slight_smile:

#5

You shouldn’t worry too much about such benchmarks, especially the kind posted in the article (they seem very ‘microbenchmarky’ and the author seems to be making some jump conclusions based on them).
There are many other C# vs. Java benchmarks where java is consistently faster in every case, like http://www.manageability.org/blog/archive/20030520%23p_the_problem_with_cameron/view (which you shouldn’t take too seriously either).

Erik

#6

The author of the article (who is obviously very C++ minded) stresses the importance of the raii idiom over and over again and the lack of support of it in the java language.
I’m wondering, do we really miss raii that much? Is it really so important? I’m no expert at this raii thing, but raii seems a solution to a somewhat C/C++ specific problem to me.

Erik

#7

[quote]The author of the article (who is obviously very C++ minded) stresses the importance of the raii idiom over and over again and the lack of support of it in the java language.
I’m wondering, do we really miss raii that much? Is it really so important? I’m no expert at this raii thing, but raii seems a solution to a somewhat C/C++ specific problem to me.
[/quote]
While in C++ it is a lot more important, because it ease memory management, it would be also useful in java, in case of limited resources allocation (database connections, file handles). Best example for this is JDBC, with Connection/PreparedStatement/ResultSet trio. With raii you would just write


try {
  Connection con = getConnection();
  PreparedStatement ps = con.prepareStatement("SELECT * FROM X WHERE Y=?");
  ps.setInt(1,10);
  ResultSet rs = ps.executeQuery();
  while ( rs.next() ) {
    System.out.println(rs.getInt(1));
  }
} catch (SQLException exc) {
  //handle sql exc
}

currently, to be really on safe side, you need to do following


Connection con = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
  con = getConnection();
  ps = con.prepareStatement("SELECT * FROM X WHERE Y=?");
  ps.setInt(1,10);
  rs = ps.executeQuery();
  while ( rs.next() ) {
    System.out.println(rs.getInt(1));
  }
} catch (SQLException exc) {
  //handle sql exc
} finally { 
   if ( rs != null ) {
     try {
             rs.close()
     } catch (SQLException exc) {}
  }
  if ( ps != null ) {
     try {
             ps.close()
     } catch (SQLException exc) {}
  }
  if ( con != null ) {
     try {
             con.close()
     } catch (SQLException exc) {}
  }
}

It is ugly and error prone. Same thing applies to open files. While finalizers do their job, you cannot rely on them being run in any bound time, so they are in fact useless for resource management. Raii solves this transparently.

#8

It’s a chalk-and-cheese comparison really. It works that way in Java because the whole language is designed so it has to work that way.

I notice that he uses tuned-to-the-eyeballs compilers to measure C++ and then when he comes across an entirely obvious tuning issue when Java mysteriously hits a brick wall in memory allocation tests he doesn’t bother adjusting the memory allocation parameters for the benchmark. Ho hum.

Cas :slight_smile:

#9

It is shame people bind themselves. If this benchmark had had “JAVA is FAST” all over it, some of the more eager members of this board would have been proclaiming the second coming of jesus himself. Past five years Java programmers have been hit in the face by Java is slow. Before it didn’t matter, since Sun didn’t claim it to be a serious media library. Perhaps it is true after all?

Currently only advatange of java is “platform intependance”, which is arguable anyway. For example SDL adds the all Java has to offer to C++ and code difference is very small.

Future only shows whether sun will pull its head out of its arsehole when pondering whether java should be a serious media library or not.

Also, Sun is teh evil company now. Microsoft is going for ansi/iso standard for C#. Where is the standard for Java?

#10

Anyone tried to tell me that Java is faster than C++ would get laughed at too.

Cas :slight_smile:

#11

Well, here’s a benchmark that shows Java to be 10 000 times faster than C++, C#, and a bunch of other languages. The validity comments on that benchmark can probably provide some perspectives on the one from the windevnet too.

#12

That benchmark is misleading: The loops will be optimized out so the java program actually does nothing.

Erik

EDIT: oops, posted too quickly without reading the whole article :-X

#13

That is rather trivial benchmark to show ‘good’ sides of Java. What about doing a squareroot or sin something in the loop and see what happens.

I have never seen a programmer who iterates through two loops to do nothing, especially rendertime.

Actually, I’m wrong, that is not a trivial benchmark, but a proof that Java encourages bad coding styles. Since when has it been ok to make stupid mistakes like unfold incredibly long meaningless loops?

#14

I think you’re missing the point of that article. It merely puts benchmarks in general somewhat in perspective. It doesn’t say anything about speed of a language at all and it certainly doesn’t proof anything about ‘bad coding encouragement of java’. Please read the article and see how silly that point really is.

Erik

#15

[quote]I think you’re missing the point of that article. It merely puts benchmarks in general somewhat in perspective. It doesn’t say anything about speed of a language at all and it certainly doesn’t proof anything about ‘bad coding encouragement of java’.

Erik
[/quote]
Then it is not a benchmark, but rather ‘witty’ practical joke.

From what I undestrood I was directed to ‘benchmark’ when I clicked that link and thus I expected one. The multitude of expression is fooling me. Sarcastic ways of non-analytic language. I will go back to analytic language now.

#16

[quote]Sarcastic ways of non-analytic language. I will go back to analytic language now.
[/quote]
Dunno what you mean by ‘non-analytic language’, but good luck to you anyway :slight_smile:

#17

[quote]Anyone tried to tell me that Java is faster than C++ would get laughed at too.
[/quote]
Yeah me too… but mainly because it is silly to say that any language is faster than another. The compiler, VMs, and task at hand will make a huge difference… easily tipping the scale one way or another. The language is just a way to express an algorithm. So if you can write the same algorithm in C++ or Java then the speed issue is out of your hands and that of the language… now it’s up to the compiler and runtime environment.

I have seen a Java application do real work faster than the exact same algorithm compiled from C++. It wasn’t complicated, but it computed the result I wanted and the EXE file that MSVC++ 6.0 spit out was slower than running the Java code on the 1.4 client VM. Of course usually it is the other way around, but it all depends on so many factors that it is just dumb to say “Java is slow” or “Java will never beat C++”… heck it’s easy enough to prove that Java CAN beat C++. But of course it won’t ALWAYS beat C++. And currently it won’t USUALLY beat C++ for many of the tasks that I’ve tied either.

Choose your language for it’s strengths, avoid using it where it isn’t suited, and move on…

There is much more to creating software than the raw speed of execution you get.

#18

The problem with such benchmarks is that developers don’t write benchmarks - they write real applications with real bottlenecks and real algorithms. These usually have much more of an impact on whether an application is ‘fast’ or not. The one thing that many people often say about Java is that it helps them get it correct faster than they could in c/c++.

I’ve never had anyone prove to me that Java is faster than c/c++, but I’ve had plenty make fools of themselves trying to tell my why Java was ‘too slow’. At the moment - Java performance is close enough to c/c++ performance for a real application on a ‘standard shipping machine’ that it doesn’t matter much anymore.