To LWGL or not to LWGL? That's the question

  1. You no longer have to worry about getting references to some “gl” instance everywhere. Simply replace all your gl.gl* calls with GL.gl, because it’s static.

  2. Everything is done in the main thread. You can safely search for any code with the word “synchronized” or “Thread” in it and most likely rewrite it, single-threaded, unless it’s something to do with networking.
    easier to get to grips with.

  3. Be prepared to understand Buffers completely, and forget all about passing arrays into GL methods.

  4. Always always always check GLCaps for GL capabilities before attempting to call functions!

  5. Running with -ea on the commandline causes the lwjgl_d.dll to be loaded which has lots of extra debugging and checking in it. In particular all GL commands automatically call glGetError() to check for errors and throw runtime OpenGLExceptions.

Cas :slight_smile:

I mean no malice. I figued that I should present the opposing view to Cas’ statements so that we understand the implications of the “pros” (i.e. a “pro” can actually be a “con”).

  1. Using statics (which I’m not against in the GL layer BTW) allows any class to insert GL calls onto the stack at any point. You cannot prevent an arbitrary class from calling into GL. This throws “defensive coding” out the window.
    Boo hoo! to all of those that can’t pass a simple reference around! That “annoyance” provides you with the knowledge of exactly what classes can access GL. This simplifies debugging and maintenance.
    Think of it like the American HIPPA standing (it’s a medical privacy law). HIPPA doesn’t care about “who accessed the data”, it cares about “who had the ability to access the data”. The difference is subtle but very important.

  2. Threads are not bad. We all need to get out of this mindset. Just because most people seriously fubar threads does not mean that we should eradicate them. Since I don’t want to waste space I will refer you to:

http://www.realityinteractive.com/software/sc/background/benefits.html

where you can get a taste of why you want threads. (No, this isn’t a product plug.)

  1. Removing the ability for a developer to make a simple (read: less error prone) call to set or retrieve a GL value via an array is just bad mojo. See the example below:

With arrays:

        
        final int[] textureIdentifier = new int[1]; 
        gl.glGenTextures(1, textureIdentifier);
        gl.glBindTexture(GL.GL_TEXTURE_2D, textureIdentifier[0]);

Without arrays:


        final ByteBuffer buffer = ByteBuffer.allocateDirect(4 /*sizeof(int)*/);
        buffer.order(ByteOrder.nativeOrder()); // pedantic
        final IntBuffer textureIdentifier = buffer.asIntBuffer();
        GL.glGenTextures(textureIdentifier);
        final int textureId = textureIdentifier.get();
        GL.glBindTexture(GL.GL_TEXTURE_2D, textureId);

Don’t forget to call rewind() between retrievals of the value from textureIdentifier. Oh! And be prepared for cryptic VM crashes and core dumps.

I should point out that glGenTextures is even worse than I’m making it out to be: there’s no number parameter. After scouring the code (as this function has no javadoc) I determined that:

nglGenTextures(textures.remaining(), textures, textures.position())

This means that if I wish to use a shared buffer of size n when I only want m textures, I need to explicitly set the position and limit of the buffer before I can make the call, so add two more calls to the already bloated code.

The bottom line is that more lines of code = larger possibility for bugs. The second bottom line is that you’re making the trade-reability-for-performance decision for me. The developer should be provided with the ability to make that decision.

  1. “GLCaps”? “GLCapabilities” too long to type? :slight_smile: Seriously now folks, why doesn’t the library protect me from making a call to an unavailable extension? More code = more bugs.

  2. The debugging is done in a separate dll? Oye! The developer does not have the ability to choose at runtime nor selectively choose which set of GL functions they want debugged. Why isn’t this done in the Java layer where more control can be handed to the developer?


Now with all of this said, I stand behind the LWJGL effort 100% and I recommend it to everyone that I talk to. Those guys are doing a great job. They're handing bug reports very quickly. There are no production bugs that I have ever run into that have ever stopped me from coding.

If you’re going to write a standalone game, use LWJGL.

Those are very good points, and mostly reflect a difference of opinion in the performance trade-offs the two libraries decided to make. In the interests of disagreeing, though, I have to be picky about this comment:

[quote]Without arrays:


        final ByteBuffer buffer = ByteBuffer.allocateDirect(4 /*sizeof(int)*/);
        buffer.order(ByteOrder.nativeOrder()); // pedantic
        final IntBuffer textureIdentifier = buffer.asIntBuffer();
        GL.glGenTextures(textureIdentifier);
        final int textureId = textureIdentifier.get();
        GL.glBindTexture(GL.GL_TEXTURE_2D, textureId);

[/quote]
That’s hardly fair! With the addition of one utility class I write something like:

FloatBuffer textureIdentifier = BufferUtil.makeFloatBuffer(1);
GL.glGenTextures(textureIdentifier);
GL.glBindTexture(GL.GL_TEXTURE_2D, textureIdentifier.get(0));

Which is basically the same as your array version, except with different words used. There’s some familiarity advantage to using arrays, but buffers really aren’t hard to get used to - really. Come to think of it, in my code I would actually write:

Texture t = new Texture(myImage);

;D

Of course you can wrap the code that I presented but the point is that the code has to be written, debugged and maintained. Instead of debugging and maintaining three lines of code, I’m dealing with six which are intrinsicly more complex.

And the way that I do it is found under Simulation Common at:

http://www.realityinteractive.com/software/oss/index.html

and essentially goes:

final Texture2DFactory texture2DFactory = new Texture2DFactoryImpl();
...
final Texture2D someTexture = texture2DFactory.createTexture(<some URL>);

So ;D to you too! :slight_smile:

I’d like to respond to your 4. point:

[quote]Seriously now folks, why doesn’t the library protect me from making a call to an unavailable extension? More code = more bugs.
[/quote]
The library does check for nonexistent function pointers, so the GLCaps check is not strictly nescessary, as your application will bomb out with an exception if the function pointer is NULL. However, not checking your (required) extensions before using them is not good OpenGL practice anyway, so I can’t see why the extra GLCaps checks (they only need to be done once per extension at startup mind you) can be avoided in a wellbehaving application.

  • elias

And regarding your second point about threads:

You’re right in that threads can be useful, my griefs are just that they’re highly overrated, really. Many programmers (including myself once upon a time) will happily use threads for anything that smells like “multiple tasks at once”, ignoring the fact that most of the time the complexity of using threads correctly outweighs the little extra performance they may give. I deliberately say ‘little’ here because it is well known that you can achieve thread-like execution without threads in common tasks related to games. Take OpenGL for example. Calling OpenGL from multiple threads will likely hurt performance, but using APIs like ARB_VBO cleverly ensures that both the CPU and the GPU is working at all times. Even Cas’ example of networking can be solved with non-blocking NIO (most of the time).

  • elias

…But I completely agree with your statement about the debug version of the libraries. It does remove some debug flexibility, but more importantly, lwjgl is about simplicity and the separate debug version simply does not fit into that goal.

So we’re slowly working on a solution that removes the need for the debug version while maintaining the same level of error checks, but without introducing too much overhead per GL call. Stay tuned.

  • elias

[quote]The library does check for nonexistent function pointers, so the GLCaps check is not strictly nescessary, as your application will bomb out with an exception if the function pointer is NULL
[/quote]
This is exactly what I’m referring to. Why force the developer to make the call to GLCaps? Shouldn’t the library be the one that’s concerned about not throwing NPE’s like they’re going out of style? Having the library log some sane message is better than an NPE any day of the week!

[quote]However, not checking your (required) extensions before using them is not good OpenGL practice anyway, so I can’t see why the extra GLCaps checks (they only need to be done once per extension at startup mind you) can be avoided in a wellbehaving application.
[/quote]
Developer correct / sane / defensive coding procedures weren’t in question and of course the developer should check the available extensions before using them. The root of what is in question is the need for the call. So let’s look at this the other way, if the library didn’t throw on an unavailable extension and instead logged the error what would the developer lose?


In response to your threads topic, I most certainly agree that calling GL from multiple threads is a recipe for disaster. Just to be clear, Cas was implying that you dump all threads (with some networking footnote) and that's what I'm calling into question. Threads can be a "good thing" if used appropriately and the link I provided shows how this may be possible. I just want to get the word out that threads aren't as bad as everyone thinks they are. Add to that the fact that multi-core processors will be quite common in the near future. Unless we start to get over our "threads are evil" hangups now, we're going to have a lot of idle processor time out there :)

In fact, LWJGL will not throw NPE’s, but OpenGLExceptions, but this is not the point. You’re saying that LWJGL should log a message, probably something along the lines of “Application tried to call nonexistent function glBlah”, right?

If that’s what you’re saying, then no, that’s simply not better than an exception, ever. I’m certainly not calling OpenGL functions because I like to see my nvidia card sweat, but because I want something drawn or a state changed. If the GL library does not support the call there’s no other choice than to halt the running program, hence the nescessity of checking with GLCaps. I like bugs to scream in my face, not hide in a log file somewhere.

  • elias

I don’t want to side-track more interesting points with this so I’ll simply say that unchecked exceptions (be it NPE’s or OpenGLException’s) are a developers worst nightmare and anything that you can do to eliminate them should be persued. Unchecked exceptions serve only to bite developers in the ass when their code goes into production.

I very much agree with Elias’ sentiment. Naturally I like a system to prevent me from writing bad code when possible, but when that’s not possible it should tell me in no uncertain terms that I’ve screwed up. Trying to use an extension that isn’t supported is a developer error, and should be treated as such.

And yes, in the past I’ve vocally opposed removing checked exceptions from Java, and will continue to do so!

This is one of those great debates that only comes along once every 6 months :slight_smile:

In response to everything here I’d like to state our goals with LWJGL:

[] Speed
[
] Simplicity
[] Ubiquity
[
] Smallness
[] Security
[
] Robustness
[*] Minimalism

and this will help explain how we got to where we are today and more importantly where we’re going and where we’re not going.

Speed
The whole point of LWJGL was to bring the speed of Java rendering into the 21st century. This is why we have:

[] Thrown out methods designed for efficient C programming that make no sense at all in java, such as glColor3fv.
[
] Made the library throw an exception when hardware acceleration is not available on Windows. No point in running at 5fps is there?
[*] Provided a separate debug library that checks GL error state after every method call, which is not something you would normally do in deployment but would definitely want to do in development.

Ubquity
Our library is designed to work on devices as small as phones right the way up to multiprocessor rendering servers. Just because there aren’t any phones or consoles yet with fast enough JVMs and 3d acceleration is neither here nor there - there will be, one day. We’re carefully tailoring the library so that when it happens we’ll have OpenGL ES support in there just like that. This means that:

[] We had to have a very small footprint or it’ll never catch on in the J2ME space at all. That’s why the binary distribution is under half a meg, and that takes care of 3d sound, graphics, and IO.
[
] Even under desktop environments having a 1-2mb download just to call a few 3D functions is daft.
[*] We’ve worked to a lowest common denominator principle rather than attempting to design for all possibilities, but we’ve made sure that 99% of required uses are covered. That’s why we’ve only got one window, and why we don’t guarantee that windowed mode is even supported (it’s officially a debug mode and hence we don’t even supply some very basic windowy abilities that you’d get in AWT) and why we don’t allow multiple thread rendering contexts.

Simplicity
LWJGL needed to be simple for it to be used by a wide range of developers. We wanted relative newbies to be able to get on with it, and professionals to be able to use it professionally, maybe typically coming from a C++ background. We had to choose a paradigm that actually fits with OpenGL, and one that fits with our target platforms which ranges from PDA to desktop level. This is why:

[] We aren’t catering for single-buffered drawing
[
] We don’t require that an instance of GL is passed around all over the place but we do not prevent this style of coding. See below for why.
[] We removed a lot of stuff that 99% of games programmers need to know nothing about
[
] We have decided that consistency is better than complexity. Rather than allowing multiple ways to call the same methods and bloating the library we’ve just said, “Right, no arrays. They’re slower anyway. Get used to buffers, as this is what buffers are meant to be used for.” See below for why this doesn’t affect you.

Smallness
See ubiquity above. We had to be small.

[] Small == simple. The less ways there are to do something, the easier it is to learn the only way that works or is allowed.
[
] Small == our code is less buggy. Wouldn’t you rather be hunting for bugs in your own code, not ours?
[] Small == downloadable. No version nightmares. LWJGL is small enough to download with every application that uses it.
[
] Small == J2ME.

Security
We realised a few months ago that no-one was going to take us seriously if we couldn’t guarantee the security of the LWJGL native libraries. This is why we:

[] No longer use pointers but exclusively use buffers instead
[
] Are gradually adding further checks to buffer positions and limits to ensure that the values are within allowed ranges to prevent buffer attacks

Robustness
Similarly to security we have now realised that a reliable system is far more useful than a fast system. When we actually had a proper application to benchmark finally we had some real data. Many of our original design decisions were based on microbenchmarks - well, you have to start somewhere! But with a real application to benchmark we now know we can throw out asserts and replace them with a proper if (…) check and a thrown exception. We know also that we can move all that GL error checking out of native code and into Java code and we will no longer need a separate DLL for debug mode.

As for runtime exceptions, they have their place. There’s not a reasonably well defined argument as to when you should use a runtime exception and when you should use a checked exception. When I made OpenGLException a checked exception all it did was end up littering my code with try {} catch {} sections - except that if you’ve got an OpenGLException there is very little sensible you can do to rectify it because it should never have occurred in the first place. That’s why it’s a runtime exception. You should simply not write code than can throw it because it is generally not recoverable nicely. However for robustness (and security) we are required to throw an exception if something is amiss. It falls, I believe, into exactly the same category of trouble as NPEs, ArrayIndexOOBs and ClassCastExceptions: should never occur but needs to be trapped somewhere.

Minimalism
This is another critical factor in our design decisions. If it doesn’t need to be in the library, it’s not in the library. Our original aim was to produce a library that provided the bare minimum required to access the hardware that Java couldn’t access, and by and large we’re sticking to this mantra. The vector math code in the LWJGL is looking mighty scared at the moment because it’s probably for the chop - well, at least, from the core library - as it’s not an enabling technology at all, and there are numerous more fully featured alternatives. We chucked out GLU because it’s mostly irrelevant to game developers except for a few functions that we really need to get redeveloped in pure Java - but basically, GLU is just a library of code built on top of the enablement layer.

Now why doesn’t this affect you?
Because we’re providing a small, minimal library that very closely mirrors the way the operating system actually works rather than imposing our own ideas about how the object oriented world works upon it, we’re letting the next layer up make a decision. In Rob’s case, that’s him!

If you want to pass a GL instance to all objects so you can track who’s calling what, it’s easy:

Create a GL interface that contains the method signatures from our GL class.
Create a concrete implementation of that class.
Pass references to the instance of the interface around.
Problem solved.

If you want checked exceptions, create a checked OpenGLException. In your GL implementation class, do:


public void glSomeMethod() throws CheckedOpenGLException {
      try {
            GL.glSomeMethod();
            checkError();
      } catch (OpenGLException e) {
            throw new CheckedOpenGLException(e);
      }
}

where checkError() is just this:


protected void checkError() throws CheckedOpenGLException {
      int error = GL.glGetError();
      if (error != 0) {
            throw new CheckedOpenGLException(GLU.gluErrorString(error));
      }
}

Save the code in a library, call it lw_bob_gl, and code to that library, not to LWJGL directly. I already do the same in Alien Flux - I automatically direct calls to EXT methods as appropriate instead of later GL 1.4 ARB methods, as appropriate. The important thing is we haven’t got in the way of getting the job done, and we haven’t sacrificed any of our goals in order to let you achieve what you wanted to achieve.

Sorry for the rambling!

Cas :slight_smile:

Three cheers for Cas! Hip hip Horray! … (you get the idea :)).

The main reason that this makes me so happy is that we can finally make decisions based on a set of goals rather than pulling them out of our butts. This ensures that RFE’s (request for enhancements) are accepted and rated based on a set of guidelines.

The only other thing that I’m waiting for are performance and size metrics. That will cut out all of the unbased claims of “we can’t do that because it will bloat the library” or “that will hurt performance”.

Cas has been modded up a few points! :smiley:

You forgot the “we can’t do that because we’re too lazy” :wink:

Actually this might be a good time to tell you what’s coming up in 0.9 (I think Brian, Elias and I are agreed on it):

Preparing for static import
Another major change in the API approaches! But hopefully, the last major change. We will be splitting up the huge GL class into separate classes for each extension, and the underlying C code will be split likewise.

We are doing this in preparation for the new language feature static import. This will make it easier to keep track of which classes use which GL extensions.

The classes will be named directly after the GL extensions string, eg. ARB_multitexture. We’ve not yet decided yet but all extensions will become classes; those that are merely interfaces full of constants will become classes too. All extensions will be marked final with private contstuctors, reflecting their intended usage.

Existing code will become hideously ugly as static import is not yet implemented in the language. This will go away as JDK1.5 becomes the normal development environment, and also as LWJGL users develop libraries on top of LWJGL to abstract things away.

OpenAL extensions
There’s been a lot of work going on in OpenAL lately and we will be catching up on the new stuff.

OpenGL 1.5 and a few more extensions
We’ll have OpenGL1.5 support too. Hurrah! There are also a few more extensions going in.

Stable Mac port
We’ve got a new developer who’s getting a Mac especially to work on LWJGL for us. As usual Alien Flux will be the “test” application.

Documentation
The increasingly-innaccurately described cruft that we claim to be “documentation” will be dealt with :wink:

NeHe
…and we’ll get these bloody NeHe tutorials ported and running properly while we’re at it.

That’s all I can think of off the top of my head. The next version’s a big one, oh yes. But the reason for that is that we’ll be going into “beta” at 0.9, meaning that we will no longer change the API save for bugfixes, and we need to get it just right this time.

Cas :slight_smile: