antialiasing question

OpenGL Development JOGL Development
#1

I would like to add antialiasing to a jogl program I made. I have a large polygon with many tiny polygons on it. The tiny polygons rarely appear correctly without antialiasing and there are lots of jagged edges where the large polygon touches the background color in my scene.

What is the best way to implement this given my situation?

I am embedding the opengl stuff in an SWT window. If I understand, this means I cannot use code like this because I don’t use a standard GLCanvas:
GLCapabilities cap = new GLCapabilities();
cap.setSampleBuffers(true);
cap.setNumSamples(2);

SWT has an object similar to GLCapabilities called GLData which is passed to the SWT version of GLCanvas. It has two variables sampleBuffers and samples which I have set to 1 and 16, respectively. I really don’t know how to proceed beyond this (or even if what I did was useful). I have seen reference to using gl.glEnable(GL.GL_POLYGON_SMOOTH) in many places online, but whatever snippets of code I try, they never seem to work. I feel I am missing something or have something present that is interfering. Antialiasing does work on my computer as I can enable it by overriding the application via my nvidia control panel.

#2

I believe GL_POLYGON_SMOOTH applies anti-aliasing to the edges of the polygons so it’s not probably not what you’re looking for. You could try looking into using the accumulation buffer to do antialiasing. This method is slower than using the multi-sampling that’s enabled through GLCapabilities because you have to re-render your scene multiple times, jittering the camera less than a pixel, and accumulating each render into the accum buffer to get the antialiased result. Even if it’s slow it’s forgiving on requirements needed to use it. I believe the opengl redbook has a good explanation of how to set it up.

#3

Wouldn’t smoothing the edges of the polygons be exactly what I am looking for? I’ve read about using the accumulation buffer in the red book before, but I got the impression that this would introduce a slight blur everywhere on the screen, not just the edges of the polygon. Plus, I am not really going to be running my program on the most powerful machines.

Specifically regarding GL_POLYGON_SMOOTH, how do I make this work? Merely using glEnable and glHint for this does not “turn on” antialiasing.

#4

I believe I heard somewhere that without setting edge flags on all of your primitives, the problem with POLYGON_SMOOTH is that the edges will be smoothed even within a mesh, between adjacent polygons, in which case it would look ugly. I don’t know how to set it up correctly, I figured the glEnable() should do the trick. Try reading the opengl spec.

#5

I got the GL_POLYGON_SMOOTH to work now, and I think I am running into the problem that you predicted. Visually I am seeing lines on the edges of the polygons that compose the large object that I am rendering. It is problematic since the object has a texture on it.

Are these the only two techniques to do antialiasing in opengl (the polygon smooth and the accum buffer)? It seems like the polygon smooth technique is not useful if it results in these visual artifacts and the depth buffer can’t be used. Everywhere I read online, I encounter warnings that using the accumulation buffer for this purpose isn’t practical for real-time rendering.

Is there a more official way to do antialiasing? And what is this used for: gl.glEnable(GL.GL_MULTISAMPLE); … it doesn’t seem to do anything when I used it.

#6

[quote=“john9231,post:5,topic:31302”]
Actually there are two methods that are used in 3D graphics these days that are considered the appropriate way to do anti-aliasing. The multisampling will do the polygon edges and the anisotropic filtering will anti-alias the polygon interiors when texturing.

Some quick google searching found me this multisample example/tutorial which looks similar to how you started doing things…
(and you might want to look into using more than 2 samples, all modern cards support at least 4, and many of them support even more)

http://www.java-tips.org/other-api-tips/jogl/how-to-use-multisampling-to-draw-anti-aliased-geometric-primi.html

#7

Well I don’t have to worry about anisotropic filtering…I already have that working. I have already seen that nehe example you posted, but I wasn’t able to get it to work. Simply enabling GL_MULTISAMPLE doesn’t do anything.

Does anyone know how to do antialiasing using these: EXT_framebuffer_multisample or ARB_multisample? I can’t compile my code with these in it. I don’t know what to import if that even is how to resolve that.

#8

[quote=“john9231,post:7,topic:31302”]
I just tried running that code on my machine and it works just fine. 2x AA might not be enough to see a major improvement, but 4 or 8 certainly should.

#9

The tricky thing about multisampling is that if you enable it but only have 1 multisample buffer available (which is the default), opengl behaves the same as normal. For multisampling to work you need to somehow set the GLCapabilities to get more sample buffers (which is already a problem for you), and then you can set the enable flag.

#10

I can set the samplebuffers variable to any number I want in the SWT-version of GLCapabilities (I don’t know if this has a real effect or not). I tried values between 1 and 16 and nothing seemed to change anything.

In my third post I said that I was able to get antialiasing to work, but it caused rather visible lines to appear where polygons touched (they were forming a giant sphere). Is this a dead-end or can I do something to remove the artifacts? I’m not sure what to do about the edge flags because I create the object in a display list.

The only two methods I have seen for AA in jogl are using this:
gl.glEnable(GL.GL_BLEND);
gl.glBlendFunc(GL.GL_SRC_ALPHA,GL.GL_ONE_MINUS_SRC_ALPHA);
gl.glEnable(GL.GL_POLYGON_SMOOTH);
(which leads to those lines on the polygons in the mesh)

and using the accum buffer (which I assume is not practical for AA in real-time). Are these the only two ways to do AA with jogl? I’m a little confused if enabling GL_MULTISAMPLE constitutes a separate method or not.

#11

I went back and successfully ran the nehe example with GL_MULTISAMPLE, but I still can’t figure out why nothing happens placing that in my code. I printed out the number of samples and sample buffers, and to my surprise they were both 0 even though in theory I was setting them to other values.

#12

Are you changing other settings in the capabilities? It’s possible you’ve picked a format that’s incompatible with AA.

#13

That’s a possibility. I don’t set anything other than double buffering, the number of sample buffers, and the number of samples via the GLCapabilities equivalent. If you want to take a look, this is what I am using:
http://help.eclipse.org/stable/nftopic/org.eclipse.platform.doc.isv/reference/api/org/eclipse/swt/opengl/GLData.html

I create this GLData object, set its variables, then throw it into the constructor of the SWT-version GLCanvas. After that nothing else is SWT-specific.

#14

Alright, so I tried getting multi-sampling to work with my gl code and there is one more step i forgot. You have to set the number of sample of buffers to 2, 4, 8, or 16 (although older cards might not be able to hold more than 2 or 4). Then in GLCapabalities you also have to setSampleBuffers(true) so that you are using them. Then in your gl code, you call glEnable(GL.GL_MULTISAMPLE). This worked for me, so if you do the swt equivalent it should work too.

#15

This is what I am doing now:

GLData glData = new GLData();
glData.doubleBuffer = true;
glData.sampleBuffers = 2;
glData.samples = 8;

The glData object (the equivalent of GLCapabilities) is then placed in the contructor of my SWT GLCanvas. Later on in the program I try to enable GL_MULTISAMPLE, but no antialiasing occurs. I use gl.glGetIntegerv(GL.GL_SAMPLE_BUFFERS,buf, 0); and gl.glGetIntegerv(GL.GL_SAMPLES,s,0); to query the buffers and samples I am using and it returns 0 in both cases. If I were to “force” antialiasing through a control panel outside my application, antialiasing does work. I have an nvidia 8400 so the problem shouldn’t be my hardware. It seems like the samble buffer and samples values that I am setting in glData are not making it through.

#16

I’m not up to date on the spec or how to actually make it work, but something that jumps out at me as potentially problematic is having sampleBuffers = 2, try setting that to 1, and samples = 4 (nearly everything that supports AA supports at least 4x) for now.

Like I said, I’m not exactly sure what sampleBuffers refers to, but if I were to make an educated guess I’d say you want it set to 1.

Edit: Actually looking at the output of the Nehe sample that’s what is happening there. My educated guess for the reason that you want 1 sample buffer is because it is only applied to the back+zbuffer and the samples are resolved on the flip to the front buffer.

#17

[quote=“lhkbob,post:14,topic:31302”]
6 is also a valid value on ATI cards. And given the number of AA modes many newer cards have, I wouldn’t be surprised if other numbers correspond to multisample+supersample modes, and lord knows how to turn on the transparency AA or the temporal AA other than through the control panel.

#18

I’ve tried 1 sample buffer for a long time until Ihkbob suggested “You have to set the number of sample of buffers to 2, 4, 8, or 16 (although older cards might not be able to hold more than 2 or 4).” I’ll go back and try to see if I skipped any sample buffer / sample combinations.

I know at least that any combination with sample buffers set to 1 or 2 and samples set to 2, 4, 8, 16 does not produce antialiasing for me.

#19

That’s very strange.

Hopefully Ken Russel will be able to add some input at some point tomorrow, if anyone can get this sorted out he’s the guy.

#20

Sample buffers are essentially multiple framebuffers that store all the pixel data for each sample, so that they can be automatically combined into the final buffer and appear antialiased.

I noticed that you set doubleBuffer to true, although it’s sampleBuffers that have to be enabled, I don’t know if this is because there was no boolean option to enable sample buffers. I’m not sure what the spec is for GLData, so I don’t know what the difference is between sampleBuffers and samples in their implementation. It could very well be that SWT isn’t as mature and can’t handle multisample buffer requests to the drivers (but that’s just a guess).