Mipmapping Frustrations

I have spent the past few weeks working to understand and implement my own alternative to TextureRenderer and though I think I’m nearly done, I’m struggling with mipmapping still. If I use GL_GENERATE_MIPMAP the quality in a zoomed out state for text is not crisp, so I endeavored to write my own mipmap generator to use BufferedImage scaling with bicubic interpolation to give a better appearance, but even with this it does not appear crisp:

http://captiveimagination.com/download/gearloop03.jpg

The relevant source code follows. I don’t know what else I can do to make the appearance better:

	public void update(GL gl, BufferedImage image, int x, int y, int width, int height, int mipmap, Component component) {
		if (image.getType() != BufferedImage.TYPE_INT_ARGB_PRE) {
			throw new RuntimeException("Unhandled BufferedImage format. Use TYPE_INT_ARGB_PRE. Type: " + image.getType());
		}
		
		// Create ByteBuffer for pixel data
		if ((this.width != width) || (this.height != height)) {
			recreate = true;
		}
		int imageFormat = GL.GL_RGBA;
		int textureFormat = GL.GL_BGRA;
		int type = GL.GL_UNSIGNED_INT_8_8_8_8_REV;
		
		gl.glBindTexture(GL.GL_TEXTURE_2D, textureId);
		gl.glDisable(GL.GL_LIGHTING);
		
		gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_LINEAR);
		gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, mipmap != Shape.MIPMAP_OFF ? GL.GL_LINEAR_MIPMAP_LINEAR : GL.GL_LINEAR);
		gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_S, GL.GL_CLAMP_TO_EDGE);
		gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_T, GL.GL_CLAMP_TO_EDGE);
		if (mipmap == Shape.MIPMAP_AUTO) {
			gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_GENERATE_MIPMAP, GL.GL_TRUE);
		}
		gl.glPixelStorei(GL.GL_UNPACK_ALIGNMENT, 1);
		int mipmaps = (int)Math.floor(Math.log(Math.max(width, height)) / Math.log(2)) + 1;
		if (recreate) {
			gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, imageFormat, width, height, 0, textureFormat, type, null);
		}
		
		updateSubImage(gl, image, x, y, width, height, 0, imageFormat, textureFormat, type);
		if (mipmap == Shape.MIPMAP_HIGH) {
			ReusableGraphic rg = ReusableGraphic.get("mipmap");
			
			int w = width;
			int h = height;
			for (int i = 1; i < mipmaps; i++) {
				w = Math.max(1, w / 2);
				h = Math.max(1, h / 2);
				
				if (w == 0) {
					w = 1;
				} else if (h == 0) {
					h = 1;
				}
				
				Graphics2D g = rg.request(w, h);
				try {
					component.rendering.setHints(g, component);
					g.drawImage(image, 0, 0, w, h, x, y, width, height, null);
					g.dispose();
					
					if (recreate) {
						gl.glTexImage2D(GL.GL_TEXTURE_2D, i, imageFormat, w,h, 0, textureFormat, type, null);
					}
					updateSubImage(gl, rg.getImage(), x, y, w, h, i, imageFormat, textureFormat, type);
				} finally {
					rg.release();
				}
			}
		}
		
		this.width = width;
		this.height = height;
		recreate = false;
	}
	
	private void updateSubImage(GL gl, BufferedImage image, int x, int y, int width, int height, int level, int imageFormat, int textureFormat, int type) {
		// Create ByteBuffer for pixel data
		int[] data = new int[width];
		WritableRaster raster = image.getRaster();
		ByteBuffer buffer = ByteBuffer.allocateDirect((width * height) * 4);
		buffer.order(ByteOrder.nativeOrder());
		IntBuffer pixels = buffer.asIntBuffer();
		for (int i = 0; i < height; i++) {
			raster.getDataElements(x, y + i, width, 1, data);
			pixels.put(data);
		}
		pixels.flip();
		
		gl.glTexSubImage2D(GL.GL_TEXTURE_2D, level, 0, 0, width, height, textureFormat, type, pixels);
	}

When I was using TextureRenderer everything was crisp and clean.

Old technique still seems best…

Crisp:


   private static BufferedImage scale1(BufferedImage full, int w, int h)
   {
      BufferedImage scaled = new BufferedImage(w, h, BufferedImage.TYPE_INT_RGB);
      Image im = full.getScaledInstance(w, h, Image.SCALE_AREA_AVERAGING);
      scaled.getGraphics().drawImage(im, 0, 0, null);
      im.flush();
      return scaled;
   }

Crap:


   private static BufferedImage scale2(BufferedImage full, int w, int h)
   {
      BufferedImage scaled = new BufferedImage(w, h, BufferedImage.TYPE_INT_RGB);
      Graphics2D g = scaled.createGraphics();
      g.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BICUBIC);
      g.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_QUALITY);
      g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);
      g.drawImage(full, 0, 0, w, h, null);
      g.dispose();
      return scaled;
   }

Nice read:

http://today.java.net/pub/a/today/2007/04/03/perils-of-image-getscaledinstance.html

Thanks for the information Riven, it did seem to make it look a lot better, but the performance drop was more than considerable, it was devastating. I think that link you gave me should help with performance though.

Thanks again for helping get on the right track.

Riven, after reading that and applying what it suggested I still don’t seem to be getting the crisp appearance I’d like.

Perhaps someone will see something I’m missing in my code?

package org.jseamless.gl;

import java.awt.Graphics2D;
import java.awt.Image;
import java.awt.RenderingHints;
import java.awt.image.BufferedImage;
import java.awt.image.SampleModel;
import java.awt.image.SinglePixelPackedSampleModel;
import java.awt.image.WritableRaster;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.IntBuffer;

import javax.media.opengl.GL;

import org.jseamless.Component;

public class GLTexture {
	private int textureId;
	private boolean recreate;
	private int width;
	private int height;
	
	public GLTexture(GL gl) {
		// Generate texture id
		int[] id = new int[] {textureId};
		gl.glGenTextures(1, id, 0);
		textureId = id[0];
	}
	
	public void update(GL gl, BufferedImage image, int x, int y, int width, int height, int mipmap, Component component) {
		if (image.getType() != BufferedImage.TYPE_INT_ARGB_PRE) {
			throw new RuntimeException("Unhandled BufferedImage format. Use TYPE_INT_ARGB_PRE. Type: " + image.getType());
		}
		
		// Create ByteBuffer for pixel data
		if ((this.width != width) || (this.height != height)) {
			recreate = true;
		}
		int imageFormat = GL.GL_RGBA;
		int textureFormat = GL.GL_BGRA;
		int type = GL.GL_UNSIGNED_INT_8_8_8_8_REV;
		
		gl.glBindTexture(GL.GL_TEXTURE_2D, textureId);
		gl.glDisable(GL.GL_LIGHTING);
		
		gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_LINEAR);
		gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, mipmap != Shape.MIPMAP_OFF ? GL.GL_LINEAR_MIPMAP_LINEAR : GL.GL_LINEAR);
		gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_S, GL.GL_CLAMP_TO_EDGE);
		gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_T, GL.GL_CLAMP_TO_EDGE);
		if (mipmap == Shape.MIPMAP_AUTO) {
			gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_GENERATE_MIPMAP, GL.GL_TRUE);
		}
		gl.glPixelStorei(GL.GL_UNPACK_ALIGNMENT, 1);
		gl.glPixelStorei(GL.GL_UNPACK_SKIP_ROWS, 0);
		gl.glPixelStorei(GL.GL_UNPACK_SKIP_PIXELS, 0);
		int mipmaps = (int)Math.floor(Math.log(Math.max(width, height)) / Math.log(2)) + 1;
		if (recreate) {
			gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, imageFormat, width, height, 0, textureFormat, type, null);
		}
		
		updateSubImage(gl, image, x, y, width, height, 0, imageFormat, textureFormat, type);
		if (mipmap == Shape.MIPMAP_HIGH) {
			ReusableGraphic rg1 = ReusableGraphic.get("mipmap1");
			ReusableGraphic rg2 = ReusableGraphic.get("mipmap2");
			
			int w = width;
			int h = height;
			boolean inverse = false;
			BufferedImage img = image;
			int pw = width;
			int ph = height;
			for (int i = 1; i < mipmaps; i++) {
				w = Math.max(1, w / 2);
				h = Math.max(1, h / 2);
				
				if (w == 0) {
					w = 1;
				} else if (h == 0) {
					h = 1;
				}
				ReusableGraphic rg = inverse ? rg2 : rg1;
				Graphics2D g = rg.request(w, h);
				try {
					component.rendering.setHints(g, component);
					g.drawImage(img, 0, 0, w, h, x, y, pw, ph, null);
					g.dispose();
					
					if (recreate) {
						gl.glTexImage2D(GL.GL_TEXTURE_2D, i, imageFormat, w,h, 0, textureFormat, type, null);
					}
					updateSubImage(gl, rg.getImage(), x, y, w, h, i, imageFormat, textureFormat, type);
				} finally {
					img = rg.getImage();
					inverse = !inverse;
					rg.release();
					pw = w;
					ph = h;
				}
			}
		}
		
		this.width = width;
		this.height = height;
		recreate = false;
	}
	
	private void updateSubImage(GL gl, BufferedImage image, int x, int y, int width, int height, int level, int imageFormat, int textureFormat, int type) {
		// Create ByteBuffer for pixel data
		int[] data = new int[width];
		WritableRaster raster = image.getRaster();
		ByteBuffer buffer = ByteBuffer.allocateDirect((width * height) * 4);
		buffer.order(ByteOrder.nativeOrder());
		IntBuffer pixels = buffer.asIntBuffer();
		for (int i = 0; i < height; i++) {
			raster.getDataElements(x, y + i, width, 1, data);
			pixels.put(data);
		}
		pixels.flip();
		
		gl.glTexSubImage2D(GL.GL_TEXTURE_2D, level, 0, 0, width, height, textureFormat, type, pixels);
	}
	
	public void draw(GL gl, int textureCoordinatesId) {
		gl.glTexEnvi(GL.GL_TEXTURE_ENV, GL.GL_TEXTURE_ENV_MODE, GL.GL_MODULATE);
		gl.glBindTexture(GL.GL_TEXTURE_2D, textureId);
		
		gl.glEnableClientState(GL.GL_TEXTURE_COORD_ARRAY);
		gl.glBindBufferARB(GL.GL_ARRAY_BUFFER_ARB, textureCoordinatesId);
		gl.glTexCoordPointer(2, GL.GL_FLOAT, 0, 0);
	}
	
	public void dispose(GL gl) {
		gl.glDeleteTextures(1, new int[] {textureId}, 0);
	}
}

If it’s any help to anyone I switched back to TextureRenderer in JOGL to grab a screenshot of what it should look like at that scale:

http://captiveimagination.com/download/gearloop04.jpg

Maybe the “wrong” mipmap is selected. You could try messing with glTexEnvf(GL_TEXTURE_FILTER_CONTROL, GL_TEXTURE_LOD_BIAS, …) and see if that helps.

Maybe a silly question… but:

why are you mipmapping? the text is not scaled, and if it is, it shouldn’t be - just render is a bit smaller

I’m creating a UI system that can be rendered once and as the screen resizes it scales. As well this is being displayed in standard projection so eventually I would like the UI to display in a full 3D context so mipmapping is ultimately necessary no matter what. I’ve been tempted to just have it regenerate the textures as the size changes, but that only solves the immediate problem, not the long-term one. It’s obvious that TextureRenderer is able to accomplish what I want, it’s just a matter of figuring out how it does it. I guess that’s where I need to go now and dig line-by-line until I figure out what it’s doing differently I was just hoping someone here might be able to point out what I’m missing.

Argh! I’ve stepped through TextureRenderer three times now and still can’t seem to figure out what it is that I’m missing…obviously something, but I’m pulling my hair out trying to figure out what it is.

Which mipmap mode are you using for your update() call, is it Shape.MIPMAP_AUTO or Shape.MIPMAP_HIGH? It might be that auto-generating the mipmaps with OpenGL provides bad results??

Looking through you code, I saw nothing wrong with the gl stuff and my only suggestions are looking into LOD bias (as mentioned before) and experimenting with the different modes.

I get about the same result whether I use HIGH or AUTO…HIGH seems slightly better, but nothing compares to what TextureRenderer generates.

I was going to start reading about GL_TEXTURE_FILTER_CONTROL, but looking through TextureRenderer it does not appear to be used there at all, so I just assumed that’s not the solution.

I’m really at a loss at this point, I can switch between TextureRenderer and my code and get this drastic difference and I’ve debugged through line-by-line comparing what TextureRenderer is doing and I can’t find the difference. I don’t suppose the person that wrote TextureRenderer hangs out in here ever? :wink: