DV Video textures

OpenGL Development JOGL Development
#1

I’d like to use DV-resolution video files as textures:

  1. Video needs to be decoded into raw RGB pixels;
  2. The texture needs to be updated every frame;
  3. Fast!

Given the current speed of loading textures from
static image files (e.g. JPGs, PNGs) into JOGL isn’t
that fast: ~0.8s ± 0.2s for a 512x384 PNG into
and bound to a GL_TEXTURE_2D, using JDK5 and
Pentium-M 1.3GHz.

I’m wondering if a native approach might be better
given the performance requirements, so it’d be
something like (on Win32):

MPEG2/WMV files -> [Native] DirectShow -> [Native] Raw pixels -> [Native] glTexSubImage2D() -> [JOGL] Do rendering

So the question I have is, can I update a texture
in C++ that’s been created in JOGL? Thanks!

I know some of you will suggest JMF, but the
truth is that Sun is not developing JMF anymore,
after v2.1.1, and it doesn’t handle MPEG2, QT MP4
and WMV files, so JMF is not an option for me.

.rex

#2

JMF is shit and didn’t work. We instead wrote our own bindings to NCTVideoControls activeX components to decode video, which worked perfectly. Although we use LWJGL rather than JOGL for performance reasons.

Cas :slight_smile:

#3

Well first of all the texture upload time you suggested seems quite high. We have made a wrapper to DirectShow for videocapturing / playback and we are uploading textures (1024x512) 25fps with lots of other things going on simultaneously. Pentium-M 1.3Ghz is not that fast though and you didnt mention the graphics card you are using. But back to the question;

What I understand you can use OpenGL from C++ simultaneously with JOGL, though you have to synch calls to texture upload. And also, you could use texture_rectangle extension to get rid of power-of-two textures (720x576 instead of 1024x512).
JMF is not an option as you said. In your case the uploading speed problem can be gfx-card or the machine itself :/.

Make a native method updateTexture(textureID) which uploads texture and call it from your Java program?

  • nG
#4

Could you post the source code for your video Texture Renderer :slight_smile:

#5

How do you share (Jogl) GL context with Java and native dll code?

This approach sounds very interesting indeed. updateTexture(textId) method could use directshow, take a frame buffer as byte array, then upload it to a texture. If i am right, this should use glTexSubImage2D() method to update texture bytes.

What I do not understand at all, how to draw some 3d objects in Java Jogl side (lets say its Swing embedded canvas), and then one rotating box has a videostream texture updated by a native code.

We really dont want to copy bytes from native to java and then to Jogl GL context.

Nehe: play AVI as a live texture example
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=35

#6

just now i m writing on some JNI stuff which integrates QuicktTime and OpenGL on the native side.
the basic idea is to do everything in Jogl (opening window, draw primitives, handletextures etc) and just shove the pixels from native QuickTime to ‘native’ OpenGL.

the jogl side would look something like this:


            gl.glBindTexture(GL.GL_TEXTURE_2D, myGLTextureID[i]);
            _myJNIWrapper.updateTexture(myTextureID[i]);
            gl.glPushMatrix();
            gl.glBegin(GL.GL_QUADS);
            /* draw plane here */
            gl.glEnd();
            gl.glPopMatrix();

as you can see you don t even need to pass the OpenGL texture ID as long as you bind the texture before you go native.
the interesting thing for me to see was, that as long as you call the native methods inside a valid jogl frame, that is from inside the display(GLDrawable) method, you can use the ‘native’ opengl and jogl interchangeably.
this research is really interesting since OTJava sucks as much as JMF sucks.
i will post the outcomes as soon as i have a working windows version, the osx side is already done.

PS
the native setup is NOT mindblowingly faster than the QTJava version but much more stable and reliable.

#7 

gl.glBindTexture(GL.GL_TEXTURE_2D, myGLTextureID[i]);
_myJNIWrapper.updateTexture(myTextureID[i]);
gl.glPushMatrix();
gl.glBegin(GL.GL_QUADS);
/* draw plane here */
gl.glEnd();
gl.glPopMatrix();
--  --- 
the interesting thing for me to see was, that as long as you call the native methods inside a valid jogl frame, that is from inside the display(GLDrawable) method, you can use the 'native' opengl and jogl interchangeably.
this research is really interesting since OTJava sucks as much as JMF sucks.
i will post the outcomes as soon as i have a working windows version, the osx side is already done.

I really appreciate if you could post a short example. OSX-QT test is ok as well, I have a one OSX machine althought I never use it. This example would be a good reason to do so :slight_smile:

Did you mean you don’t need to call any glEnable(this-and-thatn), glBegin, glEnd, glLoadIdentity, etc… if you have done so in Java side?

You could do simply:


void jni_from_Java_grabVideoFrame() {
  char *pTextureData;
  pTextureData=(char*) doMagicAndGrabFrame();
  glTextSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 256, 256,  GL_RGB, GL_UNSIGNED_BYTE,  pTextureData);
}

Basicly, native side is really a small funtion extension to Jogl library?

Cant wait to see your example and try it out.

#8

i posted a link to some code on this thread:

So you like Textures eh? Texturing Demo in JOGL.

it shows how to use native QuickTime and native OpenGL from within Java and Jogl.