I am working with Video and Java/JOGL (I also use C/C++ but prefer to work in Java, and CUDA/C).
NVIDIA provided CUDA video decoder examples (one in OpenGL and one on D3D) in the CUDA 2.3 SDK that achieves fantastic
frame rates with CPU usage of 1%-2% (yes always less than 3%). This is of course achieved
by using the V(ideo) P(rocessor) available on modern NVIDIA GPUs (ATI has a similar functionality).
I, of course, would like to see VPs supported by JOGL/JSR-231 but cannot wait for that.
So my question.
Is it possible to pass a javax.media.opengl.GLJPanel or a javax.media.opengl.GLCanvas to a JNI
function that would then:
- Decode a video
- Apply a YUV to (A)RGB transform on each frame
- Transform the (A)RGB frame to a OpenGL texture
- Render the OpenGL texture on the passed GLJPanel (or GLCanvas).
All the above functions occur in the GPU basically in a single pipeline at a cost
on the CPU side of transfering the input file, block at a time, to the GPU.
The following is possible (I have done it):
- Invoke a JNI function that:
- Decodes a video (on the GPU)
- Transforms the decoded frame from YCbCR to (A)RGB (on the GPU)
-
Reads into the CPU the (A)RGB image.
- passes back to java the (A)RGB frame.
-
Java/JOGL can then create a texture from the (A)RGB frame and display it
But the above requires moving the decoded frame from the GPU to the CPU just
to send it back again to the GPU that is a huge amount of data when 1920*1080
HD video is being rendered.
David M. Gaskin