VLCJ in OpenGL / LWJGL

OpenGL Development
#1

Yeah another thread from me about video playback. I’m just hell bent on getting video playback.

Basically, VLCJ works great - in Swing.
However there are no examples for using it in OpenGL.

Emailed with the creator, Mark Lee, and he said:

[quote]use DirectMediaPlayer and a class that implements RenderCallback - this implementation copies the native video memory to an OpenGL texture
[/quote]
http://vlcj.googlecode.com/svn-history/r679/trunk/vlcj/javadoc/uk/co/caprica/vlcj/player/direct/DirectMediaPlayer.html

I’m not even an OpenGL guy, so this is above my head - can anybody get this working with LWJGL ?

#2

From the Javadoc, the class that implements RenderCallback should retrieve the frame data from com.sun.jna.Memory.
Also from the Javadoc, if you need the RGB data, use RenderCallbackAdapter instead.

#3

You seem to be able to get a ByteBuffer from Memory, so why not directly send that to OpenGL using glTexImage2D or something?
How do you get VLCJ working? Video playback is something I’ve tried getting to work for a long time too, but I never really got anywhere. Does it require you to install VLC? Any native files?

#4

VLCJ has been used with JMonkeyEngine 3 and the LWJGL renderer as you can see in the screenshots. I hope this helps…

#5

I’m still holding out for a pure Java Ogg Theora solution :confused:

Cas :slight_smile:

#6

What about curtado?

#7

Didn’t work properly last time I tried but that was a year ago. I may revisit it.
Last release was May 2010, so I suspect it’s still not quite right.

Cas :slight_smile:

#8

jME3 has currently no video playback - it’s deprecated
http://jmonkeyengine.org/groups/general-2/forum/topic/video-playback-in-jme3/?_wpnonce=84688917db

I got VLCJ to work by setting the jna library path -> System.setProperty(“jna.library.path”, “vlib”);
in vlib I have the vlc dlls and a plugin folder which holds the codecs, and some other commands to specifically look there

new MediaPlayerFactory(new String[] {"--no-video-title-show", "--plugin-path=vlib/plugins"});

and so on. Works on Windows and Linux
Problems are here: cannot use in OpenGL; as is the problem with ALL video solutions
and well… codec shipping legal issues
If you only ship the theora codec it should be fine. However out of every codec in vlc’s beautiful codec collection, that one is not stable: http://forum.videolan.org/viewtopic.php?f=14&t=78720&p=313041#p313041

Either I get one of these working,
or program a kind of cutscene system in which I can at least show images and do effects, kinda like a visual novel
or I gotta go to XNA ;D

It’s so crazy that there is no easy way in java. Every game has videos normally. Start up, nvidia logo, company logo, intro fmv, then main menu.
And it’s just… well it’s hard doing in-game cutscenes for everything.
It’s like Final Fantasy VII, sure there are in-game scenes with stuff happening, but sometimes you need scenes of WEAPON attacking Junon, or whatever

#9

Huh? Why?

That would seem about the way to do it. Don’t use the RenderCallbackAdaptor, as you don’t want to copy the data into Java unnecessarily. Just beware that the ByteBuffer returned from JNA’s Pointer class is not a copy, and therefore is the same memory that VLC is going to keep writing into - don’t do anything daft with it! :slight_smile:

Also, speaking of legal issues, I assume you’re aware that VLC and VLCJ are GPL? If you need a more liberal solution it might be worth looking at the GStreamer-Java bindings - they’re LGPL. You could also look at the code in the GSVideo library for Processing which uses GStreamer-Java and ships with the precompiled native libs for Windows and Mac, and also has some code somewhere to directly update an OpenGL texture from the video stream.

Best wishes, Neil

#10

I’ll try it out when I get time. I’m so busy gaming right now. >_>

So because VLCJ is GPL, I’d have to release my entire game under GPL too?
This GStreamer binding (no idea what GStreamer is), can you also get access to frame memory like in VLCJ? Avoiding copying millions of pixels 24 times per second is pretty important…

#11

Yes! IANAL blah de blah ;D

hmmmm… :stuck_out_tongue:

The binding is JNA based like VLCJ, and has been around a bit longer. And yes, you can get direct memory access to the pixels in the same way.

#12

Because they are all for Swing/AWT and I am not an OpenGL programmer.

GStreamer is the only I haven’t looked into as thoroughly yet. As it seems to be similar to VLC.
And yeah I know there is a processing thing - running on 3 OS’… getting the code from that just seems like a nightmare, crawling through processing source code…

Yes, Mark Lee told me too. Not THAT much of a problem.

#13

As far as I’m aware, both GStreamer-Java and VLCJ started off rendering into whatever the native video component provided by the library is. After that, direct pixel access was provided through JNA pointers which has allowed rendering into Swing / AWT / BufferedImage or whatever. You can do what you want with that pixel data, though.

btw - this is all interesting timing as I’m working on the OpenGL renderer for Praxis, and that already uses GStreamer-Java for video playback, so I’m currently looking into the best ways of getting the data directly into OpenGL.

You don’t need to look through all the Processing code!!! GSVideo just has a few classes that deal with loading the native GStreamer libs on various platform that might be useful.

#14

Libs written for Processing import and use Processing classes like PImage, so its not portable to non-processing without effort.

Looking into some gStreamer & LWJGL stuff right now.

#15

Yes, sorry, should have been clearer. There’s stuff that may be useful, but you’ll have to hack it out! See how you get on with GStreamer-Java by itself first - the GSVideo stuff is only really useful if you want to ship the native libs yourself, and that’s doable without it.

I’ve used this binding quite a lot, so I can try and answer any questions you have about it. Maybe post them in a “GStreamer-Java in OpenGL / LWJGL” thread! :slight_smile:

gStreamer in LWJGL
#16

The maker of vlcj wrote me an email today

he wrote this, using jme3: http://code.google.com/p/vlcj/wiki/JMonkeyEngineExample

#17

RenderCallback: That looks ridiculously easy! No need for a 16-bit RGBA texture though… xD

#18

So, I’m painfully close in getting it to work.

See the link in my above post for reference (RenderCallback in VLCJ with display(Memory))

Source: http://www.java-gaming.org/index.php?action=pastebin&hex=33ad12326

almost there. Maybe I’m doing something incredibly stupid here with the buffers. But I’m not an OpenGL programmer =D

Getting:

[quote]Exception in thread “main” java.lang.IllegalArgumentException: Number of remaining buffer elements is 2785280, must be at least 6291456. Because at most 6291456 elements can be returned, a buffer with at least 6291456 elements is required, regardless of actual returned element count
[/quote]
To bottom line this: Once it works, video playback works.
You can ship VLC, just like a private JRE, including all natives for windows, linux and mac, just like with the lwjgl dlls.

Then there is only license problems, with java library and codecs.
To solve them: only include theora & vorbis codec
Make your code GPL, or at least that part that uses/communicates with VLCJ, still have to know which one

#19

The problem is that you shove a 1280x554 raster into a texture with the dimensions 2048x1024. It won’t fit.

You either have to pad a lot of pixels, or use non-POT textures. I think the latter will be the best option.

[quote]To bottom line this: Once it works, video playback works.
[/quote]
I don’t think you’re as near as you think you are. The ‘buf’ variable is never really filled with data, so you’d be rendering a black screen, although even that wouldn’t work, as you never actually create a texture, you simply call GL11.glBindTexture(GL11.GL_TEXTURE_2D, 1); where ‘1’ should be the handle that glGenTextures provided. This will result in a crash.

#20

I removed the get2Fold parts.
no error.
however no picture, the texture is white - this happens when you bind something thats not there usually

GL11.glBindTexture(GL11.GL_TEXTURE_2D, 1);

0, 1, 2, nothing works

so not sure what the problem is