Video playing in game

@OP: What holds you back giving YUNPM a try?

It has never crashed or frozen, yet :slight_smile:

That’s almost certainly a bug in your code. Try the same video in Praxis LIVE and tell me if you see the same. It looks like your code is stopping and restarting the pipeline when looping, which is not the way to do it!

Never seen that, but having taken over maintainership of the bindings for GStreamer 1.x I’d love to see a thread dump! I’m aware of one rare deadlock scenario that needs fixing, which shouldn’t be related to this unless you are running that init() code in multiple threads at the same time? Could also be an issue with GStreamer 0.10 itself which is unsupported now.

There are pros and cons of running things out of process. For the OP, I agree it’s probably the easiest way to get started, for simple use cases anyway :wink: Mind you, I guess no way of benefiting from GPU decoding using that model?

Yeah, I am too noob for these video rendering libraries. :slight_smile:

You are turning vague handwaving into a real art. There are pros and cons, sure, just like with anything, really. So what’s your point? GPU decoding is handled by ffmpeg, regardless of the export-format, obviously, as it is GPU decoding. What does he need beyond simple use cases anyway, he’s trying to play a video in game (hence the topic), not writing a full fledged media player.

First he should try getting LibGDX to work, as that ought to have the best support. If that fails, give YUNPM or gstreamer a try. YUNPM’s API is vastly simpler than gstreamer’s, but limited to solely playback of media (it does audio sync, it supports volume adjustments, it supports pausing, skipping frames), has some minor inter-process overhead (unless you go over full HD, then it becomes measurable), it’s not as simple as LibGDX’s API.

Me too, its okay x)

Yeah I should seek to position 0.
aaand I just changed the code to: getPlayBin().seek(0, TimeUnit.NANOSECONDS);
seems to be super smooth now

I am using gstreamer-java-1.6.jar
It happens like all the time with the code I wrote, once it starts, it works fine.
If you can tell me where the dump would be, I could show you, tho I suspect there is no dump, as it just takes forever and never finishes, and I force it to close.

I just tested it and its not the init function, its after that. Lemme find where it is…

Edit: Its on playbin.play();
it doesnt come back from that
In this particular example I am calling play as the last line of the create in libgdx, so the only mistake that I could have made is maybe calling it too early.
Tho is also works many times… its like 50% chance upon starting up :stuck_out_tongue:

Well, got to be good at something! ;D

But you’re bringing it back into CPU memory, right? I may be wrong, but it’s not possible / easy to GPU decode directly into a texture because it would require texture sharing across process boundaries?

There are lots of interesting things that could be done with video in a game that aren’t just about playing cut scenes - eg. ā€œscratchingā€ back and forth to create animated textures.

I figured out the problem.
This is my videoplayer class: http://pastebin.com/sdtCEnK7

I used to call play() directly after this init.
As I can see there is some async going with the threads. As such I assume it simply hasnt finished loading sometimes.

Now it never freezes with manually playing shortly after

@nsigma: He could, but he doesn’t, so who cares :slight_smile: Get it to work first, I’d say.

Which is actually a binding for the old GStreamer 0.10 series - don’t get me started on the version mismatch! :wink: The binding for GStreamer 1.x is gst1-java-core as linked above, although it’s still in development. Mind you, you’d have to replace all your native libraries too.

deep breath.
Alright, noted.
I will do it in like 2-3 weeks when I have more time and then publish it maybe, cant hurt.

I have a suggestion…

Could you guys load in the video then provide the option to EXPORT the video in a raw-ish file.

I think it would be beneficial to everyone if…

BMP-like header
Image-like pixel storage

All I want to do is grab the initial pixel data given the data from a header and invoke each change to the pixel data over a course of time specified in the header.

<3

Do you have any idea how massive the file would be?

This is exactly what YUNPM does (apart from providing a stream, instead of a file): it passes a raw RGB24 stream for the video, and a raw U16-stereo stream for the audio. You can do with it whatever you want - YUNPM also gives you this video player, but you may ignore that part.

@nsigma: the important part is that the heavy calculations (decoding) are done on the GPU. A full HD movie @ 24Hz means 142MB/sec of raw RGB24. Moving that amount from the GPU to the CPU, through the process pipes, and into the GPU again, seems highly inefficient, but it’s hard to even measure. This is why YUNPM has almost the same performance (CPU and GPU usage) as the native video players (like VLC). Only for 4k UHD the difference became noticable on my outdated PC.

Probably very big. Although I can think of several aids to that, such as arrays per line.

/header/
width=2
height=2
/endheader/
/image/
1100255255,2255100255 //first pixel to 100,255,255 and 2nd to 255,100,255
1255255255,2255255255 //both 2 pixels are 255,255,255
1125000000 //change first pixel R 125, change 2nd pixel G 125 but there is no more pixels to change
// This is a blank space - no change to this row of pixels
/endimage/

Is it open source with no natives?

Open source with ffmpeg natives (no libraries, just executables). YUNPM itself doesn’t have native code, it uses ffmpeg and lwjgl, but you can even use AWT/Swing as frontend, copying the byte[wh3] into the data-storage of a BufferedImage and using g.drawImage(…) This, btw, was slightly faster (!) than using advanced OpenGL trickery to stream frames into textures.

Yeah I dont have any personal reasons for favoring gstreamer over yunpm.
its just that I have a gstreamer solution that works with libgdx and I am not too much of an opengl guy.
Plus there were some minor licensing concerns with the codec included with ffmpeg and I dont wanna recompile myself to remove 'em and fiddle.
Not that I think one would really get into trouble for that… maybe

I need all the basics of course like pause, seek, volume control, having audio and video perfectly synced no matter what… but I assume yunpm has all that anyway.

It does all of that, except that it lacks support for seek, but it’s trivial to add.

I’m very curious, how can a ~80-100 MB video file packed into an application. I have used adviced converter, but output video files are pretty big.

I have already checked earlier posts in gStreamer topic, @Cero’s Dropbox sharing, but links are dead. What a shame!

English? Do you mean how 80-100 mb video file comes out so big? 80-100mb is pretty big in itself. If that is the output file, then it’s fine.