I would love to, just for simplicity/purity sake. But we use TextureImpl.getLastBind() and I dont speak OpenGL well enough to replace it =D
I just wrote the code to display the ByteBuffer that JVLC gave him, and it did not use PBOs, so it definitely won’t be faster on the Java side. I can’t speak for ffmpeg vs VLC though, but I don’t think there’s too much of a difference there…
No, though possibly problematic. Just had a thought related to that - if your build of FFMPEG can play h264 isn’t it GPL? (Mind you, the GStreamer libs Cero linked to also include the same GPL code, but this can be pulled out by deleting the plugin).
@theagentd - Never mind, I meant a comparison of Riven’s code with FFMPEG and Cero’s with GStreamer - had a go with that on Linux as mentioned, but was curious about Windows. Basically, what if any overhead is created by doing this in separate processes vs one (the codec code in GStreamer and VLC is from FFMPEG anyway). Not a lot by the look of it!
I deleted much in there, hopefully as much as possible for just playing theora&vorbis, certainly removed all codecs that sounded like licensed codecs I know
Version 0.7.8 of YUNPM
- Decoupled playback framerate from video framerate
[list]
[li]Allows rendering 24fps movie in game running at 60fps - Allows rendering game while movie is paused
- To showcase this feature, an annoying translucent butter smooth rotating quad is rendered on top of the video
- If you feel uncomfortable with the licensing of specific codecs in ffmpeg, please build ffmpeg yourself without said codecs
[/li]
- Discarding audio stream from video, by passing
nullAudioRenderer - Support for Fullscreen and v-sync
- Jar released for Java6 again (sorry for the previous screwup)
[/list]
Download files
[x] http://indiespot.net/files/projects/medialib/
Notes
[x]with the decoupling of movie playback and framerate, PBO performance has taken a nose dive. Currently it’s only slightly faster than directly updating textures, which means the CPU usage is significantly higher than in v0.7.7. As I’m new to the asynchronous behaviour of PBO’s, I’d appreciate it if somebody could take a look as to why this happened. In the meantime I’ll try to fix it myself, obviously.
Hilarious sample code
class VideoPlaybackTest {
public static void main(String[] args) throws Exception {
File movieFile = new File(args[0]);
boolean audioEnabled = true;
VideoRenderer videoRenderer = new OpenGLVideoRenderer(movieFile.getName());
AudioRenderer audioRenderer = audioEnabled ? new OpenALAudioRenderer() : null;
if (videoRenderer instanceof OpenGLVideoRenderer) {
OpenGLVideoRenderer opengl = (OpenGLVideoRenderer) videoRenderer;
opengl.setFullscreen(false); // uses current desktop resolution
opengl.setVSync(true);
opengl.setRenderRotatingQuad(true);
}
VideoPlayback playback = new FFmpegVideoPlayback(movieFile);
playback.setCoupleFramerateToVideo(false);
playback.startVideo(videoRenderer, audioRenderer);
/**
* oldskool controls!!
*/
BufferedReader br = new BufferedReader(
new InputStreamReader(new BufferedInputStream(System.in)));
while (true) {
String line = br.readLine();
if (line.equals("mute")) {
playback.setVolume(0.0f);
} else if (line.equals("half")) {
playback.setVolume(0.5f);
} else if (line.equals("full")) {
playback.setVolume(1.0f);
} else if (line.equals("pause")) {
playback.pause();
} else if (line.equals("resume")) {
playback.resume();
} else {
System.out.println("wait what?");
}
}
}
}
It seems that PBO performance is related to vsync and decoupling video framerate from render framerate:
[tr][td]Test 1:[/td][td]vsync[/td][td]coupled[/td][td]texture update takes 1.6ms[/td][td]25fps (video framerate)[/td][td]Java CPU usage: 7%[/td][/tr]
[tr][td]Test 2:[/td][td]vsync[/td][td]decoupled[/td][td]texture update takes 17.2ms[/td][td]60fps (vsync)[/td][td]Java CPU usage: 17%[/td][/tr]
[tr][td]Test 3:[/td][td]no vsync[/td][td]coupled[/td][td]texture update takes 1.6ms[/td][td]25fps (video framerate)[/td][td]Java CPU usage: 7%[/td][/tr]
[tr][td]Test 4:[/td][td]no vsync[/td][td]decoupled[/td][td]texture update takes 5.3ms[/td][td]180fps[/td][td]Java CPU usage: 28%[/td][/tr]
(CPU usage as reported by the task-manager on my quadcore, so multiply by four for usage on 1 core)
Some code for those interested:
private void updateTexture(ByteBuffer rgb) {
if (rgb.remaining() != videoWidth * videoHeight * 3) {
throw new IllegalStateException(rgb.remaining() + " <> " + (videoWidth * videoHeight * 3));
}
// snippet of relevant PBO code
{
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, pboHandle);
pboBuffer = glMapBuffer(GL_PIXEL_UNPACK_BUFFER, GL_WRITE_ONLY, pboBuffer);
//System.out.println(NativeHacks.getBufferAddress(pboBuffer));
pboBuffer.put(rgb);
pboBuffer.flip();
rgb.flip();
glUnmapBuffer(GL_PIXEL_UNPACK_BUFFER);
glBindTexture(GL_TEXTURE_2D, textures[textureIndex & 1]);
glTexSubImage2D(GL_TEXTURE_2D, 0/* level */, 0, 0, videoWidth, videoHeight, GL_RGB, GL_UNSIGNED_BYTE, 0);
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, 0);
}
// bind the texture, that has been filled in the previous frame (if any)
if (textureIndex++ == 0) {
currentFrameTexture = textures[0];
} else {
currentFrameTexture = textures[textureIndex & 1];
}
}
I’m getting horrible stuttering with the latest version, not only higher CPU usage. Texture updates take between 0.6ms and 50ms, very unstable. First I tried to not clear the videoUpdateBuffer after each update and that fixed it. The problem is most likely related to the GL_STREAM_DRAW flag; we’re telling OpenGL that we’ll be streaming continuously but end up doing it every X frames. That probably hits a bad path in the driver.
The proper fix was discarding the old FBO data prior to mapping. Add this:
glBufferData(GL_PIXEL_UNPACK_BUFFER, videoWidth * videoHeight * 3, GL_STREAM_DRAW);
before the call to glMapBuffer and it will be fine.
I also tried CPU asynchronous updates, as described here. That’s a decent win on CPU usage at the cost of double the memory usage (you need 2 ping-pong PBOs).
edit: The ping-pong version. Probably needs to be made optional.
I incorporated your changes, but I don’t see any improvement. I didn’t have the stuttering, so that might very well be fixed, therefore I released v0.7.9.
I even tried triple buffering (3 textures, 3 PBOs) but that gave me exactly the same results.
Well, the thing with vsync is that its implementation differs considerably between vendors and architectures. On my AMD for example I’m pretty sure it just busy-spins a lot, hence the higher CPU usage. Also, what you’re measuring as the texture update may very well include the SwapBuffers time. All GL calls are asynchronous, the corresponding action may happen long after the call returns, including SwapBuffers. The call to glMapBuffer happens to be the first call that requires some kind of pipeline flush, so you end up “paying” the vsync cost at that time. That doesn’t make PBOs slower, it simply means that timing on the CPU-side is inaccurate by nature.
Add a timer around Display.update and you’ll see that it takes less than 1ms when the PBO update takes 16+ms (16.6 ms = 60fps vsynced).
That makes sense.
16.6ms(60Hz) + 1.5ms(coupled) = 18.1ms
That 1.5ms is probably partly covered by v-sync wait/busyloop, resulting in 17.2ms per frame.
I’d expected that modern drivers are better at syncing than a busyloop, but you could argue it’s the fastest way, and nothing else matters.
I can confirm that’s probably it. I get roughly the same ‘upload time’ for 240p and 1080p videos.
The latest version here works almost as good as the “best” version. My test case is only a little worse CPU wise and well there is a much smaller start up time.
Over its working well.
I didn’t see any Mac executables on the ffmpeg website, but this one works:
http://ffmpegmac.net/
At 1080p it runs at 20 FPS and audio goes out of sync. (receiving 20ms, rendering 80ms)
I also ran into this exception when trying to resize my 1080p video. It seems to happen erratically. And occasionally when closing I get this exception.
Otherwise small files seem to work and resize properly. I’ll have to do some more tests with different codecs.
I can confirm that this exception happens. I played the video in fullscreen and tabbed out. This freezes the rendering, and when I tabbed in again it crashed with that same exception. It’s most likely occurs when some video buffer is completely filled, probably since your computer didn’t manage to display them fast enough.
EDIT:
I ca confirm this too. Measuring CPU usage is veeeery difficult when using OpenGL…
[quote=“theagentd,post:94,topic:39517”]
I recommend ARB_timer_query. Easy to use and you get very accurate timings.
For trivial rendering (like this video player) you just need to be careful with reading back the times. It will cause a pipeline flush and will skew the rest of the rendering, like SwapBuffers does to the texture update above. A trick I use is to actually measure every X frames, where X is how many frames it takes for the timing result to be available. Like so:
private int timerID;
private boolean timerAvailable = true;
{
timerID = glGenQueries();
}
private void render() {
if ( timerAvailable ) // skip if we're still waiting for the previous timing
glBeginQuery(GL_TIME_ELAPSED, timerID);
// ...the GL calls we want to time...
if ( timerAvailable )
glEndQuery(GL_TIME_ELAPSED);
// Check if the timing result is available
timerAvailable = glGetQueryObjecti(timerID, GL_QUERY_RESULT_AVAILABLE) != 0;
if ( timerAvailable ) {
long time = glGetQueryObjecti(timerID, GL_QUERY_RESULT);
System.out.println("took = " + time + " ns.");
}
}
There is already a mac port in the zip. I assumed it worked :persecutioncomplex:
The video player doesn’t drop frames. If rendering takes 80ms, you’d only ever get 12.5fps, which is not acceptable in the first place. That is goes out of sync is no surprise, given that the video can never ‘catch up’.
Regarding exception #1, that’s not supposed to happen anymore in the latest versions. I tested whether a buffer put in the pool returned true for obj1.equals(obj2) on any item already in the pool. For distinct ByteBuffers containing the same pixels, this returned true. Now I’m using ‘==’ instead. Which version (of the jars) did you use?
Regarding exception #2, that happened when ffmpeg.stdout returned an incomplete frame, which basically happens because I close the stream during closing of the player. I simply removed rethrowing the IOException as IllegalStateException.
This shouldn’t be the cause of that exception, really. I cannot reproduce it, so I hope that my latest version solves it.
Version 0.7.11 of YUNPM
- Re-enabled measuring of PBO performance, which allows fallback to plain texture updates, if the drivers pick an horrendous path. (previously reported by kappa, with ~330ms updates per frame)
- Discarded expected IOException on media player termination
Download files
[x] http://indiespot.net/files/medialib/
Well, finally got a few hours to look at tidying up the GStreamer stuff today. I initially looked at creating an alternative backend to YUNPM, but due to the difference in approach (callback vs streaming) this was leading to about 4x more code then was actually required (and I was having issues with some of the dependencies - seems a few things missing in your JAR?). So, instead we have GstLwjgl - in a separate thread so as not to pollute this one further! 
Not quite as CPU friendly yet (not too bad), but it is lacking a few optimizations.
Given our earlier comments on sync, isn’t the lack of frame-drop actually a bit of a flaw at the moment? Obviously, constant frame dropping will look terrible, but the ability to always work to the audio clock and drop the odd video frame (in case of CPU spike, etc.) would seem to be important for accurate sync - unless I’m missing something in your answer?
I own a flower shop and every wednesday a friendly guy comes in, handing out flyers for his flower shop. I offer him tea and a biscuit and we discuss unrelated business.
The smell of flower stores drives me nuts.
LOL - nice analogy! ;D Actually, you misinterpret my intentions though. I have no interest in running a flower shop, I have enough other shops to juggle at the moment! One flower shop would be fine in this town, but I disagree with growing half the product in the back garden when a reputed wholesaler can offer so much more with far less effort. Ah, you say, but I can never get hold of that wholesaler, the phone always seems to be down … here, try the number on this card instead!
You started this thread with the assertion that there was no useful code for video in Java. As I’ve pointed out before, there are lots of people away from here quite happily working with video, and have been doing for years (Processing guys particularly have this all working cross-platform). Almost all are using GStreamer, some are using VLC. Most of the code I posted above was on here months ago, and it’s a far simpler too. Use it or ignore it, I don’t particularly care - I’ll maintain it because I personally need the extra features in Praxis. Just don’t see the point in re-inventing the wheel! I personally believe that a marriage of GStreamer backend and the YUNPM rendering code is the best way forward, and while working now also offers some interesting longer term potential (hardware decoding and rendering directly to texture, streaming video from the net, etc.) If a shotgun wedding ain’t the way forward, then so be it.
Now, where’s that tea and biscuits you offered me? ;D