YUNPM - a Java Media Player

Introducing: YUNPM (Y U NO play movie!) library name suggestions are welcome…

To put it mildly, Java movie playback is a joke, and waiting for JavaFX to become the defacto standard might take yet another decade. That’s why I’ve decided to bite the bullet and make a 2nd attempt (after VLC failed) to make a minimalistic Java Video Player implementation, using an external process to decode the video and audio in a media file: this time using ffmpeg.

Approach

  • Launch ffmpeg: make it generate a mjpeg stream piped to stdout
  • Launch ffmpeg: make it generate a wav stream piped to stdout
  • Use Matthias Mann’s JPEG decoder (replacing ImageIO) to decode the chunked mjpeg stream
  • Use the audio (if any) to sync the video stream per frame, otherwise syncs to video framerate

The media playback code is fairly OO, which means you can easily swap out the current OpenGL and OpenAL renderers, and tie the library to AWT + JavaSound if that floats your boat.

Download files
[x] http://indiespot.net/files/projects/medialib/ (source included)

The archives contain an example video and launch scripts, so testing it is simply a matter of extract + click!

Hilarious sample code


class VideoPlaybackTest {
	public static void main(String[] args) throws Exception {

		File movieFile = new File(args[0]);

		boolean audioEnabled = true;

		VideoRenderer videoRenderer = new OpenGLVideoRenderer(movieFile.getName());
		AudioRenderer audioRenderer = audioEnabled ? new OpenALAudioRenderer() : null;

		if (videoRenderer instanceof OpenGLVideoRenderer) {
			OpenGLVideoRenderer opengl = (OpenGLVideoRenderer) videoRenderer;
			opengl.setFullscreen(false); // uses current desktop resolution
			opengl.setVSync(true);
			opengl.setRenderRotatingQuad(true);
		}

		VideoPlayback playback = new FFmpegVideoPlayback(movieFile);
		playback.setCoupleFramerateToVideo(false);
		playback.startVideo(videoRenderer, audioRenderer);

		/**
		 * oldskool controls!!
		 */

		BufferedReader br = new BufferedReader(
			new InputStreamReader(new BufferedInputStream(System.in)));

		while (true) {
			String line = br.readLine();

			if (line.equals("mute")) {
				playback.setVolume(0.0f);
			} else if (line.equals("half")) {
				playback.setVolume(0.5f);
			} else if (line.equals("full")) {
				playback.setVolume(1.0f);
			} else if (line.equals("pause")) {
				playback.pause();
			} else if (line.equals("resume")) {
				playback.resume();
			} else {
				System.out.println("wait what?");
			}
		}
	}
}

Version 0.7.2 of YUNPM

Changes
[x]Overwrote version in classfiles to support Java 6.
[x]Created zips for 32-bit systems and 64-bit system, where the 64 bit zip also contained the 32-bit libraries, to support the case of running 32-bit and 64-bit JVMs on a 64 bit OS.

Download files
[x] http://indiespot.net/files/projects/medialib/

Notes
[x]Friendly folks in #lwjgl verified it worked on Linux.

Under which license will you publish this ?

Am I correct to assume, since you bundle the (LGPL) ffmpeg with your code, that a user would never have to install anything to use this ?

Any flaws you’re keeping from us ?
Why hasn’t this been done before ? (Might ask the same about mapped Objects, but I guess this was less complicated)

Read your earlier posts on this, but not able to respond as on holiday and posting on here with Android is horrible! :wink:

This seems an interesting approach. I wondered if you’d compared this to using GStreamer, and what you’d consider the pros and cons of either approach? Your approach must add some extra overhead (extra encode / decode, etc.), but presumably that’s usable. Be interesting to benchmark them, particularly as the underlying codec code is most likely the same.

Most of the code to do this bundling and playback with GStreamer is hidden in this thread - I haven’t had a chance to tidy it up further since helping Cero get it working, but can hopefully have another look sometime soon.

I’m not really interested in the most optimal solution. I thought it was easy to do it with VLC, then got stuck with VLC exporting 1 image-file per frame (I forced it to be a TGA file for performance) which ended up with the requirement of a ramdisk - which is not easily achieved on Windows. Then I decided to give ffmpeg a go, which was only able to generate either mjpeg or wav, so i decided to spawn two processes, streaming files to disk and reading it back in java - an hour later i realized that ffmpeg supported writing the data to stdout, so I didn’t have to touch the disk.

Regarding gstreamer: it might work, it might work infinitely better, I don’t know, I never used it.

My library is quick 'n dirty, but it’s extremely easy to setup and run and reliable to the point that you have no way to break it (famous last words) and it handles syncing video and audio for you, which is a non-trivial problem for certain videos, as the number of audio samples per frame is not always an integer. I create (bruteforce) a pattern of swapping differently sized buffers, and usually get the sync-error below 0.01s per hour.

There is barely any API on the Java side (no seeking), but does it matter? Now we have a trivial way to playback video in Java, that’s more than what we had in the last decade.

Maybe my library will spark enough interest in ‘java video’ for somebody to write something better. In the meantime, the getting-things-done approach prevails and everything else is premature optimization.

Feel free improve on it! (and provide a gstreamer backend, while you’re at it :point:)

/*
 * Copyright (c) 2012, Riven
 *
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 *     * Redistributions of source code must retain the above copyright notice,
 *       this list of conditions and the following disclaimer.
 *     * Redistributions in binary form must reproduce the above copyright
 *       notice, this list of conditions and the following disclaimer in the
 *       documentation and/or other materials provided with the distribution.
 *     * Neither the name of Riven nor the names of its contributors may
 *       be used to endorse or promote products derived from this software
 *       without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

Indeed. ffmpeg includes all the required codecs.

You cannot seek, you cannot adjust the volume. Basically, you cannot do anything, except watching the video, until it stops. I can easily add support for pause/resume and stop, but that’d be all.

My guess is that everybody wanted to write a ‘proper’ solution, instead of this quick hack. As a proper solution is a lot of work, nobody did it. I thank the JGO community for this insight… I never used to get anything done for the very same reason.

Something a bit similar has already been done and showcased during Siggraph 2012, it works even on Raspberry Pi.

Nice job Riven ;D

Seems bound to JOGL, correct?

Nothing prevents you from porting this API to LWJGL, most of the code relies on LibAV and FFMPEG but you would need to modify a bit the native windowing toolkit of LWJGL to support some Broadcom non standard (not EGL compliant) initialization routines.

Except that a number of ‘proper’ solutions have existed for a while! eg. GStreamer-Java has been around over 5 years. The only difficult thing until recently has been getting hold of the pre-built native libs for Windows & OSX, but that’s no longer a problem. Decoding video and audio to bytebuffers is trivial as it’s all ‘in-process’, so no need for writing to / from JPEG and GStreamer handles the sync for you.

I don’t understand why with VLC you were using the write to file approach, either, rather than the VLCJ bindings which also give you direct bytebuffer access.

As an approach I find this interesting, but if the intention was ‘getting-things-done’, this seems like the long-winded way! :wink:

I’ll try and take a look at some point soonish. ;D

Then I don’t get what everybody here was complaining about :slight_smile:

Why were people struggling with it so much (according to many recent threads), calling it a disgrace that Java was still lacking video, and coding (new) solutions, giving up, etc. And why are people throwing medals in my general direction?

So many questions! Maybe it’s just lack of exposure of existing solutions, or horrific APIs.

@license thats great !

IMO all you need for video game video playback is: play, stop(in case a user cancels), isDonePlaying (to proceed with different code afterwards) and setVolume would be nice in some way or else huge volume differences may occur when ingame volume is low and stuff
pause and resume CAN be nice, when you have something like, press a button during a video -> “really skip video?” (while video is pausing)

Its all about something that works easily from the programmers view of things(Xuggler fails here), easy for the users view of things(it just works, no prev. requirements) and it has to be legal, meaning that all code and codecs have to be something like LGPL. (VLCJ is still GPL)

Technically I have video playback running with gStreamer, and its beautiful; however if this is equal in performance, then I could technically see me switching to this, because we include windows and mac libraries and no linux ones and “expect” with a very high degree of probability that gStreamer is already installed on linux machines.
This lib would make that tiny risk vanish.

I failed to find macosx builds of ffmpeg, and as I don’t have a mac, I cannot build it. Hence: no macosx support.

It was always something:

JMF - laughable, no further comment
JavaFx - javafx1 had license problems and javafx2 seems to have them too
Xuggler - “Very nice, however there is no player, and people who are much more skilled than I try to write a player still have sync problems.”
Cortado - Not reliable, skips and freezes
JGStreamer - seems fine.
VLCJ - GPL license

should be a cinch with mac, to find them and bundle them…

Added MacOSX executable in 0.7.4

Well, I wouldn’t dare to claim that some people like to be spoonfed! :wink: Actually, it’s probably mainly an exposure issue, and the fact that the solutions that have the most exposure are the least suitable. Having tried most of them, I’d claim GStreamer as probably the better option - VLC also good bar license issues.

The problems in most of those threads (half of which might have been Cero’s :stuck_out_tongue: ) were generally with obtaining and bundling native libs, and with difficulties knowing how to upload those video frames to OpenGL textures - both of which are easily solvable.

Bearing in mind that GStreamer, VLC and FFMPEG are pretty much using the same code (libav is an FFMPEG fork), then I seriously doubt that the added indirection in Riven’s solution could match the performance, but it may not be enough to be an issue. You could easily bundle the Linux libs in the same way you’re already doing mind you (just cos I said I had no interest in doing it doesn’t mean you shouldn’t - you should be able to get them from gstreamer.com or a Linux distribution). Just be careful with either FFMPEG or GStreamer that you’re not picking up the GPL builds.

Gstreamer is a total nightmare to install and is flaky as hell. It hardly ever works properly and you can’t “package it” with your game in any reasonable way. And if you do it is unlikely to work.

This solution is nice not because its ffmpeg, but because its just a file filter. We can easily use gstreamer or mplayer or whatever if it takes a some media file and spits out decoded frames and sound. The hard part of the code is done. The sync.

Media library’s out there right now are total crap. They have an api to decode a frame and some sound… even working out the colourspace and then the format of this decoded frame is a PITA, you still don’t have a solution to one of the bigger problems. Audio sync. By the time you have a binding and code that plays something… its a massive amount of work.

This is almost perfect for cut scenes and video “want to know more” stuff. I want to add skipping forward (think this is easy), pause start, restart(not really the player here), and stop (with fade perhaps). This is not a movie player.

The other thing that is fairly easy to do is have render to geometry. So i can put a TV, or movie screen in my game, not in front of the game.

Finally its simple. Here play this. Done.

I have gone through so many of the different bindings for java over the years. None of them really work properly or have all sorts of issues or require so much work just to play something even without sound. Or serious issues with performance. This puppy played 2k just fine, and with my changes its going to be able to play 1080p on pretty low end hardware.

But feel free to use all these other “easy just use this” libs out there for java movie playback.

Most games don’t need to play user provided video files. Would it simplify anything to use a custom format? Eg, Bink.

Why limit a developer provided video files to a specific format?

I can imagine not every developer would feel comfortable transcoding their video.