Best Way to Implement Music and Effects in Java 2D Game

[nsigma]

JavaSound has worked pretty well for over 10 years when the direct device support was added. That’s a generally solid low level API. I wouldn’t recommend most people use it directly, like I wouldn’t recommend most people on here use PortAudio directly, which is the low level library used in OpenAL-Soft. Likewise you wouldn’t criticise PortAudio for not being as easy to use as OpenAL. Julian assumes that JavaSound based libraries are hard to use because JavaSound itself is.

Or another way, it’s similar to what you said about it being possible to build JavaFX on top of LWJGL.

[philfrei]

I was asking around about implementations for javax.sound and JavaFX’s Media & MediaPlayer and got a slightly more specific answer than nsigma’s. Yes, Linux JavaFX uses GStreamer, but the responder points to native DirectX for Windows.

http://www.coderanch.com/t/656515/JavaFX/java/JavaFX-play-audio-PCM-stream#3040933

One thing different (improved) about the JavaFX Clip is that it is possible to play it multiple times, concurrently, without having to link to a different memory copy for each playback. As to how this relates to nsigma’s concern about opening a pipe for each iteration, that’s a bit over my head.

I really don’t want to step to far into the controversies. I haven’t used Clips in years and haven’t tested them out either. But it seems pretty likely that a problem with sound files shorter than a second would have been fixed after so many years and releases.

I’m wondering about piping a stream directly to the JavaFX sound implementation. That was the gist of the link above. Looks like I will be waiting for someone else to come up with a work around, or waiting for Java/JavaFX 9. My level of expertise is being tested enough trying to get better encapsulated envelopes working for my dang synthesizer patches…(been working on this for two days now).

[nsigma]

@philfrei

I was asking around about implementations for javax.sound and JavaFX’s Media & MediaPlayer and got a slightly more specific answer than nsigma’s. Yes, Linux JavaFX uses GStreamer, but the responder points to native DirectX for Windows.

Thanks for the assumption he’s more correct! The answer is likely both. GStreamer is a cross-platform API, which will use elements of DirectX on Windows. The bug report linked above is on Windows with AudioClip and shows errors in the GStreamer layer. Oracle maintain their own fork of GStreamer 0.10 in JavaFX. Discussions going on within OpenJFX show them looking at an update to GStreamer 1.0 across all 3 platforms.

@philfrei

One thing different (improved) about the JavaFX Clip is that it is possible to play it multiple times, concurrently, without having to link to a different memory copy for each playback. As to how this relates to nsigma’s concern about opening a pipe for each iteration, that’s a bit over my head.

That’s a guess based on the bug report linked above and the API. The code for this kind of starts here - http://hg.openjdk.java.net/openjfx/8u60/rt/file/996511a322b7/modules/media/src/main/java/com/sun/media/jfxmediaimpl/NativeMediaAudioClipPlayer.java

hmmmm … yuck! Though I love the JavaDoc comment -

[quote]This is a stop-gap solution to eliminate the JavaSound implementation. It’s not meant to be perfect, just work well enough to demonstrate functionality.
[/quote]
fills one with confidence! ;D

[princec]

Still boggles me why they didn’t just wrap OpenAL… mind you it boggles me that they didn’t put LWJGL or JOGAmp in there too and reinvented the whole wheel all over again.

Cas

[nsigma]

@princec

Still boggles me why they didn’t just wrap OpenAL… mind you it boggles me that they didn’t put LWJGL or JOGAmp in there too and reinvented the whole wheel all over again.

GStreamer does kind of make more sense given the target market for JavaFX, and that it gives them a complete media stack (eg. video too). I like it too, but as one of the GStreamer Java maintainers I’m biased!

I don’t want to see OpenAL in the JDK either - it’s too high level / usage specific. An updated low-level binding using something like PortAudio would make more sense, and allow library writers freedom to develop stuff on top. I don’t like the kitchen sink approach to the JDK. Kind of like I said above about Java NIO - I don’t want a complete server in the JDK but the access to things that allow library writers to build one. Same goes for audio.

[princec]

Hmm, and there was me thinking OpenAL was about as low-level as you could reasonably get without actually writing drivers.

Cas

[nsigma]

@princec

Hmm, and there was me thinking OpenAL was about as low-level as you could reasonably get without actually writing drivers.

IIRC OpenAL has mixing, effects, etc. I’m talking about literally the ability to open a line to the device and write bytes to it. Everything else can then build on top of it. Yes, I realise OpenAL could do that, but why enforce the added layers.

The direct devices in JavaSound are actually a direct abstraction of the underlying sound card device. It’s why some JavaSound mixers can’t actually mix - it depends on the capabilities of the underlying sound card driver.

You potentially have

Java -> LWJGL / JogAmp -> OpenAL-Soft -> (possibly PortAudio) -> native soundcard driver.

vs

Java -> native soundcard driver

Mind you, that is currently missing access to the up-to-date Windows sound APIs (blimey, how many do Microsoft need?) and the ability to work with callback-based APIs directly. I definitely ain’t saying it’s perfect!

Incidentally, another reason for keeping the JDK bit minimal and lean is the ability to extend and develop outside of the JDK development process. There’s a great video somewhere of a talk from the developer of Gervill that is now the software synthesiser in the JDK. To loosely paraphrase, the JDK is where code goes to die!

[gouessej]

@princec

Well hold on… what Julien is saying is that maybe he’s not had so much success getting Javasound to do this correctly. I know that OpenAL works just perfectly (coz I’ve been using it exclusively for the last 15 years), and I remember in the dim dark past that Javasound was just hopeless which is why I didn’t use it, so does it now work ok?

It works better now but I still have no sound on numerous machines for unknown reasons and I still have to increase the duration of short sound samples to make it work, the problem is the same for Oracle Java and OpenJDK. The libraries based on JavaSound that don’t use any Clip instance aren’t concerned by this limitation or bug but the first problem I’ve just mentioned is still here (not always caused by the webcam unfortunately :(). I don’t really see the plus value of using JavaSound for games except that it’s build-in but I don’t deny that it’s extendable and then good for Gervill and co.

[nsigma]

@gouessej

The libraries based on JavaSound that don’t use any Clip instance aren’t concerned by this limitation or bug but the first problem I’ve just mentioned is still here (not always caused by the webcam unfortunately :().

So the libraries we’re talking about aren’t affected by the Clip issue (and Clips should not be used for the purposes we’re talking about)! But does the no-audio bug affect any of those libraries? You’ve previously mentioned never actually using those libraries - that is my entire problem with what you say! There are lots of reasons why you might not get sound output via the JavaSound if the API is not being used correctly. Stop damning writers of libraries you admit to never having tried!

[princec]

This is the same sort of issue that plagued Java2D though… it was never entirely obvious how to use it correctly or performantly. There were entrails, and riddles, and bones. So that’s why we all ended up using OpenGL instead for anything remotely decent. Tellingly OpenAL doesn’t seem to suffer from these sorts of issues vs. JavaSound so you can see why Julien prefers it. And me for that matter, not to mention it’s also got a life outside of the Java ecosystem.

Cas

[gouessej]

@nsigma

Stop damning writers of libraries you admit to never having tried!

Stop accusing me of criticizing libraries of which I haven’t read the source code whereas it’s plain wrong as I read TinySound’s source code. I’m not “damning” anybody.

If you can’t find a mixer despite a correct sound setup, it’ll be hopeless, you can’t even use TinySound.init(javax.sound.sampled.Mixer.Info). I don’t have to give it a try, this library is useless if JavaSound retrieves no mixer on a machine.

@princec

This is the same sort of issue that plagued Java2D though… it was never entirely obvious how to use it correctly or performantly. There were entrails, and riddles, and bones. So that’s why we all ended up using OpenGL instead for anything remotely decent. Tellingly OpenAL doesn’t seem to suffer from these sorts of issues vs. JavaSound so you can see why Julien prefers it. And me for that matter, not to mention it’s also got a life outside of the Java ecosystem.

Yes, that’s the point. Then, someone can write a pretty abstraction above JavaSound, even a well commented one but if the underlying library fails to find my audio devices, I won’t be able to use it.

[nsigma]

@princec

Tellingly OpenAL doesn’t seem to suffer from these sorts of issues vs. JavaSound so you can see why Julien prefers it.

No one here (unless they like writing low-level audio stuff) should be using raw JavaSound is my opinion. This is not about OpenAL vs JavaSound, it’s about OpenAL vs a library built with JavaSound. It is the library writers job to cover over the entrails, riddles and bones. You should look at the never entirely obvious stuff involved in using any of the low-level audio APIs used in OpenAL-Soft. Using your argument no-one should use OpenAL either!

[princec]

Excepting OpenAL-Soft “just works”, of course 8)

So I’d be perfectly keen on an implementation of OpenAL on top of JavaSound too… provided it always just worked.

Cas

[nsigma]

@princec

Excepting OpenAL-Soft “just works”, of course 8)

As does the JavaSound lib in Praxis LIVE (see video!) I’ve never had bug reports from any platform around sound IO, and some of the underlying libs I wrote are used in various other software too. There are numerous other libraries and software that use JavaSound under the hood that also work fine. I’m a big fan of “just works”. 8)

[Spasi]

[quote=“nsigma,post:34,topic:55703”]
Except the only sane way to distribute a Java application is with a private JRE included. I don’t know anyone who doesn’t do that. Who cares about Ubuntu and LTS. Is Ubuntu the only Linux distro out there?

With a private JRE and OpenAL-Soft binaries you also have a solution that just works, why is that different from JavaSound? (not comparing the low/high-level nature of the two APIs here)

On PortAudio, it is just one of 4-5 possible audio backends that OpenAL-Soft can use (at runtime). Considering the state of sound on Linux, I wouldn’t ship an app without at least a few options. With that said, if you think using PortAudio directly would be beneficial, I wouldn’t say no to an LWJGL binding if you’ve got time to contribute one.

[nsigma]

@Spasi

Who cares about Ubuntu and LTS. Is Ubuntu the only Linux distro out there?

No. That’s just a personal decision because I ship DEB’s for Ubuntu LTS that use system OpenJDK. Whether that’s a sane decision is another matter … ;D

@Spasi

With a private JRE and OpenAL-Soft binaries you also have a solution that just works, why is that different from JavaSound? (not comparing the low/high-level nature of the two APIs here)

It isn’t! I’m not in any suggesting people don’t do that if it works for them. It depends what features people need. It’s not an either / or game.

Incidentally, what’s missing in both I believe is support for callback based audio API’s.

@Spasi

On PortAudio, it is just one of 4-5 possible audio backends that OpenAL-Soft can use (at runtime). Considering the state of sound on Linux, I wouldn’t ship an app without at least a few options. With that said, if you think using PortAudio directly would be beneficial, I wouldn’t say no to an LWJGL binding if you’ve got time to contribute one.

Yes, I realised after I first posted that OpenAL-Soft supports PortAudio and various native audio backends directly. That seems an odd decision which kind of defeats the point of having PortAudio in there, but I assume it made sense for some reason.

I wouldn’t agree with you on the Linux sound thing - a well performing ALSA backend should be fine for most uses. JACK is overkill for games, etc. (although I did write Java bindings if anyone wants them), and I don’t really see the need in supporting the PulseAudio API directly.

On PortAudio - I’d quite like a binding in JAudioLibs too. Unfortunately, if there’s one thing I haven’t got at the moment it’s time to write and support another library! :-\

[philfrei]

@nsigma

@philfrei

I was asking around about implementations for javax.sound and JavaFX’s Media & MediaPlayer and got a slightly more specific answer than nsigma’s. Yes, Linux JavaFX uses GStreamer, but the responder points to native DirectX for Windows.

Thanks for the assumption he’s more correct! The answer is likely both. GStreamer is a cross-platform API, which will use elements of DirectX on Windows. The bug report linked above is on Windows with AudioClip and shows errors in the GStreamer layer. Oracle maintain their own fork of GStreamer 0.10 in JavaFX. Discussions going on within OpenJFX show them looking at an update to GStreamer 1.0 across all 3 platforms.

@nsigma - “More specific” doesn’t necessarily mean more correct! I very much appreciate your expertise and taking the time to clarify what was an ambiguity for me. This all takes place at a level of coding where I am pretty much at sea.

[nsigma]

@philfrei

@nsigma - “More specific” doesn’t necessarily mean more correct! I very much appreciate your expertise and taking the time to clarify what was an ambiguity for me. This all takes place at a level of coding where I am pretty much at sea.

I was only joking! Although, even after what I said above, and having looked at some of the code, I still wouldn’t stake my life on it - so many layers of indirection going on in there I’m not sure I’ve clarified the ambiguity to myself!

[nsigma]

@nsigma

@Spasi

With a private JRE and OpenAL-Soft binaries you also have a solution that just works, why is that different from JavaSound? (not comparing the low/high-level nature of the two APIs here)

It isn’t! I’m not in any suggesting people don’t do that if it works for them. It depends what features people need. It’s not an either / or game.

Just to clarify that (having re-read it) I meant the feature set of whatever libraries build on top of either solution - that’s where it’s different. There’s actually a load of interesting Java related audio stuff going on at the moment, some of which might use JavaSound, some of which might use OpenAL, and some of which uses neither (JSyn (old version), Methcla, Overtone, etc.) Why arbitrarily tell people to ignore it all? OpenAL is great for what it does, but I still think it’s overkill to play the odd sound. Likewise, it’s probably not the best solution if you want to get more complicated - eg. create a soundscape (music and FX) that responds to game state in interesting ways.

What bugs me about Julien’s argument (I’ll use one more parallel) is that it’s akin to finding a bug in jME and saying don’t use libGDX because LWJGL sucks! It just doesn’t compute.