JavaSound on Linux

Hello everyone,

I’m trying to get a game working on Linux (Link: http://www.ludumdare.com/compo/ludum-dare-21/?uid=398 [yes, that’s Notch’s Ludum Dare entry]) - that is, I want to fix the sound, because the rest seems fine. The sound implementation in the game is very basic, but while trying to tinker with it, I’ve had trouble getting JavaSound to work with Linux at all (using Linux Mint 11).

So, the sound effects are used like this (taken from the source files):

package com.mojang.escape;

import javax.sound.sampled.*;

public class Sound {

	public static Sound altar = loadSound("/snd/altar.wav");
	// load a bunch more sounds...

	public static Sound loadSound(String fileName) {
		Sound sound = new Sound();
		try {
			AudioInputStream ais = AudioSystem.getAudioInputStream(Sound.class.getResource(fileName));
			Clip clip = AudioSystem.getClip();
			clip.open(ais);
			sound.clip = clip;
		} catch (Exception e) {
			System.out.println(e);
		}
		return sound;
	}

	private Clip clip;

	public void play() {
		try {
			if (clip != null) {
				new Thread() {
					public void run() {
						synchronized (clip) {
							clip.stop();
							clip.setFramePosition(0);
							clip.start();
						}
					}
				}.start();
			}
		} catch (Exception e) {
			System.out.println(e);
		}
	}

}

The input files are very short .wav files of the type “PCM_SIGNED 44100.0 Hz, 16 bit, mono, 2 bytes/frame, little-endian” (according to “ais.getFormat().toString()”).

While this works fine on Windows, it fails on my Linux machine using OpenJDK 1.6 at the line “clip.open(ais);”:

java.lang.IllegalArgumentException: Invalid format
        at org.classpath.icedtea.pulseaudio.PulseAudioDataLine.createStream(PulseAudioDataLine.java:143)
        at org.classpath.icedtea.pulseaudio.PulseAudioDataLine.open(PulseAudioDataLine.java:100)
        at org.classpath.icedtea.pulseaudio.PulseAudioDataLine.open(PulseAudioDataLine.java:289)
        at org.classpath.icedtea.pulseaudio.PulseAudioClip.open(PulseAudioClip.java:402)
        at org.classpath.icedtea.pulseaudio.PulseAudioClip.open(PulseAudioClip.java:453)
        at com.mojang.escape.Sound.loadSound(Sound.java:33)
        at com.mojang.escape.Sound.<clinit>(Sound.java:6)

So, I changed the “loadSound()” method to the following:

	public static Sound loadSound(String fileName) {
		Sound sound = new Sound();
		try {
			AudioInputStream ais = AudioSystem.getAudioInputStream(Sound.class.getResource(fileName));
			Clip clip = (Clip) AudioSystem.getLine(new DataLine.Info(Clip.class, ais.getFormat()));
			clip.open(ais);
			sound.clip = clip;
		} catch (Exception e) {
			e.printStackTrace();
		}
		return sound;
	}

This actually works, but if you try to play the same sound many times, there’s a pretty large delay (about 1 second) between each play. I’ve noticed that the “Clip.stop()” method takes a very long time on my machine, which it doesn’t on Windows. So, because playing the sound is synchronized in the “play()” method (see above), any attempts to play the sound again have to be queued up and wait until that’s finished. In the game, if you play the same sound again very quickly, that obviously doesn’t work because the action that triggered the sound is long over.

Because I don’t know very much about JavaSound, I decided to switch to the Sun JDK 1.6 and see if that works better. No matter if I use the first or second version of the “loadSound()” method, I get this at “clip.open(ais)”:

javax.sound.sampled.LineUnavailableException: line with format PCM_SIGNED 44100.0 Hz, 16 bit, mono, 2 bytes/frame, little-endian not supported.
        at com.sun.media.sound.DirectAudioDevice$DirectDL.implOpen(DirectAudioDevice.java:494)
        at com.sun.media.sound.DirectAudioDevice$DirectClip.implOpen(DirectAudioDevice.java:1280)
        at com.sun.media.sound.AbstractDataLine.open(AbstractDataLine.java:107)
        at com.sun.media.sound.DirectAudioDevice$DirectClip.open(DirectAudioDevice.java:1061)
        at com.sun.media.sound.DirectAudioDevice$DirectClip.open(DirectAudioDevice.java:1151)
        at com.mojang.escape.Sound.loadSound(Sound.java:33)
        at com.mojang.escape.Sound.<clinit>(Sound.java:7)

So, I thought, maybe the format actually isn’t supported. So I tried with a .wav file of the following format: “PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian”. I didn’t get any exceptions this time, but there’s a problem: I can’t hear anything. As I tried out various things from the forums, I looped through all the mixers on my system via “AudioSystem.getMixerInfo()”. With the Sun JDK, this gave me:

NVidia [plughw:0,0], version 1.0.23
NVidia [plughw:0,1], version 1.0.23
NVidia [plughw:0,2], version 1.0.23
Generic [plughw:1,3], version 1.0.23
Port NVidia [hw:0], version 1.0.23
Port Generic [hw:1], version 1.0.23

while with OpenJDK, I got:

PulseAudio Mixer, version 0.02
default [default], version 1.0.23
NVidia [plughw:0,0], version 1.0.23
NVidia [plughw:0,1], version 1.0.23
NVidia [plughw:0,2], version 1.0.23
Generic [plughw:1,3], version 1.0.23
Port NVidia [hw:0], version 1.0.23
Port Generic [hw:1], version 1.0.23

Now, because sound on Linux is generally a big pile of chaos, I don’t really know too much about it and I also don’t know what all those mixers are referring to. I do know, however, that I have PulseAudio, so perhaps the missing entries in the Sun JDK list explain why I got no exception, but still couldn’t hear any sound.

So, at this point, I’m stuck. To recap, I can’t play sound with the Sun JDK at all and with OpenJDK, I don’t know how to play the same sound multiple times, quickly. Is there any way I could get this running? I’d really like to stay with JavaSound and avoid any external libraries - the application is extremely simple. I mean, basically, I just want to play some sounds!

JavaSound typically being such an unstable can of worms, I advice you to do the following:

Open a 16 bit, stereo audio channel at 44100Hz, and do all mixing yourself.

Yes, it’s a lot of work, but at least it works and keeps working throughout your game.

I haven’t worked with sound specifically on Linux, but I haven’t had any Linux users complain about the various Applets I’ve posted.

Sometimes things that are “slightly incorrect” work on one OS but not on others. Also, there are dozens of ways to slip up and forget a step. I know I’ve found myself pulling out my hair because I forget something dumb like an open() step or my sound editor decides to export a blank wav and I don’t confirm there is actually the expected sound in it before I try to run it in Java. :stuck_out_tongue: (Not saying you have made such mistakes, just that, in general, a lot of double checking has to be done.)

I’m curious, how long (in frames) are these short sounds?

I suspect that the mixer issue is a red herring.

Not sure I understand your usage, with the static methods and invocation of synchronize on the clip. Do you have code that demonstrates usage that can be run as a test case? I don’t see how you select which clip is going to play if the Sound method loads “a bunch more sounds”. It looks like each load overwrites clip.

@Riven:
I guess I’ll try that, however I’m not sure how one would actually do that. Do you have any pointers/tips on how to start? Anyway, I’m going to take a closer look at it tomorrow.

@philfrei:
If you’ve successfully used sound with applets before, could you perhaps demonstrate a working example? I suppose you’ve used “AudioClip” when working with applets; I don’t know in how far that’s related to the rest of the JavaSound API, but maybe I can take a look at that as well.

Also, note that the example posted here isn’t actually my code - I’m tinkering with an open source game from the Ludum Dare competition (link in the first post) to learn some things and to get it working properly on Linux. By the way, on Windows, there are no problems with the game, and on Linux, everything is fine except for the Sound (which is basically implemented using that single class I posted).

I’d have to check to see how long the sounds are exactly, but they’re all shorter than a second. They can also be downloaded (link in the first post) if you need more information.

Concerning the mixer issue, that’s just a difference I found between OpenJDK and the Sun JDK. Since I’ve heard about JavaSound using the wrong mixer before, and because I can get sound with OpenJDK, but not the Sun JDK (with the exact same code), I thought that might be one of the causes (still not sure).

About the code: As I said, the whole game source code is available (it’s not huge, the game was made in 2 days) and is not written by me. However, I can explain how it works. An object of the “Sound” class, as it was posted, is supposed to represent a single sound effect that can be played with its only method, “play()” (very simplistic). The “loadSound()” static method loads a sound from a resource, creating a “Sound” object. This could instead be replaced with a constructor, but the author chose to do it like this for some reason. Where I put “load a bunch more sounds…”, I just cut out how all the sounds were loaded in the original code, all in the same way:

public static Sound someSound = loadSound("/snd/someSound.wav");

Whenever a sound is needed, it can be played frome elsewhere in the code with (for example)

Sound.someSound.play()

“clip” is not overwritten because each load creates a new sound object, each with its own “clip” field.
Regarding “synchronize”, this is needed to prevent multiple threads (a new thread is launched each time a sound is played) from accessing the same clip at the same time, which would cause problems. On Windows, this use of synchronization isn’t a problem for the game because the sounds are very short and the threads terminate very quickly. WIth OpenJDK, though, the call to “clip.stop()” takes ages for some reason, so if you play the same sound several times in quick succession (e.g. gunshots), it takes far too long to play all the sounds because the threads can’t quit quickly enough, and you still hear the queued-up shots with long pauses in between (during which the thread is busy doing “clip.close()”) even if the guns have long stopped shooting.

EDIT: I just found this: Java Theremin. Without having looked at any error messages, I’ll just quickly say I’m not getting any sound there, either.

I was never tempted by AudioClip. I’ve always wanted to have some control over volume.

The Theremin isn’t working? Interesting. When you drag the mouse upwards, the volume should rise. Are you able to get other sounds via other Java programs? You may have uncovered something that I have not dealt with yet!

My other Applets with sound are http://hexara.com/game.html (warning, slow load) and http://hexara.com/VSL/VSL2.htm

Princec generously posted his code listing, which includes his sound handling, if you want to poke around there for ideas.
http://www.java-gaming.org/index.php/topic,24367.msg204698.html#msg204698

Re: synchronize, there are perhaps alternative strategies to consider, such as thread isolation. If the clip is synchronized, how does one get access to stop() it? Maybe try using the getLine() method from one of your Mixers rather than from AudioSystem?

But I guess the first thing to establish (is that your volume knob is on – just kidding), for my being able to help or not, is whether it can play my Applets. If it can’t then I clearly have the same problem as you to solve. I’ll take a look at the download link you provided as soon as I get a chance.

I don’t hear anything from the Theremin on my Linux box, either. Socob is spot on with “sound on Linux is generally a big pile of chaos”, but I’m pretty sure I don’t have anything locking my soundcard at the moment.

Thx ptjt33, and Socob, too. I am glad to find this out.
I left a query over at a linux forum–but no responses yes.
The Notch Ludum Dare entry makes sound on my Windows system, but that was expected. (Now, if I could only figure out how to get out of the starting cell, in that game.)

EDIT: found this thread on similar question
http://lists.linuxaudio.org/pipermail/linux-audio-dev/2008-August/021547.html
But I’m not sure what to do with it. If I understand it correctly, it is suggesting that Java isn’t selecting the correct output card for whatever reason. But I don’t see how we can anticipate that and figure out which Mixer is the best one for a random client.

Another thread recommends prompting the User to select a Mixer.
https://forums.oracle.com/forums/thread.jspa?threadID=2230378&tstart=0&messageID=9627670#9627670
which points to this article, which looks like one of the better sound articles I’ve come across:
http://www.developerfusion.com/article/84314/wired-for-sound/

Generally, I’d agree with Riven - do your own software mixing. I’d stay well clear of Clip - they seem highly problematic (and also used to have difficulties playing sounds under 1 sec.) I don’t entirely agree that JavaSound is an unstable can of worms though - the direct audio devices generally work well if you use the API as intended. You need to be aware that the direct audio devices are just that though, direct access to the underlying soundcard driver, and can behave quite differently on different systems. eg. there is no hardware mixing on most Linux drivers, unlike Windows, so playing multiple sounds by opening multiple lines / clips will generally fail. A lot of Java games (and Pulpcore) suffer from this issue. The simplest way is to use the direct lines as they were intended - open a single line, keep it open, and mix your sounds into it yourself.

I’d recommend looking at http://www.jsresources.org/ - it’s an old site but a lot of the information is still relevant.

I posted the AudioServers code from my Praxis project up recently - info here http://www.java-gaming.org/index.php/topic,24574.0.html This abstracts out the actual audio setup code - you just have to implement a callback to fill float buffers as required. The audio routing / mixing stuff from Praxis will be up under a similar license in the near future, but is currently undergoing some API changes. I can point you at a few classes in the repo that will do the mixing bit for you though, if you want.

Incidentally, I get sound fine with philfrei’s Theremin applet - Linux Mint LTS, Oracle JDK. (it’s very crackly mind you, but that might be the synthesis algorithm! :slight_smile: )

Best wishes, Neil

Crackly?! What does that mean? I am aware of something that I assumed was kind of like anti-aliasing in the high registers, especially the square wave. Is that what you mean? Crackly as a descriptor sounds much worse than that. Does the sine wave crackle, too? That is clear as a bell, over here.

Sorry, Socob, I don’t mean to hijack your thread.

As far as mixing, I am kind of disappointed. Windows and Mac systems seem fine with running multiple SourceDataLines at the same time. Are you saying Linux can’t handle this, even if we write something that allows the User to pick their preferred sound card? :stuck_out_tongue:

Writing a live mixing routine seems like it should be doable, but a pain. The main thing, I assume is work out the relative volumes of all the source data, then add the audios, frame by frame. Yes? It’s basically just an addition operation with provisions made to keep everything aligned in time and to prevent the sums going out of bounds? I guess there is no reason the inner loop can’t have 5 different wav files all being read a buffer at a time.

Hi

As far as I know, since Java 1.5, JavaSound tries to use the best direct audio mixer. However, on some Linux machines, when some webcams are plugged, it tries to use the webcam as sound output >:( I wrote a bug report about it. On the other hand, some direct audio mixers are buggy. Some programmers try to use “Java Sound Audio Engine” to get the same behavior everywhere.

As far as I know, there is no “Java Sound Audio Engine” on OpenJDK which means that in this case, you have to use another mixer.

That’s why OpenAL (with OpenALSoft) through JOAL (JogAmp) or LWJGL is a better solution in my humble opinion, Riven’s suggestion is interesting too. These 2 options are the most viable cross-platform solutions to play sounds in Java.

[quote=“philfrei,post:9,topic:37148”]
Java Sound is quite badly maintained on Linux, it has nothing to do with Linux itself.

[quote=“philfrei,post:9,topic:37148”]
Paul Lamb is already working on its own software mixer, I hope he will succeed.

If you want to do the actual processing in Java, then Riven’s way is the only way.

[quote=“gouessej,post:10,topic:37148”]

Actually, no, it has as much to do with Linux (well ALSA) as Java. Notwithstanding that Java uses deprecated bits of the Alsa API, JavaSound only supports multiple lines on Linux when ALSA and the soundcard supports hardware mixing, which is not very often. On Windows, this is emulated in the soundcard drivers, but Java doesn’t use this feature of ALSA at the moment. IMO this is a good thing - the direct mixers do what they’re meant to, and give you direct access to the capabilities of the underlying hardware. That’s one reason why Linux has the best JavaSound performance (in terms of latency). The problem is people not using the API properly.

[quote=“gouessej,post:10,topic:37148”]

And there are many other options - Beads, Rasmus, JASS, Toot, RAPL (when I finish extracting it from Praxis), and even Gervill if you hacked it out. Software mixing is not exactly hard, particularly if you use float buffers (which any decent audio API does). As Phil says, it’s basically just a summation buffer (possibly with some -1:1 limiting or an auto gain control).

The sine wave is actually the worst of the lot. Quite harsh discontinuities in the audio. Any chance you’ve got some out-of-range numbers in there somewhere? ALSA can be rather unforgiving with that.

Well, I kind of initiated the hijack. :persecutioncomplex: I’ll post any further things in your thread.

As far as I know, Socob just wants to play some sounds, OpenAL is fine to do this.

Therefore, it is not a problem in ALSA. Non-Java applications and APIs using it work fine, better than OSS. I don’t see what you mean because ALSA supports multiple lines only if the underlying hardware supports hardware mixing.

Socob only wants to play some sounds like me. Software mixing is not hard for you but it is not the case for me. Paul Lamb has written a nice and easy-to-use API with several plug-ins for JOAL, LWJGL and Java Sound, it supports Ogg Vorbis too. The projects you quote are interesting but in my case, I need an out-of-the box solution, I have no time to hack anything about sound, I cannot spend some time in building all my tools, I have to focus on some things. With Paul Lamb Sound Library, you almost only have to know whether you want to stream your sound or not and which plug-ins you have to use depending on which file formats you need, you can learn how to use it in less than an hour, it is an excellent solution if you do not have any time to tinker existing solutions. Paul Lamb knows the limitations of each APIs he uses, he tries to work around them when possible. Therefore, I think that someone who is not a specialist of sound will have less problems and will waste less time by using it instead of using directly Java Sound or by using the APIs you suggest. I know only some of them, maybe I’m wrong.

[quote=“philfrei,post:5,topic:37148”]
Unfortunately, I can’t seem to get any sound out of any of your applets. I have no idea why it works for nsigma, considering we’re both using very similar Linux versions (maybe it’s related to the Java plugin - Sun/Oracle vs. OpenJDK - but that shouldn’t matter, right?). Regarding the sound approach in the game again (synchronization), you can’t really stop the sound once it’s playing, that’s correct. Keep in mind that the game had to be created within 48 hours, though, and that the sound is extremely simplistic. As I intend to take part in that competition eventually, as well, (with the somewhat self-imposed limit of using “pure” Java), I doubt I’ll have time or energy to deal with annoying sound issues or to implement some complex sound system.

@nsigma:
So, basically, I seem to have approached the JavaSound API in the wrong way. What I was basically looking for was the way it was used in my first post - you load a file, get a sound clip, let it play when it’s needed, and it gets done without much trouble. I don’t need any special features or anything (heck, for this, I don’t even need volume control). I haven’t had time to look at your links/suggestions yet, but I’ll see what I can get done. Still, isn’t there an easier way to do something relatively simple like this? So far, JavaSound seems terribly non-portable and completely opposite to Java’s cross-platform goals.

@gouessej:
You’ve desribed my problem pretty exactly - I don’t want/need sophisticated audio due to the circumstances. OpenAL would be a solution, but as I’ve said, I’d like to avoid native libraries. That doesn’t mean it doesn’t work, though - in fact, the Slick2D library (which uses LWJGL/OpenAL) does pretty much exactly what I want: you create a “Sound” object, call a “play” method, and you’re done. The problem is that I want that, but without native dependencies. I’ll have a look at Paul’s library, it seems to be pretty popular.

Not in pure Java it’s not!

I didn’t say it was a problem with ALSA. Non-Java applications work fine, and so do Java applications when they do what the majority of native applications do, which is software mix to a single line! ALSA (leaving aside dmix for a minute) only supports multiple lines where the soundcard itself provides them. ALSA through dmix, and also PulseAudio, provide software mixing on top of the basic ALSA drivers. However, this is to allow multiple software to play at the same time, not usually to allow any individual software to open multiple lines.

See also - http://www.jsresources.org/faq_audio.html#alsa_mixing

I understand all that, and I’m not trying to suggest everyone become an audio specialist (although I don’t believe for a minute you’d really find software mixing hard considering what else you’ve done - it’s basically summing arrays of data). Both Beads and JASS could probably work out of the box pretty much for this, though licences would be an issue for some - that’s why my intention with RAPL is to be GPL+CPE. There is a definite need for a nice audio library built on top of JavaSound that makes this simple to work with, and I gave some examples of some I already know about. Paul’s library has a nice API, but I don’t agree with you that he’s working around the limitation of the APIs, at least for JavaSound - all the mixers tell you how many simultaneous lines they can support, which on Linux will generally be 1!

Yeah, that’s the bit of JavaSound that doesn’t work very well! :slight_smile: Personally, I’d look for a simple 3rd-party replacement. From my list I’d look at Beads or JASS first, or maybe some of the audio libraries written for Processing.

Not really. The API allows you to query the underlying system for capabilities. You just can’t expect the same capabilities on every platform, but I’d rather Java echoed the underlying system than tried to paper over the cracks! Lots of things in Java work differently on different platforms - as long as the API allows you to query the difference, I don’t see the problem with that.

Ok, you’re right. If you want to avoid native dependencies, OpenAL is not the way to go. Paul Lamb Sound Library is even used by Minecraft :slight_smile: It would allow you to go on using Java Sound without needing any extra effort to switch to JOAL (except for deployment).

The only tricky part of doing the mixing yourself is that you have to convert to and from different formats.

Here is a piece of source code that will take care of a lot of convertions. It is part of the Gervill midi synthesizer.

Thank you very much for this example ;D

That is the same class used in the AudioServers code you were asking me about a month ago! If you’d bothered to follow up on my long reply to your question you’d have found it. :stuck_out_tongue:

The other ‘conversion’ code potentially of use is interpolation to handle pitch changes - not so much to handle dynamic pitch changes but for sample rate conversion. I say potentially because it makes more sense to convert all your samples to the same sample rate beforehand, but I know not everyone does. There is some good code for this in Gervill too, though from recollection the API is not as simple to use externally as it could be.

Sorry. I remind you that I’m a former participant to a silly reality TV show, it has caused me a lot of troubles and it is sometimes difficult for me to stay focused. I put this link into my bookmarks. Anyway, I still think writing a software mixer is not trivial, at least for me.

Socob, just went to check out Phil’s theremin and got no sound this time. I know the reason though - I’d just watched a Flash video. Restarting Firefox fixed this, but I’ve noticed this before - Flash can have a tendency to not release the soundcard so Java can’t get an output line. It probably doesn’t cover your issue, but it’s another one to be aware of.