Sorry to ask this, but I can’t really find a solid answer. For simple 2d games like mine using JavaSound, should I have the sound playing on it’s own thread? Meaning, should I start the sound on it’s own thread in the AudioHandler class? right now I just have a method in the AudioHandler class called handleAudio(), which is called in the main game loop. It works fine, but I don’t know if that’s the correct way to do it.
If a thread is playing sound, it will not do anything else until the sound is done. So yes, sound playback has to take place in its own thread.
I think it makes sense to organize control of the sound as a whole into a single class. Is that what you are doing with SoundHandler? Or is SoundHandler part of some sort of library that you are using?
Mostly, I’ve been making the sound handling class static. But in a couple of situations I’ve done some sound management in the game loop as well. (Your question makes me think maybe I should consider doing more of this.)
For example, game-loop based management makes sense in a situation where the volumes or panning are being updated by the positions of objects on the screen (more common with 3D, but can apply to 2D as well). Another case would be reading the mouse position and putting the [X,Y] values in variables that are only consulted once per game loop and used with the sound methods on that basis. This often makes more sense than triggering the sound with every mouse update.
I’m still learning, though. And a lot depends on the specifics of the game and the sound library you are using (I most use javax.sound.sampled).
Thanks guys. I am using javax.sound.sampled, and I wrote the audioHandler class to load and play the audio, as well as keep track of variables for it. In psudo code it looks like this:
public class AudioHandler {
//load all audio here, yada yada
public void handleAudio(){
switch(LEVEL){
case 1:
// play whatever audio or sound effects need to be played for level 1
break;
}
}
}
Then in the main game loop I call AudioHandler.handleAudio. I am going to go ahead and put it in it’s own thread since I’m not doing anything with volume or panning, strictly just playing. Thanks @philfrei and @ags1 !
In addition to the points that people have brought up here, you should never place disk access in your game loop. Disk access is both slow and unpredictable. For normal hard drives, the access time is in the order of 10ms. In addition, they’re a shared with all the other processes in your computer, so if your anti-virus software decides that there’s no better time than right now to start a full virus scan, a 0.25 second sound effect can now take 1 second to load.
Thanks @theagentd that is very useful advice. After yours and @philfrei 's advice I am for sure going to put the audio on a thread of its own. I tried to do it the other day in a rush but it broke a bunch of stuff so I put everything back. This sounds like a weekend job. Thanks all for the help.
If you are up to it, you can ignore java’s sound and use LWJGL to get OpenAL bindings. Threads don’t lock during sound.
Thanks man, but that will be for my next game.
Ok, one more question. My sound is now on it’s own thread, which I thought would help solve this other problem, but it doesn’t. The other problem is two sounds not consecutively playing when they should. For instance, when the player collects bones (think of them as coins or rings), a sound is supposed to play. If he collects 2 of them back to back, sometimes the second sound does not play. Now, I know I am using java sound and it is probably just a limitation of that…but here is how I am doing it:
if(player collides with bone) {
boneSound.stop();
boneSound.setFramePosition(0);
boneSound.start();
}
Is there a better way to do this to help this problem?
Never use Clip for this purpose. It’s not really designed for it. Clip opens a line to the soundcard every time it’s used. For good performance in a game you want to open a line once and mix sounds to it as needed. This is low level stuff - you may want to have a look at a library that does software mixing for JavaSound, such as TinySound, instead.
Ok, thank you for the explanation. I will look into TinySound.
I hate to disagree with nsigma of all people, but the Java Clip is specifically designed for being played more than once. But his statement about opening and line and keeping it open is most likely* accurate. That is what I do with my audio library, and what TinySound does as well.
The code you posted should work. If it doesn’t, it could be for various reasons. One possibility that comes to my mind is that the second iteration happens before the first has barely started, and it stops and restarts after just playing a small portion of the first, making it seem like it didn’t play at all.
To fix that (without using a library), it seems to me there might be two ways to attack the problem. One is to put in code to make sure that enough time has elapsed before starting the second iteration. It might be possible to set a variable with the start time of the play and enforce that a certain duration elapses before the second play occurs. Or, use a LineListener to flip a boolean that indicates the sound is done (and have the method poll this boolean before the replay). How accurate this will be with Java’s multithreading mechanism is not something I am competent to predict.
Another way is to make a second copy of the Clip and run it independently of the first. In this case, the two would likely play in an overlapping manner which may be acceptable or even preferable. Mixing will be handled at a lower level. But it could still be important to enforce a certain minimum wait to prevent the two from being so close together in time that they either fuse or cause some odd comb filter or phasing effect.
- @nsigma: I noticed that OpenAL seems to be continually opening and closing something when we play back sound data. I’m referring particularly to the case where one streams sound data “manually” through a Source. Do you know if there an underlying mixer in the OpenAL implementation running continually that dynamically gets inputs from the Source objects, or is this an instance where opening to a soundcard happens repeatedly?
So, Phil, one line to disagree with something I didn’t quite say, followed by four paragraphs that pretty much agree with what I said! 
I didn’t say that Clip is not designed to be played more than once, but that they’re badly specified for rapid, overlapping or synchronized sounds. This is because the API is specified as extending DataLine rather than something that writes to a DataLine. Since the (positive) move to direct audio devices, this means that it punts responsibility to the OS for mixing on AFAIK all implementations, rather than doing it internally. Also, from OpenJDK source code it looks like each Clip uses a separate underlying Thread. All this makes sense for the odd business use case for sound, but explains why application level mixing is required to get good performance for more complex uses in games, music, etc.
OpenAL-Soft uses an internal mixer to write to a single playback “line” as required. It would be possible to write a pure-Java application that did everything OpenAL-Soft does using a JavaSound line, but not using Clip.
1 Like
@nsigma Thank for answering my question about OpenAL! I apologize for any misunderstanding about the rest of my post. I didn’t intend or expect anything but my first comment about Clips to be seen as contradicting or correcting what you wrote! Thanks for clarifying my misinterpretation of what you were saying about Clips. I was mostly just intent on providing a path for solving the game requirement via use of Clips as an alternative to the overhead of bringing a sound library into the project.
To the OP:
My guess is that the cue in question is at least 1/2 or 1/3 of a second in length but not much longer than that, and that it would best be heard twice in a row, immediately back-to-back, rather than having the first playback be stopped before completion (or before barely getting started–which is what I suspect you have inadvertently coded).
TinySound permits concurrent playback of its “clip-equivalent”, but this doesn’t solve the matter arranging the two plays to follow one another instead of playing simultaneously or overlapping in time. If you wish to allow the sound to play through completely before starting the second iteration, I suggest using a LineListener to trigger the second play, because you can use the EVENT of the first cue’s STOP to handle the second playback. Maybe something like the following:
private int playAgain;
private boolean clipIsPlaying;
private Clip boneRattle;
//... some code to load and open the boneRattle cue ...
//... include adding the following LineListener to the Clip:
boneRattle.addLineListener(new LineListener()
{
public void update(LineEvent event)
{
if (event.getType() == LineEvent.Type.STOP)
{
if (playAgain > 0)
{
playAgain--;
boneRattle.setFramePosition(0);
boneRattle.play();
}
}
else
{
clipIsPlaying = false;
}
}
});
// and your play() command is done with a bit of extra coding:
public void playBoneRattle()
{
if (clipIsPlaying)
{
playAgain++; // the listener will take care of handling the next play()
}
else
{
clipIsPlaying = true;
boneRattle.setFramePosition(0);
boneRattle.play();
}
}
IF this is the issue (how to make two-or-more-playbacks-in-a-row behave in a sequential manner) then hopefully the above code will help. (I haven’t tested it.)
If some overlap is okay, or interrupting the first sound and restarting is okay, then my other suggestion is to have your play method store a timestamp of when the play() method was called, and consult this value on subsequent plays to enforce a minimum interval before your next play().
It is hard to say how much time to enforce between the starting time of play() commands without hearing the cue. Also, it depends if you are doing the stop/restart plan or the concurrent play plan.
If two copies of the same sound play concurrently, but with very small starting time differences (e.g., in the vicinity of 50 millis or less) you can get some unexpected artifacts from “comb filtering”. Enforcing a minimum difference in onset times of at least 100 or so millis should prevent that. I haven’t thought through or tested the best intervals yet though. My numbers are just ballparks.
The amount of time between starts will also depend on how much of the cue you decide should be heard in order for it to register in the ear of the person playing/listening.
TinySound allows you to play its “clip-equivalent” concurrently. With JavaSound’s Clip, for concurrent playback, you have to make two separate Clips and manage both instances yourself, which can be a bit annoying.
No need to apologize @philfrei - just amused by the contradiction in your post, which does highlight quite a few ways that Clip is not the answer to this problem - is not designed for this.
We simply should not be advising people to use Clip in this case, or trying to suggest complicated workarounds that could easily break on another system. There’s no fundamental issue with JavaSound, but ill-advised usage like this just leads to a world of pain.
If the OP wishes to stick with a pure-Java solution then they need to look at one of the libraries designed for this (or write their own). TinySound is one option, but there are others. Minim is probably worth a look (which was written for Processing but will work outside it), or Beads, or JASS, or JSyn, or …
… one day I may actually get around to making a single library out of the audio code in Praxis LIVE :persecutioncomplex:
@nsigma Ah, now I understand your post better.
But I’m not so sure that going to an audio library would be helpful in this case.
Let’s consider the following scenario: your avatar bumps into two instances of bones and you’d like to play the short bone SF/X twice in rapid succession with each playing both completely and clearly audible. Also, when you bump the bones the first time, you don’t know in advance you are about to bump them again (and a third collision may also be possible).
How, using Libgdx or LWJGL3 (in other words, OpenAL), is this to be done? AFAIK, SF/X cues play asap when you call them. There is no way to “schedule” the call. Same thing with TinySound. (Someone, PLEASE correct me if I am wrong!) The only thing I know of to do is to manually handle this in real time, which is a little dicey since Java doesn’t provide real-time guarantees: store the time when the SFX was called and refer to this and compute a sleep amount, and execute the sleep, before completing the 2nd SFX call. And this will have to be done on its own thread so as not to stop execution of everything else during the sleep.
At least with Clip, you can use the LineListener “STOP” event to manage the 2nd trigger (as coded in the earlier post.) I’m not clear how LineListener-equivalents would be coded in the OpenAL or TinySound worlds. Can they?
Thus, except for the pickup of the ability to play sounds concurrently (which isn’t what we are trying to do here), there may not be a lot gained by abandoning Java’s audio, and LineListening functionality is lost. So for me, sticking with Clips or abandoning them is not a black & white issue. There is a case to be made for both approaches. I understand we may have to agree to disagree.
Maybe one of the other libraries you mentioned has some sort of event-scheduling that would make this easier.
This isn’t about abandoning Java’s audio - the library examples I gave are all deliberately things that build on JavaSound and do this properly. I don’t see a problem with handling your scenario in any of them - it’s a pretty basic ask. They’re also going to do it with much better accuracy and performance. Seriously, this is not a reason to use Clip.
With TinySound, you could also just check in your game loop whether the sound (actually Music!) has finished and retrigger it if required (eg. count greater than 0). This won’t offer sample accuracy - given that line listeners on a Clip aren’t either and bring in a massive amount of overhead, it’s still going to work a lot better! (EDIT - you realise that for each line listener call the output to the soundcard has been stopped and needs restarting? That’s not cheap!)
I also think that OpenAL supports queuing of sounds in this way. Never used it though, so I’m not the right person to answer.
[quote]With TinySound, you could also just check in your game loop whether the sound (actually Music!) has finished and retrigger it if required (eg. count greater than 0).
[/quote]
@nsigma: If it is not too much trouble, could you demonstrate an example of the code in TinySound that tests if a sound or music cue has finished? I didn’t know this could be done.
It sounds like polling is used rather than getting a push when the cue ends. Still, I believe you, that this is less costly than happens when an open Clip is reset and restarted.
In my audio library, the equivalent Listener is frame-accurate. There is also a frame accurate event-scheduler. I followed your (nsigma) advice and output continuously via a Java SourceDataLine. Nice to hear that many of these other cool libraries also do this. I am not knowledgeable about the various libraries and their underlying code.
This week … unlikely … sorry :persecutioncomplex: However, the TinySound Music interface has a done() method. I’m assuming that works right! ;D
The listener is good as a low-level mechanism, but as a library API it’s somewhat problematic. To work it requires that the listener is called in the audio thread, is lock-free, and perhaps forces certain state not to be messed with. It’s not ideal for the end user. OTOH, a rich event scheduling API where the library user can pass in commands via a lock-free queue is generally a better approach for those unlike us who don’t want to get deep into the nitty gritty.
@nsigma
Thanks for the done() link. That is what I was asking for.
Yes, the Listener I wrote is in part of the audio thread and is lock-free. As far as using it in a sensible manner, though, it is up to the programmer to be aware they are consuming audio thread cpu’s and use efficient patterns like “loose coupling” and to avoid blocking. But it is nice to be able to have options for having sound trigger visual events.