ogg and aiff - streams vs clips

Java Sound & OpenAL
#1

I’ve got some simple sound playing using AudioClips, but I noticed in the byteArray thread a mention of AudioClip causing a stutter. Can someone clarify playing audio streams and clips for me?

  • If I have audio of about 30s or more and its an ogg, then I can decompress all of it into ram, but that will suck up lots of memory. Mostly true?

  • If instead I read the file in as a stream, then I risk eating more cpu time as its playing, since it has to decode on the fly? Still mostly true?

  • What if I read in an ogg and decompress it to an aiff, then I stream it in? This seems like a good way of doing things. Remotely true?

I am trying to avoid having a 20 meg download of my app due to audio files. If I have them all compressed, then decompress them once downloaded, that would seem the best.

Any insight from those who are wiser than I?

#2

[quote]I’ve got some simple sound playing using AudioClips, but I noticed in the byteArray thread a mention of AudioClip causing a stutter. Can someone clarify playing audio streams and clips for me?
[/quote]
For some reason, AudioClips are slow to play, and have to drain the entire sound buffer before restarting. This behavior tends to cause hiccups and pauses in gameplay when sounds are repeated at a very fast rate. (e.g. a spaceship cannon firing)

GAGESound and many other JavaSound libraries handle this problem by pre-loading the decompressed sound into a byte buffer, then playing back the sound via a SourceDataLine. A SourceDataLine is nothing more than an open mixer channel, so latency is very low and the stream can be interrupted in an instant.

[quote] - If I have audio of about 30s or more and its an ogg, then I can decompress all of it into ram, but that will suck up lots of memory. Mostly true?
[/quote]
The formula is actually pretty easy to figure out:

Uncompressed Size = Bits per Sample * Channels / 8 * Sample Rate * Seconds

Thus a 16 bit, Stereo, 44.1KHz sound that lasts 30 seconds would take:

16 * 2 / 8 * 44100 * 30 = 5292000 bytes = ~5 MB

Tremendous sizes like this are much of the reason why most sound effects are kept small, short, and low quality. e.g. An 8 bit Mono, 11Khz, 2 second sample would be:

8 * 1 / 8 * 11000 * 2 = 176000 = ~171kb

[quote] - If instead I read the file in as a stream, then I risk eating more cpu time as its playing, since it has to decode on the fly? Still mostly true?
[/quote]
More or less. It’s slightly more complex because streaming compressed audio is a very tricky thing. Not only does it take more CPU, but it also takes more I/O, more interrupts, and more bus bandwidth. The end result is that you can pretty easily outstrip a machine’s resources in attempting to stream a file from disk. Modern 2GHz+ computers have an ungodly amount of internal bandwidth to handle this sort of problem, but older machines might not be able to keep up. There are a number of ways to get around this problem, but pretty much all of them amount to stealing from Peter to pay Paul.

[quote]What if I read in an ogg and decompress it to an aiff, then I stream it in?
[/quote]
I’d actually recommend writing out raw data to disk. This would allow you to stream it back in with zero decompression or parsing costs. Also, you may want to experiment with memory mapping the file or using large, staggered buffers. This way the OS can perform a DMA load on the data while your game runs in parallel.

Good luck!

#3

Hey thanks for the input!

I didn’t say it expressly, but I meant to decompress the audio and write it to a disk file as an aiff.

I am assuming the stutter you are talking about is actually because I stop the sounds mid way through. At least they can be, depending on what the user clicks.

I’ll take a look at the GageSound you mentioned.

I went to the Jorbis site (http://www.jcraft.com/jorbis/) and the last time the source was touched was 2002. Is this the libary to use or are their newer works based on it?

#4

[quote]I didn’t say it expressly, but I meant to decompress the audio and write it to a disk file as an aiff.
[/quote]
I figured that was what you meant. My point though, was to write the non-file encoded data to the disk so that you could use more optimized streaming methods like memory mapping. If you use an AIFF file, you’re going to have to run it through the JavaSound decoders/parsers. Since you already did that with the OGG file, why do it again? :slight_smile:

[quote]I went to the Jorbis site (http://www.jcraft.com/jorbis/) and the last time the source was touched was 2002. Is this the libary to use or are their newer works based on it?
[/quote]
Those are the guys you want. I’m unaware of any newer files. If you want to have JavaSound handle all the API details for you, make sure you grab the VorbisSPI package. Simply add it to the classpath, and you’ll be able to load OGGs like they were WAV files. :wink:

#5

I thought that Aiff wasn’t encoded, but that it merely had a header or such to identify the data contents.

How does it differ from a wav then or a raw mono stream?

I am somewhat familiar with memory mapping in C/C++ land, but not java. (ie mmap) How would I do that here?

I looked at the GAGE site and figured out how you knew so much about audio. Given that you’re the author or GAGE. :slight_smile:

I need to look at the GAGE audio API as that might get me going even quicker.

#6

[quote]I thought that Aiff wasn’t encoded, but that it merely had a header or such to identify the data contents.

How does it differ from a wav then or a raw mono stream?
[/quote]
If it’s saved at the same sample rate, bit rate, and channels you wish to play it at, and if it’s in the byte order you need, and if you’re looking to write the code to read the header, then I suppose it’s not a big deal. I’m thinking from the perspective of all the low level optimizations that can be done by having the data you need start and end on 4K boundries. i.e. I personally would try to avoid the existance of the header in my memory-mapped file, but maybe that’s just the super-optimization side of me speaking. :slight_smile:

[quote]I am somewhat familiar with memory mapping in C/C++ land, but not java. (ie mmap) How would I do that here?
[/quote]
Here I am going to refer you to the JavaDocs for java.nio.MappedByteBuffer. It contains some good info that should help you get up and going.

[quote]I looked at the GAGE site and figured out how you knew so much about audio. Given that you’re the author or GAGE. :slight_smile:
[/quote]
Who? Little 'ole me? :wink:

Honestly, I gained a good portion of my knowledge by writing GAGESound. Hopefully I haven’t fallen out of date with everyone using JOAL around here. :slight_smile:

[quote]I need to look at the GAGE audio API as that might get me going even quicker.
[/quote]
Be my guest. It’s in public domain, so you can actually use it as a starting point if you’d like.

#7

I puttered around and came up with the following -



import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.Clip;
import javax.sound.sampled.DataLine;
import java.io.*;
import java.util.*;

public class AudioClipPlayer {

      static ArrayList audioClips = new ArrayList();
      static Clip currentClip;
      
      public int addAudioClip(String fileName) {
            File soundFile = new File(fileName);
            try {
                  AudioInputStream stream = AudioSystem.getAudioInputStream(soundFile);
                  AudioFormat format = stream.getFormat();
                  DataLine.Info info = new DataLine.Info(Clip.class, format);
                  Clip clip = (Clip)AudioSystem.getLine(info);
                  clip.open(stream);
                  audioClips.add(clip);
            } catch (Exception ex) {
                  System.out.println("stupid ex in audio");
            }
            
            return audioClips.size() - 1;      
      }
      
      public synchronized void play(int clipID) {
            
            if(clipID == audioClips.size())
                  return;
            
            stop();            
            
            currentClip = (Clip)audioClips.get(clipID);
            currentClip.setFramePosition(0);
            currentClip.start();
      }
      
      public void stop() {
            // Check if any audio is playing and interupt it if we need to
            if(currentClip != null) {
                  currentClip.stop();
            }
      }
}

This seems to do exactly what I want for small sound samples any way.

Correct me if I’m wrong, but this will load the audio into memory completely?

For a quick and easy way to have your buttons or whatever generate sound when pressed or rolled over, this should work great, for swing apps anyway. You can’t have more than one sound play at a time, but that’s what I wanted.

Any comments? Am I just thinking I’ve found a ‘great’ solution and its really going to cause lots of trouble later? I’ve used it for a bit and it seems pretty swell.

This doesn’t examine the ogg playing, but I’m still working on it.

#8

If all you want to do is play sounds in a Swing GUI, the code you found should do it just fine. Only a high performance game should need to use the SourceDataLine method. :slight_smile:

#9

I assume you’ve seen my chatter in a nearby thread about code to use for Clips, and each time you play one a Clip is created and played, so that the same sound can be played simultaneously with others. Can GAGESound do this? Looking at the code it looks like it can’t.

#10

[quote]I assume you’ve seen my chatter in a nearby thread about code to use for Clips, and each time you play one a Clip is created and played, so that the same sound can be played simultaneously with others. Can GAGESound do this? Looking at the code it looks like it can’t.
[/quote]
If you create a new Clip everytime, then the “Clip” objects can do this. You can’t get the same clip to play more than one stream at a time. Similarly, GAGESound allows a sound to be played once, but you can have the same clip loaded more than once. My take on this is that there tends to be very little point. Just restart the sound and no one will notice that the end got chopped off. This is where GAGESound excels. It allows instant restart of sounds, where Clips must be first drained (they block while they drain!).

#11

When you say they need to be drained, when does that have to occur? I didn’t do any draining in the code I posted above and I don’t notice any delays. I guess if I tried it with some very large files, I’d know right away.

Something did occur to me - What thread is causing the audio to be played? Since I’m not calling any type of refresh or update method, something is doing it for me.

I’m guess the swing dispatch thread, but that’s just a guess.

#12

[quote]When you say they need to be drained, when does that have to occur? I didn’t do any draining in the code I posted above and I don’t notice any delays. I guess if I tried it with some very large files, I’d know right away.
[/quote]
If you stop the Clip before it’s finished, and try to reset it, it will pause to drain. If you let the clip play through, you should have very little delay in restarting it. Granted, I haven’t tried it in 1.5, so it’s possible this problem was fixed.

[quote]Something did occur to me - What thread is causing the audio to be played? Since I’m not calling any type of refresh or update method, something is doing it for me.

I’m guess the swing dispatch thread, but that’s just a guess.
[/quote]
JavaSound creates its own background thread for use in Clips and MIDI files. You have to be careful here, because this thread can sometimes interfere with your game.

#13

[quote]Just restart the sound and no one will notice that the end got chopped off. This is where GAGESound excels. It allows instant restart of sounds, where Clips must be first drained (they block while they drain!).
[/quote]
It’s very easily noticeable in my game where you can hear the same 1 second sound coming from 3 different sources at two plays per second. When they all have their own Clip, you hear them all fine. When they’re sharing 1 Clip, you only hear a bunch of clicking.

#14

OK I assume it’s possible still to use GAGESound to create audio objects in multiple instances for the simultaneous play I’m talking about, right? Maybe? Also… it looks like the sounds only play when you’re in a loop and you tell them to. Does that mean you can constantly monitor they’re play… so that if the game loop it’s in stops, like if you pause the game, the sound will pause and be silent? If so, that’s really cool, because I can’t figure out a nice way to do that with Clips. Am I reading this correctly?

#15

[quote]OK I assume it’s possible still to use GAGESound to create audio objects in multiple instances for the simultaneous play I’m talking about, right? Maybe?
[/quote]
There’s absolutely no reason why you can’t load the same sound effect multiple times, and play these instances simultaneously. All you’d need is an engine on top that keeps track of which sounds are the same.

[quote]Also… it looks like the sounds only play when you’re in a loop and you tell them to.
[/quote]
That’s correct. If you don’t keep feeding GAGESound every frame, the SourceDataLines will underflow and should stop automatically.

[quote]Does that mean you can constantly monitor they’re play… so that if the game loop it’s in stops, like if you pause the game, the sound will pause and be silent? If so, that’s really cool, because I can’t figure out a nice way to do that with Clips. Am I reading this correctly?
[/quote]
Hmm… yes it would pause, and yes it should pick back up after you unpause. I don’t know if the results would be exactly what you expected, though. GAGESound uses a simple timing mechanism to keep track of how much data it needs to write to cover each frame of animation. Pausing the entire loop might cause issues with too much time building up in between frames. You’d have to test it to be sure. :slight_smile: