How to play a byte array ?

Hansdampf · May 10, 2004, 1:53pm


public AudioInputStream getAudioInputStream(){
    {
 ArrayList al = new ArrayList();
      double frequency = 500d;
      double rate=IntArray.rate;
      for (double time = 0; time <= rate*2; time++)
      {
        short amplitude =
  (short)(8000d*Math.sin(2*Math.PI*time/(44100d/frequency)));

        byte[] bytePair = shortToBytes(amplitude);
        al.add(new Byte(bytePair[0]));
        al.add(new Byte(bytePair[1]));
      }

byte[] array = new byte[al.size()];
for (int i = 0; i < array.length; i++) array[i] = ((Byte)al.get(i)).byteValue();
return new AudioInputStream(in,
new AudioFormat(AudioFormat.Encoding.PCM_SIGNED, 44100f, 16, 1, 4, 44100f, false),
array.length);
}

I can save it to a file :


 AudioSystem.write(getAudioInputStream(),
                      AudioFileFormat.Type.WAVE,
                      new File("sound.wav"));

then load and play it.

But I want to play it immediately, without saving to a file.

When I play it directly, I get a


java.lang.IllegalArgumentException: No line matching interface SourceDataLine supporting format PCM_SIGNED, 44100.0 Hz, 16 bit, mono, little-endian, audio data is supported.

Quite the same problem under 1.1 , generating+save+load .au works, playing it directly lags terrible. (via sun.AudioClip)

could someone help me out?

oNyx · May 10, 2004, 6:47pm


import java.io.*;
import java.net.URL;
import java.util.*;
import javax.sound.sampled.*;
import com.jcraft.jorbis.*;
import com.jcraft.jogg.*;

public class SoundManager
{
      private javax.sound.sampled.Line.Info lineInfo;

      private Vector afs;
      private Vector sizes;
      private Vector infos;
      private Vector audios;
      private int num=0;
      private static int rate=0;
      private static int channels=0;

      public SoundManager()
      {
            afs=new Vector();
            sizes=new Vector();
            infos=new Vector();
            audios=new Vector();
      }

      public int getCount()
      {
            return num;
      }

      public void addClip(String s)
      {
            try
            {
                  URL url = getClass().getResource(s);

                  byte[] audio = decodeOgg(url.openStream());

                  AudioFormat audioFormat = new AudioFormat
                  (
                        (float)rate,
                        16,
                        channels,
                        true,  // PCM_Signed
                        false  // littleEndian
                  );

                  DataLine.Info info = new DataLine.Info
                  (
                        Clip.class,
                        audioFormat
                  );

                  afs.add(audioFormat);
                  sizes.add(new Integer(audio.length));
                  infos.add(info);
                  audios.add(audio);

                  num++;
            }
            catch(Exception e){e.printStackTrace();}
      }

      public void playSound(int x)
      {
            try
            {
                  if(x>num)
                  {
                        System.out.println("playSound: sample nr["+x+"] is not available");
                  }
                  else
                  {
                        Clip clip = (Clip) AudioSystem.getLine((DataLine.Info)infos.elementAt(x));
                        clip.open((AudioFormat)afs.elementAt(x), (byte[])audios.elementAt(x), 0, ((Integer)sizes.elementAt(x)).intValue());
                        clip.start();
                  }
            }
            catch(LineUnavailableException lue){lue.printStackTrace();}
      }

      public byte[] decodeOgg(InputStream input)throws IOException
      {
            [...]
            channels=vi.channels;
            rate=vi.rate;
            [...]
            return bytearrayoutputstream.toByteArray();
      }
}

That works flawlessly. Oggs are decoded into a Clip (thus in ram) and playback is therefore instantly. However, the first time a sound is played there is a little delay, because the default mixer needs to be created (that delay should be removed by creating a mixer yourself).

[It isn’t a delay per Clip like you get with AudioClip]

Hansdampf · May 10, 2004, 9:14pm

thanks for your code, I tried it yesterday

my MAIN problem ist not the delay (evil too),
I can’t play my procedurally generated sounds.

so I don’t need a new format.

I modified your code to


public void addClip(AudioInputStream audioInputStream)
            throws IOException, UnsupportedAudioFileException, LineUnavailableException
             {
          AudioFormat af = audioInputStream.getFormat();

          int size = (int) (af.getFrameSize() * audioInputStream.getFrameLength());
           byte[] audio = new byte[size];

           DataLine.Info info = new DataLine.Info(Clip.class, af, size);
           audioInputStream.read(audio, 0, size);

           afs.add(af);
           sizes.add(new Integer(size));
           infos.add(info);
           audios.add(audio);
           num++;
             }

and passed the AudioInputStream (which I can save+load+play), I get this exception:


java.lang.IllegalArgumentException: No line matching interface Clip supporting format PCM_SIGNED, 44100.0 Hz, 16 bit, mono, little-endian, audio data, and buffers of 352008 to 352008 bytes is supported.
      at javax.sound.sampled.AudioSystem.getLine(AudioSystem.java:309)
      at kdk.sound.SoundManager.playSound(SoundManager.java:105)
      at kdk.apps.SoundSpeed.gameInit(SoundSpeed.java:126)
      at kdk.Main.run(Main.java:89)
      at java.lang.Thread.run(Thread.java:536)

is there an easy way to achieve what I want ???

oNyx · May 10, 2004, 10:47pm

Hmm… mayor headache

Well, ok. I would just return a byte array instead of a AudioInputStream… and then just use that instead of decodeOgg. Channels 1 and rate 44100… and it should just work.

Heh you already have a byte array there… just return that

oNyx · May 10, 2004, 10:58pm

Oh and I forgot this link:
http://java.sun.com/j2se/1.4.2/docs/guide/sound/programmer_guide/contents.html

There’s everything explained. In detail. It’s rather long winded, but there is everything you’ll ever need to know

Hansdampf · May 11, 2004, 4:46pm

thank you, Onyx.

My problem had to do with wrong Buffer/FrameSizes.

Now I can play all sounds but… THEY LAG

Framerate goes down 50%, jerky…


public void play(){
try{
Clip clip = (Clip) AudioSystem.getLine(info);
clip.open(af, audio, 0, audio.length);
clip.start();
}catch(Exception e){e.printStackTrace();}
}

this is the relevant part when I play a sound. AudioFormat is PCM. Is there a way to do this better/faster?

Or is the only way to go using JNI? JOAL?
And in applets? Does anywhere in the universe exist an applet game with extensive usage of sound that is playable?

BTW, when I try to play generated .au 's (works in 1.1)


AudioStream ads = new AudioStream(new ByteArrayInputStream(data));
 AudioPlayer.player.start(ads);

in 1.5b, the sound is played THREE times in a row, with pauses of one second >:(

Perhaps using Java you have to make sure not to play more than two sounds per minute.

Please convince me of the the opposite

oNyx · May 11, 2004, 5:43pm

Hm… I hadn’t problems like that.

F5-F7 kicks a sample in (if you hold the button they are repeated damn fast).

Samples are: 8bit/mono/rate 11025

other keys:
F1-F4 = zoomfactor
r = restart
F9 = showfps on/off
F10 = fps capping on/off
F12 = showtime on/off (showfps has to be turned off, because they are overlapping)

Btw if capping is disabled the key response gets totally laggy.

The SoundManager class is a bit simpler there (older heh):


import java.io.*;
import java.net.URL;
import java.util.*;
import javax.sound.sampled.*;

public class SoundManager
{
       private javax.sound.sampled.Line.Info lineInfo;

    private Vector afs;
    private Vector sizes;
    private Vector infos;
    private Vector audios;
    private int num=0;

    public SoundManager()
    {
            afs=new Vector();
            sizes=new Vector();
            infos=new Vector();
            audios=new Vector();
    }

    public void addClip(String s)
        throws IOException, UnsupportedAudioFileException, LineUnavailableException
    {
        URL url = getClass().getResource(s);
        //InputStream inputstream = url.openStream();
        AudioInputStream audioInputStream = AudioSystem.getAudioInputStream(loadStream(url.openStream()));
            AudioFormat af = audioInputStream.getFormat();
            int size = (int) (af.getFrameSize() * audioInputStream.getFrameLength());
            byte[] audio = new byte[size];
            DataLine.Info info = new DataLine.Info(Clip.class, af, size);
            audioInputStream.read(audio, 0, size);

            afs.add(af);
            sizes.add(new Integer(size));
            infos.add(info);
            audios.add(audio);

            num++;
    }

    private ByteArrayInputStream loadStream(InputStream inputstream)
              throws IOException
      {
            ByteArrayOutputStream bytearrayoutputstream = new ByteArrayOutputStream();
            byte data[] = new byte[1024];
            for(int i = inputstream.read(data); i != -1; i = inputstream.read(data))
                  bytearrayoutputstream.write(data, 0, i);

            inputstream.close();
            bytearrayoutputstream.close();
            data = bytearrayoutputstream.toByteArray();
            return new ByteArrayInputStream(data);
    }

    public void playSound(int x)
          throws UnsupportedAudioFileException, LineUnavailableException
    {
            if(x>num)
            {
                  System.out.println("playSound: sample nr["+x+"] is not available");
            }
            else
            {
                  Clip clip = (Clip) AudioSystem.getLine((DataLine.Info)infos.elementAt(x));
                  clip.open((AudioFormat)afs.elementAt(x), (byte[])audios.elementAt(x), 0, ((Integer)sizes.elementAt(x)).intValue());
                  clip.start();
            }
      }
}

If there are ~10 samples overlapping played at the same time the framerate goes also down by ~50%. But my machine is rather old (500mhz) and the framerate is approximative capped at somewhat over 60fps (in 5msec sleep steps - with my pc sleep is set to 15msec).

So it wouldn’t be a problem with better timing and a maximum of let’s say 8 sounds at the same time - even with a pc crappy as mine

Btw it’s a bit nicer if addClip returns the index number of the clip.

int blamSnd=sou.addClip("/sfx/blam.wav");

Easy to add

Malohkan · July 26, 2004, 5:17pm

oNyx, I tried using your suggestions to load ogg files for myself, and I can get the file to load and play now, but it sounds nothing like it’s supposed to. Windows Media Player plays the sound fine, but when I use the code you suggested, it sounds like I’ve lost all the quality. Instead of a “boom” sound I get a “scratch” sound. Any idea what I might be doing wrong? I’m using all the code you suggested in this thread, except my decodeOgg() method looks like this:

ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int len;
while ((len = input.read(buf)) > 0) {
      bos.write(buf, 0, len);
}
input.close();
bos.close();
return bos.toByteArray();

Thanks!

oNyx · July 26, 2004, 5:53pm

It should look more like… uhm… this:


public byte[] decodeOgg(InputStream input)throws IOException
{
      ByteArrayOutputStream bytearrayoutputstream = new ByteArrayOutputStream();
      int convsize=4096*2;
      byte[] convbuffer=new byte[convsize]; // take 8k out of the data segment, not the stack

      SyncState oy=new SyncState(); // sync and verify incoming physical bitstream
      StreamState os=new StreamState(); // take physical pages, weld into a logical stream of packets
      Page og=new Page(); // one Ogg bitstream page.  Vorbis packets are inside
      Packet op=new Packet(); // one raw packet of data for decode

      Info vi=new Info();  // struct that stores all the static vorbis bitstream settings
      Comment vc=new Comment(); // struct that stores all the bitstream user comments
      DspState vd=new DspState(); // central working state for the packet->PCM decoder
      Block vb=new Block(vd); // local working space for packet->PCM decode

      byte[] buffer;
      int bytes=0;

      // Decode setup

      oy.init(); // Now we can read pages

      while(true)
      { // we repeat if the bitstream is chained
            int eos=0;

            // grab some data at the head of the stream.  We want the first page
            // (which is guaranteed to be small and only contain the Vorbis
            // stream initial header) We need the first page to get the stream
            // serialno.

            // submit a 4k block to libvorbis' Ogg layer
            int index=oy.buffer(4096);
            buffer=oy.data;
            try
            {
                  bytes=input.read(buffer, index, 4096);
            }
            catch(Exception e)
            {
                  System.err.println(e);
                  System.exit(-1);
            }
            oy.wrote(bytes);

            // Get the first page.
            if(oy.pageout(og)!=1)
            {
                  // have we simply run out of data?  If so, we're done.
                  if(bytes<4096)break;

                  // error case.  Must not be Vorbis data
                  System.err.println("Input does not appear to be an Ogg bitstream.");
                  System.exit(1);
            }

            // Get the serial number and set up the rest of decode.
            // serialno first; use it to set up a logical stream
            os.init(og.serialno());

            // extract the initial header from the first page and verify that the
            // Ogg bitstream is in fact Vorbis data

            // I handle the initial header first instead of just having the code
            // read all three Vorbis headers at once because reading the initial
            // header is an easy way to identify a Vorbis bitstream and it's
            // useful to see that functionality seperated out.

            vi.init();
            vc.init();
            if(os.pagein(og)<0)
            {
                  // error; stream version mismatch perhaps
                  System.err.println("Error reading first page of Ogg bitstream data.");
                  System.exit(1);
            }

            if(os.packetout(op)!=1)
            {
                  // no page? must not be vorbis
                  System.err.println("Error reading initial header packet.");
                  System.exit(1);
            }

            if(vi.synthesis_headerin(vc,op)<0)
            {
                  // error case; not a vorbis header
                  System.err.println("This Ogg bitstream does not contain Vorbis audio data.");
                  System.exit(1);
            }

            // At this point, we're sure we're Vorbis.  We've set up the logical
            // (Ogg) bitstream decoder.  Get the comment and codebook headers and
            // set up the Vorbis decoder

            // The next two packets in order are the comment and codebook headers.
            // They're likely large and may span multiple pages.  Thus we reead
            // and submit data until we get our two pacakets, watching that no
            // pages are missing.  If a page is missing, error out; losing a
            // header page is the only place where missing data is fatal. */

            int i=0;
            while(i<2)
            {
                  while(i<2)
                  {
                        int result=oy.pageout(og);
                        if(result==0) break; // Need more data
                        // Don't complain about missing or corrupt data yet.  We'll
                        // catch it at the packet output phase

                        if(result==1)
                        {
                              os.pagein(og); // we can ignore any errors here
                              // as they'll also become apparent
                              // at packetout
                              while(i<2)
                              {
                                    result=os.packetout(op);
                                    if(result==0)break;
                                    if(result==-1)
                                    {
                                          // Uh oh; data at some point was corrupted or missing!
                                          // We can't tolerate that in a header.  Die.
                                          System.err.println("Corrupt secondary header.  Exiting.");
                                          System.exit(1);
                                    }
                                    vi.synthesis_headerin(vc,op);
                                    i++;
                              }
                        }
                  }
                  // no harm in not checking before adding more
                  index=oy.buffer(4096);
                  buffer=oy.data;
                  try
                  {
                        bytes=input.read(buffer, index, 4096);
                  }
                  catch(Exception e)
                  {
                        System.err.println(e);
                        System.exit(1);
                  }
                  if(bytes==0 && i<2)
                  {
                        System.err.println("End of file before finding all Vorbis headers!");
                        System.exit(1);
                  }
                  oy.wrote(bytes);
            }

            // Throw the comments plus a few lines about the bitstream we're
            // decoding
            {
                  byte[][] ptr=vc.user_comments;
                  for(int j=0; j<ptr.length;j++)
                  {
                        if(ptr[j]==null) break;
                        System.err.println(new String(ptr[j], 0, ptr[j].length-1));
                  }
                  channels=vi.channels;
                  rate=vi.rate;
                  //System.err.println("\nBitstream is "+vi.channels+" channel, "+vi.rate+"Hz");
                  //System.err.println("Encoded by: "+new String(vc.vendor, 0, vc.vendor.length-1)+"\n");
            }

            convsize=4096/vi.channels;

            // OK, got and parsed all three headers. Initialize the Vorbis
            //  packet->PCM decoder.
            vd.synthesis_init(vi); // central decode state
            vb.init(vd);           // local state for most of the decode
            // so multiple block decodes can
            // proceed in parallel.  We could init
            // multiple vorbis_block structures
            // for vd here

            float[][][] _pcm=new float[1][][];
            int[] _index=new int[vi.channels];
            // The rest is just a straight decode loop until end of stream
            while(eos==0)
            {
                  while(eos==0)
                  {
                        int result=oy.pageout(og);
                        if(result==0)break; // need more data
                        if(result==-1)
                        { // missing or corrupt data at this page position
                              System.err.println("Corrupt or missing data in bitstream; continuing...");
                        }
                        else
                        {
                              os.pagein(og); // can safely ignore errors at
                              // this point
                              while(true)
                              {
                                    result=os.packetout(op);

                                    if(result==0)break; // need more data
                                    if(result==-1)
                                    { // missing or corrupt data at this page position
                                          // no reason to complain; already complained above
                                    }
                                    else
                                    {
                                          // we have a packet.  Decode it
                                          int samples;
                                          if(vb.synthesis(op)==0)
                                          { // test for success!
                                                vd.synthesis_blockin(vb);
                                          }

                                          // **pcm is a multichannel float vector.  In stereo, for
                                          // example, pcm[0] is left, and pcm[1] is right.  samples is
                                          // the size of each channel.  Convert the float values
                                          // (-1.<=range<=1.) to whatever PCM format and write it out

                                          while((samples=vd.synthesis_pcmout(_pcm, _index))>0)
                                          {
                                                float[][] pcm=_pcm[0];
                                                boolean clipflag=false;
                                                int bout=(samples<convsize?samples:convsize);

                                                // convert floats to 16 bit signed ints (host order) and
                                                // interleave
                                                for(i=0;i<vi.channels;i++)
                                                {
                                                      int ptr=i*2;
                                                      //int ptr=i;
                                                      int mono=_index[i];
                                                      for(int j=0;j<bout;j++)
                                                      {
                                                            int val=(int)(pcm[i][mono+j]*32767.);
                                                            //                  short val=(short)(pcm[i][mono+j]*32767.);
                                                            //                  int val=(int)Math.round(pcm[i][mono+j]*32767.);
                                                            // might as well guard against clipping
                                                            if(val>32767)
                                                            {
                                                            val=32767;
                                                            clipflag=true;
                                                            }
                                                            if(val<-32768)
                                                            {
                                                            val=-32768;
                                                            clipflag=true;
                                                            }
                                                            if(val<0) val=val|0x8000;
                                                            convbuffer[ptr]=(byte)(val);
                                                            convbuffer[ptr+1]=(byte)(val>>>8);
                                                            ptr+=2*(vi.channels);
                                                      }
                                                }

                                                //if(clipflag)
                                                //  System.err.println("Clipping in frame "+vd.sequence);

                                                //System.out.write(convbuffer, 0, 2*vi.channels*bout);
                                                bytearrayoutputstream.write(convbuffer, 0, 2*vi.channels*bout);

                                                vd.synthesis_read(bout); // tell libvorbis how
                                                // many samples we
                                                // actually consumed
                                          }
                                    }
                              }
                              if(og.eos()!=0)eos=1;
                        }
                  }
                  if(eos==0)
                  {
                        index=oy.buffer(4096);
                        buffer=oy.data;
                        try
                        {
                              bytes=input.read(buffer,index,4096);
                        }
                        catch(Exception e)
                        {
                              System.err.println(e);
                              System.exit(1);
                        }
                        oy.wrote(bytes);
                        if(bytes==0)eos=1;
                  }
            }

            // clean up this logical bitstream; before exit we see if we're
            // followed by another [chained]

            os.clear();

            // ogg_page and ogg_packet structs always point to storage in
            // libvorbis.  They're never freed or manipulated directly

            vb.clear();
            vd.clear();
            vi.clear();  // must be called last
      }

      // OK, clean up the framer
      oy.clear();
      //System.err.println("locked and loaded");
      bytearrayoutputstream.close();
      return bytearrayoutputstream.toByteArray();
}

oNyx · July 26, 2004, 6:01pm

Ah well… what a mess…

Here is the whole (ready to use) thing:

http://kaioa.com/src/SoundManager.java

Malohkan · July 26, 2004, 6:49pm

holy cow. I’ll consider this my birthday present from you That was great! Works completely and now I’m all set to do what I need to do. Many many thanks

oNyx · July 26, 2004, 8:24pm

Well, keep in mind that this thing decodes the whole ogg on loading. It’s perfectly fine for sound effects, but it takes alot of ram if you decode a whole track (easily 50-60mb [the size of a wav]).

So in that case you would need something else - streaming. Jorbis has a streaming example and Tom has posted some streaming source too (openal).

Malohkan · July 28, 2004, 2:30am

yeah all of my clips are like .5 - 2 seconds, so I don’t have that problem. If I find a need for music or something long, I’ll be sure to look into that. Thanks again!