AudioMixer

Demo up!

http://www.hexara.com/VSL/AudioMixerDemoWarOfWorlds.htm

All sounds are routed through the a single Mixer.

The ‘bombs’ are actually the same clip data as the gunshot, but slowed down a LOT.
The ‘machine gun’ is also the same clip but played 6 times in quick succession. I stuck the play() command in a for loop with pauses of Thread.pause(88) in between.

The ‘RayGun’ is a little FM synthesis toy I made.

Everything is triggered by Buttons at the moment. That means, the sound won’t go off until you complete the mouse click (down & up). You can click the ‘guns’ multiple times and they will retrigger and overlap. The RayGun stuff though won’t let you retrigger until the sound ends.

132KB is the .wav file.
The code is 32KB.

That was fun! ;D

Good job! Definitely getting there. Possibly need some sort of automatic gain control on the output - running multiple sounds does cause audible clipping at times. It should be possible to get the latency down further too - it’s not bad but there’s still an audible delay after hitting the buttons.

Best wishes, Neil

@philfrei
I got this as soon as the applet loaded:



java.io.IOException: mark/reset not supported
	at java.io.InputStream.reset(Unknown Source)
	at com.sun.media.sound.SoftMidiAudioFileReader.getAudioInputStream(Unknown Source)
	at javax.sound.sampled.AudioSystem.getAudioInputStream(Unknown Source)
	at com.adonax.pfaudio.ClipTrack.<init>(ClipTrack.java:79)
	at com.adonax.pfaudio.warApplet.WarGUI.<init>(WarGUI.java:86)
	at com.adonax.pfaudio.warApplet.WarApplet$1.run(WarApplet.java:18)
	at java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.awt.EventQueue.access$000(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)
Exception in thread "AWT-EventQueue-2" java.lang.NullPointerException
	at com.adonax.pfaudio.warApplet.WarGUI.playAShortSilentTone(WarGUI.java:143)
	at com.adonax.pfaudio.warApplet.WarApplet$1.run(WarApplet.java:20)
	at java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.awt.EventQueue.access$000(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)

I get this when I click on any of the 5 buttons on the top:



Exception in thread "AWT-EventQueue-2" java.lang.NullPointerException
	at com.adonax.pfaudio.warApplet.WarGUI.actionPerformed(WarGUI.java:104)
	at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
	at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
	at java.awt.Component.processMouseEvent(Unknown Source)
	at javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.awt.Component.processEvent(Unknown Source)
	at java.awt.Container.processEvent(Unknown Source)
	at java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.awt.EventQueue.access$000(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue$4.run(Unknown Source)
	at java.awt.EventQueue$4.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)

Ray Gun and Zapper work fine for me :slight_smile:

Thanks @ra4king!

This is where the .wav file is loaded. Since it doesn’t load, that causes more errors along the way.

I’ve loaded resources from jars before! I specify a file location relative to my “AudioMixer” class, as follows:


		AudioInputStream ais = AudioSystem.getAudioInputStream(
				AudioMixer.class.getResourceAsStream(fileName));

The AudioMixer is part of the same package as the ClipTrack: com.adonax.pfaudio. The sound file is in a subfolder “audio/GunshotIndoor2.wav”.

Maybe there was something screwy with my web host? I just updated the app, and tested on IE & Firefox, and it ran okay.

Try again?
Any suggestions?

@nsigma: Lag? Largely due to GUI laziness on my part. (Shot myself in the foot?)

Slightly improved demo is now up: http://www.hexara.com/VSL/AudioMixerDemoWarOfWorlds.htm

For one, Java JButtons don’t send events until the MouseUp occurs. So the sounds will naturally seem to fall behind the clicks. Maybe I should create MouseDown sensitive “touchpads.” (:cranky: Have…other…priorities…)

I admit I used a single ActionListener, and a chain of string comparisons to interpret the ActionCommand strings. I’ve changed this to individual, anonymous ActionListeners. The RayGun has a built-in ramp-up time to prevent clicks. I sharpened that envelope by a factor of 5. But that ramp-up could still be perceived as lag. Also, since I run them (and the machine gun) on new threads, I should probably use a thread pool, which could also reduce latency. The Gun.wav could probably be edited a bit tighter. Silent samples at the front end could be perceived as lag, especially at the slower playback rates of the bigger bombs. There’s a tiny bit of distortion near the very end of the cheap, free gunshot sample as well.

All in all, my point is that it is dubious to judge lag based solely on a web demo. One doesn’t know what the programmer has done via the GUI layer. Better is to ask: is it in the ball park, and thus worth taking a more serious look?

+++++++++++++++++++++++++++++++++++++++++++++++

Here’s what the coding looks like, as far as an example of api use. The AudioMixer is turned on via a play(mixerLine) command. The null means: use the default output line. One could specify a particular output line here, for example if needed for Linux. Three MixerTracks are specified for the AudioMixer: a gunshot ClipTrack and two RayGuns. (Both ClipTrack and RayGun implement the MixerTrack interface. Yes, all the bombs & guns are on the same MixerTrack.) By passing the AudioMixer as a parameter in their creation, the tracks are created automatically.


		audioMixer = new AudioMixer();
		audioMixer.play(null);
		audioMixer.setMasterVolume(0.65);
		
		// create Mixer tracks, one for gun and two RayGuns
		try {
			gun = new ClipTrack(audioMixer, 
					"audio/GunshotIndoor2.wav", true);
		} catch (UnsupportedAudioFileException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		
		ray1 = new RayGun(audioMixer, 500, 0.6f, 12, 5);
		ray2 = new RayGun(audioMixer, 1250, 0.5f, 50, 4);

To play the various is also pretty straightforward. The ClipTrack play() optionally take a factor that controls playback rate. For example play(0.5) plays at half speed, and play(2) plays back at double speed. RayGuns are ‘continuous’ so I use a start() and stop(). I didn’t explain the RayGun parameters here because I haven’t settled on the specifics of that class. [Parameter list: AudioMixer, frequency (in Hertz), volume, LFO modulation rate (in Hertz), modulation depth (in wavetable samples :P)]


		// ActionListeners added to JButtons:
		bigBomb.addActionListener(new ActionListener()
		{
			@Override
			public void actionPerformed(ActionEvent e) {
				gun.setVolume(0.8f);
				gun.play(0.24);
			}
		});
		mediumBomb.addActionListener(new ActionListener()
		{
			@Override
			public void actionPerformed(ActionEvent e) {
				gun.setVolume(0.7f);
				gun.play(0.30);
			}
		});
		smallBomb.addActionListener(new ActionListener()
		{
			@Override
			public void actionPerformed(ActionEvent e) {
				gun.setVolume(0.6f);
				gun.play(0.38);
			}
		});
		gunshot.addActionListener(new ActionListener()
		{
			@Override
			public void actionPerformed(ActionEvent e) {
				gun.setVolume(0.5f);
				gun.play();
			}
			
		});
		machineGun.addActionListener(new ActionListener()
			{
				@Override
				public void actionPerformed(ActionEvent e) {
					PlayMachineGun pmg = new PlayMachineGun();
					Thread t = new Thread(pmg);
					t.start();						
				}
			}
		);
		rayGun1.addActionListener(new ActionListener()
		{
			@Override
			public void actionPerformed(ActionEvent e) 
			{
				DeathRay dr = new DeathRay();
				Thread t = new Thread(dr);
				t.start();
			}
		});
		rayGun2.addActionListener(new ActionListener()
		{
			@Override
			public void actionPerformed(ActionEvent arg0) 
			{
				RayGunBlast rgb = new RayGunBlast();
				Thread t = new Thread(rgb);
				t.start();	
			}
		});

	class PlayMachineGun implements Runnable
	{
		@Override
		public void run() 
		{
			gun.setVolume(0.6f);
		
			for (int i = 0; i < 5; i++)
			{
				gun.play(1.25);
				try {
					Thread.sleep(88);
				} catch (InterruptedException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}
			}
		}	
	}
	
	class DeathRay implements Runnable
	{

		@Override
		public void run() 
		{
			ray1.start();
			try {
				Thread.sleep(1000);
			} catch (InterruptedException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
			ray1.stop();
		}
	}
	class RayGunBlast implements Runnable
	{

		@Override
		public void run() 
		{
			ray2.start();
			try {
				Thread.sleep(350);
			} catch (InterruptedException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
			ray2.stop();
		}
	}

Notice: no need to deal with SourceDataLines or AudioInputStreams, etc. And with this ClipTrack, one can leverage a single .wav resource to a considerable extent.

Meanwhile, I DO look forward to the appearance of other mixers that have been promised and are in the works. It will be awesome to have multiple options.

Nope, same errors when initializing and for the top five :frowning:
Strangely, the bottom two are working. Try looking into anything different you are doing with the bottom two from the top 5?

The bottom two buttons don’t use .wav data. They generate the sound on the fly. The top five ALL rely on GunshotIndoor2.wav being loaded. And the error says this file couldn’t be located.

The ClipTrack instantiation uses the fileName string: “audio/GunshotIndoor2.wav”

The resource load uses:

		AudioInputStream ais = AudioSystem.getAudioInputStream(
				AudioMixer.class.getResourceAsStream(fileName));

The structure of the unpacked jar:
/com/adonax/pfaudio/

this location holds:
audio/GunshotIndoor2.wav
warApplet/WarApplet.class
AudioMixer.class
ClipTrack.class

The html spec:

    var attributes = {code:'com.adonax.pfaudio.warApplet.class',
                      archive:'AudioMixerDemoWar.jar',
                      width:700, height:100} ;

Why shouldn’t the “getResource” work in one case and not in another?
It should be okay to use the class AudioMixer as the reference point, yes? Even though we are executing code in ClipTrack? (They are in the same directory.)

Doh! It says “mark/reset not supported”

[quote]java.io.IOException: mark/reset not supported
at java.io.InputStream.reset(Unknown Source)
at com.sun.media.sound.SoftMidiAudioFileReader.getAudioInputStream(Unknown Source)
at javax.sound.sampled.AudioSystem.getAudioInputStream(Unknown Source)
at com.adonax.pfaudio.ClipTrack.(ClipTrack.java:79)
at com.adonax.pfaudio.warApplet.WarGUI.(WarGUI.java:86)
[/quote]
Why would my browsers have no problem with mark/reset?
And isn’t this the correct way to get an audio resource?

I have never used mark/reset methods ever in my life.

I wonder if the .wav that was saved by Audacity (I trimmed some of the leading silence off) is somehow not what is expected.

AFAIK, Java found the file and tried to read it. Normally, .wav’s are read just fine by this method. (getResourceAsStream)

We need a smiley icon for “bangs-head-against-wall”.

Windows media plays the .wav just fine.

Is there a problem loading resources if the file is too small? (It is 132KB, about 3/4 of a second.) I’ve heard of problems with Clips under a second before, but I thought that had to do with the playback buffer, not from loading the data from file.

[EDIT: reloaded jar with a larger .wav file, just over a second in length. It adds 20KB to the jar.]
[EDIT2: didn’t help. Meanwhile here’s a clue: investigating if it is a Java 7 issue, that InputStream is perhaps “pickier” than before about what it considers a “markable” file. This effectively breaks “loadResourceAsStream()” for audio if it is true! Yikes! I hope it isn’t true.]

[EDIT3: SOLVED! (I hope)
I changed the way the audio file is being read into the program. Old way is commented out. Preferred form is below.]


//		AudioInputStream ais = AudioSystem.getAudioInputStream(
//				AudioMixer.class.getResourceAsStream(fileName));

		URL url = AudioMixer.class.getResource(fileName);
		AudioInputStream ais = 
			AudioSystem.getAudioInputStream(url);

Don’t take things too critically! ;D

I’m just saying that there is some room for improvement, but you’re definitely in the ballpark. However, I doubt any of this is due to your GUI code. You’d have to do something really ‘lazy’ for that to make much difference.

Your code for timing events could be improved though - maybe do the initial play() / start() whatever in the actionPerformed() method, and then post subsequent events into a Timer or ScheduledExecutorService.

I’m planning on some of those for JAudioLibs (recently been having conversation with another audio developer about adding some useful GUI widgets) but that won’t be until the new year. Like you - other priorities.

However, quick fix for your applet might be to add mouse listeners to the JButtons instead of action listeners, and fire on mousePressed().

I’m sure we used to have one of those?!

@nsigma Thanks for the suggestion about attaching a MouseListener to the JButtons. I didn’t realize that was an option. :clue: The demo works much better! I also put the starts of the RayGuns prior to the Thread creation. Seems to help. I don’t think it would be wise to do this to the Machine gun. Because the timing of the shots needs to be very even, I want it all in one place.

I added an ‘Options’ MenuBar for picking an output line, in case there is a Linux user or someone else who is having troubles such as we’ve discussed before with output lines.

The only other red flag was from ra4king. I’ve confirmed that other people have hit the same issue with Java 7 and backwards compatibility with audio. There’s already thread on it at the Oracle Forum for Java Sound. They are calling it a bug and have registered it with Oracle, but I suspect my fix (avoiding using InputStream) is going to be fine. Am awaiting any confirmation that this fix works! Hopefully ra4king tries again or someone else with Java 7 gives the demo a try and let’s me know if it works now or not.

It works now!! Hooray!!! :slight_smile:

Whoo hoo! You made my day.
Now I go to the 6 other forums where I was asking about this and follow up…