Hi all,
I’m starting to work on a music/metronome application in Java and I’m running into some problems with the timing and speed.
My test program plays two sine wave tones together at regular intervals and updates two visual displays (one display for each tone). At slower speeds the two tones will play in sync for a little while, then slightly out of sync for a few beats, then they will play back in sync again, and this cycle continues. At faster speeds, the synchronization is all over the place.
From researching good metronome programming, I found that Thread.sleep() is horrible for timing, so I completely avoided that and went with checking System.nanoTime() to determine when the sounds should play.
I’m using AudioSystem’s SourceDataLine for my audio player and I’m using a thread for each tone that constantly polls System.nanoTime() in order to determine when to play the tones and update the visual display. I create a new SourceDataLine and delete the previous one each time a sound plays, because the volume fluctuates if I leave the line open and keep playing sounds on the same line…I’ve read that you’re not supposed to leave the SourceDataLine open when not playing anything, but I wanted to see what happened if I did. Unfortunately the process of creating a new player each time is costing a lot of time, I timed the process for creating the player and it was upwards of about 270ms. However, I’m not sure what else to do with it.
In theory this seemed like a good method for getting each sound to play on time, but in reality it seems like the audio hardware isn’t able to keep up with the method that I’m using and is causing timing problems.
At the moment this is just a simple test in Java, but my goal is to create my app on mobile devices (Android, iOS, Windows Phone, etc)…however my current method isn’t even keeping perfect time on a PC, so I’m worried that certain mobile devices with limited resources will have even more timing problems. I will also be adding more sounds to it to create more complex rhythms, so it needs to be able to handle multiple sounds going simultaneously without lagging.
Another problem I’m having is that the max tempo is controlled by the length of the tone since the tones don’t overlap each other. I tried adding additional threads so that every tone that played would get its own thread…but that really screwed up the timing, so I took it out. I would like to have a way to overlap the previous sound to allow for much higher tempos.
One final point…my first attempt I used a larger buffer (loading multiple tones separated by silence) and played the buffer using SourceDataLine. However, the problem with this method was that I wasn’t able to know exactly when each tone played so that I could update the beat counter visual display. I do have a working example using this method, so if needed I could post my code for this as well.
Any help getting these timing and speed issues straightened out would be greatly appreciated. Thanks!
Here is my code…
SoundTest.java
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;
import java.io.*;
import javax.sound.sampled.*;
public class SoundTest implements ActionListener {
static SoundTest soundTest;
// ENABLE/DISABLE SOUNDS
boolean playSound1 = true;
boolean playSound2 = true;
JFrame mainFrame;
JPanel mainContent;
JPanel center;
JButton buttonPlay;
JLabel[] beatDisplay;
int sampleRate = 44100;
long startTime;
SourceDataLine line = null;
int tickLength;
boolean playing = false;
SoundElement sound01;
SoundElement sound02;
public static void main (String[] args) {
soundTest = new SoundTest();
SwingUtilities.invokeLater(new Runnable() { public void run() {
soundTest.gui_CreateAndShow();
}});
}
public void gui_CreateAndShow() {
gui_FrameAndContentPanel();
gui_AddContent();
}
public void gui_FrameAndContentPanel() {
mainContent = new JPanel();
mainContent.setLayout(new BorderLayout());
mainContent.setPreferredSize(new Dimension(400,400));
mainContent.setOpaque(true);
mainFrame = new JFrame("Sound Test");
mainFrame.setContentPane(mainContent);
mainFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
mainFrame.pack();
mainFrame.setVisible(true);
}
public void gui_AddContent() {
JPanel center = new JPanel();
center.setOpaque(true);
buttonPlay = new JButton("PLAY / STOP");
buttonPlay.setActionCommand("play");
buttonPlay.addActionListener(this);
buttonPlay.setPreferredSize(new Dimension(200, 50));
beatDisplay = new JLabel[2];
beatDisplay[0] = new JLabel("~ SOUND 1 ~");
beatDisplay[0].setPreferredSize(new Dimension(400, 50));
beatDisplay[0].setHorizontalAlignment(SwingConstants.CENTER);
beatDisplay[0].setOpaque(true);
beatDisplay[0].setFont(new Font("Serif", Font.BOLD, 40));
beatDisplay[1] = new JLabel("~ SOUND 2 ~");
beatDisplay[1].setPreferredSize(new Dimension(400, 50));
beatDisplay[1].setHorizontalAlignment(SwingConstants.CENTER);
beatDisplay[1].setOpaque(true);
beatDisplay[1].setFont(new Font("Serif", Font.BOLD, 40));
center.add(buttonPlay);
center.add(beatDisplay[0]);
center.add(beatDisplay[1]);
mainContent.add(center, BorderLayout.CENTER);
}
public void actionPerformed(ActionEvent e) {
if (!playing) {
playing = true;
if (playSound1)
sound01 = new SoundElement(this, "Sound1", 0, 800);
if (playSound2)
sound02 = new SoundElement(this, "Sound2", 1, 1200);
startTime = System.nanoTime();
if (playSound1)
new Thread(sound01).start();
if (playSound2)
new Thread(sound02).start();
}
else {
playing = false;
}
}
}
SoundElement.java
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;
import java.io.*;
import javax.sound.sampled.*;
public class SoundElement implements Runnable {
SoundTest soundTest;
// TEMPO CHANGE
// 750000000=80bpm | 300000000=200bpm | 200000000=300bpm
long nsDelay = 750000000;
long before;
long after;
long diff;
String name="";
int soundNumber;
int clickLength = 4100;
byte[] audioFile;
double clickFrequency;
SourceDataLine line = null;
long audioFilePlay;
// GUI
int beatCount=1;
boolean green=true;
public SoundElement(SoundTest soundTestIn, String nameIn, int soundNumberIn, double clickFrequencyIn){
soundTest = soundTestIn;
name = nameIn;
soundNumber = soundNumberIn;
clickFrequency = clickFrequencyIn;
generateAudioFile();
}
public void generateAudioFile(){
audioFile = new byte[clickLength * 2];
double temp;
short maxSample;
int p=0;
for (int i = 0; i < audioFile.length;){
temp = Math.sin(2 * Math.PI * p++ / (soundTest.sampleRate/clickFrequency));
maxSample = (short) (temp * Short.MAX_VALUE);
audioFile[i++] = (byte) (maxSample & 0x00ff);
audioFile[i++] = (byte) ((maxSample & 0xff00) >>> 8);
}
}
public void run() {
createPlayer();
audioFilePlay = soundTest.startTime + nsDelay;
while (soundTest.playing){
if (System.nanoTime() >= audioFilePlay){
play();
gui_UpdateBeatCounter();
destroyPlayer();
createPlayer();
audioFilePlay += nsDelay;
}
}
try { destroyPlayer(); } catch (Exception e) { }
}
public void createPlayer(){
AudioFormat af = new AudioFormat(soundTest.sampleRate, 16, 1, true, false);
try {
line = AudioSystem.getSourceDataLine(af);
line.open(af);
line.start();
}
catch (Exception ex) { ex.printStackTrace(); }
}
public void play(){
line.write(audioFile, 0, audioFile.length);
}
public void destroyPlayer(){
line.drain();
line.close();
}
public void gui_UpdateBeatCounter(){
soundTest.beatDisplay[soundNumber].setText("" + beatCount);
if (green){
soundTest.beatDisplay[soundNumber].setBackground(Color.GREEN);
green = false;
}
else{
soundTest.beatDisplay[soundNumber].setBackground(Color.YELLOW);
green = true;
}
if (beatCount == 4)
beatCount = 1;
else
beatCount++;
}
}
Next window of opportunity will be Sunday. Will be looking forward to it!