Hi all,
I’m currently working on a metronome app and thanks to this awesome community (special thanks to philfrei!) the audio section is working really well. Now I’m working on syncing a visual display to the audio and it’s not working quite right. For testing purposes the visual is two JLabels that display the current beat and change color.
I’m basically trying to get these two visual displays to update at exactly the same time, but they don’t always change at the same time, especially at faster tempos. This is essentially the problem I was previously having with the audio because I was trying to play multiple files simultaneously via different threads. Philfrei’s fix for the audio was timing it to the audio hardware which worked perfectly, but I don’t believe I can use that same method with the visual and I have to rely on Java’s timing to update the visual when it is supposed to.
My goal is to eventually move this over to mobile devices and I’d prefer not to use 3rd party libraries so that I know how it functions and I can move it over to mobile when the time comes.
I created a short example that I could post to the forum which illustrates the problem I’m having. Here’s a brief explanation of each of my java files:
TimingCheck.java creates and displays the GUI and it also generates and plays the metronome audio tone. The audio part is different from my actual metronome program, but for the purpose of posting this is a lot shorter.
The visual part is essentially the same as my metronome program, but for simplicity of making this example I did just “copy and paste” the same code over again where I placed the comments “VISUAL 1” and “VISUAL 2” instead of looping through arrays.
VisualCalc.java is a loop that calculates when the visual should change and adds the time in milliseconds to an ArrayList in Visual.java. The value that is added to the ArrayList in Visual.java is offset by the value of “visualBuffer”, in order to match the latency created by adding the audio tone and playing it. Without this delay the visual would display way before the audio tone plays.
Visual.java is a loop that checks the system time to see if the visual needs to be changed and it updates the JLabel in TimingCheck.java.
Before posting my sample code, I wanted to mention that I’ve also been doing some research into game engines and I was wondering if something like that would be better suited for my metronome application? My current approach kind of feels like the audio and visual are totally separate elements each doing their own thing on their own time, but I want these two things to be tightly connected so they don’t drift apart over time. I’m not totally sure if trying to develop a game engine would work better or if I’d be better off trying to get my current approach to work.
I’d greatly appreciate any help in getting the visual display synced up to the audio. Thanks! 
Here’s my sample code:
TimingCheck.java
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;
import java.io.*;
import javax.sound.sampled.*;
public class TimingCheck implements ActionListener {
static TimingCheck timingCheck;
static Visual visual;
static VisualCalc visualCalc;
// CHANGE TEMPO
int tempo = 60;
JFrame mainFrame;
JPanel mainContent;
JPanel center;
JButton buttonPlay;
JLabel[] beatDisplay;
double frequencyOfTick = 800;
int sampleRate = 44100;
int bufferSize = 2000;
byte[] buffer;
int tickBufferSize;
double[] tickBuffer;
SourceDataLine line = null;
int tickLength = sampleRate / 10;
boolean isPlaying = false;
long systemStartTime;
public static void main (String[] args) {
timingCheck = new TimingCheck();
SwingUtilities.invokeLater(new Runnable() {
public void run() {
timingCheck.guiCreateAndShow();
}
});
}
public TimingCheck(){
tickBufferSize = (int)((60 / (double)tempo) * sampleRate);
visual = new Visual(this);
visualCalc = new VisualCalc(this, visual);
generateTick();
}
public void guiCreateAndShow() {
guiFrameAndContentPanel();
guiAddContent();
}
public void guiFrameAndContentPanel() {
mainContent = new JPanel();
mainContent.setLayout(new BorderLayout());
mainContent.setPreferredSize(new Dimension(500,500));
mainContent.setOpaque(true);
mainFrame = new JFrame("Timing Check");
mainFrame.setContentPane(mainContent);
mainFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
mainFrame.pack();
mainFrame.setVisible(true);
}
public void guiAddContent() {
JPanel center = new JPanel();
center.setOpaque(true);
buttonPlay = new JButton("PLAY / STOP");
buttonPlay.setActionCommand("play");
buttonPlay.addActionListener(this);
buttonPlay.setPreferredSize(new Dimension(200, 50));
beatDisplay = new JLabel[2];
beatDisplay[0] = new JLabel("~ SOUND 1 ~");
beatDisplay[0].setPreferredSize(new Dimension(400, 50));
beatDisplay[0].setHorizontalAlignment(SwingConstants.CENTER);
beatDisplay[0].setOpaque(true);
beatDisplay[0].setFont(new Font("Serif", Font.BOLD, 40));
beatDisplay[1] = new JLabel("~ SOUND 2 ~");
beatDisplay[1].setPreferredSize(new Dimension(400, 50));
beatDisplay[1].setHorizontalAlignment(SwingConstants.CENTER);
beatDisplay[1].setOpaque(true);
beatDisplay[1].setFont(new Font("Serif", Font.BOLD, 40));
center.add(buttonPlay);
center.add(beatDisplay[0]);
center.add(beatDisplay[1]);
mainContent.add(center, BorderLayout.CENTER);
}
public void actionPerformed(ActionEvent e) {
if (!isPlaying) {
isPlaying = true;
createPlayer();
systemStartTime = System.currentTimeMillis();
new Thread(visual).start();
new Thread(visualCalc).start();
new Thread(play).start();
}
else {
isPlaying = false;
}
}
public void generateTick(){
tickBuffer = new double[tickBufferSize];
for (int i = 0; i < tickLength; i++)
tickBuffer[i] = Math.sin(2 * Math.PI * i / (sampleRate/frequencyOfTick));
for (int i = tickLength; i < tickBufferSize; i++)
tickBuffer[i] = 0;
}
public void createPlayer(){
AudioFormat af = new AudioFormat(sampleRate, 16, 1, true, false);
try {
line = AudioSystem.getSourceDataLine(af);
line.open(af);
line.start();
}
catch (Exception ex) { ex.printStackTrace(); }
}
public Runnable play = new Runnable() { public void run() {
buffer = new byte[bufferSize];
int b=0;
int t=0;
while (isPlaying) {
if (t >= tickBufferSize)
t = 0;
short maxSample = (short) ((tickBuffer[t++] * Short.MAX_VALUE));
buffer[b++] = (byte) (maxSample & 0x00ff);
buffer[b++] = (byte) ((maxSample & 0xff00) >>> 8);
if (b >= bufferSize) {
line.write(buffer, 0, buffer.length);
b=0;
}
}
destroyPlayer();
}};
public void destroyPlayer(){
line.drain();
line.close();
}
}
VisualCalc.java
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;
public class VisualCalc implements Runnable{
private static TimingCheck timingCheck;
private static Visual visual;
int visualBuffer = 3100;
long msQuarterNoteDelay;
long currentTime;
boolean isPlayingTEMP=false;
// VISUAL 1
long visual1_msAdd=0;
int visual1_subdivision=1;
// VISUAL 2
long visual2_msAdd=0;
int visual2_subdivision=1;
public VisualCalc(TimingCheck timingCheckIn, Visual visualIn){
timingCheck = timingCheckIn;
visual = visualIn;
msQuarterNoteDelay = (long)(60000 / timingCheck.tempo);
}
public void run(){
visual.visual1_msUpdateArray.add((long)visualBuffer);
visual.visual2_msUpdateArray.add((long)visualBuffer);
while (true){
currentTime = getCurrentProgramTime();
if (timingCheck.isPlaying){
// VISUAL 1
if (currentTime >= visual1_msAdd){
addNextVisual1();
}
// VISUAL 2
if (currentTime >= visual2_msAdd){
addNextVisual2();
}
}
}
}
public long getCurrentProgramTime(){
return System.currentTimeMillis() - timingCheck.systemStartTime;
}
// VISUAL 1
public void addNextVisual1(){
visual1_msAdd += (long)(msQuarterNoteDelay / visual1_subdivision);
visual.visual1_msUpdateArray.add(visual1_msAdd + visualBuffer);
}
// VISUAL 2
public void addNextVisual2(){
visual2_msAdd += (long)(msQuarterNoteDelay / visual2_subdivision);
visual.visual2_msUpdateArray.add(visual2_msAdd + visualBuffer);
}
}
Visual.java
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;
import java.util.ArrayList;
import java.util.Vector;
public class Visual implements Runnable{
private static TimingCheck timingCheck;
long currentTime;
// VISUAL 1
ArrayList<Long> visual1_msUpdateArray = new ArrayList<Long>();
int visual1_subdivision=1;
int visual1_maxBeatCount = 4;
int visual1_beatCount = 1;
boolean visual1_green = true;
// VISUAL 2
ArrayList<Long> visual2_msUpdateArray = new ArrayList<Long>();
int visual2_subdivision=1;
int visual2_maxBeatCount = 4;
int visual2_beatCount = 1;
boolean visual2_green = true;
public Visual(TimingCheck timingCheckIn){
timingCheck = timingCheckIn;
}
public void run(){
while (true){
currentTime = getCurrentProgramTime();
if (timingCheck.isPlaying){
// DOES NOT WORK WITHOUT THIS SLIGHT DELAY
try { Thread.sleep(1); } catch (InterruptedException ie){}
// VISUAL 1
if (!visual1_msUpdateArray.isEmpty()){
if (currentTime >= visual1_msUpdateArray.get(0)){
updateVisual1();
visual1_msUpdateArray.remove(0);
}
}
// VISUAL 2
if (!visual2_msUpdateArray.isEmpty()){
if (currentTime >= visual2_msUpdateArray.get(0)){
updateVisual2();
visual2_msUpdateArray.remove(0);
}
}
}
}
}
public long getCurrentProgramTime(){
return System.currentTimeMillis() - timingCheck.systemStartTime;
}
// VISUAL 1
public void updateVisual1(){
timingCheck.beatDisplay[0].setText("" + visual1_beatCount++);
if (visual1_green){
timingCheck.beatDisplay[0].setBackground(Color.GREEN);
visual1_green = false;
}
else{
timingCheck.beatDisplay[0].setBackground(Color.YELLOW);
visual1_green = true;
}
if (visual1_beatCount > visual1_maxBeatCount){
visual1_beatCount = 1;
}
}
// VISUAL 2
public void updateVisual2(){
timingCheck.beatDisplay[1].setText("" + visual2_beatCount++);
if (visual2_green){
timingCheck.beatDisplay[1].setBackground(Color.GREEN);
visual2_green = false;
}
else{
timingCheck.beatDisplay[1].setBackground(Color.YELLOW);
visual2_green = true;
}
if (visual2_beatCount > visual2_maxBeatCount){
visual2_beatCount = 1;
}
}
}