Music

Anyone who’s thinking of trying to squeeze some music into their game is likely to find this blog post interesting: http://countercomplex.blogspot.com/2011/10/some-deep-analysis-of-one-line-music.html

It might be worth making a compatibility list of OSes and VM versions to see how widely available 8kHz lines are; I suspect they will be widely supported, but I’d be more confident with 16kHz.

Oh man that’s good reading material. Thanks!

Because I’m not a fan of 8-bit music (whatever conclusions you may have drawn from me posting the link above), I’ve been spending a lot of time working on compact softsynth. (Need to spend a bit more time on the actual game!) Here for anyone to copy, modify, etc. is a standalone class which generates a snare drum patch and plays it once.

import java.io.*;
import javax.sound.sampled.*;

public class SynthSandbox implements LineListener {
	public static void main(String[] args) throws Exception {
		new SynthSandbox().preview();
	}

	public void preview() throws Exception {
		final int frameRate = 16000;
		final int bytesPerFrame = 2;

		// Snare using Karplus-Strong filter. 1 second long.
		int numFrames = 16000;
		byte[] data = new byte[numFrames * bytesPerFrame];

		// Linear congruential PRNG. This seed is conveniently equal to a constant
		// that we already have in the pool, and I think it sounds better than many.
		int rnd = 16000;
		int[] ksbuf = new int[200];
		for (int i = 0; i < ksbuf.length; i++) {
			ksbuf[i] = 0x4000 + ((rnd >> 16) & 0x3fff);
			rnd *= 1103515245;
		}
		for (int i = 0, off = 0; i < numFrames; i++) {
			int nextOff = (off + 1) % ksbuf.length;
			int y = (ksbuf[off] + ksbuf[nextOff]) / 2;

			if ((rnd & 0x10000) == 0) y = -y;
			rnd *= 1103515245;

			ksbuf[off] = y;
			off = nextOff;
			data[2 * i] = (byte)y;
			data[2 * i + 1] = (byte)(y >> 8);
		}

		AudioFormat format = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED,
		    frameRate, bytesPerFrame * 8, 1, bytesPerFrame, frameRate, false);

		DataLine.Info info = new DataLine.Info(Clip.class, format);
		Clip clip = (Clip)AudioSystem.getLine(info);
		clip.addLineListener(this);

		clip.open(format, data, 0, data.length);
		clip.start();

		// Busy-wait until the listener calls System.exit.
		while (true) Thread.sleep(50);
	}

	public void update(LineEvent le) {
		if (le.getType() == LineEvent.Type.STOP) System.exit(0);
	}
}

Here’s another micro-sound-system that just plays an ugly beep. However, I guess it’s pretty much the smallest that it can get (I chose some other features instead of sound, though). Maybe it’s useful for someone as a starting point!

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.Clip;
import javax.sound.sampled.DataLine;

public class TSS {

	public static void main(String args[]) {
		TSS tss = new TSS();
		tss.run();
	}

	public void run() {
		try {

			// Buffer for the audio sample
			byte audioData[] = new byte[4000];

			// Generate a simplistic beep
			float amp = 0.5f;
			byte b = 0;
			for (int i = 0; i < audioData.length; i++) {
				audioData[i] = (byte) (b++ * amp);
				amp -= 0.5f / (float) audioData.length;
			}

			// Play the beep
			AudioFormat audioFormat = new AudioFormat(16000, 8, 1, true, true);
			DataLine.Info info = new DataLine.Info(Clip.class, audioFormat);
			Clip music = (Clip) AudioSystem.getLine(info);
			music.open(audioFormat, audioData, 0, audioData.length);
			music.start(); 

			Thread.sleep(1000);

		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}

@Grunnt
your code looks clean and simple. however if I want to change the sound, which part should be modified? amp?

Nah, you will need a bit more creativity :slight_smile: If you want a different sound you need to fill the audioData buffer with a different pattern, in the for-loop below the “// Generate a simplistic beep” comment. Most other solutions out there actually make it easy to generate different sounds, this one is just to show the most minimal possible version.

Oh crap, it needs creativity :stuck_out_tongue: may try to use Strings from Synth4K, turn it to byte and play it with your code. -brute way, cause I don’t understand sound in Java yet, until now-

If you don’t mind 8-bit sound, below is a class I wrote to test my sound generation. Note that it uses basic sine waves; I tried a few other wave forms, but either made mistakes with them or else they didn’t sound very good. I coded it to use the same frequencies as the Western standard (A440), so each note maps directly to a standard piano key (including sharps/flats.) So Middle C is note -9. The generateNote() method generates a given note of a given duration and appends it to the byte array. It also extends the array as needed, so you can write very small code to generate a lot of notes very easily. Notice the two commented out examples, one showing a simple three note sequence, and the other showing a repeating pattern that increases frequency and then in tempo.

The default, uncommented code uses another method I wrote, parseMusicStream(), to parse a byte stream in a format I created for the Java4K competition. In it, each note is represented by 1 byte, with the top two bytes representing the duration (shifted up by one, it gives you a range of 1-4) and the bottom 6 bytes representing the note (giving you a range of 0 to 62, with 63 reserved for a Rest note.) Those bytes compress really well in the final jar, probably because of repeated notes and repeated sequences within the music (eg: a chorus.) I’ve got a second program that takes a comma-separated list of notes in {duration}{octave}{note} format and outputs the notes encoded in the previous byte format. So I can write music like this:

14C,13G,14Eb,24C,00R,14A#,00R,14C,

And have those notes encoded into just 8 bytes before compression. I’ll paste the encoder code in another comment.

Rick


package ca.townsends.games.templates;

import java.applet.Applet;
import java.awt.*;
import java.awt.event.KeyEvent;
import java.awt.event.MouseEvent;
import java.awt.image.BufferedImage;
import java.io.InputStream;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.Clip;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.LineUnavailableException;

/**
 * Generate Sound
 * 
 * Ideas:
 * - Pre-generate the sound so it doesn't slow down framerate.
 * - Loop notes in groups of 8
 * - add changes by:
 *   - increasing/decreasing tempo
 *   - adjusting notes up/down by a set amount
 *   - play up down scale
 *   - repeat two notes
 * 
 * - Maybe pre-store some music in a byte string, using first six bits as the note, and the last two bits as the length?
 * 
 * - Note that with the current implementation, note 0 is the lowest Treble A, so -9 is middle C.
 * 
 * @author Rick Townsend
 * @version 1.0
 *
 */
public class Sound4K extends Applet implements Runnable {

	private static final int SCREEN_WIDTH = 800;
	private static final int SCREEN_HEIGHT = 600;
	private static final long LOOP_DELAY = 16; // 16 = approx 60 FPS

	// in hz, number of samples in one second 
	private static final int SAMPLE_RATE = 44100;
	 
	// this is the time BETWEEN Samples 
	private static final double SAMPLE_PERIOD = 1.0 / SAMPLE_RATE;
	
	// volume
	private static int AMPLITUDE = 20;
	
	// Incoming notes are between 0 and 63:
	// - Shifting down by 40 gives a range of -40 to +22, with 23 reserved for silent beat.
	// - Since -9 is middle C, that leaves (-40 - -9) = 31 notes below middle C, and (22 - -9) = 31 notes above middle C
	// - This also means incoming, encoded notes should have 31 as middle C (so it translates to shifted value -9).
	private static final int NOTE_SHIFT = -40;
	
	// Incoming durations are between 0 and 3
	// - The code shifts them to be between 1 and 4
	// - This multiple is how many seconds a duration of 1 represents.
	private static final double DURATION_MULTIPLE = 0.2;
	
	/** array of keys (to detect which ones are currently being pressed */
	private boolean[] keys = new boolean[65536];
	private int mouseButton = MouseEvent.NOBUTTON;
	private int dMouseX = 0;
	private int dMouseY = 0;
	

	public void start() {
		new Thread(this).start();
	}

	public void run() {
		// Turn on event handling
		this.enableEvents(
				AWTEvent.KEY_EVENT_MASK |
				AWTEvent.MOUSE_EVENT_MASK |
				AWTEvent.MOUSE_MOTION_EVENT_MASK);

		// Set the screen size
		setSize(SCREEN_WIDTH, SCREEN_HEIGHT); // For AppletViewer, TODO: remove later.

		// Set up the graphics stuff, double-buffering.
		BufferedImage screen = new BufferedImage(SCREEN_WIDTH, SCREEN_HEIGHT, BufferedImage.TYPE_INT_RGB);
		Graphics2D g = (Graphics2D) screen.getGraphics();
		Graphics appletGraphics = this.getGraphics();
		
		long fps = 0;

		int iteration = 0;
		
		byte[] pcmMusic = new byte[0];

		/* e.g.: 
		 * Play three fixed notes, each for 0.3 seconds
		 */
		/*
		rawOutput = generateNote(rawOutput, -15, 0.3);
		rawOutput = generateNote(rawOutput, -17, 0.3);
		rawOutput = generateNote(rawOutput, -19, 0.3);
		 */
		
		/* e.g.:
		 * Generate a repeating sequence of:
		 * - two notes
		 * - played four times
		 * - repeated three times, each time the two notes are three notes higher than the previous two
		 * - repeated six times, each time the two notes are played for 0.1 seconds shorter than the previous time
		 */
		/*
		for (int k = 0; k < 6; k++) {
			double duration = 0.6 - (k * 0.1);
			for (int i = 0; i < 4; i++) {
				for (int j = 0; j < 4; j++) {
					pcmMusic = generateNote(pcmMusic, i * 3 - 8, duration);
					pcmMusic = generateNote(pcmMusic, i * 3 + 4, duration);
				}
			}
		}
		*/
		
		/* e.g.:
		 * Load the music from a file (each byte is a note: top two bits are the duration, bottom 6 bits are the note to play.)
		 */
		InputStream stream = this.getClass().getResourceAsStream("MusicBoxDancer.music");
		pcmMusic = parseMusicStream(stream);
		
		// Play the music
        AudioFormat audioFormat = new AudioFormat(SAMPLE_RATE, 8, 1, true, true);
        DataLine.Info info = new DataLine.Info(Clip.class, audioFormat);
        Clip music;
		try {
			music = (Clip) AudioSystem.getLine(info);
	        music.open(audioFormat, pcmMusic, 0, pcmMusic.length);
	        music.loop(Clip.LOOP_CONTINUOUSLY); 
		} catch (LineUnavailableException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}
		
		// Game loop.
		while (isActive()) {
			iteration++;
			long lastTime = System.currentTimeMillis();
			
			// Clear the screen
			g.setColor(Color.WHITE);
			g.fillRect(0, 0, SCREEN_WIDTH, SCREEN_HEIGHT);
			
			// TODO add some update logic here.

			
			// TODO: Animation goes here
			
			g.drawString("FPS: " + fps, 20, SCREEN_HEIGHT);
			
			// Game title
			g.setFont(new Font("Arial", Font.BOLD, 26));
			// Draw a drop-shadow first
			g.setColor(Color.GRAY);
			g.drawString("Sound 4K v1.0", SCREEN_WIDTH - 180, SCREEN_HEIGHT - 3);
			// Then the main title
			g.setColor(Color.BLUE);
			g.drawString("Sound 4K v1.0", SCREEN_WIDTH- 180, SCREEN_HEIGHT - 3);

			// Draw the entire results on the screen.
			appletGraphics.drawImage(screen, 0, 0, null);

			//Lock the frame rate
			long delta = System.currentTimeMillis() - lastTime;
			if(delta < LOOP_DELAY)
			{
				try {
					Thread.sleep(LOOP_DELAY - delta);
				} catch(Exception e) {
					// Nothing to do here
				}
			}
			if (iteration % 10 == 0) fps = 1000 / (System.currentTimeMillis() - lastTime);
		}
	}

	/**
	 * Generates a new note, and appends it to the original array.
	 * 
	 * @param originalArray An array of bytes representing pcm music (can be empty/zero length)
	 * @param note The note to generate.  There are 12 notes per octave: -9 = middle C, -8 = C#/Db, -7 = D, etc. 
	 * @param duration How long to play the note, in seconds.
	 * @return A new byte array containing the originalArray followed by the pcm bytes for the new note. 
	 */
	private byte[] generateNote(byte[] originalArray, int note, double duration){
		
		int durationInSamples = (int) Math.ceil(duration * SAMPLE_RATE) + 1;
		
		byte[] noteArray = new byte[durationInSamples * 2];
		                    			
		double time = 0;
		int i = 0;
		
		// Put in a pause if this note is the max value
		if (note == 63 + NOTE_SHIFT) {
			i = durationInSamples;
			
		} else {
			
			double tone = 440 * Math.pow(2,note/12f);
			
			boolean finishedNote = false;
			
			// Generate the note using a SINE wave.
			for(int j = 0; j < durationInSamples || !finishedNote; j++, i++) { 
				noteArray[i] = (byte) (AMPLITUDE * Math.sin(2 * Math.PI * tone * time));
				time += SAMPLE_PERIOD;
				// Keep going until the wave is finished at zero
				if (i > 0) finishedNote = (noteArray[i - 1] < 0 && noteArray[i] >= 0);
			}
		
		}

		// Append the note
		byte[] finalArray = new byte[originalArray.length + i];
		System.arraycopy(originalArray, 0, finalArray, 0, originalArray.length);
		System.arraycopy(noteArray, 0, finalArray, originalArray.length, i);
		
		return finalArray;
	}
	
	/**
	 * Parses the bytes of an InputStream and returns a byte array containing the represented
	 * notes in pcm format.
	 * 
	 * - The InputStream must be in "ca.townsends.music" format, where each byte represents a note and its duration.
	 * - For each byte, the top 2 bits are the duration, the bottom 6 bits are the note to play.
	 * - The duration is multiplied by the duration multiple and shifted up by 0.1.  So a duration of 0 = 0.1, 1 = 0.1 + (duration * multiple), etc.
	 * - The note is shifted up by NOTE_SHIFT, which lets you adjust the note range up or down as desired. For example,
	 *   to shift the music being parsed up by an octave, just increase NOTE_SHIFT by 12 (the number of notes in an octave.)
	 * 
	 * See ca.townsends.games.templates.EncodeMusic for a "music" encoder.
	 * 
	 * @param stream An InputStream in "music" format.
	 * 
	 * @return A byte array with the parsed music in pcm format.
	 */
	private byte[] parseMusicStream (InputStream stream) {
		byte[] music = new byte[0];
		
		int b;
		try {
			while((b = stream.read()) != -1) {
				double duration = (b >> 6) * DURATION_MULTIPLE + 0.1;
				int note = (b & 63) + NOTE_SHIFT;
				// XXX: System.out.println(note + ", " + duration + ", " + (byte)b);
				music = generateNote(music, note, duration);
			}
		} catch (Exception e) {
			// TODO: Handle an error here
			e.printStackTrace();
		} finally {
			try {
				stream.close();
			} catch (Exception io) {
				// Can't do anything here
				io.printStackTrace();
			}
		}
		
		return music;
	}
	
	/**
	 * Just standard key handling for a Java4K applet.
	 */
	public void processKeyEvent(KeyEvent e) {
		switch (e.getID()) {
			case Event.KEY_PRESS:
			case Event.KEY_ACTION:
				// skip keypresses more than 0.1 seconds old, because they indicate bad lag time
				if (System.currentTimeMillis() - e.getWhen()  > 100)
					break;
				keys[e.getKeyCode()] = !keys[e.getKeyCode()];
				break;
		}
		
	}
	
	/**
	 * Just standard mouse handling for a Java4K applet.
	 */
	public void processMouseEvent(MouseEvent e) {
		dMouseX = e.getX();
		dMouseY = e.getY();

		switch (e.getID()) {
			case MouseEvent.MOUSE_PRESSED:
				// mouse button pressed
				mouseButton = e.getButton();
				break;
			case MouseEvent.MOUSE_RELEASED:
				// mouse button released
				mouseButton = MouseEvent.NOBUTTON;
				break;
			case MouseEvent.MOUSE_MOVED:
				break;
			case MouseEvent.MOUSE_DRAGGED:
				break;
		}
		
	}
	/**
	 * Just standard mouse handling for a Java4K applet.  Delegates to processMouseEvent().
	 */
	public void processMouseMotionEvent(MouseEvent e){
		processMouseEvent(e);
	}
}

As promised, my EncodeMusic class, which generates the “music” format that Sound4K plays. I’ve tried to comment it so that you can understand what’s going on, but you’re welcome ask questions or just use it without understanding. :slight_smile:


package ca.townsends.games.templates;

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.HashMap;
import java.util.StringTokenizer;

/**
 * Takes a csv-like file of notes in {duration}{octave}{note} format, and outputs one byte per note.  The input
 * ignores all whitespace, including linefeeds and carriage returns, so you can group music as you please.  For example,
 * the first few stanzas of the song "Music Box Dancer" are:
 * 
 * 13C,13E,13G,13E,14C,13G,13E,13G,
 * 13C,13E,13G,13E,14C,13G,13E,13G,
 * 13C,13E,13G,13E,14C,13G,13E,13G,
 * 13C,13E,13G,13E,14C,13G,13E,13G,
 * 
 * 14C,13G,14C,14E,14C,14E,14G,14C,
 * 15C,15B,15A,34G,20R,
 * 14G,14F,14D,14B,13G,14C,14D,14F,
 * 14E,14C,15A,34G,20R,
 * 14C,13G,14C,14E,14C,14E,14G,14C,
 * 15C,15B,15A,34G,20R,
 * 
 * Note that the input accepts notes in any mixed-case, and recognizes both sharps
 * and flats (e.g.: "A", "C#" and "Db" are all valid notes within an octave.)  "R" is used for a Rest note.
 * 
 * I've named the output format "ca.townsends.music", with a ".music" extension, but that's purely arbitrary.
 * 
 * 
 * @author Rick Townsend
 * @version 1.1
 * 2012-02-25
 */
public class EncodeMusic {

	/** Notes are between 0 and 63:
	 * - Shifting down by 40 gives a range of -40 to +22, with 23 reserved for silent beat.
	 * - Since -9 is middle C, that leaves (-40 - -9) = 31 notes below middle C, and (22 - -9) = 31 notes above middle C
	 * - This also means incoming, encoded notes should have 31 as middle C (so it translates to shifted value -9).
	 */
	public static final int NOTE_SHIFT = -40;
	
	/**
	 * Main method
	 * @param args Not used
	 */
	public static void main(String[] args) {
		HashMap<String,Integer> noteMap = new HashMap<String,Integer>();
		
		// The recognized notes within an octave
		noteMap.put("A",	0);
		noteMap.put("A#",	1);
		noteMap.put("Bb",	1);
		noteMap.put("B",	2);
		noteMap.put("C",	3);
		noteMap.put("C#",	4);
		noteMap.put("Db",	4);
		noteMap.put("D",	5);
		noteMap.put("D#",	6);
		noteMap.put("Eb",	6);
		noteMap.put("E",	7);
		noteMap.put("F",	8);
		noteMap.put("F#",	9);
		noteMap.put("Gb",	9);
		noteMap.put("G",	10);
		noteMap.put("G#",	11);
		noteMap.put("Ab",	11);
		noteMap.put("R",	63); // Rest note
		
		InputStream inStream = null;
		OutputStream outStream = null;
		
		// If we get an IO exception at any time, there's no point in continuing.
		try {
			// You could change these to be parameters from the args[] array, instead of hard-coded.
			inStream = EncodeMusic.class.getResourceAsStream("MusicBoxDancer.csv");
			outStream = new FileOutputStream("src/ca/townsends/games/templates/outputMusic.music");
			byte[] bytes = new byte[4096 * 4]; // 4K notes is a very, very long song... But if you overrun it, just make this bigger.
			
			// Grab all the bytes from the file
			inStream.read(bytes);
			
			// Dump it to a String so we can parse it more easily
			String data = (new String(bytes)).toUpperCase();
			
			// Split on commas
			StringTokenizer st = new StringTokenizer(data, ",");
	        
			// Parse all the notes
	        while(st.hasMoreTokens()) {
	        	// Trim off whitespace
	        	String token = st.nextToken().trim();
	        	
	        	// Skip any empty values (ie, where the input was only whitespace between two commas, or simply ",,")
	        	if (token.length() == 0) continue;
	        	
	        	// Grab the duration, octave and note
	        	int inDuration = Integer.parseInt(token.substring(0, 1));
	        	int inOctave = Integer.parseInt(token.substring(1, 2));
	        	String inNote = token.substring(2); // This grabs all text from the third character onward, so it will get both single ("C") and double ("C#) character notes
	        	
	        	//System.out.println(inNote);
	        	// Grab the notes value from the map
	        	int noteVal = noteMap.get(inNote);
	        	
	        	// Calculate the actual note value (0 - 62), by multiplying by the octave and shifting down by 7 so that middle C is dead center at note 31. 
        		int outNote = (byte)(inOctave * 12 + noteVal - 7);
        		// Shift the duration up by 6 bits
	        	int duration = inDuration << 6;

	        	// Hard-code the Rest note as note 63.
	        	if (noteVal == 63) outNote = 63;
	        	
	        	// Write the note in the lower 6 bits and the duration in the top 2 bits to both the output stream and system.out
	        	outStream.write((byte)(outNote | duration));
	        	System.out.print((byte)(outNote | duration) + ",");
	        	//System.out.print((char)(outNote | duration));
	        	//System.out.print("0x" + Integer.toHexString(outNote | duration).toUpperCase() + ",");

	        }
		} catch (IOException e) {
			e.printStackTrace();
		} finally {
	        try {inStream.close();} catch (Exception e){};
	        try {outStream.close();} catch (Exception e){};
		}
		
	}

}