Java Theremin

Hi again - I posted an “in progress” Theremin. Can try it here: http://www.hexara.com/VSL/JTheremin.htm

A real Theremin is a contraption that you play by waving your hands in the air over two metal bars – I’m not sure of how the electronics works – but one vector is pitch and the other is volume. The most famous uses are maybe in “Good Vibrations” by the Beach Boys, or in the soundtracks for “The Red Planet” or “The Day the Earth Stood Still”.

Biggest plus: the GUI is pretty dang responsive, if I say so myself, for a Swing Applet running sound, real time. I’m using the RTESmoother I presented in this post: http://www.java-gaming.org/index.php/topic,24605.0.html to capture and ship the mouse events evenly and smoothly to the inner buffer loops of the javax.sound.sampled SourceDataLine output.

This is my first attempt at wave-table synthesis. There are definitely some issues with the aliasing in the upper octaves of the square and sawtooth waves. I’m using wave table arrays of 720 floats. Maybe need more? I was thinking 2 per degree (360 * 2) and using linear interpolation, both between the samples when moving through at various rates and between the sine & other selected wave form, if you use an intermediate point on the slider.

Very fun to play around with!!!

Thanks for trying it out @ra4king! I’m happy to hear you enjoyed playing with it a bit.

Quote from: nsigma on another thread [quote]
Quote from: philfrei on 10 hours ago [quote]Crackly?! What does that mean? I am aware of something that I assumed was kind of like anti-aliasing in the high registers, especially the square wave. Is that what you mean? Crackly as a descriptor sounds much worse than that. Does the sine wave crackle, too? That is clear as a bell, over here.
[/quote]
The sine wave is actually the worst of the lot. Quite harsh discontinuities in the audio. Any chance you’ve got some out-of-range numbers in there somewhere? ALSA can be rather unforgiving with that.
[/quote]
Now that I am listening for it, I can hear a bit of fuzz to the sine tone, but it is really faint. I confirmed that the sine table ranges from -1 to 1. A volume float is applied (multiplied), but it ranges from 0 to 1. The value from the toneCtrl also ranges from 0 to 1, for mixing the sine with the selected wave shape. With a pure sine, the value is 0. I think the conversion to bytes is pretty standard.

I scanned for highest and lowest audio values (prior to conversion to bytes), I get “high:28611.75767117732 low:-28611.76278736017” (Took me by surprise, I thought I would get something closer to 32767.)

The following is the guts of the TDL read.


while(framesToRead-- > 0)
{
    pitchVal = pitchCtrl.tick();
    av1 = sineTable.get(pitchVal);
    av2 = currentWT.get(pitchVal);
    toneVal = toneCtrl.tick();
    audioVal = av1 * (1 - toneVal) + av2 * toneVal;
			
    audioVal *= volumeCtrl.tick();

    // 16-bit, signed
    audioVal *= 32767;
    
    // stereo, little-endian
    buffer[bufferIdx++] = (byte)((int)(audioVal) & 0xff);
    buffer[bufferIdx++] = (byte)((int)audioVal >> 8);
    buffer[bufferIdx++] = (byte)((int)(audioVal) & 0xff);
    buffer[bufferIdx++] = (byte)((int)audioVal >> 8);
}

I’m happy to receive suggestions to improve the above.
Wondering what other sources of discontinuities there might be…

P.S. I found some needless work (packing and unpacking the values for the volume calc) that has just been eliminated. I’ve reposted the Applet. I don’t know if what I did will have much off an effect.

That does seem odd - you’d expect when everything was pretty much 1 you’d get a lot nearer 32767.

To try and explain the sound I’m hearing (rather difficult in words!) - it’s a quick click like a sharp discontinuity, and is happening at a regular rhythm. Rather bizarrely that rhythm is a consistent da da da dum, rather than just a continuous pulse, and occurs constantly at the same speed regardless of pitch or volume settings. I’ve not noticed anything like it before. What format are you opening the output line with (sample rate, endianess, mono / stereo, etc.) - I wonder if it’s a conversion issue.

@nsigma (Or anyone else running Linux!)
I made a stripped down version, just a test tone. I’m curious if this also exhibits the problem you cited on Linux playback.

http://hexara.com/VSL/TestTone.htm

It’s 122.5 Hz, a pretty low tone. I’m going through the sine wave table (720 floats) skipping every second member.
The buffer is currently sized at 10240 bytes (2560 samples), which comes to 17 or so buffer loads per second.

Output (to speakers) audio format is defined as follows:


    AudioFormat audioFmt = new AudioFormat(
            AudioFormat.Encoding.PCM_SIGNED, 
            44100, 16, 2, 4, 44100, false);		
    Info info = new DataLine.Info(SourceDataLine.class, audioFmt);
    SourceDataLine sourceDataLine =  (SourceDataLine) AudioSystem.getLine(info);

Meanwhile, I’ve made a couple small updates to the JTheremin: clearer octave bars, running a short zero-volume tone as part of the startup, and fixed what turned out to be a GUI error that was miscalculating and limiting the top volume to 28611.+ when it should be 32767.

coughbartlebycough

The TestTone doesn’t work for me. Windows 7 x64bit.
All I hear is crackling when I press the mouse then it crackles again after I release mouse.

@ra4king Thanks for trying it.

Classic, though, the test fails, though it was supposed to be a simpler version, used for troubleshooting. :stuck_out_tongue:

I raised the pitch a couple octaves, to something that should be audible. It is plausible that the tone was too low for a laptop speaker. It was pretty low.

It IS pretty much guaranteed to crackle at the start and finish, as the raw sound comes on and goes off with no cushioning or transitions of any sort. I wanted to just be sending a single tone and be doing as little else as possible. But as long as you hold the mouse down it should sound loud and clear.

So, I’m curious if it works on Linux, and whether there are some things I can do as diagnostics to run, to get info that would help figure this Linux/JavaSound problem.

Yep it works now. I hear 1 tone everywhere I click and there’s a really tiny crackle when i click and release.

Both applets work well on my netbook (dual core Atom, 64-bit Ubuntu Linux, latest Oracle Java). The only problems with the sound were expected: some of the low-pitched audio is too low for the speakers to play correctly, and the audio for the test tone applet starts and ends with a click due to the lack of cushioning.

Interestingly, the sound plays out of the netbook’s internal speakers, rather than the USB headphones (which are set as the audio output device in the OS settings). This isn’t a problem with your code, it’s actually due to the fact that the Java Sound Audio Engine is missing on this machine, so there is no way (that I’m aware of) to determine which is the “correct” audio device to pick from the list of available hardware mixers.

The test tone applet sounds OK for me, but the theremin still exhibits the same problem.

I’m on 64-bit Linux Mint LTS (basically Ubuntu 10.04). By getting the applet to throw an exception (not letting it open the soundcard) I can see that it’s trying to use the HeadspaceMixer, which is the one for the Java Sound Audio Engine. I guess this difference may be why Paul and I are getting different results. He mentions using latest Oracle Java (which I assume means 6u27 not 7?) - I’m using the latest security release, which is 6u26.

Any chance you could add in a combo box to the applet to select which mixer is used? This would narrow it down for sure. The Java Sound Audio Engine is horribly buggy and outdated, so that might be the cause of the audio artefacts, although why the test applet would be OK is a mystery - are you writing the data at the same rate / (fixed?) buffer size in the theremin?

Depends exactly where and how you’re setting the “default” device. The mixers in JavaSound should be in the same index order as alsa, so you need to force your headphones to be at position 0 (ie. plughw:0,0).

[quote=“nsigma,post:11,topic:37109”]
Right. I hadn’t notice that 7 was out. I’ll have to give it a try when I have some time.

[quote=“nsigma,post:11,topic:37109”]
Right, I meant from within Java ;D

Whoo hoo! That’s great. We can troubleshoot!

Yes, I am using the same buffer size for both apps.
I will make additional versions and post them, see if we can nail this down.
Have a composing (!) deadline right now…have to get back to that. Hopefully will have a version with Mixer selection within a few days.

EDIT, 9/11/11, COMMERCIAL BREAK:
http://adonax.com/Carmilla/2011SFCCO_Practice_LaughAtLocksmiths.mp3
http://adonax.com/Carmilla/2011SFCCO_Practice_ISlept.mp3

These are two of three songs just finished. Send out parts yesterday. It will be premiered as part of a “Halloween” concert of the San Francisco Composers Chamber Orchestra on Oct 15 & 16. (http://news.sfcco.org/category/news/) They are for Soprano (covered by the flute in the clips) clarinet, cello & piano. These clips are being played by Finale’s “Kontakt” add-on. (Finale is music notation software.) Meant to help the performers learn the music. Text is from a ghost story called “Carmilla” by J. Sheridan Le Fanu.

I put in a mixer selector via a menubar. The menu bar gets populated with all the mixers found, and includes an entry for “use default mixer”. I didn’t bother to filter, just reported everything. I put this mixer selector in both the TestTone app and the JTheremin app. There are some diagnostics sent to the console when a mixer is selected or when a sound is started. But I couldn’t figure out any way to identify what the AudioSystem chooses via getSourceDataLine(AudioFormat format). (Used when one selects the “default” on the mixer options that I put in.)

http://hexara.com/VSL/TestTone.htm
http://www.hexara.com/VSL/JTheremin.htm

The clicking that nsigma reported occurs on my JTheremin app if I select the “Java Sound Audio Engine”. So, THAT aspect is probably a “Java Sound Audio Engine” issue rather than a Linux issue. The clicking doesn’t occur on the “TestTone” app, though there is some very intermittent crackle. I plan to go ahead and start trouble shooting and see at what point these clicks start appearing.

If anyone has a suggestion for further modifying the “TestTone.htm” applet to make it into a more generally useful sound diagnostic program, I’m happy to consider implementing it.

Yes it is! Setting the ALSA direct mixer this works clear as a bell theremin! ;D

Yay! ;D Perhaps I needn’t worry about Linux issues, then, and just recommend that people avoid “Java Sound Audio Engine”? My suspicion is that it isn’t very high-powered and that most systems will have “Direct Audio Devices” that perform better.

I tightened up the Menu Option a bit, so that only playback options are included. No more Ports or “incoming” lines reported. Rejected Mixers are displayed on the Java Console.

I’ve posted some updates to the Java Theremin.

[I worry a bit about showing audio work here, since it is NOT strictly in a game form yet. It could be considered a “toy” perhaps? I’m also thinking in terms of SF/X engines that could be used in games in a way that is more open-ended and efficient than using prerecorded PCM. And I do appreciate the advice I’ve received at this forum!]

New features:
(1) the pitch range of the working area can be varied, from two to six octaves.
(2) Also, it can be retuned to focus on higher or lower frequencies.
(3) A piano keyboard display now overlays, to help one find pitches when attempting to play a melody.
(4) A Pulse wave was added, and does it sound nasty in the upper registers! Really need to put in some anti-aliasing filter or something.
(5) I’ve added an Echo, but am still a few steps short, as there is no Main:Echo volume ratio control in the GUI yet.

Next post: I will talk about some problems that have come up with dropouts and with the Echo implementation. (This screen is starting to jump around while I type.)

Since putting in the Echo, I’ve started to get dropouts. I’m concerned, because this represents the addition of only a few operations per frame. If something as simple as this causes problems, I worry that there is no “room” to add more DSP like putting in even a simple filter to deal with the aliasing problems.

Also, the needed latency is already kind of high compared to what I’ve heard can be done with audio. So I want to track down the culprits.

(Could be something like the GUI, too, taking up CPU. I redraw just the play-area display with every mouse event. Could try and limit the refresh further, but the mouse can move from end to end so fast, that trails are left behind. Idea, maybe try eliminating the keyboard overlay and see if that helps. I DO do a lot of precalculating to speed up the keyboard display, though, so all the rectangle are ready to go.)

To clearly hear the dropouts, try an echo of length 5000 msec (5 seconds). They are rarer with more normal echo lengths (e.g. 500 msec). Why should the size matter? Because more RAM is being used up to hold the echo data?

By the way, it is possible to get some rather interesting and radical sounds by setting the feedback to 99 and the echo length to 50 or shorter, and 100% Sawtooth.

import java.util.concurrent.LinkedBlockingQueue;

public class Echo {

	private final LinkedBlockingQueue<Double> samples;
	private final double feedback;
	private final int PADDING = 44100;
	
	public double getFeedback() { return feedback;}
	public int getEchoLength() 
	{
		return samples.size() - PADDING; 
	}
	
	Echo(int echoLength, double feedback)
	{
		this.feedback = Math.min(1.0, 
				Math.max(0, feedback));
		samples = new LinkedBlockingQueue<Double>(
				echoLength + PADDING);
		for (int i = 0; i < (echoLength - 1); i++)
		{
			samples.add(0.0);
		}
	}
	
	public double tick(double audioVal){
		double soundVal = samples.poll();
		samples.add(soundVal * feedback + audioVal);
		return soundVal;
	}
}

Next is the Read() for the TargetDelayLine which provides the audio data. I’m wondering if I should structure it to work on more than a single frame at a go. Maybe getting blocks of 256 samples, for instance, would make more sense. The routine which reads the TDL is currently calling for 22050 bytes per read. Any less makes the dropout problem worse.

	public int read(byte[] buffer, int idxBuf, int bytesToRead) 
	{
		if (!running) return 0;

		int frToIdx = bytesToRead/4;
		int bufferIdx = idxBuf;		

		while (frToIdx-- > 0)
		{
			pitchVal = pitchCtrl.tick();
			av1 = sineTable.get(pitchVal);
			av2 = currentWT.get(pitchVal);
			toneVal = toneCtrl.tick();
			audioVal = av1 * (1 - toneVal) + av2 * toneVal;
			
			double mouseVol = volumeCtrl.tick();
			audioVal *= mouseVol;
			audioVal += echo.tick(audioVal) * mouseVol;
			
			audioVal *= 32767;
			audioVal = Math.max(-32700, Math.min(32700, audioVal));
			buffer[bufferIdx++] = (byte)((int)(audioVal) & 0xff);
			buffer[bufferIdx++] = (byte)((int)audioVal >> 8);
			buffer[bufferIdx++] = (byte)((int)(audioVal) & 0xff);
			buffer[bufferIdx++] = (byte)((int)audioVal >> 8);
			
		}
		return (bufferIdx - idxBuf);
	}

Don’t worry about being offtopic. You’re doing the gruntwork of getting reliable audio in java, which is valuable for lots of us. Regarding the jumpiness, if you’re using MSIE, please consider another browser. The SMF guys can’t be bothered to fix it, and neither can I. I dropped support for MSIE a long time ago, when I noticed even IE9 didn’t produce correct renderings. So far for me being offtopic…

I think it was a great toy, no problems about it not being a game :slight_smile:

I think the volume of the echo should be based upon the volume you were at when the sound got produced, and when releasing the mouse the echo should go on as well.

Mike