Yep it works now. I hear 1 tone everywhere I click and there’s a really tiny crackle when i click and release.
Both applets work well on my netbook (dual core Atom, 64-bit Ubuntu Linux, latest Oracle Java). The only problems with the sound were expected: some of the low-pitched audio is too low for the speakers to play correctly, and the audio for the test tone applet starts and ends with a click due to the lack of cushioning.
Interestingly, the sound plays out of the netbook’s internal speakers, rather than the USB headphones (which are set as the audio output device in the OS settings). This isn’t a problem with your code, it’s actually due to the fact that the Java Sound Audio Engine is missing on this machine, so there is no way (that I’m aware of) to determine which is the “correct” audio device to pick from the list of available hardware mixers.
The test tone applet sounds OK for me, but the theremin still exhibits the same problem.
I’m on 64-bit Linux Mint LTS (basically Ubuntu 10.04). By getting the applet to throw an exception (not letting it open the soundcard) I can see that it’s trying to use the HeadspaceMixer, which is the one for the Java Sound Audio Engine. I guess this difference may be why Paul and I are getting different results. He mentions using latest Oracle Java (which I assume means 6u27 not 7?) - I’m using the latest security release, which is 6u26.
Any chance you could add in a combo box to the applet to select which mixer is used? This would narrow it down for sure. The Java Sound Audio Engine is horribly buggy and outdated, so that might be the cause of the audio artefacts, although why the test applet would be OK is a mystery - are you writing the data at the same rate / (fixed?) buffer size in the theremin?
Depends exactly where and how you’re setting the “default” device. The mixers in JavaSound should be in the same index order as alsa, so you need to force your headphones to be at position 0 (ie. plughw:0,0).
[quote=“nsigma,post:11,topic:37109”]
Right. I hadn’t notice that 7 was out. I’ll have to give it a try when I have some time.
[quote=“nsigma,post:11,topic:37109”]
Right, I meant from within Java ;D
Whoo hoo! That’s great. We can troubleshoot!
Yes, I am using the same buffer size for both apps.
I will make additional versions and post them, see if we can nail this down.
Have a composing (!) deadline right now…have to get back to that. Hopefully will have a version with Mixer selection within a few days.
EDIT, 9/11/11, COMMERCIAL BREAK:
http://adonax.com/Carmilla/2011SFCCO_Practice_LaughAtLocksmiths.mp3
http://adonax.com/Carmilla/2011SFCCO_Practice_ISlept.mp3
These are two of three songs just finished. Send out parts yesterday. It will be premiered as part of a “Halloween” concert of the San Francisco Composers Chamber Orchestra on Oct 15 & 16. (http://news.sfcco.org/category/news/) They are for Soprano (covered by the flute in the clips) clarinet, cello & piano. These clips are being played by Finale’s “Kontakt” add-on. (Finale is music notation software.) Meant to help the performers learn the music. Text is from a ghost story called “Carmilla” by J. Sheridan Le Fanu.
I put in a mixer selector via a menubar. The menu bar gets populated with all the mixers found, and includes an entry for “use default mixer”. I didn’t bother to filter, just reported everything. I put this mixer selector in both the TestTone app and the JTheremin app. There are some diagnostics sent to the console when a mixer is selected or when a sound is started. But I couldn’t figure out any way to identify what the AudioSystem chooses via getSourceDataLine(AudioFormat format). (Used when one selects the “default” on the mixer options that I put in.)
http://hexara.com/VSL/TestTone.htm
http://www.hexara.com/VSL/JTheremin.htm
The clicking that nsigma reported occurs on my JTheremin app if I select the “Java Sound Audio Engine”. So, THAT aspect is probably a “Java Sound Audio Engine” issue rather than a Linux issue. The clicking doesn’t occur on the “TestTone” app, though there is some very intermittent crackle. I plan to go ahead and start trouble shooting and see at what point these clicks start appearing.
If anyone has a suggestion for further modifying the “TestTone.htm” applet to make it into a more generally useful sound diagnostic program, I’m happy to consider implementing it.
Yes it is! Setting the ALSA direct mixer this works clear as a bell theremin! ;D
Yay! ;D Perhaps I needn’t worry about Linux issues, then, and just recommend that people avoid “Java Sound Audio Engine”? My suspicion is that it isn’t very high-powered and that most systems will have “Direct Audio Devices” that perform better.
I tightened up the Menu Option a bit, so that only playback options are included. No more Ports or “incoming” lines reported. Rejected Mixers are displayed on the Java Console.
I’ve posted some updates to the Java Theremin.
[I worry a bit about showing audio work here, since it is NOT strictly in a game form yet. It could be considered a “toy” perhaps? I’m also thinking in terms of SF/X engines that could be used in games in a way that is more open-ended and efficient than using prerecorded PCM. And I do appreciate the advice I’ve received at this forum!]
New features:
(1) the pitch range of the working area can be varied, from two to six octaves.
(2) Also, it can be retuned to focus on higher or lower frequencies.
(3) A piano keyboard display now overlays, to help one find pitches when attempting to play a melody.
(4) A Pulse wave was added, and does it sound nasty in the upper registers! Really need to put in some anti-aliasing filter or something.
(5) I’ve added an Echo, but am still a few steps short, as there is no Main:Echo volume ratio control in the GUI yet.
Next post: I will talk about some problems that have come up with dropouts and with the Echo implementation. (This screen is starting to jump around while I type.)
Since putting in the Echo, I’ve started to get dropouts. I’m concerned, because this represents the addition of only a few operations per frame. If something as simple as this causes problems, I worry that there is no “room” to add more DSP like putting in even a simple filter to deal with the aliasing problems.
Also, the needed latency is already kind of high compared to what I’ve heard can be done with audio. So I want to track down the culprits.
(Could be something like the GUI, too, taking up CPU. I redraw just the play-area display with every mouse event. Could try and limit the refresh further, but the mouse can move from end to end so fast, that trails are left behind. Idea, maybe try eliminating the keyboard overlay and see if that helps. I DO do a lot of precalculating to speed up the keyboard display, though, so all the rectangle are ready to go.)
To clearly hear the dropouts, try an echo of length 5000 msec (5 seconds). They are rarer with more normal echo lengths (e.g. 500 msec). Why should the size matter? Because more RAM is being used up to hold the echo data?
By the way, it is possible to get some rather interesting and radical sounds by setting the feedback to 99 and the echo length to 50 or shorter, and 100% Sawtooth.
import java.util.concurrent.LinkedBlockingQueue;
public class Echo {
private final LinkedBlockingQueue<Double> samples;
private final double feedback;
private final int PADDING = 44100;
public double getFeedback() { return feedback;}
public int getEchoLength()
{
return samples.size() - PADDING;
}
Echo(int echoLength, double feedback)
{
this.feedback = Math.min(1.0,
Math.max(0, feedback));
samples = new LinkedBlockingQueue<Double>(
echoLength + PADDING);
for (int i = 0; i < (echoLength - 1); i++)
{
samples.add(0.0);
}
}
public double tick(double audioVal){
double soundVal = samples.poll();
samples.add(soundVal * feedback + audioVal);
return soundVal;
}
}
Next is the Read() for the TargetDelayLine which provides the audio data. I’m wondering if I should structure it to work on more than a single frame at a go. Maybe getting blocks of 256 samples, for instance, would make more sense. The routine which reads the TDL is currently calling for 22050 bytes per read. Any less makes the dropout problem worse.
public int read(byte[] buffer, int idxBuf, int bytesToRead)
{
if (!running) return 0;
int frToIdx = bytesToRead/4;
int bufferIdx = idxBuf;
while (frToIdx-- > 0)
{
pitchVal = pitchCtrl.tick();
av1 = sineTable.get(pitchVal);
av2 = currentWT.get(pitchVal);
toneVal = toneCtrl.tick();
audioVal = av1 * (1 - toneVal) + av2 * toneVal;
double mouseVol = volumeCtrl.tick();
audioVal *= mouseVol;
audioVal += echo.tick(audioVal) * mouseVol;
audioVal *= 32767;
audioVal = Math.max(-32700, Math.min(32700, audioVal));
buffer[bufferIdx++] = (byte)((int)(audioVal) & 0xff);
buffer[bufferIdx++] = (byte)((int)audioVal >> 8);
buffer[bufferIdx++] = (byte)((int)(audioVal) & 0xff);
buffer[bufferIdx++] = (byte)((int)audioVal >> 8);
}
return (bufferIdx - idxBuf);
}
Don’t worry about being offtopic. You’re doing the gruntwork of getting reliable audio in java, which is valuable for lots of us. Regarding the jumpiness, if you’re using MSIE, please consider another browser. The SMF guys can’t be bothered to fix it, and neither can I. I dropped support for MSIE a long time ago, when I noticed even IE9 didn’t produce correct renderings. So far for me being offtopic…
I think it was a great toy, no problems about it not being a game 
I think the volume of the echo should be based upon the volume you were at when the sound got produced, and when releasing the mouse the echo should go on as well.
Mike
@Riven – Thanks! ;D
@Mickelukas – I was kind of thinking along similar lines! It would be a more graceful way to end a tone to have the echo continue. I do want to get a control for some sort of ratio for the original tone vs the echo tone, as well.
I found a significant inefficiency in my MouseMotionListener code, extracting the info from the echo TextFields repeatedly. Doh! I changed it so that the number to be shipped is extracted just once and ready to go in a variable. This one change let me lower the buffer size somewhat. It’s now 16384 which comes to 4096 samples or just under 1/10 of a second.
This seems to work OK for “normal” echoes (under 1 second in length) but there are definitely dropouts as we get into multiple seconds.
I’m wondering what a “realistic” target buffer size might be. (Still in the queue, adding to the prospective cpu load: vibrato which entails FM(!), and a filter to deal with the aliasing.)
What you’re trying to do is easily achievable performance-wise in Java. However, the code you’ve posted looks a little suspect. Why are you using LinkedBlockingQueue and Double? This is going to create a huge amount of overhead. Standard practice would be to use a float[] or double[] to store the delay buffer, sized to the maximum size of your delay, and 2 int fields to track read and write positions. I can point you to some code that implements this in JAudioLibs if you want, though that’s probably overkill for your needs as it linear interpolates the read position. A delay effect like this should be (almost) a constant time operation - delay length should make no practical difference to your performance.
Also, I’m wondering if the LinkedBlockingQueue means you’re trying to multithread your audio code? Simply put - don’t! 
It’s really not worth it for what you’re trying to do.
Conservatively, 512 (samples not bytes). Less is achievable - Praxis will achieve 32 on my laptop with Windows and Linux, though there is a little extra overhead in the soundcard driver itself.
Incidentally, you’d be better performing all your operations on float or double arrays and just converting to bytes at the end.
Good luck with it!!! ;D
Best wishes, Neil
I’ve done polyphonic anti-aliased low latency synthesizers in Java many times eg:
http://www.angryoctopus.co.nz/synth2/synth.php
Just be very careful about how your audio generation thread works.
- Don’t use any form of synchronization at all in, there are many lockless ways to safely pass information from the GUI thread to the generation thread.
- Avoid branching if possible (no If’s in your main generation loop).
- Don’t use Java math functions, sine/cos lookup tables are much faster.
- Don’t use linked lists to buffer samples, arrays (power of 2 length for easy wrapping) are much faster.
+1 on everything you’ve said there. Some great synths on your site. Really impressed! ;D Are you planning on doing anything else with them besides applets? Don’t suppose the source is available? Would love to hook them through JNAJack.
I have lots of plans that will likely never materialize. If the Android audio latency ever gets sorted (see http://code.google.com/p/android/issues/detail?id=3434) I was planning to write a simple music creation app (similar to Korg’s DS-10). I have no objections to open-sourcing stuff once I get round to cleaning it up, I’ve been asked this a few times so I really should get on with it.
@nsigma
I wasn’t planning on using the Echo in a concurrent fashion. At first I had separate public poll() and add() methods, and this led to a reflex concern about concurrency. Since these have been combined into a single public entry point: tick(), as in a tick of a clock for a single sample, this isn’t a concern.
But I realize that the concurrency overhead eliminated (for example, by using a simple LinkedList as a queue) was not the only performance issue. By indexing into an array as you suggest, and overwriting instead of adding and deleting array elements, the dropouts have disappeared. Thank you!
As for the use of doubles, I have run into some confusion and trouble trying to keep track of when it is okay to use a double and when it is better to use a float. So, instead of solving this in a real way, I thought I’d try just making everything a double and leave the optimization to floats to a later time. I will reconsider this.
Maybe this guideline will work (floats vs doubles):
audio data - float (that is what is in the WaveTable now, I’m just returning the float into a double currently)
index into wave table audio data (e.g., for interpolation locations for pitch variation) - double
math ops, such as multiplication by a volume value between 0 and 1 - float is OK? that is where I started using doubles.
@ShannonSmith
Good checklist! Thank you.
(1) - “Don’t use any synchronization at all”
I may have one exception to this. There was no need of any synchronization in my implementation of the tone generator. But I think there is a possible concurrency danger in the conversion of real-time mouse data to pitch and volume values. It is impossible to control when mouse events will be added to the Queue where they are stored and converted into per-sample values. But there are probably efficiencies to be gained in the design of the class I wrote that does this: RTESmoother, with I put in the SampleCode area. I’d be interested in any advice you might have about that code, which can be seen here:http://www.java-gaming.org/topics/an-audio-control-helper-tool-using-a-fifo-buffer/24605/view.html
ACTUALLY (Doh!) based on what I just learned from Neal, a plain array used as a circular queue would be a big pickup. I can overwrite mouse events rather than creating and destroying them!! Must do this. But this will still need a bit of concurrency protection as writes and reads will be occurring potentially at the same time. I’ll post the improved version in the above link.
(2) “Avoid branching” – check, done.
(3) Avoid Java Math functions. – Does this include Math.min() & Math.max()? (I guess, per your next, I should also check for ways to keep the min and max values powers of two.) Otherwise, the only Math function I’m using is Math.sin(), when the WaveTable is first generated. From then on, all references are lookups into this WaveTable.
(4) “Don’t use linked lists to buffer samples…” – check, done, now that I’ve changed the echo to a simple array. That was the only spot. As for the power of two aspect, I will add that. I made my WaveTable 720 samples large as being a multiple of 360 makes the generating math a little easier, but I can see that making it 1024 and doing a bit-level AND operation (vs bounds testing) would be a performance pickup.
@ShannonSmith
Yes, fun synths! I remember playing with them when I found your site about a year ago. I also have had the “cloud” Content Creation PtI tutorial bookmarked for a long time. Very much meaning to pore over it, except I got sidetracked by working on audio.
Ahhh! I’ve long wanted a varispeed echo. Twisting the delay length knob on an analog echo effect on a held note source is a different sound than varying the pitch over a constant delay length. (And beautiful when handled well!)
Interpolation into the echo data might be part of it…but have to get my head around how the data points in the echo array would be stored.
Oh! One can use linear interpolation to store info in the array positions that were skipped over!