Java Theremin

@Riven – Thanks! ;D
@Mickelukas – I was kind of thinking along similar lines! It would be a more graceful way to end a tone to have the echo continue. I do want to get a control for some sort of ratio for the original tone vs the echo tone, as well.

I found a significant inefficiency in my MouseMotionListener code, extracting the info from the echo TextFields repeatedly. Doh! I changed it so that the number to be shipped is extracted just once and ready to go in a variable. This one change let me lower the buffer size somewhat. It’s now 16384 which comes to 4096 samples or just under 1/10 of a second.

This seems to work OK for “normal” echoes (under 1 second in length) but there are definitely dropouts as we get into multiple seconds.

I’m wondering what a “realistic” target buffer size might be. (Still in the queue, adding to the prospective cpu load: vibrato which entails FM(!), and a filter to deal with the aliasing.)

What you’re trying to do is easily achievable performance-wise in Java. However, the code you’ve posted looks a little suspect. Why are you using LinkedBlockingQueue and Double? This is going to create a huge amount of overhead. Standard practice would be to use a float[] or double[] to store the delay buffer, sized to the maximum size of your delay, and 2 int fields to track read and write positions. I can point you to some code that implements this in JAudioLibs if you want, though that’s probably overkill for your needs as it linear interpolates the read position. A delay effect like this should be (almost) a constant time operation - delay length should make no practical difference to your performance.

Also, I’m wondering if the LinkedBlockingQueue means you’re trying to multithread your audio code? Simply put - don’t! :wink: It’s really not worth it for what you’re trying to do.

Conservatively, 512 (samples not bytes). Less is achievable - Praxis will achieve 32 on my laptop with Windows and Linux, though there is a little extra overhead in the soundcard driver itself.

Incidentally, you’d be better performing all your operations on float or double arrays and just converting to bytes at the end.

Good luck with it!!! ;D

Best wishes, Neil

I’ve done polyphonic anti-aliased low latency synthesizers in Java many times eg:

http://www.angryoctopus.co.nz/synth2/synth.php

Just be very careful about how your audio generation thread works.

  • Don’t use any form of synchronization at all in, there are many lockless ways to safely pass information from the GUI thread to the generation thread.
  • Avoid branching if possible (no If’s in your main generation loop).
  • Don’t use Java math functions, sine/cos lookup tables are much faster.
  • Don’t use linked lists to buffer samples, arrays (power of 2 length for easy wrapping) are much faster.

+1 on everything you’ve said there. Some great synths on your site. Really impressed! ;D Are you planning on doing anything else with them besides applets? Don’t suppose the source is available? Would love to hook them through JNAJack.

I have lots of plans that will likely never materialize. If the Android audio latency ever gets sorted (see http://code.google.com/p/android/issues/detail?id=3434) I was planning to write a simple music creation app (similar to Korg’s DS-10). I have no objections to open-sourcing stuff once I get round to cleaning it up, I’ve been asked this a few times so I really should get on with it.

@nsigma
I wasn’t planning on using the Echo in a concurrent fashion. At first I had separate public poll() and add() methods, and this led to a reflex concern about concurrency. Since these have been combined into a single public entry point: tick(), as in a tick of a clock for a single sample, this isn’t a concern.

But I realize that the concurrency overhead eliminated (for example, by using a simple LinkedList as a queue) was not the only performance issue. By indexing into an array as you suggest, and overwriting instead of adding and deleting array elements, the dropouts have disappeared. Thank you!

As for the use of doubles, I have run into some confusion and trouble trying to keep track of when it is okay to use a double and when it is better to use a float. So, instead of solving this in a real way, I thought I’d try just making everything a double and leave the optimization to floats to a later time. I will reconsider this.

Maybe this guideline will work (floats vs doubles):
audio data - float (that is what is in the WaveTable now, I’m just returning the float into a double currently)
index into wave table audio data (e.g., for interpolation locations for pitch variation) - double
math ops, such as multiplication by a volume value between 0 and 1 - float is OK? that is where I started using doubles.

@ShannonSmith
Good checklist! Thank you.

(1) - “Don’t use any synchronization at all”

I may have one exception to this. There was no need of any synchronization in my implementation of the tone generator. But I think there is a possible concurrency danger in the conversion of real-time mouse data to pitch and volume values. It is impossible to control when mouse events will be added to the Queue where they are stored and converted into per-sample values. But there are probably efficiencies to be gained in the design of the class I wrote that does this: RTESmoother, with I put in the SampleCode area. I’d be interested in any advice you might have about that code, which can be seen here:http://www.java-gaming.org/topics/an-audio-control-helper-tool-using-a-fifo-buffer/24605/view.html

ACTUALLY (Doh!) based on what I just learned from Neal, a plain array used as a circular queue would be a big pickup. I can overwrite mouse events rather than creating and destroying them!! Must do this. But this will still need a bit of concurrency protection as writes and reads will be occurring potentially at the same time. I’ll post the improved version in the above link.

(2) “Avoid branching” – check, done.

(3) Avoid Java Math functions. – Does this include Math.min() & Math.max()? (I guess, per your next, I should also check for ways to keep the min and max values powers of two.) Otherwise, the only Math function I’m using is Math.sin(), when the WaveTable is first generated. From then on, all references are lookups into this WaveTable.

(4) “Don’t use linked lists to buffer samples…” – check, done, now that I’ve changed the echo to a simple array. That was the only spot. As for the power of two aspect, I will add that. I made my WaveTable 720 samples large as being a multiple of 360 makes the generating math a little easier, but I can see that making it 1024 and doing a bit-level AND operation (vs bounds testing) would be a performance pickup.

@ShannonSmith

Yes, fun synths! I remember playing with them when I found your site about a year ago. I also have had the “cloud” Content Creation PtI tutorial bookmarked for a long time. Very much meaning to pore over it, except I got sidetracked by working on audio.

Ahhh! I’ve long wanted a varispeed echo. Twisting the delay length knob on an analog echo effect on a held note source is a different sound than varying the pitch over a constant delay length. (And beautiful when handled well!)

Interpolation into the echo data might be part of it…but have to get my head around how the data points in the echo array would be stored.

Oh! One can use linear interpolation to store info in the array positions that were skipped over!

Your main problem was actually Double vs double rather than double vs float - you were creating a Double object for every single sample. Java object creation may be fast, but it’s not that fast! :slight_smile: Not to mention the garbage!

As for double vs float, you should be fine with float for most things, except possibly for certain cases where values are feeding back into themselves (such as in filters). This does happen in a delay, but to be honest I’d probably still keep it as a float[]. There’s an article somewhere on the CSound site which talks about 6 or more DSP operations before the difference in accuracy between float and double may become audible.

That was my suggestion for your audio data in your audio thread, and to avoid object creation. For your communication between GUI and audio threads, then use one of the concurrent BlockingQueue implementations - you’ve got objects anyway, and those queues should be the most optimal - they’re damn well meant to be! :slight_smile:

Sounds like that could be pretty cool! If you do get around to opening up some or all of the code, though, let me know. I’m currently working on a project coming out of Praxis to try and look at some common audio interfaces (including a LADSPA / (simple) VST-like API) for Java, and porting a range of things from other projects to it, as well as looking to get other Java audio devs interested. This is and will be up at http://code.google.com/p/java-audio-utils/

Best wishes, Neil (with an ‘i’ :stuck_out_tongue: )

http://code.google.com/p/praxis/source/browse/audio.ops.impl/src/org/jaudiolibs/audioops/impl/VariableDelayOp.java

It’s partly ported from Gervill (I just realised it hasn’t got a credit in there - must fix that before release).

btw - this isn’t quite the same as those analog echo units - love that sound! ;D

I made some changes, putting items in arrays via overwriting rather than via adding and dropping objects. After doing so, I was able to lower the latency from 4096 to 1536 samples. (An improvement–but still a ways to go to reach 600.) Below that, I’m getting a slight sfzt sound at the start. At least, the echo and its length are no longer an issue.

With a 1024 sample buffer the sfzt start is pretty prominent, but other than that, the JTheremin runs fine. So I’m assuming there is something about how I start up the sound that needs improving more than anything else.

Neil, the routine which receives from the GUI (RTESmoother) only gets a double. It then adds a time stamp. [Hmmm. Maybe it would be more correct to issue the time stamp immediately and pass it in as a second parameter.] What I’ve done is to get rid of the LinkedBlockingQueue and put in two arrays, one a double[] for the GUI values and the other a long[] for the time stamps. The add() method and the block of code where the value is read are both enclosed by a “lock” Object and synchronized. Neither occur very often relative to how short they both are, so I think this should be okay.

I KNOW the GUI add() calls are not causing the starting sfzt via blocking, because I can prevent any add() calls, and the sfzt still is audible, is unchanged.

I’ll post the “improved” RTESmoother one its thread in the “Shared Code” area, for the curious.

You echo code looks interesting. I can’t totally imagine how it sounds (I’m not the best at reading code) but I see the interpolation happening.

I had this idea for an echo array that allows detuning the echo: let the “read-head” progress incrementally by 1’s, but allow the “write-head” to lag behind by a variable increment. One can hold onto the previous write value and its location and the current write value and its location, and interpolate to post into all the intervening array slots.

I suppose this could be combined with a read-head that can progress a varying amount as well. Not sure what one gains that way, though. Seems like one can do one or the other, no need to do both. I can get my head around the varying write-head model a little easier than a varying read/write-head. (Dating myself, using a tape deck metaphor?)

I remember once using a digital echo VST that allows pitch-bending, but it was written in C++ I’m pretty sure. I can’t recall what I did with it though, and I miss the effect. My analog units are rather noisy and colored. So I’m pretty motivated to make one for myself. Another good learning project. Whoopee!

Are you starting the line each time the sound starts? You’d be better holding the line open and writing silence to it.

I owe you a big thank-you here, as you’ve inadvertently pointed me to a bug I introduced in refactoring Praxis a while back. I was about to write and tell you that you’ve probably made this slower by adding in a lock, but actually to be lock-free you should have been using ConcurrentLinkedQueue and not LinkedBlockingQueue anyway. I’m always getting those two mixed up for some reason. You could try the previous code you had with a ConcurrentLinkedQueue and see what the performance is like then - locking is possibly more than the object overhead here.

EDIT: Actually, scratch the previous paragraph. Looking again at LinkedBlockingQueue it uses two locks - put and take. There shouldn’t be much thread contention at all, so it is likely to be better than the locking strategy you just implemented.

I think there’s a subtle bug in your code as well (though I’m no good at reading code either! :slight_smile: ) in that you should poll() the queue in a while loop to ensure it’s drained on each tick - not that it’s likely to get swamped by the GUI.

I think it’s better to let the write-head progress normally (by 1) and let the read-head read from a variable moment back in time. This is closer to an actual tape-delay, I think. The write-head is recording the present and the read-head is playing back some moment in the past.

PS. I’m old enough to be perfectly happy with the tape-deck metaphor. ;D

Thats awesome! Works really nice and fun to play with. Are you planning on using this in another project, like a game or something else?

The echo effect is really cool, but it stops as soon as you let go of the mouse. :frowning:

@davedes

[quote]The echo effect is really cool, but it stops as soon as you let go of the mouse. Sad
[/quote]
Mickelukas had the same suggestion, and I agree that would be nice to have. I am thinking I will try and implement Neil’s suggestion to keep the Theremin “running” but silent even if the mouse is released, and include that feature then as well as some sort of volume ratio or independence between the source and the echo.

Maybe the mouse position should drive the overall echo volume even when the mouse is up.
Or the echo setting could be static, on the control panel, rather than tracking the mouse. :point:

@fruitmaze

[quote]Thats awesome! Works really nice and fun to play with. Are you planning on using this in another project, like a game or something else?
[/quote]
Thanks! Yes in terms of using it in “another project” but I don’t have any concrete plans yet. Any suggestions?

I’m still in the learning-project stage. Right now, I’m working on another sound app for a friend for a few days, that plays a “tree” of sine waves, where I am learning about how to create and use envelopes (got a simple Attack-Release working so far), and about using a JTable in the GUI for holding/editing the info used to configure the sines. Am very pleased that I figured out how to play multiple sine waves all from the same WaveTable.

On the queue for the Theremin: vibrato. Which as a learning-project means I get to implement a bit of FM! (FM is my main synth, via DX-7 and Native Instruments FM7 [FM8 if I upgrade]. I’ve done a lot of high level FM programming (musician user level).)

Also on wish list: ability to record, save & playback the data currently generated by the GUI, as this is much lighter than raw audio.

Also on wish list: implement a filter to get rid of the aliasing! I think if you go down SteveShannon’s links below you will find a demo where you can hear how important that is.

Also on wish list: have this light and fast enough that it can be used/integrated into a game as a SF/X source, for example, with tracking the motions of some object moving on a display. The results could be more varied and expressive of game action, potentially, than canned sounds, if done right. Though at the moment, the simplest (conceptually) would be if you had a use for a slide-whistle or trombone slide that could be tied to game state, or game events.

But there is also the potential for some nice, sinister “industrial” background sounds that have more expressive variability than a prerecorded loop. Try this:
100% Sawtooth
Lowest possible pitch setting
99% Echo feedback
Echo delay setting in the range 10msec to 50msec.
This could be very cool if variable echo delay rates get programmed. :slight_smile:

Turned out to be really simple to make the Echo continue independently of the Theremin volume.
Done.
Everything still goes silent if you let go of the mouse. But if you drop down to the volume floor, the Theremin is silent, but the echo (assuming it is on, of course) will linger.

@fruitmaze

[quote]Thats awesome! Works really nice and fun to play with. Are you planning on using this in another project, like a game or something else?
[/quote]
Thanks! Yes in terms of using it in “another project” but I don’t have any concrete plans yet. Any suggestions?

Well as a game maybe something surrealistic and atmospheric, thats the feeling I get when I hear the sounds. Or a game about a bee :slight_smile: Another idea could be to combine graphics and sound, like when you move the mouse to create different sounds the background changes as well.