Hello, all. My next big project is to write my own software audio mixer, so that I can play multiple audio inputs on a single SourceDataLine. I need to be able to do the following things with PCM audio data:
Mixing multiple inputs into one 2-channel (left/right) output, maintaining synchronization between the channels
Converting between (or mixing directly) various audio formats (different sample rates/sample sizes/channels)
Changing frequency (pitch)
Changing gain (volume) independently on each channel
I want to be able to do this in pure Java, completely programatically (i.e. no linking with 3rd-party libraries). I’m looking for all tips, source code, tutorials, that kind of thing. I am proficient in several programming languages, so references don’t necessarily have to be written specifically for Java.
Converting between sample rates and changing frequency. Both of those are not trivial to get high quality results. There are a few pretty good sites for audio stuff, musicdsp.org has some good code examples also the dsp forum at KVR is a good place to ask questions. Columbia uni has a very good mailing list on audio dsp that I tend to search with google
eg: sample rate conversion site:music.columbia.edu
From my experience, I’d suggest using small, constant size floating-point buffers throughout your audio pipeline. This is the method used by all the Java audio libraries I know of, including my own, and will make a lot of the mixing stuff a lot easier and reduce any problems with different sample sizes. Sample rate conversion (and changing frequency) require interpolation between a source table and your destination buffer - I’d recommend going with some sort of pluggable interpolation strategy and starting with a basic linear interpolator - quite nasty but possibly good enough, and you can then plug in better alternatives later.
For mixing (and gain) and any other sort of control change, I’d also recommend linear interpolating values through your buffer period as it’s easy to get nasty clicks otherwise.
I am planning on opening up some of the audio libraries under Praxis http://code.google.com/p/praxis as GPL w/CPE in the near future, but I know you don’t like that licence! Feel free to look around the repo though - audio.ops, audio.servers and rapl are the 3 things to look under. Just be warned it’s a little untidy in there, and there may be dragons!
For low latency, worth having a look at this page http://frinika.wikispaces.com/JavaSoundAudioServer I’m using a similar principle in Praxis, it’s not that tricky, and it gives good results. In fact, Frinika is worth a look for what’s possible with Java audio.
Hope that’s vaguely useful, and good luck with it.
One approach to doing the pitch shifting is to use a phase vocoder. With this you can slow down/speed up a sound without changing the pitch. By resampling up or down respectively, you can then change the longer or shorter sounds into ones with the original duration but different pitch.
The quality of this might not be as good as you need, but since I had most of this lying around already I thought I’d put it together anyway. So here’s a basic framework for a phase vocoder in Java. The code’s all mine, and you’re welcome to take any bits you want and chuck a BSD license or whatever on them.
Non-periodic sounds (noise) is where the phase vocoder tends to fail. Also, to handle low frequency sounds you will need to use a larger window size, which increases the processing overhead.
Feel free to look around the repo though - audio.ops, audio.servers and rapl are the 3 things to look under. Just be warned it’s a little untidy in there, and there may be dragons! 