3D Sound Engine

Java Sound & OpenAL
#161

Agreed. So it “alters the pattern of the sound wave”, but what does that mean really? I’m thinking this isn’t something that can just be thought up theoretically - it will require some actual measurements and comparisons to first recognize what those pattern differences are, and second to come up with a filter that can recreate them, which can be applied to the audio data. How realistic can a system be without this component? Well, I’ll just have to see, I suppose. I’ll continue working on the other components, and maybe come back to this one later.

To counter my earlier argument, the skull actually isn’t “more or less spherical”. More like an upside-down bowl (a better representation might be a half-sphere with a flat end). The ears themselves are positioned toward the lower-back, not smack in the middle. So the echo will take longer to return to the ear if a sound is coming from behind than if it is coming from the front (and longer if it is coming from below than if it is coming from above). Likewise, it will pass through more space and attenuate more if coming from behind or below. So even without the pinna component, the brain can probably make the distinction by taking the phase and attenuation differences between ears and comparing that to the phase and attenuation differences between the initial sound and its echo for each side.

All this is really driving home to me just how complex positional audio is in the real world. As far as we’ve come with 3D graphics and virtual reality, on the audio side we are still practically in the stone age as we continue to simulate positional sounds using the cosine function!! It is about time some advances were made in this area.

#162

You have to threat it like photon-mapping: you hit the skull with a bunch of ‘audio rays’ which bounce around the skull, producing more rays. The more accurate the shape of the head/ears, the more realistic the result. The nice part is that you don’t actually have to use any complex math anywhere, ‘pattern manipulation’ is a side effect of what I just described.

#163

I think you have to incorporate the frequency absorbtion and resonance on every bounce, too. Additionally there is also the sound (mostly lower frequences) that is contributed by the skull itself, but I don’t know if this would be significant for spatial detection, since lower frequences usually are considered less important (hence you only have one subwoover, but multiple satelite speakers)

#164

[quote=“Riven,post:162,topic:33341”]
Doing this on the fly would require mixing a massive number of lines if you are going for a truly “photo-realistic” audio effect. I’m not all that sure it could be done without tens or even hundreds of milliseconds of buffering. That being said, I think this type of rig could be used to come up with that “complex math” that I’d need to formulate a filter that could be used on the fly (along the lines of texture mapping a low-poly model by using a high-poly version).

#165

I think this is what you should research :

there are various papers in the subject in google, but as far as I looked, they are very very complex :confused:

#166

I was taught, in a PsychoAcoustics class on Hearing, 20-something years ago at UCBerkeley, that for low pitches, phase differences are used by the ear to stereo-locate, and that for high pitches (size of the wave < size of the head) the relative amplitudes are used for stereo-location.

The point was that waves that are larger than the head in size will simply go around it, and smaller ones will tend to be blocked/attenuated. Probably not a sharp cutoff between the two regions, but then, it is hard to make a sharp cutoff with digital filters anyway. Still a high-pass filter that attenuates L or R based on angle might be sufficient for game programming 3D. Or maybe in combination with a slight phase-emulating delay for low-pass filtered sounds (delay compensates for speed of sound in air from one ear to the other).

Echoes matter, but it is the first sounds that reach the ear that are the most important for binaural hearing.

This refers to how you would treat a mono sound to be added to a mix, if you wanted to go beyond normal panning which seems sufficient for most 3D games. Theoretical on my part, as I have never experimented with this. But maybe I will after reading a few more chapters of this awesome book! It is a little dated, but the explanations are the clearest I’ve found on DSP: http://www.dspguide.com/pdfbook.htm

#167

Yes, my current plan is to go with a 3-phase process:

  1. Overall logarithmic attenuation based on distance from the listener, plus Doppler effect if enabled (the normal way of doing 3D audio, minus the panning)

  2. Phase and attenuation adjustments per side (as described earlier) based on the direction vector, using average values for speed of sound, attenuation, and distance between the ears

  3. Additional filtering per side to simulate echoing (either by using a formula derived from “audio ray tracing” a high-poly model of the ears and skull, or by doing the ray-tracing and mixing real time if fast enough)

#168

That sounds awesome. Such a project I would donate too.

#169

Me too.

#170

I will actually say that something strange has happened. When running the compatibility check, I found that my SoundSystem was actually not compatible with JavaSound. It’s weird since this is considered a “backup” library. JOAL worked just fine… But this is still kind of strange to me as I (think?) use Javasound to play sound effects right now in some way. Wondering if anyone had thoughts.

#171

Which version of Java are you running, and what operating system?

Could you provide the console output from after running the following applet:
Bullet / Target Collision Applet
(I feel silly pushing this dumb applet, but it’s the easiest test-case for LibraryJavaSound I have at the moment)

#172

Running Java 7 on Windows 7 64 bit. Can’t figure out where to obtain the console output but I heard sounds being played by the applet.

#173

Oh, well that applet uses the LibraryJavaSound plug-in. Make sure you are using the latest version in your project:

LibraryJavaSound.jar

If that doesn’t work, could you put together a simple test case that experiences the problem, and post the code? Could be a bug I haven’t encountered yet.

#174

Weird, I made a new project that works when I force the soundsystem to be instantiated with libraryjavasound. I’ll do a bit of fiddling to see why the compatibility would say no in my main project.

In other news, my game is 2D, and I actually position the listener at playerx, playery, -25 and play my sounds at z=0, otherwise there is no black and white as to what speaker is playing the sound, which isn’t the most realistic (it should play partially out of each speaker, with a percentage based on how far it is to the left or right of the listener in 2D?) The -25 z fix worked alright, but I’m wondering if anyone has had similar experiences?

#175

The left-right thing is due to the way I calculate the panning (it’s just a simple cosine formula). In a 2D situation where you want to utilize panning based on position on the screen, the way you are doing it is the way a number of others have done it as well - by playing with the z value (closer values pan faster, further values less). You can also change the attenuation to zero if you are having a problem with the sounds being too quiet at the distance you are listening from. It is a 3D sound library, so you sometimes have to get a little creative when you use it for 2D.

#176

Yeah I’m thinking of just sticking with the Z value. For anyone else playing with this, our world coordinates are based on pixels (one screen width = 1440 world units), and I’m achieving a decent pan with a default rolloff of .003, ROLLOFF for attenuation mode, and positioning the listener to Playerx, playery, -25, and playing all sounds from sourcex, sourcey, 0.

I’m having a weird issue though, with certain .wav files not obeying the formulas I’ve set up. I can be well over 2000 units away from a particular wav source and still hear it as though I’m using ATTENUATION_NONE. I’m guessing this is just a flaw in the wav file, because for certain files it is working fine.

#177

For attenuation to work, be sure to use monotone files (can’t remember if that just affected the OpenAL plug-ins only or also the JavaSound plug-in). If that doesn’t help, if you post a link to one of the problem files, I’ll take a look to see if there is anything “special” about it.

#178

If they’re not “monotone”, would there be an easy way with audacity or something to correct the issue with the file?

#179

[quote=“Rejechted,post:178,topic:33341”]
In Audacity, I believe it is an option in the Tracks menu (something like “convert stereo to mono”). Note: monotone is only required for point sources (ones you want to pan and attenuate). For ambient sources like music, use stereo instead.

#180

[quote=“paulscode,post:179,topic:33341”]

Right this makes sense. I am assuming that this is definitely why the issue is happening, I can think of no other logical reason why.

In that case, if you have a monotone sound and play it without your sound mod and it comes out both speakers, does it just play the sound out of both speakers at 50% volume each or something?