Positional audio with camera attached to parent mesh

[FunFetched]

Hi there! I'm relatively new to Babylon.js, but loving it so far! I'm currently developing a simple FPS, and had some issues getting the positional audio to work at first. I found FreeCamera's control scheme to be inadequate for my purposes, and decided to instead assign a parent mesh (representing the player's "head") to my camera, so it would follow along perfectly. However, the positional audio didn't work with this configuration, as I discovered that the engine uses the camera's local coordinates (always 0, 0, 0 in this case) to determine the listener's position. I changed the following line (#18730) to use the camera's globalPosition instead, and all is well.

audioEngine.audioContext.listener.setPosition(listeningCamera.globalPosition.x, listeningCamera.globalPosition.y, listeningCamera.globalPosition.z);

I would imagine that this could break some other projects, but it seems to me that there should be some kind of option to enable this behavior. Perhaps a check to see if the camera has a parent? Maybe a simple bool flag?

[GameMonetize]

Ping @davrous

[davrous]

Hi! Thanks for your feedback.

So if I've understood correctly, you'd like to create a third-person camera, so the audio engine should update the position and orientation based on the mesh representing the head, correct?

This is an interesting use case I've never thought about before. I'm going to add an option on the scene to let you define a listening object instead to be able to set something else as the current active camera. 

[FunFetched]

In my case, it's a first-person camera, but one that is attached to the "eye" mesh by means of parenting. Here are snippets from a couple of different files so you can see what's going on:

// Setting up the player's 'actor' mesh
this.mesh = new BABYLON.AbstractMesh('player', scene);

this.bodyMesh = Meshes.body.clone('body', this.mesh);
this.bodyMesh.position.y = 0.32;
this.bodyMesh.parent = this.mesh;

// Head pivots with mouse look.
this.head = new BABYLON.AbstractMesh('head', scene);
this.head.parent = this.mesh;
this.head.position.y = 0.3;

// Separate, empty 'eye' mesh for attaching camera.
// This is for controlling recoil and other 'shake' animations
this.eye = new BABYLON.AbstractMesh('eye', scene);
this.eye.position.y = 0.1;
this.eye.parent = this.head;

...

// Attaching camera to the player's 'eye'
this.camera.parent = this.me.actor.eye;

As you can see, there's some hierarchy here that puts a few layers between the camera itself and the world coordinate system. Since the camera's local coordinates never stray from 0, 0, 0, the audio listener's position never moves.

FreeCamera was a little too "special purpose" for my needs, as I wanted to roll my own control scheme. I also didn't want the camera to control the player mesh; I wanted the player mesh to control the camera. I tried a few of the other built-in cameras, TargetCamera, ArcRotateCamera, etc., and attempted to make them follow my eye mesh, but couldn't get them to track smoothly and reliably. They all seem to be hard-wired to follow just behind their target with some interpolation, while I wanted the camera to be locked firmly to my mesh. Fiddling with the parameters to eliminate the interpolation just caused them to go haywire, so I decided to eliminate all the bother and set the camera's parent to my 'eye' mesh and be done with it.

Ideally, I guess I'd propose another type of camera altogether; say, a "FixedCamera", or "MeshCamera". No fancy controllers or interpolators, just a means of locking its global position and orientation to a mesh (or AbstractMesh in this case). I might take a stab it myself at some point, but my work-around is doing fine for now, and I don't have quite enough time at the moment to dig in there and figure that out.