Jump to content

Creating fun & immersive audio experiences with Web Audio


Recommended Posts



 I’ve written an article dumping almost everything I know about Web Audio and what I’ve learned while writing the audio stack of Babylon.js: Creating fun & immersive audio experiences with Web Audio. I’m also explaining how we’ve built the Music Lounge demo being hosted on the MS Edge dev site.


 It also turns out to be the best up-to-date documentation on our audio stack (I'm talking about the new audio streaming feature for instance). 









Link to comment
Share on other sites

Very interesting and cool !

I wanted a sorta solar system with each planet emitting a sound, but in a musical/rhythmically synced way, and with positional audio.
Three.js worked but I didn't manage to sync audio in a simple way...
Babylon.js is perfect in the way it implements Web Audio capabilities from the start, and with really easy coding.
I tried but failed at writing directly with the web audio API scripting.
But I was able to make things work with Babylon.js, thanks to the very good tutorials, playground and forums.
Doc could be improved though it's another topic ... (for example, having a Q&A section ala 'gamefaq.com' would be nice, in the classes doc : i'd like to know default values or possible values more often given...)
There is something very important I noticed :
upon creating scenes with audio looping (much like in your MusicLounge example) I noticed that .mp3 sounds do NOT loop well ! There is latency between each loop ! This problem does not happen when using .wav files for sound.
That is a big problem of course, as .wav files really are NOT web-friendly in terms of file size :/
I don't know if this comes from Babylon.js or from the Web Audio capabilities but this is a crucial thing to know, and I haven't seen this info anywhere...
I also noticed that no sound comes out using an iPad2 either with safari app or chrome app on iOS 9 ... 
Thanks for all the good work anyway, open source tools this quality and easy to use is a gift to the world :-)
Link to comment
Share on other sites

It would be a very good idea to add the simple information that .mp3 do not loop correctly in your tutorials, that would make it even more straightforward and complete ;-) !


- So is there a way to actually have sound coming out on iOS9 Safari ?


- Is there gonna be any more Web Audio capabilities implemented through Babylon.js ? As an audio lover, I'd like to be able to play with reverb, delay, EQ, etc ... ?


- Would you recommend Babylon.js framework if I'm gonna use Cocoon.js to turn my website into an iOS/Android/WindowsPhone app ?

Link to comment
Share on other sites

The mansion demo runs at 9fps on my iPad2(retina) with iOS 9.1, so hard to know :/, but I can't hear anything ...

This "music lounge" demo doesn't work for audio either, but I could hear sound from the Flight Simulator demo !

I found those informations :



Alas I'm not good enough as a dev to implement these, and I'm not sure about the problem :/


See one of my simple experiments here :


it uses sound.attachToMesh(mesh) and actions. The position of the meshes and positional audio are used to control the volume of each sound.


I still have a long way to go, with the assets loader, etc ...

Link to comment
Share on other sites

Any news regarding this issue ?


I've updated the link in my previous post for the sound demo I'm making (http://www.orbis-somnium.com/geometric_sampler/)


the code is a mess, now I realized i have used 2.2.0 version ...

var box3 = BABYLON.Mesh.CreateCylinder("maintrigger", 2*(((3/(2*Math.sqrt(3)))*Math.sqrt(6))/3), 0, 2, 3, scene, true);

And now I just see this :


2.3.0: Major updatesNew Mesh.CreatePolyhedron() method 




I worked out my maths anyway >_<

Link to comment
Share on other sites

  • 3 weeks later...

I need to work on iOS 9 issue. I've managed to find one of the only framework that seems to work on iOS 9: howler.js . I'll try to implement a similar hack.


For the VirtualJoysticksCamera, it works on iPhone / iOS 9, I've just tested on our main website. You need to reference jQuery PEP polyfill. Did you reference it?

Link to comment
Share on other sites

@davrous - I've not heard of  jQuery PEP polyfill either. When do you recommend using this instead of hand.js, and in what circumstances would you use either - as well as does it make any sense to reference both and use conditionally?

Link to comment
Share on other sites

THis question is more for Davrous as he built the virtual joystick camera


but as you can see here:



Joystick registers pointer events but is not blocking them


Perhaps you should just try to hook your event on VirtualJoystick.vjCanvas as well

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...