Web Audio Use with BJS

[gryff]

I've been building a little scene that is driven by a number of sound files (9) using this kind of code:

         sound1 = new BABYLON.Sound("sound1", "sound1.mp3", myScene, soundReady, { loop:  false,  volume: theVolume  });
	sound2 = new BABYLON.Sound("sound2", "sound2.mp3", myScene, soundReady, { loop: false, volume: theVolume  });
	sound3 = new BABYLON.Sound("sound3", "sound3.mp3", myScene, soundReady, { loop: false, volume: theVolume  });
	
        ...

	sound9 = new BABYLON.Sound("sound9", "sound9.mp3", myScene, soundReady, { loop: false, volume: theVolume  });

	
	function soundReady() {
        soundsReady++;
        
        ....

        }

Everything worked nicely when I tested it on my XP machine in Firefox. But then I started testing it in a variety of browsers I have on my machines - and then I started getting problems.

So Windows 7 - Chrome and Firefox  only

Windows 10 - Chrome, Firefox and Edge

Samsung Tablet (Android 5 - "Lollipop") - Chrome, but only if repeatedly I clear out the browser cache. ( Drove me crazy for several days )

Today I found this list : Can I Use : Web Audio API

Is that the best summation of the current situation?

And is it telling me that Apple devices will be a problem without fancy coding?

cheers, gryff

 

 

[Nabroski]

Someone should code an Audio Engine for Babylon

[meteoritool]

I remember that WebGL was not even possible before iOS 8, compatibility is going fast ! Just have to be a little patient ...
The only problem i've had with Apple devices is that the user has to initiate manually a sound before you can hear anything, but that's not a problem to me, since any webdesigner will tell you that auto-playing sound from a new webpage is the WORST design idea, end-users hate that !
WebAudio API seems like a powerful tool, I can't wait to have its more advanced features into Babylon.js !

[aWeirdo]

Hi @gryff

Personally, i use html5 audio tags instead atm, plan to change once it gets better.

e.g.

  <!-- Audio -->
  <!-- Loops -->
  <audio id="AmbientLoop01" src="assets/sounds/AmbientLoop01.mp3" loop></audio>
  <audio id="AmbientLoop02" src="assets/sounds/AmbientLoop02.mp3" loop></audio>
  <audio id="AmbientLoop03" src="assets/sounds/AmbientLoop03.mp3" loop></audio>
  <audio id="AmbientLoop04" src="assets/sounds/AmbientLoop04.mp3" loop></audio>

  <!-- no Loop --> 
  <audio id="soundEffect01" src="assets/sounds/soundEffect01.mp3"></audio>
  <audio id="soundEffect02" src="assets/sounds/soundEffect02.mp3"></audio>
  <audio id="soundEffect03" src="assets/sounds/soundEffect03.mp3"></audio>
  <audio id="soundEffect04" src="assets/sounds/soundEffect04.mp3"></audio>

 

With modern javascript, all you have to do;

AmbientLoop01.play(); //Play
AmbientLoop01.pause(); //Pause
AmbientLoop01.volume = 0.5; //Set volume to 50% (100% by default)

It's a good idea to keep track of playing loop music, e.g for ambient loops;

The key to the following code is that we actually pass the html element, and not "just" the id of the element, so we don't even have to do document.getElementById, etc.

var ambientPlaying = undefined;

funtion playLoop(newAmbient){
  // Avoid two loop audio's at same time.
  if(ambientPlaying) ambientPlaying.pause();

  //Start new ambient audio
  newAmbient.play();

  // Set the new audio as ambientPlaying.
  ambientPlaying = newAmbient;
}

//Script start
playLoop(AmbientLoop01);

//so AmbientLoop01 is playing from start and after some time, we want to change to AmbientLoop02
//...
playLoop(AmbientLoop02);

 

Disclaimer; the code above was writting out of memory, i may have made a small misstake somewhere or forgotten something

[Nabroski]

I also  dont know what is the benefit to use HTML5 audio. (?) You can also pass the header to of the soundfile to your cpu like in the good old times

[JCPalmer]

I think that web page you reference is in-complete in that it does not does not even mention which browsers handle which types of sound files.  That may be part of your results (check your console when it does not work).  Might actually be below file type, in the codec chosen.

BABYLON.Sound does take an array of sound file names now, I believe.   Does mean you need convert your files to other formats to take advantage of, though.

I am integrating sound tightly with animation, and also in need of voice recordings.  I added voice recording / saving to my code base.  Knowing that my recordings would need to be in multiple formats, I decided to fire all formats and make my own.  I added saving to embedded Javascript or Typescript.  The code needed to decode is minimal, and means the sound can be put in a script tag.  This means you know it will be ready to play immediately.  Not good for streaming, so reply on a different format for that.

Am thinking about a slightly lossey change, so I am also saving everything in .WAV too.  Can read external files of any type the browser supports & re-save in a more compact manner later.  Here is what a .TS file looks like (base 64 data shortened):

public MySound(scene: BABYLON.Scene) : BABYLON.Sound {
    var toBuf = function(base64){
        var bStr = window.atob(base64);

        var len = bStr.length;
        var ret = new Float32Array(len / 2);
        var b1, b2, asShort, isNeg;
        for(var i = 0, j = 0; i < len; i += 2, j++){
            b1 = bStr.charCodeAt(i);
            b2 = bStr.charCodeAt(i + 1);
            isNeg = b1 >> 7 === 1;
            b1 = b1 & 0x7F;
            asShort = 256 * b1 + b2;
            if (isNeg) asShort -= 0x8000;
            ret[j] = asShort / 0x7FFF;
        }
        return ret;
    }

    var context = BABYLON.Engine.audioEngine.audioContext;
    var audioBuffer = context.createBuffer(1, 13517, 44100);
    audioBuffer.getChannelData(0).set(toBuf("AzADDQK0AkQCjAMTAqoB0QGsAi...==") );
    var snd = new BABYLON.Sound("MySound", null, scene);
    snd.setAudioBuffer(audioBuffer);
    return snd;
}

 

[gryff]

14 hours ago, meteoritool said:

The only problem i've had with Apple devices is that the user has to initiate manually a sound before you can hear anything

@meteoritool; Yeah I run across that. Apple likes to do it their way.

 

14 hours ago, aWeirdo said:

Personally, i use html5 audio tags instead atm, plan to change once it gets better.

@aWeirdo : Well I used to use howler.js which I gather automatically falls back to HTML5 audio tags if Web Audio is not supported. But I just felt I'm using BJS, I should used its sound code. Ty for the example code.- always helps this coder dummy

 

10 hours ago, JCPalmer said:

BABYLON.Sound does take an array of sound file names now, I believe.   Does mean you need convert your files to other formats to take advantage of, though.

@JCPalmer : Jeff,  I tried it with and without .ogg files too - same issues. And of course, I'm well aware of your efforts with your own format. Be nice to have more on that . (Is that what gryff is always saying to JCP? )

 

Anyway thank you all for the suggestions and thoughts, and being the stubborn sort, I went ahead and published my efforts anyway:

The Man From Babylon thread in Demos and Projects forum.

cheers, gryff