Jump to content

Precompiling Shaders


royibernthal
 Share

Recommended Posts

I'm working on a feature to contribute to the bjs core - precompiling shaders for materials. It'd be great if you could help with issues that arise as I develop it.

Many thanks to @Deltakosh for his support and detailed responses in the mail.

 

The problem

There are many parameters that can trigger shader recompilation for a material, e.g. using bones, adding a new light, etc.

Since the shader compilation is a sync process, it can be problematic if it happens during gameplay, as it'll result in lags that're noticeable to the user.

For instance, imagine you have a mesh with bones, suddenly walking in the game into a certain point light for the first time, at that moment, a new shader will have to be recompiled for a mesh with bones and with that point light.

Now imagine having multiple meshes, many lights and other parameters.

 

The solution

Precompiling all possible shader combinations for each mesh's material, so when a certain shader combination is suddenly required during the game, it already exists and doesn't need to be compiled again.

Here's a list of all the possible parameters that affect shader precompilation:

https://github.com/BabylonJS/Babylon.js/blob/master/src/Materials/babylon.standardMaterial.ts#L2

 

Of course, shaders for all the possible combinations even for a single material is an unacceptable number.

More practically, each game has certain requirements, which are usually not too much. Ideally it'd have a few fixed parameters - e.g. a hemispheric light that's always on, and a few varying parameters, e.g. a point light which can be turned on and off.

The idea is to only precompile shaders that would've been compiled during gameplay anyway, and thus precompiling a very acceptable number of shaders.

Each game will be able to define for itself what parameters it should take into account, and specify for each parameter the possible values that should be precompiled.

A single value means the parameter is fixed, which makes the calculation for it almost negligible, as it doesn't require more than 1 combination, e.g. a light that's always on.

 

What I have so far

At the moment I'm developing it in a separate project, and once it's complete I'll migrate it to the bjs core.

shaderCompiler.ts - actual implementation

shaderCompilerTest.ts - how it should look to the user

 

I intend to start with a few common parameters (hemipheric lights, point lights and bones), and extend the options with time. (hopefully with your help)

Each supported parameter is a class implementing the IShaderCompilerEntity interface, at the moment there's only ShaderCompilerHemisphericLight.

 

What's next

I'm struggling with the heart of the solution - calculating all the possible combinations for a specified configuration. In other words, implementing ShaderCompiler/getConfigurations().

For example, if I have 3 varying lights, each can be on or off, I'd need to calculate all the possible combinations:

0, 0, 0
0, 0, 1
0, 1, 0
0, 1, 1
1, 0, 0
1, 1, 0
1, 1, 1
1, 0, 1

I'm not sure how to calculate all possible combinations when I have more parameters, with each parameter having its own number of possible values (which can be more than just 2 values - on/off).

Any idea how such a thing can be achieved?

Link to comment
Share on other sites

4 hours ago, royibernthal said:

I intend to start with a few common parameters (hemipheric lights, point lights and bones), and extend the options with time. (hopefully with your help)

Each supported parameter is a class implementing the IShaderCompilerEntity interface, at the moment there's only ShaderCompilerHemisphericLight.

 

What's next

I'm struggling with the heart of the solution - calculating all the possible combinations for a specified configuration. In other words, implementing ShaderCompiler/getConfigurations().

For example, if I have 3 varying lights, each can be on or off, I'd need to calculate all the possible combinations:

0, 0, 0
0, 0, 1
0, 1, 0
0, 1, 1
1, 0, 0
1, 1, 0
1, 1, 1
1, 0, 1

I'm not sure how to calculate all possible combinations when I have more parameters, with each parameter having its own number of possible values (which can be more than just 2 values - on/off).

Any idea how such a thing can be achieved?

Hey this is a great Idea, I added the procedural materials library , Post-process library and the procedural textures library to the UNITY exporter (coming soon for the next version of the UNITY toolkit exporter).\

I created this scene with most of the materials without pre-compiling (water, lava, grid, gradient , fur etc..shaders)

Now I would like to understand better your sample so I can tested in my code and here is a java script combination library we may use for this or something similar

module ASSETS {

	export class ShaderCompilerTest {

		constructor() {
			var mesh: BABYLON.Mesh;
			var shaderCompiler: ASSETS.ShaderCompiler = new ASSETS.ShaderCompiler(mesh);

			var lightA: BABYLON.HemisphericLight;
			var lightB: BABYLON.HemisphericLight;
			var lightC: BABYLON.HemisphericLight;

			shaderCompiler.compile([
				new ASSETS.ShaderCompilerTask(new ASSETS.ShaderCompilerHemisphericLight(lightA), ASSETS.ShaderCompilerHemisphericLight.ON),
				new ASSETS.ShaderCompilerTask(new ASSETS.ShaderCompilerHemisphericLight(lightA), ASSETS.ShaderCompilerHemisphericLight.ALL),
				new ASSETS.ShaderCompilerTask(new ASSETS.ShaderCompilerHemisphericLight(lightA), ASSETS.ShaderCompilerHemisphericLight.ALL)
			], () => console.log('complete'), (progress: number) => console.log('progress', progress));
		}

	}

}

But I dont understand if we are talking about the same shaders, can you please explain further with another example ?

 

Link to comment
Share on other sites

Hey, Is there a way for me to view your code there?

It seems that in your scene your materials only need 1 configuration, e.g. you apply a lava material which uses certain parameters and these parameters don't need to change. Correct me if I'm wrong.

In that case, all you need to do is wait for material.isReady() for each material, which is what I refer to as "precompiling", regardless of how complex that material is, it only consists of 1 combination of parameters to compile.

In the test I attached there is 1 mesh, which has 3 lights, 1 is always on, which means it doesn't affect the number of combinations, and the other 2 can be either on/off, meaning there are 4 (2^2) combinations and thus 4 shaders that need to be precompiled.

That combinations library looks good, how'd you use it in this case? It looks like it can possibly be part of the solution but there're still gaps to fill.

Link to comment
Share on other sites

6 hours ago, royibernthal said:

Hey, Is there a way for me to view your code there?

It seems that in your scene your materials only need 1 configuration, e.g. you apply a lava material which uses certain parameters and these parameters don't need to change. Correct me if I'm wrong.

In that case, all you need to do is wait for material.isReady() for each material, which is what I refer to as "precompiling", regardless of how complex that material is, it only consists of 1 combination of parameters to compile.

In the test I attached there is 1 mesh, which has 3 lights, 1 is always on, which means it doesn't affect the number of combinations, and the other 2 can be either on/off, meaning there are 4 (2^2) combinations and thus 4 shaders that need to be precompiled.

That combinations library looks good, how'd you use it in this case? It looks like it can possibly be part of the solution but there're still gaps to fill.

Sure here is the module I used to create the shaders and the images ZIP dont forget to install the libraries in the library folder and ref on the index 

/* Babylon this.scene Controller Template */
/* <reference path="{*path*}/Assets/Babylon/Library/babylon.d.ts" /> */

module PROJECT {
    export class SceneController extends BABYLON.SceneController {

        public ready(): void {
            // this.scene execute when ready
            var scene = this.scene;
            let camera :BABYLON.Camera = new BABYLON.UniversalCamera('gamepad', new BABYLON.Vector3(0,0,0),this.scene);
            camera.position = this.scene.activeCamera.position;             
            this.scene.activeCamera = camera;
            this.scene.activeCamera.attachControl(this.engine.getRenderingCanvas());
            this.manager.enableUserInput();
            // // 6 SIDED SKYBOX ////////////////////////////////////////////////////////////////////////////////////////////////////////////// 
            let skybox = BABYLON.Mesh.CreateBox("skyBox", 1000.0, scene);
            let skyboxMaterial = new BABYLON.StandardMaterial("skyBox", scene);
            skyboxMaterial.backFaceCulling = false;
            skyboxMaterial.reflectionTexture = new BABYLON.CubeTexture("SkyBox/TropicalSunnyDay", scene);
            skyboxMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;
            skyboxMaterial.diffuseColor = new BABYLON.Color3(0, 0, 0);
            skyboxMaterial.specularColor = new BABYLON.Color3(0, 0, 0);
            skyboxMaterial.disableLighting = true;
            skybox.material = skyboxMaterial;


            // SKY  MATERIAL //////////////////////////////////////////////////////////////////////////////////////////////////////////////
            var skyMaterial2 = new BABYLON.SkyMaterial("skyMaterial", scene);
            skyMaterial2.backFaceCulling = false;

            var skybox2 = BABYLON.Mesh.CreateBox("skyBox", 1000.0, scene);
            skybox2.material = skyMaterial2;


            // LAVA MATERIAL //////////////////////////////////////////////////////////////////////////////////////////////////////////////
            let ground = BABYLON.Mesh.CreateGround("ground", 1000, 1000, 10, scene);
            let lavaMaterial = new BABYLON.LavaMaterial("lava", scene);
            lavaMaterial.noiseTexture = new BABYLON.Texture("scenes/cloud.png", scene); // Set the bump texture
            lavaMaterial.diffuseTexture = new BABYLON.Texture("scenes/lavatile.jpg", scene); // Set the diffuse texture
            // lavaMaterial.diffuseTexture = new BABYLON.Texture("scenes/water.jpg", scene); // Set the diffuse texture
            lavaMaterial.speed = 1.5;
            lavaMaterial.fogColor = new BABYLON.Color3(1, 0, 0);
            ground.material = lavaMaterial;

            // WATER MATERIAL //////////////////////////////////////////////////////////////////////////////////////////////////////////////

            let groundMaterial = new BABYLON.StandardMaterial("groundMaterial", scene);
            groundMaterial.diffuseTexture = new BABYLON.Texture("Scenes/ground.jpg", scene);
            // groundMaterial.uScale = groundMaterial.diffuseTexture.vScale = 4;

            ground = BABYLON.Mesh.CreateGround("ground", 512, 550, 32, scene, false);
            ground.position.y = -1;
            ground.material = groundMaterial;

            let waterMesh = BABYLON.Mesh.CreateGround("waterMesh", 512, 512, 32, scene, false);
            let water : any = new BABYLON.WaterMaterial("water", scene);
            water.bumpTexture = new BABYLON.Texture("Scenes/waterbump.png", scene);

            // Water properties

            water.colorBlendFactor = 0.5;
            water.windForce = -5;
            water.waveHeight = 1.3;
            water.windDirection = new BABYLON.Vector2(1, 1);
            water.waterColor = new BABYLON.Color3(0.1, 0.1, 0.6);
            water.colorBlendFactor = 0.3;
            water.bumpHeight = 0.1;
            water.waveLength = 0.1;

            // Add skybox and ground to the reflection and refraction
            water.addToRenderList(skybox);
            water.addToRenderList(ground);

            // Assign the water material
            waterMesh.material = water;

            // GRID MATERIAL ////////////////////////////////////////////////////////////////////////////////////////////////////////////// 
            var defaultGridMaterial = new BABYLON.GridMaterial("default", scene);
            defaultGridMaterial.majorUnitFrequency = 5;
            defaultGridMaterial.gridRatio = 0.5;

            var sphere = BABYLON.Mesh.CreateSphere("sphere", 20, 9, scene);
            sphere.position.y = 12;
            sphere.position.x = -6;
            sphere.material = defaultGridMaterial;

            var knotMaterial = new BABYLON.GridMaterial("knotMaterial", scene);
            knotMaterial.majorUnitFrequency = 8;
            knotMaterial.minorUnitVisibility = 0.45;
            knotMaterial.gridRatio = 0.3;
            knotMaterial.mainColor = new BABYLON.Color3(0, 0, 0);
            knotMaterial.lineColor = new BABYLON.Color3(0.0, 1.0, 0.0);

            var knot = BABYLON.Mesh.CreateTorusKnot("knot", 3, 1, 128, 64, 2, 3, scene);
            knot.position.y = 30.0;
            knot.position.x = 6;
            knot.material = knotMaterial;

            var groundMaterial2 = new BABYLON.GridMaterial("groundMaterial2", scene);
            groundMaterial2.majorUnitFrequency = 5;
            groundMaterial2.minorUnitVisibility = 0.45;
            groundMaterial2.gridRatio = 2;
            groundMaterial2.backFaceCulling = false;
            groundMaterial2.mainColor = new BABYLON.Color3(1, 1, 1);
            groundMaterial2.lineColor = new BABYLON.Color3(1.0, 1.0, 1.0);
            groundMaterial2.opacity = 0.98;

            var ground2 = BABYLON.Mesh.CreateGroundFromHeightMap("ground", "Scenes/heightMap.png", 100, 100, 100, 0, 10, scene, false);
            ground2.material = groundMaterial2;

            var skyMaterial = new BABYLON.GridMaterial("skyMaterial", scene);
            skyMaterial.majorUnitFrequency = 6;
            skyMaterial.minorUnitVisibility = 0.43;
            skyMaterial.gridRatio = 0.5;
            skyMaterial.mainColor = new BABYLON.Color3(0, 0.05, 0.2);
            skyMaterial.lineColor = new BABYLON.Color3(0, 1.0, 1.0);
            skyMaterial.backFaceCulling = false;

            var skySphere = BABYLON.Mesh.CreateSphere("skySphere", 30, 110, scene);
            skySphere.material = skyMaterial;


            // NORMAL MATERIAL //////////////////////////////////////////////////////////////////////////////////////////////////////////////
            var sphere2 = BABYLON.Mesh.CreateSphere("sphere2", 16, 20, scene);
            sphere2.position.y = 12;
            sphere2.position.x = 6;
            var normalMaterial = new BABYLON.NormalMaterial("normalMat", scene);
            sphere2.material = normalMaterial;


            //   FUR MATERIAL //////////////////////////////////////////////////////////////////////////////////////////////////////////////
            var sphere = BABYLON.Mesh.CreateSphere("sphere1", 48, 10, scene);
            sphere.position.y = 25;
            sphere.position.x = 12;
            // Fur Material
            var furMaterial = new BABYLON.FurMaterial("fur", scene);
            furMaterial.furLength = 4;
            furMaterial.furAngle = 0;
            furMaterial.furColor = new BABYLON.Color3(1, 1, 1);
            furMaterial.diffuseTexture = new BABYLON.Texture("Scenes/fur.jpg", scene);
            furMaterial.furTexture = BABYLON.FurMaterial.GenerateTexture("furTexture", scene);
            furMaterial.furSpacing = 6;
            furMaterial.furDensity = 10;
            furMaterial.furSpeed = 200;
            // furMaterial.furGravity = new BABYLON.Vector3(0, -1, 0);

            sphere.material = furMaterial;

            // Furify the sphere to create the high level fur effect
            // The first argument is sphere itself. The second represents
            // the quality of the effect
            var quality = 20;
            var shells = BABYLON.FurMaterial.FurifyMesh(sphere, quality);

            furMaterial.highLevelFur = false;


            // FIRE MATERIAL //////////////////////////////////////////////////////////////////////////////////////////////////////////////
            var sphere3 = BABYLON.Mesh.CreateSphere("sphere1", 48, 10, scene);
            sphere3.position.y = 25;
            sphere3.position.x = -12;
            var fire = new BABYLON.FireMaterial("fire", scene);
            fire.diffuseTexture = new BABYLON.Texture("Scenes/fire.png", scene);
            fire.distortionTexture = new BABYLON.Texture("Scenes/distortion.png", scene);
            fire.opacityTexture = new BABYLON.Texture("Scenes/candleOpacity.png", scene);
            fire.speed = 5.0;
            sphere3.material = fire;

            // GRADIENT MATERIAL //////////////////////////////////////////////////////////////////////////////////////////////////////////////
            var sphere4 = BABYLON.Mesh.CreateSphere("sphere", 32, 2, scene);
            sphere4.position.y = 10;
            sphere4.position.x = -12;
            var gradientMaterial = new BABYLON.GradientMaterial("grad", scene);
            gradientMaterial.topColor = BABYLON.Color3.Blue(); // Set the gradient top color
            gradientMaterial.bottomColor = BABYLON.Color3.Red(); // Set the gradient bottom color
            gradientMaterial.offset = 0.5;

            sphere4.material = gradientMaterial;

        }


    }
}

 

Link to comment
Share on other sites

16 hours ago, royibernthal said:

Not sure I understand how this is related, care to elaborate?

Hey man, so did the code work or you need it a more like playground example??

I have many samples of shaders that I'm currently working on to create an standard shader to choose from and combined different materials and options,  here is my first PBR material shaders but I stop doing this because I'm using the shader program @MackeyK24 created on the unity toolkit. so I put this together with the unity toolkit. 

if you want to work on this and need help let me know send me a PM may I can help

Link to comment
Share on other sites

16 hours ago, royibernthal said:

Hey, to the best of my understanding it's not really related to what I'm doing. Correct me if I'm wrong.

Hey man. so what do you want to do is a pre-compile materials library using the shaders programs from babylon?

 

Link to comment
Share on other sites

I do have the same problem.
I'm not using babylon's sceneoptimizers, I have different render qualities insted like low, normal, high and ultra.
different qualities have different maximum lights, shadow qualities and material settings. Different quantity of lights are switched on. (from 2 point lights to 16)
The game switches between qualities every 1s if the FPS is out of 40 - 55 range. It works, the game runs steadily after a while on different computers from 4 old laptop to gamer PC, but in the first 30s, when I recompile all shaders again and again with calling isReady(), sometimes it takes compiling 5s (!!!!!) on a gamer PC, sometimes just 100ms.
And sometimes on a laptop, it grind halts the operating system with ventillators stirring up until it stops dead with too high temperature....

I plan to solve this somehow, maybe storing the different "versions" of every material with every quality setting.
I tried this, but even if i freeze the material, it still recomplies the shader because I enable different number of point lights with differrent quality settings.

If you have some progress of pre-compiling or caching different versions of shaders (Or should I say materials?), please let us know!
 

Link to comment
Share on other sites

Now I had some success with caching materials. The problem solved by setEnabled -ing lights on and off before compiling material.
I store every material in the scene, for every possible render parameter in a material cache.
This works, and now I switch precompiled shaders without slowing down, the only problem is now that reattaching standard pipeline and other post-processes still takes time. But that is only a few msecs, not really a problem.
I use material.markDirty() and material.freeze() now, so materials are freezed after compilation.

Parts of my code:
 

// Renderparams store things like shadow map size, max point lights etc.
// when renderparams change, I set every parameter like shadow map size, turn lights on/off, and // after that I run this:
//(calculates a hash of every parameter, so when it is the same, the pre-compiled materials come // from the cache)

...

	if (renderparams_change) {
		this._prev_renderparam_hash = this._renderparam_hash;
		this._renderparam_hash = JSON.stringify(this.renderparams)+'saltypepper';
		this._renderparam_hash = this._renderparam_hash.hashCode();
		this._refresh_materials();
		this.log( 'RENDERPARAMS CHANGE:' + rpc + ' /// hash:'+this._renderparam_hash);
	}


_refresh_materials: function () {
	for ( var i = 0; i < this.scene.materials.length; i ++ ) {
		var mat = this.scene.materials[i];
		var matnew = this._get_cached_material(mat);
		var save = false;
		if (!matnew) {
			matnew = mat.clone();
			if (!mat.nocache) {
				mat.markDirty();
				mat.freeze();
			}
			this.scene.materials[i] = matnew;
			save = true;
		}
		if (save) {
			this._save_cached_material(matnew);
		}
	}
	},


//The caching functions:

this._materialcache = {};

_get_cached_material: function(mat) {
	var hash = this._renderparam_hash;
	if (typeof(this._materialcache[hash]) == 'undefined') {
		this._materialcache[hash] = {};
		return(false);
	}
	var matfound = this._materialcache[hash][mat.id];
	if (typeof(matfound) == 'undefined') {
		return(false);
	} else {
		return(matfound);
	}
},

_save_cached_material: function(mat) {
	var hash = this._renderparam_hash;
	if (typeof(this._materialcache[hash]) == 'undefined') {
		this._materialcache[hash] = {};
	}
	this._materialcache[hash][mat.id] = mat;
},

//and a simple string hash:

String.prototype.hashCode = function() {
	var hash = 0;
	if (this.length == 0) return hash;
	for (i = 0; i < this.length; i++) {
		char = this.charCodeAt(i);
		hash = ((hash<<5)-hash)+char;
		hash = hash & hash; // Convert to 32bit integer
	}
	return hash;
	};



 

Link to comment
Share on other sites

Taking a break from shoveling out for 2 feet of snow & 3 foot drifts (dreading what the street and sidewalk plows have done to me).  If shadows are not critical to you, you might try a different tack.

Have as many static lights, shining on non-moving meshes as you like.  If the mesh does not move, then the lights are not going to change.  No problem for these meshes.

For meshes that move, assign one of the 4 layermasks out of range of the default layermask.  The static lights will then never affect these meshes, so no recompile.

Assign a light or perhaps 2 or 3 points, that move with the camera.  For timing purposes, you need to move them using a scene.beforeCameraRender().  Assign the same Layermask to the lights, and they will only shine on the moving meshes, and always perfectly light them. Again, no recompile.  I am not yet building full scenes, so not using layermasks for lighting yet.  Here is code I am using now.

var camera = new BABYLON.ArcRotateCamera("Camera", -Math.PI / 2, 1.6, 15, BABYLON.Vector3.Zero(), scene);
camera.wheelPrecision = 50;
camera.fov = 0.265103 // 120mm focal length
camera.angularSensibilityY = Infinity;
camera.attachControl(canvas);   

var camlight = new BABYLON.PointLight("Lamp", BABYLON.Vector3.Zero(), scene);        
scene.beforeCameraRender = function () {
    var cam = (scene.activeCameras.length > 0) ? scene.activeCameras[0] : scene.activeCamera;
    // move the light to match where the camera is
    camlight.position = cam.position;
    camlight.rotation = cam.rotation;
};

Lighting is not the only cause for recompile, but the most common.  Can solve most of the need for something like this, and is a lot less complicated.

Link to comment
Share on other sites

What a great idea!
I will use a combination, where static lights affect only static objects, and dynamic lights affect dynamic objects + the closest static lights and the sunlight is simulated using dynamic lights.
(I used only dynamic lights sorted by intensity and distance, to simulate the closest static lights.)
This will work.
Thank you!

Link to comment
Share on other sites

Also, layermask for cameras does not look like it is being cascaded down to sub cameras.  That means in order to use any 3D rigs with a layerrmask, would probably want to change layermask to a setter, which also sets any sub-cameras.  Assigning the current layermask at the time the rig is assigned would make it leak proof.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...