Combining baked and dynamic shadows nicely

[erixon]

Hello, I wonder if there some possible solution for this in babylonjs, probably too tricky but I give it a try.

Baked shadows look nice and smooth but very static, and dynamic shadows easily become expensive, so question is how to mix these to come somewhere near a realistic result.

Can you merge lightmap-textures and dynamic with shadows with like "darken" without getting the double effect? What I can see it become very complex with two renderings, mixing and postprocessing and stuff.

kind regards
Tomas

[Wingnut]

Hi @erixon... cool idea.

Allow me to clarify.  When there is a baked shadow (let's pretend it is on the ground material)... you don't want an incidental dynamic shadow... to cast a "second layer" of shadow upon the baked shadow area?   In other words, wherever there is a baked shadow, don't allow a dynamic shadow?  I think that is what you are talking about.   (sorry, if I got it wrong).

If this is correct clarification, then I have an idea.  Don't bake the shadows into the ground texture.  Instead... make a separate ground texture and ground plane... that contains ONLY the shadows... on a transparent background (probably an 8-bit-color or maybe a 2-bit-color black and white image - it will be small bytes-sized - good).

Now use alpha transparency on the "static shadowMap" and position it a tiny bit above normal ground plane/material.  It should "look like" baked shadows on the ground texture... but it really isn't so.  Instead, it is a "shadow layer" atop a standard ground/texture.

THEN... turn OFF ground.receiveShadows (used in line 57 of this playground demo) and/or perhaps turn off myStaticShadowMapPlane.receiveShadows, too.  

There is a problem with this.  ground.receiveShadows = false... will disallow ALL dynamic shadows for entire ground.  So, you might want to make multiple ground planes and shadow maps... tiled.  Some parts of the ground... you would allow to .receiveShadows, and some parts... not.  *shrug*

Will your ground be heightMapped?  Bumpy?  If so, tiling heightMaps is not overly easy.  I am currently fighting-with a method to do this (make tiled sections of heightMap from a single-image FULL heightMap)... but I have run into troubles with that.   (You can read backwards up that thread, where I tell every little problem I have hit, so far.  I'm a talker.  heh.)

Just some ideas... probably not very good ones.    I'll keep thinking.  Others will comment soon, too, likely.

[Wingnut]

Ya know, after doing more thinking... maybe you would NOT need to turn-off ground.receiveShadows... only myShadowLayerPlane.receiveShadows = false.

(yay, no tiling needed!)

Know why?  Because... even if a static shadow IS being cast on the ground UNDER a shadowMapLayer shadow... it won't be seen.  The shadowMapLayer texture is not transparent at that location (it is only transparent where there ISN'T any shadow). 

In theory, a ground-based dynamic shadow (possibly being cast by a walking character)... will not double-shadow... even when parts of that dynamic shadow are beneath a static shadowMapLayer shadow.  (in theory)    (maybe).

Update:  Oh wait, the shadows on a shadowMapLayerTexture... WOULD need to be somewhat transparent, right?  Or else we would not see the grass and leaves of the ground's texture... within the shadow area.  CRAP!  hrmph!   hmm.  *sigh*.  I'll keep thinking. 

[erixon]

Sorry for not being clear, I have baked shadow/light information in a secondary texture on all environment. So if a table has baked shadows 50% darker on the floor, I don't want the dynamic shadow of player add another 50% making it darker. This sounds though like quite complex shader programming. In theory I could render out everything in hard black shadows and mix this 50% with another render that has no shadows at all, causing shadow opacity of 50%. With baked light/shadows you can render out endless complex smooth lighting, the problems come when you want to add dynamic shadows. Adding dynamic shadows to a whole scene is often too expensive.  

 

 

[GameMonetize]

If you just want a mesh to not receive shadows, them set mesh.receiveShadows = false

 

[V!nc3r]

It seems that you want the equivalent of dual lightmapping system of Unity 4 for example ?

That what they said about :

Quote

Dual Lightmaps

Dual lightmaps is Unity’s approach to make lightmapping work with specular, normal mapping and proper blending of baked and realtime shadows. It’s also a way to make your lightmaps look good even if the lightmap resolution is low.

Dual lightmaps by default can only be used in the Deferred Lighting rendering path. In Forward rendering path, it’s possible to enable Dual Lightmaps by writing custom shaders (use the dualforward surface shader directive).

I don't know if it is even possible to do that in WebGL ? It looks difficult to implement

[GameMonetize]

This requires WebGL2 multi render to be efficient 

[BitOfGold]

I had the same exact problem, and solved it. And lost that hacked shader...
As I remember, there is this LIGHTMAP definition in default.fragment:

#ifdef LIGHTMAP
   vec3 lightmapColor = texture2D(lightmapSampler, vLightmapUV).rgb * vLightmapInfos.y;

#ifdef USELIGHTMAPASSHADOWMAP
   color.rgb *= lightmapColor;
#else
   color.rgb += lightmapColor;
#endif
#endif

this adds or multiplies the lightmap after composition, which was wrong. light should be added at the lighting phase. Oh I have fouund the hacked shader here:
 

    // Lighting
   vec3 diffuseBase = vec3(0., 0., 0.);
   lightingInfo info;
#ifdef SPECULARTERM
   vec3 specularBase = vec3(0., 0., 0.);
#endif
   float shadow = 1.;

    //Light map (Taci)
#ifdef LIGHTMAP
   vec3 lightmapColor = texture2D(lightmapSampler, vLightmapUV).rgb * vLightmapInfos.y;
#endif

#include<lightFragment>[0..maxSimultaneousLights]

    //Light map (Taci)
#ifdef LIGHTMAP
#ifndef USELIGHTMAPASSHADOWMAP
diffuseBase += lightmapColor;
#ifdef SPECULARTERM
specularBase += lightmapColor;
#endif
#endif
#endif

Then only one thing left, in the lightFragment code there was a hack to modify all the DirectionalLights (which were pre-baked) to add only to the the float shadow if LIGHTMAP was defined.
I hope I can rewrite it again. It looked very cool 

 

[BitOfGold]

There are several other forum topics about calculating only the shadow and not the light, like this:

The solution is the same. leave out light calculation only add to shadow value in the lightFragment code.

[BitOfGold]

Can someone help? If I could add only one property to every light: .lightMapsExcluded
If this is defined on the light (LIGTHMAPSEXCLUDED{X} in lightFragment.fx and LIGHTMAP on the material) then you can leave out the light calculation, and only add to the shadow.

Where can I define this new property?
(not in babylon.light.ts)

Quote

Edit: ok I found it in materialHelper.ts

 

[BitOfGold]

Well guys it was not easy. But it works perfectly now.
https://www.bitofgold.com/lightmaptest/
A little demonstration to show what is wrong with the previous ligthmap calculation - no ambient lighting, no other light, no specular highlights.

erixon: the mode you are looking for is mode 3 (no light calculation, only shadow. mixing baked lightmap and dynamic shadows)

I will make a pull request for this.
Does not break the previous lightmap calculation.

 

material.lightmapTexture needs to be set to activate light map.

light.lightmapExcluded
if true, gets de diffuse lighting from the light map. adds specular highlights mixed widh baked and dynamic shadows.

if false, lightmap calculation is not affected. ("Old" lightmap, material.useLightmapAsShadowmap works as before (simply multiplying color.rgb after composition). Does not look right.)

light.lightmapNoSpecular
if true, does not even calculate the light functions, only the baked lightmap + dynamic shadow. (no specular then from this light, only the baked diffuse)
 

[V!nc3r]

Great job ! In your demo only the key Enter works here ; space or 1 doesn't, i don't know why.

[BitOfGold]

Thanks it works now after refresh! (in Firefox too)
 

[GameMonetize]

This is really great!

I just have concerns regarding naming. Names are not self explanatory enought to me

[BitOfGold]

I just refreshed my demonstration:
name of option of the light is .lightmapMode
https://www.bitofgold.com/lightmaptest/