Is there Support for Object Space Normal Maps?

[mr_pinc]

Is there a way to use object-space normal maps instead of tangent-space normal maps?  If not  do you plan to support object-space normal maps, or world-space normal maps, in the future builds?

[GameMonetize]

This is not in our roadmap but I must admit that the roadmap is kind of flexible.

Can you explain the use case? or perhaps give us some examples?

[mr_pinc]

From my technical artist:
 

our typical project showcases a very complex mechanical object, or set of objects.

we are constrained by draw culls and by content size in megabytes, due to the fact that our app needs to download quickly even with a mediocre connection speed.

since the amount of pieces in our objects is high, the triangle count we can afford per piece has to be minimal. what makes matters worse is often most pieces have unique shapes, so instancing does not completely solve the problem.

we are using normal maps to fake some details, but we run into the common issue of lowpoly geometry smoothing which produces visual artifacts. we can provide examples later if required; but just think of your keyboard which has no definite smoothing seams, but lots of smooth angles and bevels, many of which have to have a geometrical representation in addition to pixels.

 

utilizing the normal compensation approach we can bake the geometry smoothness data, invert it, and combine with the normal map to restore the visual smoothness where required (check handplane.com for some nice examples). we are also averaging the geometrical normals to reduce the artifacts.

but this can only take us half-way, since this approach does not always work 100%. in such cases, using object-space normal map instead would give us full control over the look of our assets - and those arealways either static, or animated on object level.

of course the same could be achieved with world-space normal maps, but this would result in wrong visuals for instanced objects (think of two instanced bolts facing different directions); thus we would have to revert to unique meshes with unique UVs and correct world space normal data in the texture.

 

the cost of one extra channel does not seem too high (meaning we could use B of tangent space normal map for storing something useful, and that of course would not be possible with OS or WS). the precision difference would be negligible as well. we are also not using any detail normals currently, so combining normals would not be an issue. 

now, we understand that adding the OS option could lead to refactoring in many unexpected places. but we keep our hopes up.

 

The attached image  is for illustration purposes only - example of smoothness artifacts resulting from geometry; adding tangent-space normal map here would not hide the issues completely.   this asset in general also illustrates how the artist had to introduce numerous hard edges across bevels to preserve the smoothness. this introduces a lot of sharp lines where things need to be smooth, and adds extra cost to geometry, multiplying the number of verts to be exported

 

[GameMonetize]

Can you provide examples where tangent space normals do not work?

[kenhennen]

Hi Delta. They work, but they have issues, like in the car example above. This is a knows problem of TS; as mr_pinc explained, there are workarounds, but in some cases OS is preferable.

[GameMonetize]

Completely understand

 

Do you have a simple example of ..let's say, a cube and the associated OS texture? To understand what it could take to add it in the engine

[kenhennen]

Here you go Deltakosh

mid that TS normal used in preview is already taking into account the lowpoly normals, so the distortion is minimal (but still present) and the texture is already not reusable for other geometry, as in the OS by default of course.

blablabla_mesh.obj

[GameMonetize]

OK I can see the distorsion

Do you know which maths need to be used?

Here is the shader we use so far:

https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/ShadersInclude/bumpFragment.fx

https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/ShadersInclude/bumpFragmentFunctions.fx

[kenhennen]

right now you're taking normal of the triangle. the missing step (which would be optional with default == false) would rotate this normal based on object pivot or world

it would be in this section:

 

// get edge vectors of the pixel triangle

vec3 dp1 = dFdx(p);

vec3 dp2 = dFdy(p);

vec2 duv1 = dFdx(uv);

vec2 duv2 = dFdy(uv);

// solve the linear system

vec3 dp2perp = cross(dp2, normal);

vec3 dp1perp = cross(normal, dp1);

vec3 tangent = dp2perp * duv1.x + dp1perp * duv2.x;

vec3 bitangent = dp2perp * duv1.y + dp1perp * duv2.y;

[GameMonetize]

I'm absolutely fine if you want to PR something (We can then discuss implementation detail)

[kenhennen]

I really don't [= This is a nice feature to have, but as said before, this is not a blocker and more of a wishlist item for us.

[GameMonetize]

Ok let' s see if someone is willing to help

Anyway, I keep this in my todo list

[V!nc3r]

sorry for the up of this thread but I just see this github notification https://github.com/BabylonJS/Babylon.js/pull/3706 and it seems that object space normals are now support since February (havn't noticed that before) :), so guys here could probably be interested.