canvas 2d Track Node and vrDeviceOrientation Camera problem/issue/question

[jinxsterz]

hey i seems to have a problem of the tracknode.

so i used to example from this playground http://babylonjs-playground.com/#1N9RJY#5

and i used VRDeviceOrientationFreeCamera.

however upon testing, the trackNode doesnt stay at the position of the mesh instead the position seems very off.

any idea how to fix this?

[Wingnut]

Hi Jinxsterz!  Interesting issue.

Let's ping @Nockawa, the author of Canvas2D, and show him your issue.

http://babylonjs-playground.com/#1N9RJY#57

(switch vr cams in line 34/35, as wanted, everyone)

VR cams and ScreenSpaceCanvas2D's (with Group2D trackNode/trackedNode) do not work so well.  I'm not sure if it CAN be "fixed".  Maybe Nockawa will have some words about it.

http://babylonjs-playground.com/#1N9RJY#56

There is a "work-around" that is really not good, but it works.  (mouse control-drag to move-around Arc Cam target)

Let's tour the #56 playground:

Line 4 - the createLabel() function, and below that... it has a helper function called createText().  CreateLabel() is called from line 126.  It is called once for EACH cube.

Line 5 - Notice we use a worldSpaceCanvas2D and NOT a screenSpaceCanvas2D.  Each worldSpaceCanvas2D (let's call it wsc2d)... has a "trackedNode" built-in, automatically.  It is called a worldSpaceCanvasNode.   You MIGHT be able to set its .showBoundingBox = true... to better see it (if you wish).  Now we will use it.

Line 15 - set the worldSpaceCanvasNode (let's call it WSCN) .position... 20 units UP.  This places the labels ABOVE the cube/mesh.
Line 16 - set WSCN billboardMode to all-axes.  This causes the label to always face the camera... or try-to.
Line 17 - set WSCN parent to be mesh, which ==  a cube.

Lines 22-31 - create a Rectangle2d.  In this case, it is ONLY used to make the white border around each label.

Lines 33-54 - set many many variables... preparing to make the actual text and its background, border, color gradients, etc.  (Text2d)

Lines 57-73 - CALL the createText(many parameters) function.  It is located at line 80.

Lines 80-98 - A very-powerful createText() function.  It does many things. It sets text border with color gradients, sets text background with color gradients, sets fonts, positions text within rect2's white border, etc.  Remember... that most of the adjustments for the Text2D part of the label... are done in lines 33-54. 

These WSC2D labels work okay, eh?  I don't have any VR devices to test-with... but I think it will work "better" than SSC2D labels.

Let's look at another playground:

http://babylonjs-playground.com/#1N9RJY#58

Here, I have switched back-to VR Free Cam in lines 105/106. (It WAS set to VR ARC cam in #56 demo).  Also, I disabled billboard mode on the WSCN (the invisible trackedNode that WorldSpaceCanvas2D automatically uses)... in line 16. 

Things look different, now.  Camera is a FREE instead of ARC, so IT acts differently.  And labels no longer automatically try to face the camera, so they act differently, too.

Possibly, with line 16 activated (billboardMode ON)... you will quickly get cross-eyed or wall-eyed... when using VR cameras.  You might get motion sickness.    Be careful 

Canvas2d system is still being developed, though it is already quite powerful.  We MIGHT see future improvements for using ScreenSpaceCanvas2D with VR cameras... but perhaps not.  For now, using worldSpaceCanvas2D (instead of screenSpaceCanvas2D) might be the only solution. 

In general, notice that ScreenSpaceCanvas2D method uses ONE ScreenSpaceCanvas2D... and many Group2d children, each with trackNode.

Conversely, WorldSpaceCanvas2D method uses MANY WorldSpaceCanvas2D (each has a worldSpaceCanvasNode - an invisible mesh that can be parented to other mesh), and uses no Group2D. 

WSC2D still CAN use Group2D, but there was no need, for our demo.  We need no Group2D trackNode.  We use the WorldSpaceCanvas2D.worldSpaceCanvasNode instead.  We simply parented the WSCN to any mesh we wish (in line 17).

I hope this helps.  Sorry for the inconvenience.  Perhaps Nockawa will have comments about the future of ScreenSpaceCanvas2D... when used with VR cameras.  Be well.

[Nockawa]

I did not test ScreeSpaceCanvas with VR Camera and I bet there're some incompatibilities issues, I've filled a bug and I think I'll have time to check your PG and the bug early next week.

[jinxsterz]

thanks for the help guys