ADT fullscreen depth in 3d world

[royibernthal]

I'm not sure if this would even make sense - but would it be possible for a full screen ADT to always be rendered in front of a skybox and behind everything else?

I'm not talking about placing the adt on a mesh in the 3d world which would miss the point in my case - long story short - I need 2d gui lines to be drawn behind meshes and in front of skybox without drawing them on a 3d plane, which would not look good and cause all sorts of other problems.

[Wingnut]

Hi RB.  Doon well?  I hope so.  What a cool and interesting question you have asked! 

The fullscreen ADT would be in-background mode, yet NOT so far in-background... as to be blocked by the 'far-wall' of the skybox!  COOL! 

Kind of needed, eh?  *nod*  (Later, if you get bored, could you give more details about why a PLANE-based ADT... would not work for you?  thx.) 

Possibly problems keeping it in position?  It needs to be 'welded' to the camera frustum maxZ, so it moves PERFECTLY with the camera all the time, yet behind all other mesh? 

But things go to hell as you move camera nearer to walls of skybox, and ADT plane goes outside the skybox and out of view, huh?  SUCKS, eh?

 

I'm pretty inexperienced, but I would guess... no, that cannot be easily done, or at all.  (but wait for smarter people to comment, at least thru mid-week, of course) 
 

Wanna hear a retarded idea?  I knew ya did.

#1 - try to determine if a Babylon.Layer that is set isBackground = true... can accept a renderTargetTexture (RTT)... as its backdrop "picture".  IF IT CAN...

#2 - move your skybox away from main scene... perhaps at position 5000, 0, 5000.  Put another cam inside it... the one that will produce the RTT.
     note:  I think, using renderingGroups/Layers, you can leave skybox in place, and put cam2 atop cam1 - no special off-site positioning needed.  Set it so only cam 2 can render the skybox, and only cam 1 can render the backgroundLayer.  (You knew that already, I bet.)

#3 - Lock the orientation of the RTT cam... to your main scene cam.  Wherever cam 1 moves, aims, zooms, so does cam 2.

#4 - Somehow feed cam 2 RTT into background Layer... full speed... repeatedly, I suspect.    Video feed from the skybox cam. 

#5 - Test... to make sure your "fake skybox" RTT-fed Babylon.Layer backdrop looks exactly like it did before.  It looks like cam 1 is inside a skybox, but it really isn't.

THEN... you have a CHANCE... to use a full-screen background-mode ADT (advanced dynamic texture - for those who are in-question)... with some Z-index playing.  Maybe. 

Is this slow image-buffer-passing... a good job for GPU or SIMD or something? (increased-horsepower/reduced-instructions)   hmm.  (like I have a clue... pffft.) 

Here's a Babylon.Layer demo that does SOME (left/right) panning-with-cam-drag, so it slightly resembles being in a skybox.  http://playground.babylonjs.com/#1XQKP1#11

With the RTT-generated background layer idea (above), you would need to "stream" the RTT picture... into the background layer... continuously.  Could be bog.  Might not work.  Could be bad Wingnut idea.  heh.  More comments/ideas needed, please.  (Don't let my worthless answer prevent QUALITY answers.)

[royibernthal]

Hey Wingnut, how're you?

That sounds very convoluted

 

I was going for maybe something like defining the order in the rendering pipeline according to a depth property - such a thing could make sense, but it'll require some modifications in the bjs source - I don't think there's anything like that at the moment. (correct me if I'm wrong)

For instance - you could have skybox at depth 0, adt at depth 1, all other meshes at depth 2.

in this case adt will always render in front of skybox and behind all other meshes regardless of camera position and rotation since it's not actually placed in the 3d world.

 

I'm not sure if such a thing is hard or easy to implement though.

[Wingnut]

Skyboxes are boxes, mesh-wise, I think.  https://www.babylonjs-playground.com/#1MLVL#1

You say "in front of skybox"... but don't you really mean... in front of "back wall" of skybox?

If your ADT was 'in front of skybox', then, in this playground, you would need to mousewheel-back until you completely leave the skybox.  THEN you would see the ADT.

ADT needs to be in-front-of the BACK of the box, yet behind the FRONT of the box.  Am I making any sense?  You're wanting a box mesh... to have two different z-indices.

I seems to me... a plane-based ADT... parented to camera... exactly 0.5 +Z ahead of camera lens (check cam.minZ)... would work fine.  hmm.

Maybe I'm confused.  Let's hope smarter people than I... comment soon. 

If you want me to use moderator power to clean this thread... back-to 0-replies... to make other forum helpers pay more attention to it, just say so, no probs.

[royibernthal]

The 3d scene is translated into a 2d image which can be displayed on a 2d screen.

I was thinking that the 2d translation of the skybox from the camera's view point could always be rendered before everything else and is practically in the back, then the adt which is a 2d layer on top of it, then the 2d translation of the rest of the scene.

If you place the camera at the center of the skybox, the rest of its planes that are not within the camera's FOV would not be part of the skybox's 2d translation.

I'm not going for having a different z in the 3d world, which would not make sense as you mentioned, but rather a different z of 2d translations of the 3d world.

It's highly likely that I'm simply misunderstanding the concept or implementation of the rendering pipeline.

 

21 hours ago, Wingnut said:

I seems to me... a plane-based ADT... parented to camera... exactly 0.5 +Z ahead of camera lens (check cam.minZ)... would work fine.  hmm.

Parenting to camera is an idea, I'll need to give that a try

 

21 hours ago, Wingnut said:

Maybe I'm confused.  Let's hope smarter people than I... comment soon. 

If you want me to use moderator power to clean this thread... back-to 0-replies... to make other forum helpers pay more attention to it, just say so, no probs.

You're a smart guy yourself don't worry about it

[Wingnut]

1 hour ago, royibernthal said:

implementation of the rendering pipeline.

Yeah, you could be right.  But maybe the other mesh in the scene... could screw that up.  GPU/post-processing might find it difficult to determine WHICH is skybox mesh (pixels) and which is other mesh.  Interesting.

Thx, R.   I hope the parent-to-cam works for you. 

After experiments, can you tell us... success/failures - in making the plane-based ADT "appear" to be behind assorted scene mesh? 

Likely, the cam-parented plane-based ADT... appears to be in-front-of ALL scene mesh (unwanted).  (caused by parenting plane to camera, thus plane is near to cam)

Forum helpers - this feature idea still needs experts/answers.  (sorry for cluttering the thread).

I think we need a new term.

"Skyback"?    Is that a good term?   A "skyback" is the 2d image used as scene background image... whenever cam is inside-of a skybox, no matter the viewing angle. 

Then, to restate, if I may - Royi would like the full-screen ADT to be in-front-of skyback, but behind all other mesh in the scene.   (I think it would be a nice feature, if possible.)

Issue remains open.     

[Guest]

Also you can have multiple scenes

One scene for the ADT and one for your main scene, and thus you can decide who you want to render first

[Wingnut]

Oooh, 2 scenes.  That might be interesting.  hmm.

[Guest]

You can create two scene and just call

 engine.runRenderLoop(function(){
   scene0.render();
   scene1.render();
}) 

 

 

[royibernthal]

Multiple scenes sounds like a classic solution for this  Thanks

[JohnK]

21 hours ago, Deltakosh said:

You can create two scene and just call

Interesting idea and tried it out but could not get it to work. No idea, at all, how to do it on a PG so tried locally. Copy and paste code below to run on a local computer. As it stands it shows skybox plus gui. Now remove comment from line 107 (//scene2.render()) and scene2 only can be seen. What am I missing? How do you superimpose the two scenes so that the box from scene2 appears on top of the skybox + gui scene?

<!DOCTYPE html>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

    <title>test</title>

	<script src="https://cdn.babylonjs.com/babylon.js"></script>
	<script src="https://preview.babylonjs.com/gui/babylon.gui.min.js"></script>
    <style>
        html, body {
            overflow: hidden;
            width: 100%;
            height: 100%;
            margin: 0;
            padding: 0;
        }

        #renderCanvas {
            width: 100%;
            height: 100%;
            touch-action: none;
        }
		
		
    </style>
</head>
<body>
    <canvas id="renderCanvas"></canvas>
    <script>
if (BABYLON.Engine.isSupported()) {
    var canvas = document.getElementById("renderCanvas");
    var engine = new BABYLON.Engine(canvas, true);
	
var createScene = function () {
    var scene = new BABYLON.Scene(engine);

    // Light
    var spot = new BABYLON.PointLight("spot", new BABYLON.Vector3(0, 30, 10), scene);
    spot.diffuse = new BABYLON.Color3(1, 1, 1);
    spot.specular = new BABYLON.Color3(0, 0, 0);

    // Camera
    var camera = new BABYLON.ArcRotateCamera("Camera", 0, 0.8, 10, BABYLON.Vector3.Zero(), scene);
    camera.lowerBetaLimit = 0.1;
    camera.upperBetaLimit = (Math.PI / 2) * 0.9;
    camera.lowerRadiusLimit = 30;
    camera.upperRadiusLimit = 150;
    camera.attachControl(canvas, true);


    // Skybox
    var skybox = BABYLON.Mesh.CreateBox("skyBox", 800.0, scene);
    var skyboxMaterial = new BABYLON.StandardMaterial("skyBox", scene);
    skyboxMaterial.backFaceCulling = false;
    skyboxMaterial.reflectionTexture = new BABYLON.CubeTexture("https://www.babylonjs-playground.com/textures/skybox", scene);
    skyboxMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;
    skyboxMaterial.diffuseColor = new BABYLON.Color3(0, 0, 0);
    skyboxMaterial.specularColor = new BABYLON.Color3(0, 0, 0);
    skyboxMaterial.disableLighting = true;
    skybox.material = skyboxMaterial;

    // GUI
    var advancedTexture = BABYLON.GUI.AdvancedDynamicTexture.CreateFullscreenUI("ui1");

    var panel = new BABYLON.GUI.StackPanel();  
    panel.width = 0.25;
	panel.height = 0.25;
    panel.horizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_CENTER;
    advancedTexture.addControl(panel);

    var button1 = BABYLON.GUI.Button.CreateSimpleButton("but1", "Click Me");
    button1.width = 20;
    button1.height = "240px";
    button1.color = "white";
    button1.cornerRadius = 20;
    button1.background = "green";
    panel.addControl(button1);

    return scene;
}

var createScene2 = function () {
    var scene2 = new BABYLON.Scene(engine);
	
	// Camera
    var camera = new BABYLON.ArcRotateCamera("Camera", 0, 0.8, 10, BABYLON.Vector3.Zero(), scene2);
    camera.lowerBetaLimit = 0.1;
    camera.upperBetaLimit = (Math.PI / 2) * 0.9;
    camera.lowerRadiusLimit = 30;
    camera.upperRadiusLimit = 150;
    camera.attachControl(canvas, true);

    var light = new BABYLON.HemisphericLight("hemi", new BABYLON.Vector3(0, 1, 0), scene2);

    var box = BABYLON.MeshBuilder.CreateBox("box", {size: 5}, scene2);
	
	return scene2;

}

var scene = createScene();
var scene2 = createScene2();

engine.runRenderLoop(function () {
  scene.render();
  //scene2.render();
});


window.addEventListener("resize", function () {
    engine.resize();
});
        }
    </script>
</body>
</html>

 

[Wingnut]

Well said. 

I was just about to go to work on https://www.babylonjs-playground.com/#1AP18W#4

I can't fathom HOW the 2-scene idea could work.  Royi bought it, but I'm a little more reluctant to swallow that potential solve.  Thanks for the testing, JK!

Here's my lame diagram if anyone needs it.  http://urbanproductions.com/wingy/babylon/misc/royi01.jpg

I removed it from an earlier post, because Royi didn't LIKE or approve it as accurate... so I got worried about Wingnut not understanding the issue, and talking where he had no business talking.  (common for me).   

Ya know, we love Deltakosh's "lunch-time-scan" of forum concerns, but I think, sometimes, he dangles his entire sandwich from his mouth (to free his typing hands), and then he can't see half of his keyboard anymore.  

[Guest]

You should have learnt to trust me since the time we work together!

PG version: https://www.babylonjs-playground.com/#1AP18W#5

(The trick is to use a setTimeout to disable the renderLoop generated by the PG so only our own custom renderLoop is used)

 

@JohnK in your example you have to make sure that the second scene is set to scene.autoClear = false or it will wipe the rendering done by first scene

[Wingnut]

1 hour ago, Deltakosh said:

You should have learnt to trust me since the time we work together!

I DO trust in your genius, DK... but I suspect we have a misunderstanding of the goal. 

You haven't finished it, yet.  https://www.babylonjs-playground.com/#1AP18W#6

The button needs to be behind the mesh, but in-front-of the skybox-made background (the sky-back).  Full-screen ADT.

Can do?  (Wingnut hands DK his wizard's hat and bowl of wizard dust)    Royi?  Am I describing this correctly?

[royibernthal]

Wingnut there you go

https://www.babylonjs-playground.com/#1AP18W#7

 

Deltakosh - Is there a way to share assets between scenes? I'd like to have a single AssetsManager for all scenes.

[Wingnut]

COOOOOOL!   Ok, good, I'll shut up now, I promise. 

Oh wait... why is the sphere acting like it is parented to the cam?  I need to study this.  :/

[Guest]

Assets cannot be share across scene unfortunately. We recently introduced AssetContainers to be able to organize assets INSIDE a scene though

[royibernthal]

I guess could play a bit with AssetContainer in order to move 2 sets of assets to different scenes from the first scene into which I'll load everything.

However it'd require me to hard code which assets are moved, plus it doesn't really feel like a "pretty" solution.

On a side note, perhaps AssetContainer could extend KeepAssets instead of repeating the properties?

 

I'll most likely need to have 3 scenes in the game (another one to display an adt in front of a post process).

I have 1 preloader before the game starts.

Is the best way to solve this to have a different AssetsManager for each scene and manage them in a single preloader?

[Guest]

Yes I think that 3 assetsManager is the way to go