Billboard RayMarch Plane

[Pryme8]

https://www.babylonjs-playground.com/#QJE3YB#20

trying to figure out how to keep the rayMarched object in the center of the plane and keep it looking at the camera.

I also want to keep the camera position update able on the shader so you can "orbit" the objects.

[GameMonetize]

Hello, can you rephrase it? I'm not sure to get what you try to achieve (Shader looks gorgeous by the way!)

[Pryme8]

I basically want to take a plane that always faces the camera if its in its frustum, and raymarch a object on it like lets say a tree.  Then be able to move around in the scene and keep the raymarched object in the middle of the plane but update the position of the camera on the shader so you can view it from all angles.

Im thinking it might just be smarter to use a square.

[GameMonetize]

Can billboard work?

https://www.babylonjs-playground.com/#QJE3YB#24

[Pryme8]

That method will help for sure, I am just getting more familiar with rayMarching environments now :https://www.babylonjs-playground.com/#ZFUV4U#5
https://www.babylonjs-playground.com/#ZFUV4U#8

So I can try this whole idea out.

[Pryme8]

Now I need to figure out how to keep the camera's position/view movable in the scene but still keep the rayMarching happening on in the 'same place'.

[Pryme8]

https://www.babylonjs-playground.com/#ZFUV4U#12

Looks like I just need help with the "lookAt" Math.   I need some sort of calculation that will make it so its not like looking through a window, but at an object and so the axis arrows stay in the center of the plane no matter what your view angle is.
https://www.babylonjs-playground.com/#ZFUV4U#14 <- for anyone who wants to just move around the raymarched scene.  The camera controls are attached to it.

https://www.babylonjs-playground.com/#ZFUV4U#18 <- Basic Planet/Moon

[Pryme8]

I need some math help...
someone please give me an idea here...
 

So let me re-explain. We have a plane that is billboarded and on that plane is a shader that is rayMarching another scene or object.  If we move the camera around in the scene and look at the raymarch plane it is like your looking through a window (kinda).

What I am trying to do is figure out the projection math behind keeping an object centered in the rayMarched scene but still keep the ability to move the camera around it.  So effectively I could make the plane look like a cube that you can walk around, but is only really two polygons with a shader on it.

Im thinking I need to get a reflect angle and get the difference between the cameras target and the output plane then apply the inverse to the rayMarch camera matrix? 

https://www.babylonjs-playground.com/#ZFUV4U#22
^^^ Ive kinda got it, but there seems to be a "wiggle" or like some sort of parallax effect on the position of the rayMarched Object.

NEEED INPUT!!! JOHNNY 5 NEEDS INPUT!

[Pryme8]

https://www.babylonjs-playground.com/#ZFUV4U#24

Kinda got it where I want it... I should be able to make a tree or something now and give that a test.

[GeorgeCostanza]

@Pryme8 Sorry if I'm necroposting. I stumbled across this forum thread when searching through google for a way to ray march over billboards. I'm not using Babylon.js, instead I'm using a different game engine that still uses OpenGL. I'm a bit unclear on the "lookAt" part. The following line in particular:
 

shader.setVector3('camTarget', camera.getFrontPosition(10));

I'm not sure what does the following line do? What I'm currently doing is just passing in the camera direction and the camera position to the shader, but it seems like I'm running into an issue with the ray marched sphere moving with the camera. Attached is my attempt at ray marching over billboards.

 

[Guest]

Which engine?

[GeorgeCostanza]

Jmonkey Engine

[GeorgeCostanza]

I guess the better question is what is the relation of the camera Target and camera Direction?

[Pryme8]

this is what is needed to determine your ray direction.  If you look at the main section of the shader code you will see:

vec3 rayDir = getRayDir(camPos, normalize(camTarget - camPos), p);

which does this function:

vec3 getRayDir(vec3 camPos, vec3 viewDir, vec2 pixelPos) {

vec3 camRight = normalize(cross(viewDir, vec3(0.0, 1.0, 0.0)));

vec3 camUp = normalize(cross(camRight, viewDir));

return normalize(pixelPos.x*(camRight*-1.) + pixelPos.y*camUp + CAM_FOV_FACTOR*viewDir);

}

so the cam target, which is just a positions in front of the 'physical' camera in the scene.  So with quick maths you subtract positions and normalize to get your vector for forward.   I think you could essentially just pass the camera forward (global) forward vector but his works well.

Think of it as the cameras view direction as denoted in the getRayDir function.

A Normalized Global Camera Rotation Vector.

 

[GeorgeCostanza]

Thanks so much for the much needed help. I think I understand now. My issue, apparently it seems was the "Y" component of my camera position and the camera target needed to flipped, as in I needed to multiply by -1. So weird, could be something to do with the face  that I'm using GL_POINT instead of quads.

[Pryme8]

Glad my research ended up helping!

I should publish the end result of this, which was raymarching organics like trees and bushes.  I was never fully able to make things look truly organic but I was getting close.  The idea was to be able to eventually have fully 3D trees being rendered on a billboard, or an outside scene through a window.

[GeorgeCostanza]

Hello! Back again, now with more questions! So now that I was able to successfully implement that ray marched cube I'm running into a different issue now, something I also noticed  that seems to appear also in both mine and your implementation. The issue that seems to occur is that when the ray marched objects are far away, maybe >30 world units, upon moving or rotating the camera the ray marched object seems to "shift" towards the edges of the billboard, in more extreme cases completely going out of the billboard.

Here is the use case in Babylon.js where the ray marched object is given a white background to better visualize the issue that is occurring. As show in the picture, the camera is rotated downward and is some ways away from the ray marched object. The object seems to be "shifted" towards the top edge of the billboard. This issue seems to be more pronounced the further away the camera is from the ray marched object.