Getting distance of mesh in front of camera

[1glayfan]

This is certainly a newbie question: what is the most performant way to get the distance of a mesh which appears right in front of the camera (in the gaze direction of the camera) ?

Thanks

 

[1glayfan]

Looks like the following approach would work, but the trouble is the api is also picking up ray coming from the controller. I want only the headset. 

 

vrHelper.raySelectionPredicate = (mesh: BABYLON.AbstractMesh) => {
            return true;
        };

vrHelper.onNewMeshPicked.add((pickInfo: BABYLON.PickingInfo, eventState: BABYLON.EventState) => {
            if (pickInfo.hit) {
                console.log(`**** picked mesh: name=${pickInfo.pickedMesh.name}, distance=${pickInfo.distance}`);
            }
        });

 

[waverider]

hy

I think this will be quicker and less expensive compared to ray

var dist = BABYLON.Vector3.Distance(camera.position, mesh.position)

[1glayfan]

Thanks for the tips!

Ok, that's for calculating the actual distance. But first I need to find out which mesh appears in front of the camera though.

[Dad72]

Or:

var dist = camera.position.subtract(mesh.position);

 

[Pryme8]

var sphere = BABYLON.Mesh.CreateSphere("sphere1", 16, 2, scene);

var projectionMatrix = scene.activeCamera.getProjectionMatrix();

var frustumPlanes = BABYLON.Frustum.GetPlanes(projectionMatrix);

console.log(sphere.getBoundingInfo().boundingBox.isInFrustum(frustumPlanes));

for if its visible.

[1glayfan]

Thanks to all who replied!

Well, I still need to determine which mesh which appears in front of the camera.

Simple brute force approach is to enumerate each mesh in the scene and apply the distance-calculating codes which you guys supplied to get the distance. The shortest distance is hopefully the mesh I want.

But is there a faster way to get that mesh which appears in front of the camera ?

[Rodrix3]

Hi 1glayfan, did you find any other solution? It would be awesome if you could post your full final code here to share it with the community for people who have the same question.

 

I would really appreciate it. Thanks!

[1glayfan]

@Rodrix3 Well I haven't found the solution yet unfortunately.

Just to summarize, the real question here is about how to find a mesh in front of the camera and it ideally should be a performant one.
Calculating the distance is not the real issue as you can easily do this once the mesh has been detected.

The approach I mentioned above actually did not work very reliably. It sometimes does not detect the mesh, plus it works on the controller too (which I don't want).

[Dad72]

@1glayfan This should solve the problem. This creates a ray in front of a mesh and as this ray collides with a mesh, it returns the object and the distance. (Performance level is good)

You can change this code for your convenience :

castRay(scene, mesh, camera, showRay)
{
	let ray = new BABYLON.Ray();
	let rayHelper = new BABYLON.RayHelper(ray);
	rayHelper.attachToMesh(camera, new BABYLON.Vector3(0, 0.1, 1), mesh.position, 1000);
	if(showRay) { rayHelper.show(scene); }
	let hit = scene.pickWithRay(ray) || null;	
	if(hit) {
	    let dist = camera.position.subtract(hit.pickedMesh.position);
	    return {"meshFrontCamera": hit.pickedMesh, "distMesh": dist};
	}
	return false;
}

 

[1glayfan]

@Dad72 Thanks! That looks quite compact too.

Question for the code: what should be the value for mesh.position ? At this point we don't know the mesh (in front of the camera) yet, so I am guessing this is position for something else ?

thanks

 

[Pryme8]

http://www.babylonjs-playground.com/#FLP6GY

I could not bare to see you struggling with this anymore.

 

function checkMeshPos(target, inCam, scene){

var frustumPlanes = BABYLON.Frustum.GetPlanes(scene.getTransformMatrix());

if(!target.getBoundingInfo().boundingBox.isInFrustum(frustumPlanes) && inCam){

return false;

}

return ((scene.activeCamera.position.clone()).subtract(target.position.clone())).length();

}

[1glayfan]

@Pryme8 thanks for your reply. But ... yes unfortunately I am still struggling with your code. 

There can be many meshes in the scene, you cannot simply pass in a particular mesh (like a sphere here).

The question is how to programatically determine which mesh. Do you understand the issue ? or am I not making myself very clear.

[Pryme8]

Iterate through an array with the same function and parse the results.

[1glayfan]

It works but it is pulling down the fps tho. 

[alexoy]

I understand you so much. Sometimes in the internet I also ask something like "What time is it?" and the response is like "This color is red"

But don't worry, some people just don't have a complete solution yet, so they provide at least some ideas/code examples to solve at least some part of your question.

 

Here is my try, I hope complete - https://www.babylonjs-playground.com/#KNE0O#82

I've combined ray and distance calculation. So the idea is to cast a ray from your camera, get picked mesh, calculate the distance between camera and mesh, print results in a console. White dot in the screen center is to help to see camera's direction.

[Dad72]

9 hours ago, 1glayfan said:

Question for the code: what should be the value for mesh.position ? At this point we don't know the mesh (in front of the camera) yet, so I am guessing this is position for something else ?

thanks

 

In fact mesh position was the position of the character who looks in front of him. But you can use the camera instead.

[Pryme8]

How many meshes are you testing?
Separate them into chunks, and have it only iterate through sections at a time.  If you are doing like 1000+ do maybe 200 count chunks and switch which one you are processing after every frame.

A PG with your setup would help... This is a fairly simple problem.

[1glayfan]

@Pryme8 That was good suggestions in general, I agreed. I have only about 200 meshes, but the rendering loop is already doing some other computations plus I want to reserve some space for future stuff too (animation, AI etc).

It looks like the approach using Ray from @alexoy would work really well as it does not iterate any meshes.

[1glayfan]

@alexoy Hehehe I like your sense of humor, I thought it was pretty funny.

Everyone is trying to help me, so despite occasional misunderstandings, I really appreciate every single reply.

Well back to your codes, it looks very promising!! Well written and I don't see any issues from first looking at it.

I will give it a try now. Thank you!!

[Pryme8]

http://www.babylonjs-playground.com/#FLP6GY#2

I see no fps drop when testing 200 meshes.

http://www.babylonjs-playground.com/#FLP6GY#3

Hell there is no FPS drop when I run 2000 meshs.

[adam]

4 hours ago, 1glayfan said:

would work really well as it does not iterate any meshes.

Underneath the hood it does.

[1glayfan]

I am sure it does but probably does it more efficiently? 

[alexoy]

Hi, @Pryme8, @1glayfan,

I've got 2 news:

1. My testing shows that with 2000 meshes using a Ray is a little slower than manual iterating through all visible meshes;

2. Iterating through all visible meshes and returning the closest one - it doesn't mean this mesh is right in front of camera. Function will return a mesh even if it's somewhere in a screen corner.

 

So probably both solutions can be potentially improved.

[BlackMojito]

I think you can use a customized acceleration structure like BVH...