Getting strange results for vertices

[UP_AND_ATOM]

I'm a JS developer using Babylon.js for the first time, and right now I'm just trying develop an understanding of what is happening under the hood. I've been programming forever, but have limited 3D experience, so Babylon.js looks like a perfect learning opportunity.

 

I put together a simple program that takes a mesh, finds the screen coordinates of each of its vertices, and overlays triangles on a canvas on top of Babylon.js's renderCanvas. I'm getting some strange results, though. Most of the vertices are right where they should be, but others are completely wrong. When I do this with a cube, all the verticies look right, but if I open up the vertices array, what I'm seeing on-screen doesn't match the data. Screenshot: http://i.imgur.com/POtcnBr.png

 

It's a bit simpler with a plane. All four vertices make a square using only X and Y, which is exactly what I would expect. For some reason when I run my program, the scene coordinates for vertices 0 and 1 end up floating in space, while 2 and 3 are right where they should be. Screenshot: http://i.imgur.com/s4e0HlH.png

 

Not sure if it helps, but it gets even weirder with a sphere. Screenshot: http://i.imgur.com/pQU9yXg.png

 

I guarantee I'm just missing something simple, probably in the getScreenCoords function, but so far I haven't had any luck. Am I just misusing the indices array? It seems that each number in the indices array corresponds to a specific vertex. I have a feeling that's what is failing, but so far I haven't been able to nail it down.

 

Here's the function that does the work:

  engine.beginFrame = function() {    box.rotation.y += 0.005;    box.rotation.x += 0.003;    ctx.clearRect(0, 0, drawCanvas.width, drawCanvas.height);    ctx.fillStyle = 'rgba(255, 43, 0, 0.25)';    ctx.strokeStyle = 'black';        var vertexCoords = [];        var vertices = box.getVerticesData(BABYLON.VertexBuffer.PositionKind);    var indices = box.getIndices();    for (var i=0, len=indices.length; i<len; i+=3) {      for (var v=0; v<3; v++) {        var index = indices[i+v];        if(!vertexCoords[index]) {          vertexCoords[index] = getScreenCoords(BABYLON.Vector3.FromArray(vertices, index), box);          ctx.fillRect(vertexCoords[index].x-4, vertexCoords[index].y-4, 8, 8);        }      }      ctx.beginPath();      ctx.moveTo(vertexCoords[indices[i+2]].x, vertexCoords[indices[i+2]].y);      ctx.lineTo(vertexCoords[indices[i+0]].x, vertexCoords[indices[i+0]].y);      ctx.lineTo(vertexCoords[indices[i+1]].x, vertexCoords[indices[i+1]].y);      ctx.lineTo(vertexCoords[indices[i+2]].x, vertexCoords[indices[i+2]].y);      ctx.stroke();      ctx.fill();      ctx.closePath();    }  };      var getScreenCoords = function(vertex, mesh) {    var coords = BABYLON.Vector3.Project(      BABYLON.Vector3.TransformCoordinates(vertex, mesh.getWorldMatrix()),      BABYLON.Matrix.Identity(),       scene.getTransformMatrix(),       camera.viewport.toGlobal(engine)    );    return coords;  };

[fenomas]

Just a guess but it doesn't look like the camera's transform is getting used. I'd have thought you might need the camera's view matrix (or projection matrix?) multiplied onto the transform you pass to Project. Does rotating the camera affect the outlines?

 

By the way, check out the BJS playground. If you code up your example in there, you can easily share us a link that we can play with, fork and send you updates, etc. It's honestly one of the greatest things about BJS.

[RaananW]

You are using indices correctly, as far as I see. Try with this function:

var getScreenCoords = function(vertex, mesh) {	  var transformationMatrix = camera.getViewMatrix().multiply(camera.getProjectionMatrix());    var coords = BABYLON.Vector3.Project(vertex,      mesh.getWorldMatrix(),       transformationMatrix,       camera.viewport.toGlobal(engine)    );    return coords;  };

It is using the camera's view and projection matrix to get the correct transformation matrix, and is not first transforming the vertex using the transformation matrix, but using it in the project function. Easier to debug, if you use the playground - http://www.babylonjs-playground.com/#S7Z2Z , as fenomas said :-)

 

EDIT:

 

Found your mistake in the calculation. Your getScreenCoords function is fine... You are missing one little thing:

vertexCoords[index] = getScreenCoords(BABYLON.Vector3.FromArray(vertices, index *3 ), box);

index * 3 is the correct position. That was the problem here - http://www.babylonjs-playground.com/#S7Z2Z#1

Edited May 22, 2015 by RaananW

[UP_AND_ATOM]

That's it exactly. Thanks so much

[UP_AND_ATOM]

Just out of curiosity, is there a recommended way to check if a given vertex is visible to the camera? I've tried using scene.pick(x, y).distance and comparing the actual distance between the camera and vertex, but that seems a little hacky and probably won't scale too well for larger meshes.

[RaananW]

do you mean - if it is in the camera's frustum? or is it behind a different polygon?

[UP_AND_ATOM]

I'm trying to find if it's being blocked by anything - other meshes or parts of the same mesh.

[UP_AND_ATOM]

I think the way to go is checking the mesh's normals and seeing if they're facing the camera. I'm working on that now.

[fenomas]

If it's a convex mesh and you just want to know which vertices are hidden by that mesh itself, then basically that's backface culling. In general I think you'd need to check normals per face though, not per vertex.

If you want to find whether vertices are hidden by anything, including other meshes, AFAIK you'd need to analytically check the vertex against each poly that could obscure it, or else use something like a depth buffer (so, basically render the scene).

[RaananW]

The only way I can think of is raytracing (technically, what scene.pick is doing). maybe with fast check (meaning - if the first mesh that was hit by the ray is not his mesh, then it should stop searching for further meshes). But, as You said, would probably not scale the best.