Jump to content

Correct way to manually deform models with skeletons in BabylonJS


Mazax
 Share

Recommended Posts

Hello everyone!

I'm working on a BabylonJS project and having problems with the movement of the 3D-model. I have used tens of hours to find a solution with poor results. The projects goal is to get 3D-model/avatar move according to the movement data from Kinect-sensor. I'm using a C#-server to send Kinect data through WebSocket to JS and BabylonJS for visualization.

The problem what I'm having is that I don't know how to correctly move the model. BJS documentation seems to focus only to ready-made animations and not to the manual deforming of meshes with skeletons. 

I've moved and deformed the skeleton in BJS, but cannot get the meshes to move with the skeleton. Everything works fine in Blender (moving the skeleton deforms the mesh), but not in BJS. I've also tried to make custom animations for the skeleton in BJS to animate the model frame by frame with the data, but it's so complex I'm having a hard time understanding it (with no proper tutorial).

What is the correct way to manually deform models with skeletons in BabylonJS? If the right way is to make custom animations can you help me understand how to build an animation for a skeleton in BJS? (I'm familiar with the BJS tutorials and documentations for animating meshes with no skeletons).

My previous post about the problem

Here is a bit creepy custom animation I'm working on. (How to delete the ready-made animation?)

Here is one debug-model I'm using. How to make skeleton animation from scratch if your model doesn't have one?

 

Thank you for your help.

Mazax

Link to comment
Share on other sites

  • 2 weeks later...

Hiya Mazax...

   Man, sorry you have gotten no replies to this.  hmm.  Not much recent activity in your previous thread, either.

I wish I knew about skeletons, but I am weak in that area (and many other areas)  :)  A forum search for 'bone', 'skeleton', 'weighted', 'armature', etc... might bring results.

Have you had any success/advancement in this issue?  Any satisfaction?  I will try to "assemble" a decent response... and provide some playground links... soon.  Sorry for the delay.  Maybe others will respond to the "bump", too.

Link to comment
Share on other sites

Hi again and thank for your answers!

@Wingnut  I'm glad to be able to inform you that I've found part of the answer I was looking for myself from the BabylonJS GitHub-site. 

I found out that the new BJS 2.5 features include bone rotation and scaling (Added to GitHub 5 days ago, Oct 29 2016 by abow. Link). There I found basically the same demo that @Deltakosh linked previously and used that information to find the solution to set transformations, rotations and scales to the bones in my project. 

What i found out is:

  • You need to update the bone.getLocalMatrix to set the values to bone, however I cant get the bone.getWorldMatrix work the same way. (Why?)
  • To update the mesh with it's skeleton data you need to call the bone.markAsDirty(). This was my main problem what I was having and it isn't told in the BJS documentation (I didn't find it at least).
  • The new BabylonJS 2.5 (now in beta) version has at least some support of the manual bone transformation (Bone.rotate and Bone.scale). Hopefully the new version will cover more about the subject (Bone.translate maybe? And hopefully something to ease the work with the matrices like in Bone.scale).

The code I'm using now to deform the model is:

 

//get translation matrix
var matrix = bone.getLocalMatrix(); 
//set position values
matrix.m[12] = joint.x;
matrix.m[13] = joint.y;
matrix.m[14] = joint.z;
//set mesh to be updated
bone.markAsDirty();

The rotation and scale values can be set in BJS 2.5 by calling the Bone.rotate and Bone.scale.

This problem now solved, my project faces a new issue: I noticed that the skeleton I'm using doesn't have the right rotation coordinate system compatible with Kinect v2 data and it might be constructed wrong in some other way too (I think I know how to fix it and am currently working with it. I'm new to blender, but learning fast!). The next step in my project would be to fix that skeleton. If someone have one spare Kinect v2 compatible Blender armature I'm more than happy to have it ;) 

I'll update my progress again when I get something major done.

 

Thank you for your help.

Mazax

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...