Jump to content

Skeletal Animation, Picking Info, and Morph Shapes


Recommended Posts

Whoa! Okay, there are two parts to this.


Case/Issue 1: Scene picking info for animated rigs only obtains picking information from the first frame of the rig animation.


I've setup a test case, http://jsfiddle.net/7x0nkh0g/3/ , where there's this cube that is being animated and moved by this bone. You can click on the cube to change it's colour between green and red. Clicking only works if you click on the mesh where it was at the first frame of animation. Doesn't canvas picking info casts a ray into the scene and looks for active faces everytime? It appears that it's using the first frame of the rig as if it was not being animated at all.

This issue occurs from files exported from both Babylon and Tower Of Babel exporter from blender.

Case/Issue 2: Using the Morph Extension (ver 1.1) and Tower Of Babel to import and animate shape keys created in Blender, on a mesh that also has skeletal rig and animation, morph shape offsets are relative towards the Rest Position (edit mesh position in Blender before the mesh has been set to a pose by a rig).

Using the previous jsFiddle example: http://jsfiddle.net/7x0nkh0g/3/, clicking on the blue circle morph's the cube's shape into a pyramid. Only the top four keys are morphed. Notice how the top four keys try to move towards the rest position of the original mesh instead of following the translated skeletal rig's first frame.

After discovering this, I created another animation rig but this time, I set the first frame of the rig to match the original rest position of the mesh and then have my usual looping animation after that. In Babylon, I then set the rig animation to skip over that first frame. This is sort of a work-around but it actually works! See: http://jsfiddle.net/7x0nkh0g/4/ The cube is being morphed to a pyramid correctly now, while still being animated by a skeletal rig.

I've only tried this trick out on the test case. Can't wait to try it out on my facial shapes on top of a skeletal animation.  :D 

I read in some old threads that JCPalmer might have a working copy of the Morph Extension version 1.2 hasn't been released? I'm getting depreciation messages from BJS using this old Morph Extension: "BJS - [22:36:45]: Mesh.updateVerticesDataDirectly deprecated since 2.3."

On a sidenote for those test cases I've created, if you click on the blue circle to morph the shape of the cube first, then you can't click on it to change it's colour later. JS gives an error "Uncaught TypeError: Cannot read property 'subtractToRef' of undefined". Click on the cube to change its colour first and then clicking the morph next works fine however.

If you need the blender file, I've attached it.


Link to comment
Share on other sites


1. Oh, how interesting. Is there a way to update the CPU before the a picking request? What about casting a ray or collision picking? Are they done on the CPU side and are only aware of the initial state? Surely there must be a way to update them.

2. I guess too  :)

Link to comment
Share on other sites

Glad you are interested in Morphing.  I have a major new release in the works.  This version may still not work correctly against a mesh with a skeleton using the standard BJS animation system, but MORPH (renamed to QueuedInterpolation, QI) will have an integrated bone interpolator right inside the beforeRender.  BABYLON.Animation objects can loaded directly from Motion capture files (only Acclaim format right now) without running them through Blender.  I still use a BABYLON.Animation object for data storage, but it is designed to work best if you only include key frames.  In your example, I think you would only need 1 frame, the last one.  The final pose will be where it ends up, no matter what state is left after the earlier queued end state.
I looked at the behavior you are getting with skeletons & meshes with ShapeKeyGroups.  Part of the problem may be that you are using BJS 2.3.  The MORPH in the repository still calls updateVerticesDirectly().  I have done a lot of work in 2.3 to ensure things work together.  First was upgrading the standard updatingVertices() to be optimized to work with Float32Arrays.  updateVerticesDirectly() is no longer needed, so it was not updated & depreciated.
Next I updated the cpu skinning process, such that morphing & it use the same Float32Arrays to inter-operate as best as possible.
Have done a lot of testing using MakeHuman 1.1.  One can have 50 facial shapekeys.  I have converted my proof of concept VoiceSync tool to be for MakeHuman meshes.  I have not gotten all the face shapes coded for each sound yet, but you can interactively develop them using slider controls, then write the typescript snippet needed to put back into the program onto the console.  It is the first scene I know of where you can actually direct a mesh deforming using controls.post-8492-0-04030600-1448900312_thumb.pn
I also have an interactive tool to bring in Motion Capture files from Carnegie Mellon Graphics Lab http://mocap.cs.cmu.edu/

This is not likely to be released though, in a pre-built way at least.  You have to down load a 1 GB zip file of the entire library, then put the html, JS, and texture files in the top level directory.  You then launch the html via double click.  The individual online files are not CORS enabled.


Really close to getting this to turn over.  Not going to spend time to get the error you have fixed, sorry.

Link to comment
Share on other sites

In your example, I think you would only need 1 frame, the last one.

Assuming you were referring to shape key frames, so that means it would only require the one shape key frame data? instead of having the Base Shape as well? This would make it more efficient on data size.



Really close to getting this to turn over.  Not going to spend time to get the error you have fixed, sorry.


I'm glad to hear that it's coming along well and it sounds like the updated version would fix many issues with the current repository and to be in line with BJS 2.3. Would QueuedInterpolation still be an extension like Morph or would it be integrated directly into BJS?

That's quite already, I figured it would be better to debug from the new version and the old errors can be disregarded. I spent the weekend doing a lot more testing more shape keys and ended up getting more errors with incorrectly morphed shapes and how adjusting one shape key in blender messes up another when exported using TOB and using the Morph extension in BJS. I finally called it quits for the old version when I found out that you can animate individual bones on top of an existing running skeletal animation. That's what I will be using for now.

Thank you.

Link to comment
Share on other sites

  • 5 months later...

@Dal, the exporter, Tower of Babel, is in the extensions.  Shape key groups for morphing have been done for a while.  Vismes are getting close.  The skeleton interpolator is not operational.  I did use it in a test scene recently.  No skeleton or actual talking, but exercising the FACE & WINK shape key groups.  The Eyes are using a rotation POV animation.

I have abandoned motion capture in favor of inverse kinetics inside of Blender.  Not sure when this will be done, sorry.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...