Bvh directly to babylon.js

[benoit-1842]

The wisdom of wingnut strongly suggest me to change the title of my previous topic changing animation. I am using a kinect to capture bvh data of my body and I am using those data to put in an avatar in Blender that i transform to a .babylon file and voila i have an awesome avatar moving with my movement. The only little problem is when i am building the .babylon file in blender it takes around 15 minutes to have a new moving avatar. I would like to present my workflow to an expo this summer but I would like to be able to capture the bvh of the movement of the people and put it directly in my model in babylon.js..... I will work on that project and show you the details of my process. If anybody have an idea of maybe a workflow that can be helpful please send me a sign...

Merci,

Benoit

[JCPalmer]

It is possible in theory.  You would need to put the mesh, which has the MatricesWeights & MatricesIndices, and a skeleton in a .babylon or .js from a Blender exporter.  That is just a mesh with the armature, but no animation.

 

If you could write javascript that took this data source, then construct & attach animations for each bone.  Here is a sample where the animation is constructed with inline data.  You would have to do this from your data.

function getBone(skeleton, nameOfBone){    for (var i = skeleton.bones.length - 1; i>= 0; i--){        if (skeleton.bone[i].name === nameOfBone) return skeleton.bone[i];    }    return null;};var skeleton = scene.getSkeletonByName("my name");var bone = getBone(skeleton, "left-arm");var animation = new BABYLON.Animation("anim", "_matrix", 30, 3, 1);animation.setKeys([{frame: 1, value: BABYLON.Matrix.FromValues(0.9967,0.0513,-0.0632,0,-0.0495,0.9984,0.029,0,-0.0646,0.0258,-0.9976,0,0,0.9036,-0.0008,1)},{frame: 2, value: BABYLON.Matrix.FromValues(0.9972,0.0652,-0.0372,0,-0.0637,0.9972,0.0401,0,-0.0397,0.0376,-0.9985,0,0,0.8986,-0.0016,1)},{frame: 3, value: BABYLON.Matrix.FromValues(0.9983,0.0569,-0.0135,0,-0.0564,0.9976,0.0395,0,-0.0157,0.0387,-0.9991,0,0,0.8937,-0.0025,1)},  . . .{frame: 45, value: BABYLON.Matrix.FromValues(0.998,0.035,-0.0523,0,-0.0344,0.9993,0.0132,0,-0.0527,0.0114,-0.9985,0,0,0.9578,0.0024,1)}]);bone.animations.push(animation);

[dbawel]

The only way to do this withminimal delay( approx 70ms) is to build a plugin using the kinect sdk using Websocket IO through a node.js server. Further details than this would take a couse in new server language to teach.  There is currently no other way to go from the kinect to babylon unless you write out .babylon file or create your own file parser for .bvh.

[benoit-1842]

Hi guys thanks for your interest.....  What I want to be able to do it`s to take a bvh and "put it" in a model in babylon without having to pass by Blender.  My workflow : Brekel or mmc ----> .bvh -------> animated .babylon model.  What I want is a type of drag and drop bvh that change the animated model very quickly instead of             Brekel or mmc -------> bvh ----------> Blender (makewalk plugin) ----------------------> create a .babylon file 15 minutes.  I am pretty sure that we can bypass the Blender phase and have directly a drag and drop in a babylon context.  My insight is to have two models one that`s only the bvh and the other one is the actual model with the same bones name has the bvh model file....  

 

Thanx a lot,

 

Benoit

[Wingnut]

Cooool!  I thought you were talking about .bvh files.  

Realtime Kinect---> BJS?  Yeeeah, I like that idea.  Max Headroom, v7.5    I imagine that the more vertices in the skinning mesh, the worse the performance.  Keep your skins simple and blocky, and get many skeletons on-screen at the same time, maybe.  Interesting project.

http://playground.babylonjs.com/#1NSCXV#4

Our 4-skeleton PG demo does ok, performance-wise... but I'm on a brand new box, so EVERYTHING looks wonderful to me, at this time. 

[benoit-1842]

Yeah Wingnut I am talking about drag and dropping bvh files......  But this idea of Kinect and babylon.js looks like a cool project and this is something that inspire me a lot......  Because my dream it`s to keep everything internally in babylon.js........ But yeah my first idea was to take .bvh files that I took elsewhere made with a kinect and drag it in a model in a babylon.js context..

But wow kinect + babylon.js is promising...

 

Thanx,

Benoit

[dbawel]

It's probably easier to pass data directly to BJS rather than loading a table and parsing.  We are working on realtime multiuser apps, and once you place a node.js server communicating using a Websocket IO connection, this is doable.  However, you should be able to load the x,y,z transforms in a loop - but it will be a HUGE script.  This would be a much better function as a switch working inside the parsing of a .babylon file.  I don't believe worker threads are really working yet, but this would also be necessary as the overhead of an uncompiled script parsing so much data for a skeleton might be CPU intensive.  If you get real-time running, you'l easily be able to drag and drop as the overhead will already be managed.

[JCPalmer]

db,

Are you asolutely positive that a server is required?  I saw something on CocoonJS HTML5 feature list http://support.ludei.com/hc/en-us/articles/200807787-HTML5-feature-list that led me to think you can do websocket IO directly from the game.  I have not this from a JS environment before.

 

Jeff

[dbawel]

You are correct, you don't require a node.js server to stream, just the Websoket IO connection.  However, I highly recommend moving from a Linux server set to a node.js server as it is FAR more efficient for streaming.  Also, there are many scripts available once you make the switch and don't have to reinvent the wheel on this.  It has certainly been an eye opener to me and the group I work with, and we now can write whatever we need to connect and stream, without all of the overhead of old server technology.