Loading animations on the fly

[Dad72]

Hello,

I have over 50 animations of a character, but the final file is larger than 25MB I would like to (if possible) to export the character with the skeleton without animation and Babylon, loading animations on the fly as required.

Have you any ideas? Is there not something in Babylon which would loading animations on a skeleton. Because it costs a lot to export a character with much animation.

 

[JCPalmer]

I have been trying to work this problem.  It is really 2 problems.

First is the lack of interpolation of bone animations.  This means you cannot just represent taking a step with just a couple of frames, and let BJS fill in the intermediate frames.  This makes the issue massively bigger.

Second, is separation of the animation from the skeleton in the loading process.  I have added a method in skeleton to copy animation ranges from one skeleton to another.  So the animation could be in a separate .babylon file.  Blender is the only exporter which supports animation ranges so far.  As soon as I saw @Deltakosh had implemented then, I knew that I was going to greatly improve them & finally support multiple Blender Actions using them.

Other things I have done were try to load mocap files on the fly.  I have that mostly implemented for .bvh (parse is done), but I have abandoned that.  I just decided to improve the Blender exporter more, so that it was friendly to inverse kinetics.  With IK, multiple actions, and only sending the key frames, I am close to finishing (hopefully) a massively improved animation system. 

This is even without mentioning multiple independent shape key groups, think Face for expressions & speech, eye lids, left hand & right hand.  With these being independent, your animations of the skeleton can be combined with these at runtime.  No multiple walking animations, 1 with right hand holding something, another with fists, and another giving you the finger.  I cannot get the bone interpolator to work to date though.

From a 3DMax standpoint, if you implemented animation ranges in the exporter, then you have a better chance at some improvement.  Of course, 3DMax would have to allow multiple animations per character, like Blender Actions.

[Dad72]

Thank you for all your information JC. it seems not an easy task for what I understand.

You said that Babylon does not support multiple animations simultaneously? arf, if so we want to have face expression with hand movements, it is not possible.

I think it'll be significant obstacles for my new project.

Thank you

[JCPalmer]

If you are implementing hands / expressions with a skeleton, it can be very difficult.  Actually, in an old thread, I saw that you could implement different animations concurrently, but only at a bone not as a whole skeleton.  That is probably not a simple operation. You might have to invent something like a bone group, that only starts a skeleton animation on certain bones.

[Dad72]

I think of an idea.

If we export a character with all the animations and serves as reference. Then each extracted animations contained in the character.babylon file to separate files json (walk.anim, run.anim). So when you export the same character without animations, with Babylon can add animations on the fly with loadJson(walk.anim) and push() on character.babylon. I think it could work ! What do you think ?

 

[DezOnlyOne]

If you get this working, please let me know. I am in desperate need of something like this. my babylon files, including animations get over 100MB.  

[GameMonetize]

Remember that babylon files love being gzipped

[DezOnlyOne]

10 minutes ago, Deltakosh said:

Remember that babylon files love being gzipped

While that would be great for controlling file sizes, it doesn't help for memory issues. Having a mesh that can just have animations applied to it would be a better solution. GZIP of my files did not help as much as hoped for file sizes

[Dad72]

Yes, gzip is one idea also.

But as said DezOnlyOne and are 100 MB file, gzip will not be enough. and the scenes with heavy load will be 2, 3 characters. the lag is inevitable as well.

I think that the separation of animations one by one and load them on the fly as needed would be the best choice when you have a large list of animation which are not all necessary for a character to other.

 

[JCPalmer]

40 minutes ago, Dad72 said:

I think of an idea.

If we export a character with all the animations and serves as reference. Then each extracted animations contained in the character.babylon file to separate files json (walk.anim, run.anim). So when you export the same character without animations, with Babylon can add animations on the fly with loadJson(walk.anim) and push() on character.babylon. I think it could work ! What do you think ?

 

I would not change the extension.  Keep the .babylon extension & put the skeleton in each animation range file, where the name of the file would be "rangename.babylon".  Load it using SceneLoader.Append(). In the callback, call permSkeleton.copyAnimationRange(tempSkeleton, "rangename").  Delete the tempSkeleton.

The skeleton is not very big relative to the frames, and the skeleton is used as the directions to do the copy.  It also warns when the skeleton does not match.  I could already see that noobs would try to load animation ranges from dissimilar skeletons.

You could also cut out many steps if you just made a "split off animations" option in the exporter, which did everything.  If you could modify the exporter yourself, you do not control your own destiny.

[gryff]

Following this with interest. My "Blue Lady" had 900+ frames and the file size went up to 9+Mb

So being able to load animations on demand would be nice.

Quote

I could already see that noobs would try to load animation ranges from dissimilar skeletons.

Yes Jeff, I can see that coming

Quote

With IK, multiple actions, and only sending the key frames, I am close to finishing (hopefully) a massively improved animation system. 

I hope it will be soon Jeff. I have created an IK rig on the Make Human "game rig". Looking forward to this.

Will be following this thread.

cheers, gryff

[JCPalmer]

@gryff, Thought this was a 3DMax thread, but screw them. 

FYI, Make Human 1.1 is either out or set as the stable dev.  The unstable dev is now 1.2.  I have not gotten the final 1.1 yet.  Think I am September snapshot.  The reason I mention it is you should definitely not put anything more into 1.0.  As far as the "game rig".  I have made my decision to go with the 53 bone, 1.1 Game rig from MH itself, and not try to modify something built by MHX2.  There are 30 finger bones, but plan to dissolve them post Blender import.  That leaves 23.  Think that should work on mobile, but have not tested.

Right now, I am trying to convert a Blender Pose library to shapekeys for the fingers.  Trying to do a pose, apply as shape, re-parent mesh to armature wt automatic weights in a loop causes a bug.  Only the first time works.  Once you have the shapes, copying them is also not easy.  I have written a key to a file in the same format as MHX2 uses to set the face shapes on import.  It is a little hacky without any code changing.

Another copy method that does not rely on MHX2, is 'Transfer Shape Key'.  Means you have a source mesh though.  You can install a zip file instead of single .py file for an addon.  I could put a .blend where the conversion was done into a zip file with TOB.  Would still have to find a place to call a script to do all the transfers.  I see putting geometry in blends inside a zip file is how Manuel Bastioni, mod of Make Human forum, made his new system work.

 

[Dad72]

If I extract all the skeleton part of babylon file to add another file and loads after my character is loaded, is that it makes a difference in load times?

Then just make clones of the skeleton to load on other characters who obviously carries the same skeleton without animations.

The goal is to reduce loading time. At the present time, I do not see any other solution. 

[Dad72]

I make a fun a quick form to retrieve all animations with the skeleton and add them to a file created with PHP

index.html

<!DOCTYPE HTML>
<html>
<head>
	<title>AnimationsExporter</title>
	<meta HTTP-EQUIV="CACHE-CONTROL" CONTENT="NO-CACHE" />
	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />	
	<script src="http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js" type="text/javascript"></script>
</head>
<body>		
	Babylon file:<br /><br />
	<input type="file" id="meshAnim" name="meshAnim" value="">  <button>Valider</button>
	<script type="text/javascript">	
		document.getElementById("meshAnim").addEventListener('change', function(e) {			
			var file = e.target.files[0];
			if (!file) { return; }
			var reader = new FileReader();
			reader.onload = function(e) {				
				var contenue = e.target.result;
				var sceneToLoad = $.parseJSON(contenue);				
				var animationKeysAll = sceneToLoad.skeletons[0].bones;				
				//treatment here to create the file .animation.babylon				
				$.ajax({ type: "POST", url: 'write.php', data: "anim="+JSON.stringify(animationKeysAll)});				
			};
			reader.readAsText(file);
		});
	</script>	
</body>
</html>

write.php

<?php
$animation = $_POST['anim'];
file_put_contents("./skeletons.animation.babylon", $animation);
?>

.htaccess (necessary to override php limit)

php_value post_max_size 2000M
php_value upload_max_filesize 2500M
php_value max_execution_time 6000000
php_value max_input_time 6000000
php_value memory_limit 2500M

Use:

BABYLON.SceneLoader.ImportMesh("", "character/", "character.babylon", scene, function (newMeshes, particleSystems, skeletons)
		{	  
			var mesh = newMeshes[0];			
			
			if(mesh.isReady() && skeletons[0]) {
				$.getJSON("characters/skeletons.animation.babylon", function(json) {
					skeletons[0].bones = $.parseJSON(json);
					var skelette = skeletons[0];
					scene.beginAnimation(skelette, 0, 100, true, 1.0);
				});	
			}
		});

Remains to load the file once and use it when you create the characters that have a skeleton without animations and add animation to load the file only after the character is displayed.

The goal is that the character is created faster.

I post this, it might be useful to someone.  

[gryff]

Ty for posting that code Dad No experience of using AJAX - but maybe I can spend some time pondering your code.

So many things to do - and so little time.

cheers, gryff

[gryff]

@JCPalmer - yeah I have seen the bastoni plugin for Blender but not tried it. As for MH 1.1, I was told back in October 2014 that it would be out in November 2014. When I see it - I will believe it

And finger bones have always been a problem

cheers, gryff

[Dad72]

It's fixed all post Griff and tested ok. I add an .htaccess file to override php limits

 

[Dad72]

I add also post its use:

        BABYLON.SceneLoader.ImportMesh("", "character/", "character.babylon", scene, function (newMeshes, particleSystems, skeletons)
		{	  
			var mesh = newMeshes[0];			
			
			if(mesh.isReady() && skeletons[0]) {
				$.getJSON("characters/skeletons.animation.babylon", function(json) {
					skeletons[0].bones = $.parseJSON(json);
					var skelette = skeletons[0];
					scene.beginAnimation(skelette, 0, 100, true, 1.0);
				});	
			}
		});

On a character of 14 MB, the animations file (skeletons.animation.babylon) is 12 MB. My character that load quickly and animations are loaded without the user see.

Test Ok

[gryff]

@Dad72: Encore une fois merci pour vos réponses et des solutions.

Maintenant, je dois trouver un peu de temps pour expérimenter

cheers, gryff

[Dad72]

I feel that this does not work : skeletons[0].bones = $.parseJSON(json);

We can replace a skeleton [0] .bones by others having the same numbers of bone, but different animations?

Because I did this, I get an error: TypeError: o.getParent is not a function

Thank you

[JCPalmer]

Parsing of JSON data is only the first step.  Think you are replacing BABYLON.Bone objects, that ran through a constructor, have prototype methods,  & internal state with a nested dictionary,{}, with stuff in it.  The animation frames are just an array of 16 Javascript numbers.  BABYLON.Matrix objects also have prototype methods, & are back by a Float32Array, so they are SIMD for Javascript 7.

I cannot read the language you show, but you need to use Sceneloader.Append(), not a raw call to "JSON", to process the info into usable BABYLON objects.  Do your swap in the callback.

[gryff]

More interesting thoughts on this issue Jeff.

cheers, gryff

[Dad72]

4 hours ago, JCPalmer said:

but you need to use Sceneloader.Append(), not a raw call to "JSON", to process the info into usable BABYLON objects.  Do your swap in the callback.

I'll try that. thank you JC for information.

[JCPalmer]

Gryff, if this is where you are listening for animation news, the Make Human 1.1 release is starting update the packaging process.  Ubuntu was first.  I tried it & though I like getting it from a repository, I got something ancient (you cannot make replies to announcements, so created response in bugs forum).  Point is that 1.1 is essentially done.  Time to believe.

I have also got my work process for finger shape keys to make it all the way into BJS, in spite of a bug in Blender.  I have written my own data file in the same format as the face shapes file.  This is by far the best way to go.  The MHX2 add-in takes care of whether or not it is a high or low poly mesh, and I do not need put stuff in a blend file hidden somewhere.  The shapes I have are not good yet, but they are in a pose library, so they will get improved at some point.

Had to modify the MHX2 program to load another file, but now working on pulling that out.  Am planning on just calling the MHX2 code from buttons I put in TOB.  Will put in only the bare minimum to get the buttons on the data properties tab of the exporter itself.  Code that calls the MHX2 code will be in a separate file (TOB moving to a zip file to support this).

[gryff]

Jeff, only tried using the mhx2 format once with MH 1.02. Waiting until 1.1 is released as I want to make sure that the MakeClothes and MakeWalk plugins will still work in Blender, and see what rigs will finally be available. I still like the "game rig", but I guess that that MH1.1 may include a "CMU rig" which should work nicely with all the CMU .bvh files that I have as it should be possible to load mhx2 files into MH1.02.

Dad72: take a look at at this post by Jeff in the "generic animations" thread  about loading animations and two different skeletons

cheers, gryff