Evaluating Babylon.js for interactive 3d heatmap but seems slow

[vegabook]

I am a data scientist needing to visualize large multi-dimensional tensors, and flat-plane geometries are no longer sufficient for this as the data has become too big. I am thus (just beginning) exploring browser-based 3d engines because three dimensions plus animation can help to go to four dimensional data monitoring. However my initial tinkering seems to show that Babylon.js may be too slow for me. 

I have this code to draw 20*20*20 = 8000 boxes using CreateBox:

                var carrier = []

                var nb = 20;

                for(x = 0; x < nb; x++) {

                  carrier[x] = [];

                  for(y = 0; y < nb; y++) {

                    carrier[x][y] = [];

                    for(z = 0; z < nb; z++) {

                      carrier[x][y][z] = BABYLON.Mesh.CreateBox('sphere' + x + y + z, 0.1, scene);

                      carrier[x][y][z].position.x = x;

                      carrier[x][y][z].position.y = y;

                      carrier[x][y][z].position.z = z;

                    }

                  }

                }

which replaces the createSphere and sphere.position lines in the basic Playground example which can be found here.  

Here it is in the Playground.

However on a 3 year old i7 mac mini, admittedly with no dedicated GPU, I am only getting about 7 fps, whereas this example using three.js draws 150k, cubes, so 20x more, and on the same machine I easily get up to 40fps quite reliably. 

So is three.js a couple of orders of magnitude faster? Or is my code above too naive? I can see a situation where I will need 100 x 100 x 100 = 1e6 cubes on the page, each coloured slightly differently and I will need to be able to "fly around" and inside the cube relatively fast so that I can explore it for data features. Is Babylon.js going to be able to do this (assuming a faster machine)? I would like to hope so because I love the documentation and what I see on the forums so far. 

Please assume that I am a complete beginner to 3d animation. 

 

Edited June 11, 2016 by vegabook
added Playground example

[Samuel Girardin]

Hello and welcome !

You should give a try to the spsPartcile https://doc.babylonjs.com/overviews/Solid_Particle_System . Maybe it could solve you perf forum.

 

 

[jerome]

Actually the SPS (solid particle system) is designed in this intention : animate a great amount of similar (or not) 3D shapes. It uses, in a easiest way for the final user, the same process than the one used in the ThreeJS link you mentionned : a single big geometry whom you can access its subparts (the solid particles), one draw call.

The SPS can easily manage 8000 boxes at 60 fps. For 1E6 boxes, I'm afraid you will be over the javascript/webGL limitations for now... it's really a huge number, but maybe you don't need to display or animate all of them at once each frame : over some limit, no user eye will be able to see the difference between, say, 20K, 30K, and more instances, just because they can hardly be seen at the same time and because of the screen resolution. 

 

doc here : http://doc.babylonjs.com/overviews/Solid_Particle_System

(the http doc site  has the "eye" icon links working, not the https one currently)

 

[EDIT] please have a read to this recent topic about someone who had quite the same need than you... and who finally succeeded with BJS : 

followed here : 

 

[JohnK]

Welcome to the forum.  Related to the project Jerome just mentioned is this playground http://www.babylonjs-playground.com/#2FPT1A#92

This is from this related topic

You would I assume be using colours not textures but you can see what can be done.

By the way when I studied maths at university tensors and tensor algebra was that one step too far for me.

[jerome]

maybe this one is just a bit less aggressive for epilecptic people : http://www.babylonjs-playground.com/#2FPT1A#91 

 

are speaking about these tensors : https://en.wikipedia.org/wiki/Tensor ?

seems quite complex indeed

[vegabook]

Thanks you for the responses. I have looked at the SPS particle system and indeed it is exactly what I need.

On my modest hardware here at home, I have already gotten very credible results with this suggestion.  We're in the same performance envelope as the three.js sample I cited so I am very happy. As jerome said I will not need to view 100x100x100 simultaneously, but it was just for a sense of the scale of the webgl and Babylon.js technology at the moment. I think I will be viewing far fewer than 100k at the moment, let alone 1 million, though I am keen to be able to see a lot because our brains are quite good at discerning patterns even if there are very many thing we are looking at at the same time. Think about grains of sand on a beach or grass: we are able to find even subtle patterns in these (if they exist), even though we may be looking at a million of them or more.

Indeed, JohnK,  tensors are highly unintuitive because we can no longer rely on our physical-world intuitions, which are designed for 3d and maybe 4d if you consider time. However they are mathematically enormously useful as we may use projections to "slice" our data into views which can be extremely interesting, and there are major algorithmic benefits if we can keep all the factors affecting our problem in a single memory structure, as the (potentially nonlinear, and potentially very subtle) relationships are more easy to find. Needless to say, however, the memory use is potentially enormous, and of course all the calculation will not be done browser side. Indeed one of the challenges is fast communication between browser and server. It is to manage these data complexities that so many "big data" technologies are now being developed. I'm a bit old fashioned however and am simply adding RAM. 256 Gigabytes RAM in my current compute server machine. Some big companies have petabytes. I personally will not need this (yet). But one area which is massively being left behind, is the visualization side of this scale of data. 3d + animation + interactivity to "slice-on-demand" is going to be mandatory. 

I will post as and when I do interesting things will Babylon.js and I thank you for showing me the SPS.

For an example of "projecting" up to 6d onto a 2d screen, have a look at this (fairly old but excellent) video. Note the (exponentially increasing) use of the time dimension to animate in order to see dimensions 4 through 6. Interactivity and slicing would help here too of course.

 

[jerome]

Glad to hear you could achieve what you want 

Please feel fre to share your (lighter) prototype in our playground (aka PG) : http://www.babylonjs-playground.com/ if you need some help to tweak the SPS or any other parts of BJS.