Rendering another WebGL canvas on BabylonJS canvas: geographic data visualization with MapBox GL and deck.gl

[Veksi]

Hi!

I was looking some geographic visualizations created with MapBox GL ES and deck.gl and I was wondering if someone has done something similar before or could pass directions how could one go about combining these with BabylonJS. How could other WebGL canvases used with BabylonJS? Is it possible? In the following some background wondering.

Here is one titled Mapbox GL JS + AR.js + A-Frame (VR) = Mapbox AR/VR and at https://blog.mapbox.com/launching-custom-layers-with-uber-2a235841a125 is an example of Three.js being used to import a glft model into a scene (this) and more on integrating MapBox and deck.gl at https://medium.com/vis-gl/deckgl-and-mapbox-better-together-47b29d6d4fb1 and a note on camera things too.

To my brains this feels like really cool way to visualize geographic data sets and even cooler would be to add virtual reality things here. I could assume that since MapBox and deck.gl work together and render into a WebGL canvas and the first link demonstrates capturing the buffer as an image and project it to AR.js cube, something like that could be done with BabylonJS too. Maybe avoid the capturing the WebGL buffer as a image altogether. The second link also shows how ThreeJS gltf model was imported into map data.

I can somewhat follow the examples how MapBox and deck.gl are used together, but I have very little clue how could I bring BabylonJS in the future to make the stuff even sweeter! That is, I'm thinking to start with data visualization with MapBox and deck.gl as far as the examples go, but sooner or later move towards BabylonJS. I'm not sure how possible is it to use BabyloJS (hopeful due to AR.js example) or what should I know APIwise or otherwise with this endeavour.

Some help or nudge or whatnot on offer?

 

[ssaket]

Sure ! There's an awesome project here -

pinging @The Leftover who created it !!

[Veksi]

Hey, thanks for the note @ssaket. I took a look at Illuminated city and it looks it could use something like deck.gl rendered on MapBox cancas that is then rendered on BabylonJS canvas, or something. ? I edited the headline as I think in general I'm asking if it's possible to render to WebGL canvas/buffer and then use BabylonJS to render that. I think at least, asking and hopefully learning while going -- have to have a project that is sufficiently long term and interesting.

[Veksi]

Spelunking around, UtilityLayerRenderer and gizmos might be something to use might be something to use here. Difficult to tell!  I also wonder that if Mapbox/deck.gl component has these rotation, scrolling and picking helpers that what if BabylonJS code moves the objects and if Mapbox would work after that... There probably is a way to do to that, at least by wiring the BJS thingies back to Mapbox/deck.gl.

[Guest]

A good read about combining multiple GL engines: http://doc.babylonjs.com/resources/babylonjs_and_pixijs

[Veksi]

@Deltakosh That was a useful one! Regarding to my other ponderings and specifically Mapbox, there is a discussion about exposing Mapbox camera coordinate system and specifically the discussion is about Threejs and in general feedback on how one would like to have it (I don't know if BabylonJS projections are as in ThreeJS, for instance). The issue is at https://github.com/mapbox/mapbox-gl-js/issues/7395 and I provide it here in case your or someone has an opinion in that regard. That about Pixijs is just freshly implemented in https://github.com/mapbox/mapbox-gl-js/pull/7039 and I provide that too here in case someone finds it useful. There's discussion about coordinate systems, performance and occasional notes about loading gltf models (likely not implemented) and in that sense hints how engines like BabylonJS etc. could augment geographic engines. One particular thing seem to be also rendering volumetric data.

<edit: Also, custom layers examples wanted at https://github.com/mapbox/mapbox-gl-js/issues/7330

[The Leftover]

Gentlemen and ssaket, I apologize for the slow response.  I will try to remedy it with a short summary of what I do.

The biggest factor for Illuminated City is that I don't need a continuous map.  The subject is always a bracketed geography.  Another factor is that the Lat/Lon must be precise, down to 20 meters or so.  I overlay police districts and crime incidents.  They need to line up.

To do this, I create the biggest, fattest tile the browser will handle.  8K pixels squared roughly.  That's all the map is, a fat image.  (There are potential future enhancements including multiple tiles but I don't use that in production.)  Boring, huh?  

There is some pre-processing.  The map is stored on the server (and injected into Babylon) as an SVG.  Several factors here:
  ~ Map size (from server to client) is always less than 2Mbytes
  ~ I can control styling

There is no dynamic connection to an outside data source.  I get the maps custom tailored, per city, and organize Illuminated City styling in CSS and "survey" them for precise fit.  The styling is important to me.  The one you see now is called "Retro-minus".  I may want to do a "night" view.  I would only have to redefine some CSS and voila!  Night.

The result looks good and is very controllable.  Adding a new city map is a straightforward recipe but takes a couple hours.

Again, apologies for the slow response.  If I can aid your discussion (or learn from your experiences) I am now paying attention!

https://blog.illuminated.city

[The Leftover]

Three views of Baltimore, MD, with different CSS declarations.