Jump to content

Panoramic View in BJS: 360° Equirectangular Photos and Videos?


Recommended Posts

Hello Everyone, I want to use Babylon.js (great library!!) to show 360 panoramic view for equirectangular photos like http://www.babylonjs-playground.com/textures/equirectangular.jpg

The example code http://www.babylonjs-playground.com/#11GAIH is perfect for my needs, but result scene looks like mirrored (look at the child in the original photo and in the rendered scene)... I can suppose the main reason (skyboxMaterial.backFaceCulling = false and camera inside the box)...

However, does exist a way to show in a 360 panoramic view an equirectangular photo avoiding a mirrored result? 

Thanks for your help :)

Link to comment
Share on other sites

Hiya @Fabri, welcome to the forum!  Sorry it is taking so long to get replies.  Although I have found no solutions, I've certainly tried a few (hundred) things.  I'm no expert, though.

This is an interesting challenge/issue!  No matter what I tried (so far), I couldn't get that dark blue speedboat to move to the OTHER side of the pier.  :)  hmph.

BUT... I ONLY tested material and texture things (which I suck-at).  This MIGHT BE a job for the view matrix. In other words, do something to the camera->world relationship... instead-of the skybox/texture (I suck-at matrices even more).  :)

I dunno.  I think there's a matrix transformation that could do something magical, for this.  hmm.

All in all, good find, good challenge, not a bug, but a missing feature or yet-to-be-discovered adjustment.  Let's hope we get some more replies/help, soon. 

Welcome again, good to have you with us.  We'll talk again soon.  I'm going to keep goofing-around with your nice playground (thx for that, too), and I'll report discoveries, here.  Be well, party on!

Link to comment
Share on other sites

Hi Wingnut, I am happy to have received a response from you today...(I was almost losing every hope to get an answer)

I agree with you that what I found is not a bug but may be just a missing feature...unfortunately I need such feature...

I am completely new to 3D world and its terminology (...view matrix? camera->world relationship?)...so please be patient if I don't understand deeply your suggestions... :(

As far as I could verify with Three.js https://threejs.org/examples/webgl_panorama_equirectangular (just drag our equirectangular texture http://www.babylonjs-playground.com/textures/equirectangular.jpg into the page)

it works like a charm but with a little trick: geometry.scale( -1, 1, 1 )...in practice Three.js allows to invert the mesh "inside-out" by setting geometry.scale.x = -1;

Anyway I don't like Three.js's spherical view and I prefer much more Babylon.js's skybox view...

Do you know how to invert a mesh "inside-out" in Babylon.js?

I think that Babylon.js authors or any other expert, could solve my little problem in a few minutes, by checking how Three.js is working...

As a I said, I'm just a beginner and for me everything is something magic :)

Hope to hear from you soon!


Link to comment
Share on other sites

Ooh, negative scaling, what a good idea.  I tried some experiments with it... no change.  Darn.  :)  http://www.babylonjs-playground.com/#11GAIH#1 (lines 31-33)

In webGL, there is a "matrix" (a data structure used to store orientation data... used by webGL shaders... to render a scene)... called worldViewProjection.  It is "derived" by multiplying the worldMatrix * viewMatrix * projectionMatrix.  You can see a diagram of it... here... https://www.eternalcoding.com/?p=113 in the section called The Voodoo Behind Matrices.

It is MY theory... that reversing (180-degree "transform") of the camera viewMatrix... might fix our issue.  I'm not sure how to do that, but I will continue investigating it.

Many of the BJS "big dogs" will be returning from holiday vacations, soon... so maybe we will get responses from smarter people than I, soon.  Thanks for your patience and kind attitude.  We'll find the answer... or die trying.  heh.

To recap, we wish to reverse (rotate 180 degrees around Y axis) the image... as viewed from inside the skybox (current cam position).  All help highly welcomed!  Thanks!

Link to comment
Share on other sites

Hi Wingnut, I did my homework and read the excellent document that you suggested!

Waiting for BJS "big dogs" (as you call them), I found a temporary workaround (just a copy and paste of parts of babylons.js code and examples about shaders...), thanks to your very valuable suggestions!!!

You can take a look at http://www.babylonjs-playground.com/#11GAIH#2 , now the dark blue speedboat is on the correct side of the pier ;)

A trick to flip the texture is "coords.x = 1.0 - coords.x;" [line 30] 

Anyway I keep on waiting for a better approach and final solution from our BJS Experts...hope they will find time for us.

Thanks again!!!

Link to comment
Share on other sites

WOW!  Excellent!  You're a genius!  Talk about taking the bull by the horns... holy crap!   Script-based shader code... nice!  (I think Babylon.ShadersStore would work, too.)  (There's also an on-the-fly shader code assembler... called shaderBuilder - external JS extension, but that's for another day).

Amazing solution, Fabs!  What a dive-into hacking!  Have you been lurking for 2-3 years?  :)  Hacking shader code is certainly not a "noob" thing.  I predict that you are going to become a local BJS superstar. 

Thanks for sharing that... and congrats.  Yeah, the big dogs will be looking into this, including a new big dog named Fabri.  :)

Link to comment
Share on other sites

The feature is quite new so there might be bugs. Apparently in one of my projects a texture on a ground disappeared but I didn't look into why that happened, because I don't need the right handed coordinate system there, just tried it.

There "shouldn't" be any limitations caused by changing the coordinate system as far as I know, but I didn't add the feature so I don't really know if it is fully supported yet.

Alternatively, if the change of coordinate system proofs problematic. You could also load the image into a canvas first, mirror it horizontally, generate a data url and give this data url to babylon. But you'll have to wait for the image to load of course. I played with it, but it's tricky to do in the playground. I'll look into it later or tomorrow and give you an update ;)

Link to comment
Share on other sites

Excellent jschwuch!

I like your solution to flip the image via canvas. I updated your good code changing flipping just horizontally (you were right about it):  
        context2.scale(-1, 1);

and setting option invertY to "true" during Texture creation:  var tex = new BABYLON.Texture("data:flipped", scene, true, true,........


If BJS authors would add an option for Texture similar to invertY also to "invertX" or something similiar then all  extra effort we are doing to flip would be avoided...may be...just an idea...

What do you think??

Thanks a lot for strong support!!!


Link to comment
Share on other sites

Ahh! I missed the invertY option :D Good to know. Looked into the source and flipping Y is done with an option for the pixelstorage directly in WebGL. WebGL doesn't have an option there to also flip X, so i suspect it won't be supported. And I wouldn't want to implement it with a canvas there as this might cause some performance problems. Maybe we should think about an other mode (like FLIPPED_FIXED_EQUIRECTANGULAR_MODE) for the texture so we can flip the texture coordinates directly in the shader. Maybe a dev could say a word or two about that idea, I'd implement it if it's a reasonable approach.

Link to comment
Share on other sites


Hey DK... welcome back from caroling!  Everyone missed you.

Nice job with the forum catch-up, thanks for that.

DK, if YOU would have needed this, how would YOU have done it?

Perhaps with a .isBackground Layer ...where texture.uOffset bound-to camera.alpha?

Okay, maybe not that way.  :D

What would deltakosh have done... for a quick, one-time, no-framework-change -solve for this issue?

(If ya got time.)  (thx)

Right-handed was certainly easy, but that seems like it could have FRR.  (far-reaching ramifications)  Thoughts?

Link to comment
Share on other sites

Thank you all, my new friends, I really appreciate your consideration and support!!

Well, I don't know if such new mode/feature is a common need as regards gaming development,

but for those users that would use BJS in a different kind of web application

(for example to create a 3D user interface to navigate a virtual environment and then to show some real 360 photographic details....without mentioning augmented reality experiments....)

such feature could be useful (or simply fundamental).

So I think that it depends on BJS future objectives and how much universal and multi-purpose aims to become in future ...if it wants to expand its horizon or not?? ;)

I'm really interested in all your opinions about that !!!   :)

Link to comment
Share on other sites

:)  Hope ya'll had a swell holidays.  DK, did you bury your little girl in Christmas gift wrappings?  FUN!

Hey Fabri... it isn't JUST about expanded horizons, but it's also about adding yet another parameter to BJS Texture constructor.  One more thing to teach and learn.  That's why invertX parameter is weighed for usefulness.  Will it be used often enough for warranting a parameter addition?  hmm.  :)

We could get fancy... and bog the Texture code.  We already have invertY parameter.

In the invertY tester code...

IF (invertY instanceof Boolean)  then apply as boolean.

IF (invertY instanceof Vector2)  then apply as 2D transform.  (an X and Y rotation)

Nah, that's shoddy overloading... demented.  :)

How 'bout a new dataType... called a Twolean or Duolean...  two booleans side by side.  So if invertY were set to truetrue or 11 (Two-lean), both X and Y would be inverted.

Nahhhhhhhhhhhh.  hehe.  Goofy.  But I'm trying trying trying... to avoid that added parameter to Texture constructor.  :)

Man, canvas2.toDataURL()... @jschwuch just smoothly moving from canvas/Image() to BJS texture... nice.  Who IS this guy/gal?  That's some pretty advanced stuff for a 9-post noob.  Phew!  Cool!

A one-image skybox.  hmm.  "Equirectangular projection", eh?  hmm.  Looks like it might have a big future.  img.src is dripping with security blockades, though.  So, it's back to the old way... so we can view some cool web-shared equi-pics... without being security-blocked.  :)  Fun!

Link to comment
Share on other sites

Hey, thanks @Wingnut for your kind words :D

This guy is new to BJS but has some years of pure JS experience. I'm currently using BJS for a production project at work, really doing everything learning by doing (had computer graphics lectures at university though) and abusing BJS way too often :rolleyes: (like calculating transformation matrices myself and setting them on private fields "Ouch!"). But I'm planning to stay here a bit longer and do some private projects too. And I'm interested in developing BJS once I learned enough too feel comfortable with it.

Now back to topic. I wouldn't use another parameter for BJS Texture constructor either. Because WebGL doesn't support it when creating the texture and we would have to use a canvas in the BJS Texture constructor to do the flipping. It's most likely much more efficient to do the flipping via the shader. That's why I supposed another coordinateMode for Textures, like this:

skyboxMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.FLIPPED_FIXED_EQUIRECTANGULAR_MODE;

@Deltakosh I'm supporting this functionality too for uses like 360 photos and such. Question is, should I jump in at the deep end and code the changes myself and send a pull request or will you (or some other experienced developer) do the changes?

Link to comment
Share on other sites

  • 2 months later...

Hi @MrVR, yours is a brillant idea! I really don't know if it is possible use (so I let experts to reply about it) a Video Texture for a panorama view in order to recreate a realistic animated virtual environment...but I agree with you that it would be incredible!!! The only test I could do is to insert a video texture in our sample static panorama: http://www.babylonjs-playground.com/#11GAIH#19

Let's wait for our experts suggestions!



Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...