Custom VR orientation camera using Rigs ?

[dsman]

Current VRDeviceOrientationFreeCamera   is created as post-process. Hence we are loosing anti-aliasing. 

So is there any way we can create rig with 2 camera and side by side output. ( @JCPalmer created demo here http://playground.babylonjs.com/#CV4CX#1 ) And have camera rotate based on device orientation camera ?

I assume this will be slower (as device has to render two camera) . But it will be visually better , since we will not loose the antia-aliasing ?

In that case, how do we get device orientation camera ? Is there any library or is there any way we can use part of code for VRDeviceOrientationFreeCamera    somehow ?

 

[JCPalmer]

Technically, yes by adding after instancing:

camera.setCameraRigMode(BABYLON.Camera.RIG_MODE_STEREOSCOPIC_SIDEBYSIDE_PARALLEL,  { interaxialDistance: 0.0637});

Or instancing the single camera version BABYLON.DeviceOrientationCamera, then setting the rig.

Think there is going to be a problem, since only the VR, barrel distortion, uses quaternions.  I think it might not work right without.  I had the opposite problem trying to use the barrel distortion on a passive scene with camera animation.  Setting camera.rotation does nothing.

I briefly attempted to rework Camera to work the same way as Mesh with respect to rotation (euler or quaternion), but this was beyond me.

[JCPalmer]

On the phone based VR subject 2 thoughts / questions:

1. Has anyone tried the Google Pixel with webGL?  It can record @ > 60 fps from cameras, but not seen what it can do using GPU?
2. Think Note 7 has killed phone based VR for a while.  Morpheous & Hololens could be the beneficiaries, but not sure how that helps us, unless using webGL on XBox, which no one talks about.

[dsman]

@JCPalmer Consider me noob. I still doesn't have clarity if using Camera Rig with SBS + DeviceOrientationCamera  ,  I can create VR view for smartphone based VR device just like VRDeviceOrientationCamera does. But just without aliasing .

Also what do you mean by Note 7 has killed phone based VR . 

We haven't tested VRDeviceOrientationCamera in Galaxy GearVR. But I assume it will work just like in any other normal phone with CardboardVR based VR headset.  Am I right ?

 

[JCPalmer]

Try it, I do not have the equipment to definitively tell you if it will operate correctly, but just switching the rig is how to do it.

As far as the Note 7 crack, I have ABSOLUTELY NO intention in risking my sight, face, or hair (which is close to my clothes) to play a game.  I do not know if the Note 7 even fits in the GearVR, but I am not comfortable (& think others may make this leap too).  @dbawel actually talked about putting a cold pack on the phone, which in retrospect provides little comfort.

I was actually thinking about a Note just a month ago.  Am going wait.  If the Pixel breaks the 60 fps for webGL, it would be better anyway.

 

[dbawel]

@JCPalmer - I would definately avoid waiting for Samsung to release the new note 8 or whatever they release even though they once again claim to have fixed the very serious hot battery problems. Howevr, currently my S6 and S7 phones work great, and the S7 phone runs much cooler than the S6. However, the S6 performance is only about15% slower than the S7 - and this is only on CPU and GPU intensive VR apps (only VR apps for GearVR) which is hardly noticable, and only overheats on a very select few which are rendering allot of polys, animations, deformations, fine anti-aliasing, etc, as well as post processing functions simultaniously. Otherwise most of the time it does run a bit hot, but doesn't shut down due to overheating except in the few previously described VR apps - and only Oculus VR apps due to the enclosure. So these are now fairly inexpensive (I can buy the S6 phone for about $300US currently, and they perform fantasic. And would select the S6 over the S7 as there's very little difference except for price. And I generally don't require using the gel packs which don't need refrigeration first, as they simply act as a heat sink 9definately don't freeze, but you can refrigerate if you want.) But again, I rarely need to use these as I know which VR apps are pushing the limits of the hardware - as it's not the battery which gets hot, but the internal systems when they're clocking at etremely high speeds.

So again, I highly recommend the S6 phone as the resolution is fantastic and the performance is great as well. Waiting for the replacement or new device isn't worth the cost in my opinion, as there's nothing my S6 (phone) won't do with great performance. And the GearVR is only $99 - which you can find cheaper if you look. And as much as I dislike the current 3D screen formats such as Oculus and Vive, they do offer some fantasic games which I can list if anyone is interested. But the real benefit is to write for the Oculus format currently as it's very inexpensive, and with the MOGA bluetooth controller, it's really often a fun experience to play as well. The controller works and pairs perfectly, and costs about $30US. But this has already provided me with allot of experience in preparation for the release of the Microsoft Halolens which is a step above as an AR device, and tsoon we will be ahead in developing for the new genre of truly fantastic devices such as Magic Leap - which is definatly in a catagory of its own.

And a bit off topic, but relevant to this current discussion - by deciding to jump right into WebGL 2 years ago, as of about 6 weeks ago, simply posting my resume online on select sites, I receive job offers for independant work at home at least 2 offers a week or more - half are asking to relocate with long term contracts such as Verizon who is currently seeking a large # of experiencd WebGL developers and cannot find anyone to fill their many positions in several different US cities. And right now, I'm able to select among several jobs whenever I need, and another is waiting - and I'm averaging $75 - $100 per hour without any negotiation since these companies cannot locate any available competent developers currently. So if anyone on this forum needs work, it's now very easy to find here in the US - and I'm certain many companies would allow anyone in any country to work freelance as there truly are no WebGL developers with experience. And the jobs I get now, are almost always beyond my skill level - that is until I finish and deliver the work - thus my skillset and skill level is growing so much faster now than I ever thought possible by selecting work which I can find few or no examples ever published online - and certain that most of these are first to accomplish. My current project is for a trade show for a pharmacuetical company in which I have now learned to run Web Workers to calculate more than 1000 sphere imposters rendering at a constant 60fps running on a local server, and syncing the BJS render framerate using an IndexedDB - which I had no previous experience with whatsoever. But all is currently well and working, and I finish this job and deliver is 3 weeks. And there's already another job waiting for me to begin as soon as I finish this one - unbelieveable. So for anyone who's looking for work, just write your resume to highlight WebGL development, and I guarantee the emails for work will begin flying in. There are many recruiters looking for the larger contracts, but they don't know how to come to look on the forum in the jobs section.

But for the best news, all but two companies I've been contacted by recently are dead set on using Babylon.js, and not Three.js, Pixel.js - or any other WebGL framework. They all are asking for BJS experience which is fantastic, and they all tell me how rare we are - which we already know. Anyway, sorry to stray so far from the initial subject as my fingers are simply flying on the keyboard tonight. However, it all comes back to taking a serious look at the S6 phone and GearVR - since it's providing me with unique VR and AR experience, as well as additional income. And my own company 3rd Brain just received 2 proposals this week to fully invest in our full $ ask for capital investment on very reasonable terms. And we have one more seperate proposal coming early next week. So I fully expect to have our unique proprietary server technology in use by Weta and Lockheed Martin within 3 weeks (finally), and a public release 1st quarter next year - regardless of our accepting funding. And e're expanding the server API to be able to license for use to others - which provides unlimited multi-user simultanious real-time media sync and dynamic streaming of media and events in a single stream - I can explain further if anyone is interested. But this is nothing anyone else currently has built as we have almost 15 years in building this for use with our OpenGL and DirectX layers - which we've now almost fully support WebGL with the same functionality. 

So it appears we'll be receiving full funding with reasonable terms as we also have built a unique WebGL product app ready for release soon - so investors no longer consider us pre-revenue as the app is working well, and I'll share with a select few soon - you already know who you are - and I'm really looking forward to sharing, as we coudn't have done any of this without all of you. I've always said that I believed babylon.js to be the future of WebGL, and it's now coming true in all respects. Even Verizon is spending over $400 million US in WebGL development next year for a new VR platform solely using babylon.js. So we know we're all blessed by the work @Deltakosh and of course @davrous began, and all of the other brilliant minds now contributing to the framework. I'm very happy to say that I'm finally beginning to make what I consider to be simple modifications to the engine, and will also work very hard to add to the features as I better understand the core engine - as I have jobs coming which require new features, and will now always first attempt to build myself as I feel like I'm possibly ready to begin to share in this awesome opportunity to contribute to build the very best WebGL engine currently available - and make certain babylon.js remains so.

Cheers,

DB

Sorry for the long read - as you know how I can go on and on - and on sometimes.

[JCPalmer]

DB, what timezone are you in?  Think Hololens is not going to work with this code base anytime soon.  Something experimental with OpenGL ES though. 

I plan on waiting for now.

[dbawel]

@JCPalmer - You are correct, however as with Oculus, Hololens will support at least 2 browsers, and I currently and simply expand the chrome browser on my Samsung phones to full screen and use the MOGA bluetooth controller, and it looks as good as anything Oculus and their 3rd party developers have built. And if I want to use a scene within the Oculus app itself, I launch the Samsung browser which is enabled in the Oculus app - but I prefer chrome without launching the Oculus app. The Hololens will function similarly, as my Brother recently left Microsoft as a lead patent attorney with a mechanical engineering and computer science degree as well. So I've had the scoop on this for quite some time. 

Should be no problem using WebGL with Hololens.

Cheers,

DB

[dsman]

@dbawel  So does MOGA  controller work off the shelf  in a Babylon app running inside chrome browser  any android smartphone ?   ( I assume, using gamepad camera ) 

I want ability to control camera position using controller while the smartphone is inside cardboard based VR headset. 

[dbawel]

@dsman -

All 3 MOGA bluetooth controllers work right out of the box - however one stands out as it has two modes for bluetooth connectivity. The two other controllers may require some configuration depending on the device you're linking to. The oe controller that works with every device I've tested and consistanly in type of control is the MOGA Pro Power controller. Make sure you purchase this model over the other two bluetooth controllers as one mode of connection works on 80% of all devices, and the other 20% work with the second mode (either mode A or B.) Actually both modes work with all devices, however there are differences in the support of controller buttons and features pairing right out of the box - no configuration necessary.

If you require more info, just let me know. I've been developing using hand and other controllers for many years, and there is no better and more compatible controller that works as you designed your app right out of the box upon connection. Also, if you look at the features on this controller, it functions as well as any current console controller with even more features - as well as both analogue and digital components - and is USB rechargable. 

DB

[dsman]

@dbawel 

How do we use gamepad camera if we are already using VRDeviceOrientationCamera  ?  We need controller to control the camera when smartphone is inside the VR Box. 

Any way to get events for gamepad buttons ?

 

[dbawel]

@dsman -

You'll be identifying camera orientation from controller events in your application regardless - so I don't understand the question. If you're considering using on both VR devices and 2D displays, then again the answer is clear.

DB