Please welcome..the Physically based rendering material!

[dbawel]

@Vousk-prod - thanks for the info.

 

I've been working on other projects these past few years, and out of any serious coding. So I obviously need to catch up. The only rendering process I've been working with lately is Light Field Rendering (LFR), or attempting to emulate or build an efficient optimized model/process to emulate LFR. LFR is the way that natural light is reflected and/or absorbed on physical objects in the real world, and why Magic Leap displays are completely natural and realistic to the human eye - far beyond any 2D or 3D simulated renderer. No one has ever seen anything like this, unless you've actually used the Magic Leap prototypes - which there are only two currently - one at Magic Leap's home office in Florida, and one at Weta in NZ.

 

I can't say much else at this time, however, those of you who know me on this forum, know that I've been saying I believe this technology will truly and dramatically change the world as we know it, and far more than any other invention in the last few decades. So if you want to better understand the future of rendering and displays, study Light Field Rendering. As for now, what DK, David, and others (I'm not certain who else contributed), what they have currently developed as the PBR renderer looks fantastic! And I'm really looking forward to personally understanding the renderer better and the functions we might add in the future to achieve even more realism in the BJS rendering engine. For anyone who is reading this, you are definately in the right place and working with the framework and standards of the future; the near future - which is really taking off globally right now. We're still on the edge, and this is a great place to be as the rest of the world will be playing catch up soon.

[jessepmason]

@dbawel  thanks for sharing that info! I can't wait to see more of what Magic Leap has been working on!

[RichardN]

Light field rendering has almost nothing to do with PBR (except that both try to emulate physical aspects of light). Unless the definition has changed since the introduction of LFR?

From Wikipedia:

Quote

Light field rendering: By extracting appropriate 2D slices from the 4D light field of a scene, one can produce novel views of the scene (Levoy 1996; Gortler 1996). Depending on the parameterization of the light field and slices, these views might be perspective, orthographic, crossed-slit (Zomet 2003), general linear cameras (Yu and McMillan 2004), multi-perspective (Rademacher 1998), or another type of projection. Light field rendering is one form of image-based rendering.

Here is a really nice article about LFR: https://www.fxguide.com/featured/light-fields-the-future-of-vr-ar-mr/ 

 

@dbawel "LFR is the way that natural light is reflected and/or absorbed on physical objects in the real world" - I think this description is where the confusion with PBR comes from, that could pretty much describe PBR as well. It is also not a good description of LFR. LFR - the way the magic leap and related devices do it - is trying to recreate the light field, the reverse of what light field cameras such as the Lytro are doing by capturing the light field. That means that not only one "slice" of light is projected, but, like a hologram, "slices" of light at different depths. It is trying to recreate "the real world" by approximating what a hologram does. 

PBR on the other hand is trying to create realistic-looking materials (rendered on a flat screen, not "in the air") by describing the properties of the materials, and calculating how materials look, in a physics-inspired way. 

 

[Carharttguy]

Hello

I'm pretty new to 3D in general, but just to be sure: Should I just skip learning Materials and use PBR always, everywhere?

Cheers

[RichardN]

On 3/15/2016 at 2:20 PM, Carharttguy said:

I'm pretty new to 3D in general, but just to be sure: Should I just skip learning Materials and use PBR always, everywhere?

It depends. PBR has way more options and is a bit more difficult to master, but you can get very realistic results with it. It is also a standardised(ish) way to describe materials that is used by many other 3d software programs, so once you learn it, you will find it easier to use other tools.

But it also helps to have a basic understanding of the standard material, which uses some other very common ways to describe materials, also shared by a couple of other tools. Additionally, you probably can't escape the standard material if you import 3D meshes from other programs. The OBJ importer for example will create standard materials on your imported model.

If you're new to all of this, I would start with the standard material, and then move on to PBR. They actually have a lot in common, but sometimes use different words to describe the same thing.

http://doc.babylonjs.com/tutorials/Materials

http://doc.babylonjs.com/overviews/Physically_Based_Rendering_Master

https://www.marmoset.co/toolbag/learn/pbr-theory

[RichardN]

Maybe @Sebavan knows this: Is there any 3d model importer for Babylon that supports PBR already? So if I create a model in some modelling tool and use PBR materials there, is there an easy way to get that into a Babylon scene? At the moment I'm manually "translating" from standard to pbr material after importing a model.

I know FBX can theoretically store PBR material info, for example.

 

[Sebavan]

Hello,

There are no exporters yet for the PBR Material. On the other hand, It already supports serialization and deserialization so most of the framework parts are present.

It would be really nice if somebody could make the max exporter (I am running out of license to work on it unfortunately),

Most of the file format can store PBR information by convention (even albedo in diffuse, reflectivity in specular and Microsuface in SpecularPower) but nothing is really standardised so you will need to precise at export time the workflow you want to use.

 

[Wingnut]

Hey guys, just want to say "hi" and welcome you to the forum.  Sebavan... I see some git activity from you, which means you are already contributing to framework goods... THANK YOU!  Did you "introduce" your contributions somewhere?  Got a URL to some talking about your projects?  (sorry if I have missed the obvious... I'm not overly bright) 

To everyone nearby that I have not said hello-to... HELLO!   Good to have you with us!  You guys sound like you are "dancing on the bleeding edge"... pretty cool.  Talk talk talk, guys.  We all want to hear about your adventures in PBR (and loading/launching issues).  Thanks!   Party on!

[RichardN]

Hi @Wingnut!  I'm currently working on making an object look like its real-life counterpart (or at least like a very nice render), and the pbrMaterial and the HDRCubeTexture came at the perfect time for that. 

 

I found some more nice sites on PBR, so I'm just going to drop them here for others who are interested:

https://www.marmoset.co/toolbag/learn/pbr-theory - PBR theory

http://www.marmoset.co/toolbag/learn/pbr-practice - PBR practice explained in great detail, with material charts

http://blogs.unity3d.com/2015/02/18/working-with-physically-based-shading-a-practical-approach/ - PBR in Unity, with some more useful material charts

http://www.marmoset.co/toolbag/learn/pbr-conversion - converting from traditional materials to pbr materials

http://doc.babylonjs.com/overviews/Physically_Based_Rendering_Master - and of course for anyone who missed it, the excellent Babylon.js tutorial on PBRMaterial by @Sebavan 

 

 

[dbawel]

 

Quote

 

@dbawel "LFR is the way that natural light is reflected and/or absorbed on physical objects in the real world" - I think this description is where the confusion with PBR comes from, that could pretty much describe PBR as well. It is also not a good description of LFR. LFR - the way the magic leap and related devices do it - is trying to recreate the light field, the reverse of what light field cameras such as the Lytro are doing by capturing the light field. 

 

 

@Nuitari,

I commented on LFR in my own words and from actual practical experience working with Magic Leap to build an interface and content to support their displays - however I don't know anyone else on this forum who has had any experience working to integrate LFR into a rendering application other than myself. So I don't believe you are qualified to comment on my post on the subject of LFR from simply looking at definitions you found in a Google or other Web search. When you can direct me toward the work you have done using Light Field Rendering in an application, then I will be happy to contribute to such a discussion. Otherwise, please don't ever comment that I don't know what I'm speaking about unless you can comment from a position of knowledge.

I'm currently working with Light Field Rendering, and was making a comment about PBR in my own words - as PBR is not something I personally have experience with - and would never post a comment about your posts concerning PBR since I am not qualified. So I expect the same from you and other users on this forum. Please don't speak to my words on LFR unless you can demonstrate where you have contributed to the study and building of an LFR renderer. I have been working with content creators at Weta in support of the Magic Leap display device and hardware for 2 years now, and I know you cannot represent the same as our group is a very small group and are not able to share our current work - so I don't know how you might be working on LFR (or have any practical experience with) since there is no other display device capable of supportng LFR rendering. So again, I and everyone else on this forum would appriciate if you and anyone else please restrain from commenting on something you most likely know nothing about. The point of this forum is not to post whenever you believe you can, but to contribute to the advancement of the Babylon.js framework - so I suggest you think first before you post coments on something you know very little about.

DB

[RichardN]

Hi @dbawel,

I started working with light fields over a decade ago. Your definition of "Light Field Rendering" is not one I have encountered before. I didn't mean to offend you. There is a clear definition of LFR in the literature, it's a form of image-based rendering. See the original 1996 paper titled "Light Field Rendering" at http://cseweb.ucsd.edu/~ravir/6160/papers/p31-levoy.pdf. 

Maybe Magic Leap, or you, use their own definition (and who knows, maybe the Hololens calls what they are doing LFR as well).

When talking about light fields, I generally think about the plenoptic function, and about capture or recreation of all light rays in a 3D space. Like a hologram. (Which is possibly why the magic leap uses a diffraction grating, like a hologram? Magic Leap filed several hundred publicly available patents, that particular one is here http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220150016777%22.PGNR.&OS=DN/20150016777&RS=DN/20150016777 ). The biggest breakthrough of the Magic Leap seemed to me their ability to project in a way that lets the eye focus at different depths.

And as such, like I said, there is little overlap between LFR, or light fields, and PBR. Except that PBR also works with some physical properties of light.

I understand that making the objects that the Magic Leap projects blend seamlessly and realistically with the real background behind them is probably a big challenge regarding the lighting, one that must be similar to what people try to achieve with PBR. Is that perhaps what you are working on, and why you draw the comparison?

Instead of lecturing me on what this forum is about and how little I know, why don't you tell me why you think my ideas are wrong? You say we should all study LFR, do you have some resources you can point us to? I'm happy to learn.

-N

 

 

[JohnK]

 

@Nuitari As someone who knows nothing about LFR and PBR other than they appear to both have something to do with the use of light in a scene I can see why @dbawel would be offended by this portion of the post

On 15/03/2016 at 6:22 PM, Nuitari said:

 

@dbawel  I think this description is where the confusion with PBR comes from, that could pretty much describe PBR as well.

and as just an interested reader with no axe to grind I must say I found the tone of the post to imply that @dbawel knew very little about LFR whether this was intentional or not. I had thought about commenting earlier after you post but had decided that @dbawel was perfectly capable of defending himself. Nowhere within @dbawel's post

On 15/12/2015 at 9:57 PM, dbawel said:

@Vousk-prod - thanks for the info.

 

I've been working on other projects these past few years, and out of any serious coding. So I obviously need to catch up. The only rendering process I've been working with lately is Light Field Rendering (LFR), or attempting to emulate or build an efficient optimized model/process to emulate LFR. LFR is the way that natural light is reflected and/or absorbed on physical objects in the real world, and why Magic Leap displays are completely natural and realistic to the human eye - far beyond any 2D or 3D simulated renderer. No one has ever seen anything like this, unless you've actually used the Magic Leap prototypes - which there are only two currently - one at Magic Leap's home office in Florida, and one at Weta in NZ.

 

I can't say much else at this time, however, those of you who know me on this forum, know that I've been saying I believe this technology will truly and dramatically change the world as we know it, and far more than any other invention in the last few decades. So if you want to better understand the future of rendering and displays, study Light Field Rendering. As for now, what DK, David, and others (I'm not certain who else contributed), what they have currently developed as the PBR renderer looks fantastic! And I'm really looking forward to personally understanding the renderer better and the functions we might add in the future to achieve even more realism in the BJS rendering engine. For anyone who is reading this, you are definately in the right place and working with the framework and standards of the future; the near future - which is really taking off globally right now. We're still on the edge, and this is a great place to be as the rest of the world will be playing catch up soon.

does he show any confusion between LFR and PBR. He talks about his work and then switches to talking about the topic  saying no more than "PBR renderer looks fantastic" and implying that working with BabylonJS is a great place to be.

In your recent post there are also comments I balk at somewhat "why you draw the comparison?" and "You say we should all study LFR". Again nowhere in any of @dbawel's posts do I find he makes any comparison about LFR and PBR and he states "So if you want to better understand the future of rendering and displays, study Light Field Rendering" way off telling everyone to study LFR.

 

9 hours ago, Nuitari said:

@dbawel

And as such, like I said, there is little overlap between LFR, or light fields, and PBR. Except that PBR also works with some physical properties of light.

I understand that making the objects that the Magic Leap projects blend seamlessly and realistically with the real background behind them is probably a big challenge regarding the lighting, one that must be similar to what people try to achieve with PBR. Is that perhaps what you are working on, and why you draw the comparison?

.......You say we should all study LFR,

 

Now what I do read is that you have an issue with @dbawel's definition of LFR and you see that this definition means that LFR and PBR could have an overlap and be compared though this is never actually done within @dbawel's posts.

As someone who is not clever enough to understand @dbawel's definition of LFR nor the paper's you give I hope that there is room for  amicable discussion.

 

[dbawel]

@JohnK - I appriciate your understanding of my post which I was providing a very simplistic comment on Light Field Rendering, and it was never intended to be a definition of LFR - as LFR is a highly complex process, and one which is mainly on paper and extremely difficult to put into practical application. However, I've been personally fortunate to work directly with Richard Taylor who owns Weta and sits on the board of Magic Leap - and who introduced me to Magic Leap's display device and technology over 4 years ago. As there are only two of their devices currently in existence, I have also had the unique fortune to be one of a handful of people to ever have the experience of viewing imagery through a device projecting light fields diectly onto my retina. And I cannot divulge how the device works due to strict NDAs, however, they are able to manufacture head mounted displays which seperate from the processing unit weigh in at only a few ounces - which if I were able to explain how they are able to reduce what is now 50 lbs down to approximately 5 ounces, and as I was describing to DK, it belongs in a Star Trek world as it's so far beyond belief once you undrstand how they are building the components required for such complex projection.

So only once you understand the required components and how they are manufactured, only then can you begin to comprehend how we might build the authoring environment and rendering application for such a device. I was working with Mental Images (a German company) in the late 1980s and early 1990s on ray traced rendering when it was a brand new technology and mostly theoretical and on paper at the time - similar to what LFR is currently - wrapping my head around actually building an authoring API and renderer for light field projection was a huge challenge mentally for myself personally. And this is after reading many of the white papers on building an LFR application. And I can also tell you that if you read the patents filed by Magic Leap which are available (many are not yet available online), I doubt these will provide you with any practical information on how the applications in development actually function.

So all that I wanted to say in my post was that I'm very impressed with PBR, and that I recommend at some point that the user community should begin looking at what LFR is as once Magic Leap hits the market, EVERYTHING CHANGES. Oculus and all of the other un-natural "3D" display devices will die entirely with the exception of social experiences such as feature films for a time, as LFR is highly computationaly expensive, and it will take time for all media industries to catch up - but they will much faster than anyone might expect. By the time Magic Leap releases their first head mounted display, processors will be many times faster than today, as the new processing technologies are lightning fast and are required to render quality LFR in real time. So it will be post rendered media for a short time until the entertainment industries catch up. But once you get a taste of a true LFR display, there is nothing that will compare, and you'll throw out every stereoscopic display. Mark Zuckerburg has already distanced himself from Oculus, and obviously he hasn't spent much time at all discussing his huge mistake in spending 2 billio dollars for existing technology - which by the way, the founder of Oculus was my PA at Emersion Technologies, and we happened to have been building stereoscopic displays for the military - so I wonder where he got his ideas from, hmmm....

So again JohnK, thanks for understanding what I was trying to say, and why I might have been a little annoyed by comments from someone with no applied knowledge that I'm aware of - and I can generally say this as I know most all of the players in applied LFR development.

 

@Nuitari -  To produce an actual definition of LFR requires far more than a sentence or paragraph, and I was mentioning this briefly as I'm very impressed by the work done on PBR within the babylon.js framework currently being developed - and I wanted to comment that the BJS community should begin looking at LFR in the future to stay ahead of the other frameworks which are competing with babylon.js - just as we were working on the babylon.js stereoscopic camera in late 2014. I also get a little annoyed at users on this forum who offer opinions on topics without contributing to a discussion in a productive way and/or criticize other users on the forum, as this fosters a negative environment and adds nothing positive to the discussion. I'm not saying that this is the case for your comments concerning PBR, however, I certainly am qualified to speak about LFR, and am actually working on an LFR project with Weta in NZ. So of course I take offense at someone who questions my comments on a subject which they themselves could not be involved at any level on an actual LFR application, as I have had the good fortune to be.

Let's all show due respect to all of the users on this forum, as we all contribute to the evolution of this fantastic framework, and are all fortunate to be able to use babylon.js and to advance it to be the very best development environment in the WebGL standard.

DB