closed PBR on iOS

[paleRider]

Hi there:

I'm trying to have done a cross-platform development these days.

After some problems here and there I have my development running OK on:

1. desktop - windows - web - ff
2. desktop - windows - web - chrome 
3. desktop - macosx - web - safari
4. mobile - android - web - chrome
5. mobile - android - hybrid app (cordova)
6. mobile - iOS - web - safari

Sadly it doesn't work as presumed on the last platform I'm targeting, of course: mobile - iOS - hybrid app (cordova).

Under this last scenario a have several "hard" issues:

• my PBR materials are completely invisible;
• and some of my Standard materials (textures on planes) are shown all black.  

The other Standard material I am using (particle textures) are shown OK.

Any help or advice with this?

Thanks for your time.  

[BlackMojito]

Me too I am using only PBR materials. I haven't tested my viewer on IOS yet. Let's see if @Deltakosh has some comment about it. 

[paleRider]

Hi BlackMojito:

The case is that the geometries that use those PBR materials are loading OK (I'm providing a .manifest file for them), they are created properly, and sniffing into the Hybrid APP by means of Safari Developer Tools shows no error on console.

The only thing is that geometries are not shown at all (they are transparent I guess).

Certainly I don't know how to debug this. Thanks.

[V!nc3r]

Do you have a link of your app to take a look? And which BJS version do you use?

[Guest]

We definitely need more details.

Console logs, network profilem, bjs version, does it work outside of cordova, etc...

[paleRider]

First of all, thank you for your time, V!nc3r and Deltakosh:

Ok, let's see

As said my development shows OK the "problematic" meshes with their PBR materials on:

• web desktop (MS Win10 on FF and Chrome, macOS Sierra on Safari);
• web mobile (Android 6.0 and iOS 11);
• hybrid app (cordova 8.0.0) for Android (again Android 6)

But  however that same meshes are not shown, or let's say are shown as transparent, when a hybrid app (cordova 8.0.0) for iOS is built and ran. BTW my testing machine is a recent iPad Mini.

I don't get assets loading console errors by means of Safari Remote Device Developer Tools.

My APP doesn't use remote commutation APIs, only JQM and BJS as third parties libraries/frameworks.

Under Xcode I only get the typical deprecation warnings on JQuery Mobile (3 about setStatusBarStyle) and Cordova (11). Nevertheless GUI is working OK.

Finally, I'm using BJS 3.1 (I prefer to use only production versions) but changing to BJS 3.2 shows the same "bad" behavior.

P.D. As said above in this same thread, I have two problems with BJS in this app, the not visible meshes (PBR Material) and the ones shown in black (Standard Material png texture over a plane). FYI both malfunctioning meshes are parented.

[JCPalmer]

You might assign one of the materials showing Black on a plane to a mesh which is currently showing a standard material.  Also change the material of a Black mesh plane to a working material.

I know its not really an inspired answer, but sometimes you do not always get the expected results, and you then know to look in a different or at least a more specific place from where you can get better search hits.

Also, black could have something to with lighting.

[paleRider]

Hi JCPalmer, and thank you very much for your time:

Only as a help, let me share a screen capture of the app as shown in almost-all platforms:

 

But in the iOS hybrid app you get this:

 

 

As said, keep in mind that the hand is the material of a plane parented with the tap.

The other textures I'm using (water drops by means of particle sprites) are shown OK.

Of course, the two (related) clues we have here are:

• Standard Material shown as black (mesh parented with not working PBR material meshes);
• PBR Materials not shown (transparent or so). 

As all this seems to do with light, the question is if is there any known issue with the kind of light I'm using in my scene (hemispheric) and iOS' WebViews?

All help will be welcome!

[paleRider]

Following the advice of JCPalmer, I'm starting to tweak a bit the materials.

Using on some sub-meshes of the tap the same mat used in the hand, shows it in the same black-fashion:

 

Even not parenting the hand with the tap or changing the type of light has not effect.

[paleRider]

Hi everybody attending to this thread.

I come back with the last info I think I can provide at the moment: the case is that I've reverted the tap material to PBR (and so is again invisible), but now I've also deactivated (commenting its lines) the Highlight Layer you can view "pulsing in organge" in the first upload image of this thread.

Well, with that Highlight Layer out-of-service the hand is now shown OK:

What do you think of all this?

[V!nc3r]

What is the file format of your environmentBDRFTexture ?

[Guest]

ok some ideas:

- Highlight Layer requires stencil. Try to use GlowLayer instead. Or nothing just for the sake of isolating the issue

- How many lights?

- Can you try to use a standardmaterial with disableLighting = true and emissive = white? (for the pipes)

- 

[paleRider]

HI V!nc3r a and Deltakosh, and again thank you very much for your support and interest.

Only for let it clear, what I'm trying to view in this development is a consistent behavior in order to be able to consider BJS as a suitable engine/framework for cross-platform (web - hybrid apps) projects, as several articles in the internet suggest as a fact. Well, ...looking on the bright side, we are almost there, as we have 6 out of the marked 7 intended platforms running OK .

Said that, answering to V!nc3r (by the way the beautiful Toulouse and Zaragoza are twinned towns ) my environment is in DDS format.

Now about your questions, Deltakosh:

- as proved with my previously posted image, when I disable Highlight Layer the "hand" (remember Standard Material) is shown OK (orange and not blacked), so must I suppose that stencil is/maybe let's say "conflictive" on iOS? or even in WebGL implementations? WebGL 1.0, WebGL 2.0?

- I've only using a hemispheric one but, as long as I understand, it is only affecting Standard Materials ("hands" and "droplets" in my case), being the rest of meshes (chromed look ones) shown by means of PBR materials, and so auto-lit with the environment texture used. As said my light is defined this way:

var light1=new BABYLON.HemisphericLight("HemiLight",new BABYLON.Vector3(0,1,0),_Scene);
light1.intensity=1.5;

 - using the following material:

var matDeltakoshTest=new BABYLON.StandardMaterial("Material_STD-ThermoGizmos",_Scene);
matDeltakoshTest.disableLighting=true;
matDeltakoshTest.emissiveColor=new BABYLON.Color3(1,1,1);

only in the "hose" and letting the other PBRs untouched) I get this:

...so definitively there's something wrong with (my?) BJS' PBR materials on iOS (Cordova hybrid app way), but only on iOS (?).

 

P.S.: In order to have it clear, this last posted screenshot is obtained from the iOS version (Cordova hybrid app) with:

• Highlight layer "off", having the effect of showing OK the "hand" (Highlight layer "on" shows a hand in black);
• Standard material (following the specifications of Deltakosh) on the hose instead of the PBR one;
• rest of meshes, all let with their PBR Material, and so following being invisible.

 

 

 

 

[Guest]

So Cordova on iOS is not webGL compliant

 

It seems that something in the DDS file is not accepted by the webview used by Cordova. 

We need to find which property is the problem and you may need to start by creating PBR material by hand from simple to more complex. (btw: hemispheric lighting affects the PBR)

Try something like this instead of my stdmaterial:

https://www.babylonjs-playground.com/#49Y355

 

Please note that intensity is set to a big  value because of this: http://doc.babylonjs.com/how_to/physically_based_rendering_master#light-setup

[paleRider]

Hi Deltakosh and as always thank you very much for your time.

About your statement that Cordova on iOS is not webGL compliant, excuse me but is the first time I heard of it and these are, definitively really bad news for me (and my plans to rule the world).

As said this is a key concept in my SW development framework, so could you please explain what part of BJS is not going to work on iOS (Cordova)? and what are the (non-cordova) options in order to have done a cross-platform-code BJS development?  (maybe trying to use WKWebView component ?)

Said this, certainly I've been coming to the same conclusion about environment textures.

Anyway, as the mine is generated by IBLBaker (as suggested in https://doc.babylonjs.com/how_to/physically_based_rendering) only for exclude it as a cause of the issue, I'm testing my code with others environments found in BJS 3.1 SDK ("environmetSpecular.dds" and "backgroundSkyBox.dds" via BABYLON.CubeTexture.CreateFromPrefilteredData, and "room.hdr" via BABYLON.HDRCubeTexture) with the same disappointing result.

By the way, when I use the basic PBR Material you propose, I get transparent (not visible) meshes.

Moreover, compiling an hybrid iOS app (Cordova) version for the classic PBR BJS' example (the one with the sphere in front of the Chinese Theatre) shows all invisible (w/o error).

I'm going to try to generate the environment DDS with the following compresion formats:

• DXT for desktop and mobile browsers (Win, macOSX and Linux);
• PVR for iOS hybrid apps;
• and ETC1 for Android hybrid apps.

Any advice about the tool to use (sadly DDS plugin for Gimp 2.0 doesn´t generate all these formats)?

Best regards.

[paleRider]

Hi again:

I've realized that I was using (per convenience and ergonomics) a general scene environment, instead of a per-material one, by means of:

_Scene.environmentTexture=textEnvironment;

Well, if I eliminate (comment) that line I finally can see (of course in black) all the meshes on iOS hybrid app:

I think this is the final proof that a not-compatible iOS-native-app environment DDS compressed format is preventing the meshes from being shown.

Any help with this please? It seems we are closer to a solution.

 

 P.S. By the way, I define my PBR materials this way:

matMono=new BABYLON.PBRMaterial("Material_PBR-Mono",_Scene);
matMono.albedoColor=new BABYLON.Color3(1,1,1);
matMono.metallic=1.0;
matMono.bumpTexture=texMonoNormal;
matMono.microSurface=0.96;
matMono.roughness=0;
matMono.useRoughnessFromMetallicTextureAlpha=false;
matMono.useRoughnessFromMetallicTextureGreen=true;

 

 

[V!nc3r]

Just to check: is this demo works on your iOS? http://www.babylonjs.com/demos/flighthelmet/

[paleRider]

Hi V!nc3r:

Yes it works OK on my iOS, but remember this is not a problematic platform for my development (as stated at the beginning of this thread).

My problem with, basically, not shown meshes when a PBR material is used on them is only experimented on iOS-hybrid-apps (Cordova generated), but not in iOS-browsed-webs.

I think that iOS devices GPUs doesn't like (understand) the same kind of DDS compression format accepted by the rest of intended-for-my-project platforms.

I've read that the compression format must be PVR (or PVRTC). The problem is that I can´t find the way or tool for generate such a compatible environment DDS at the moment.

 Best regards.

 

P.S. ThreeJS seems to support PVR files, at the same time than DDS, in order to avoid this kind of iOS-platform-specific issues.

[Guest]

our DDS are not compressed to avoid problem with IOS

So the compression is not the problem.

The weird thing is that Cordova obviously uses the iOS webview which in turn uses webkit the same way Safari does.

 

I'm wondering if this could be a file issue. Like Cordova not finding the DDS file. Can you try to check if the file is correctly loaded?

Instead of calling CubeTexture.CreaterFromPrefilteredData, please use this:

new CubeTexture(url, scene, null, false, null, function() {console.log("loaded");}, function() {console.log("error");}, undefined, true);

[paleRider]

Dear Deltakosh:

By means of macOSX Safari Development Tools I can open a web console attached to the hybrid app running on my iOS device and, yes, I get the "Error" output.

I don´t understand why it doesn't load with iOS-hybrid-app (?), as obviously the texture is located where I'm trying to load from (and is loaded correctly in the rest of platforms):

texEnvironment=BABYLON.CubeTexture.CreateFromPrefilteredData("assets/textures/environment.dds",_Scene);

or with your suggested code:

texEnvironment=new BABYLON.CubeTexture("assets/textures/environment.dds",_Scene, null, false, null, function() {console.log("loaded");}, function() {console.log("error");}, undefined, true);

Thank you for your time.

[paleRider]

Excuse me, but I have an extra-info very important to share: I've just managed to get the all think working on iOS-hybrid-app, only using the DefaultEnvironment helper:

_Scene.createDefaultEnvironment();

instead of the environment texture:

_Scene.environmentTexture=textEnvironment;

Obviously I get default background color, ground and environment texture that I need to disable/change:

Sadly, when I try to customize the DefaultEnvironment with this code:

_Scene.createDefaultEnvironment({
  environmentTexture: "assets/textures/environment.dds"
});

I suffer again not PBR meshes:

So obviously the problem here lies in the fact that the environment texture path need to be expressed (on iOS-hybrid-app) in a different way that is done in the case of the other textures of the development.

Any advice on this?

[Guest]

OK!!!!!!!!!!!!!

So it is a url location issue  Great! 

Can you try to reference your env texture with an absolute url like here: https://github.com/BabylonJS/Babylon.js/blob/6b41cf3db8ce5f245807eb14f9e6428c2a621127/src/Helpers/babylon.environmentHelper.ts#L189

[paleRider]

Hi Deltakosh, and again thank you very much for your support on this:

As expected, to use an absolute url for the Environment texture:

textEnvironment=BABYLON.CubeTexture.CreateFromPrefilteredData("https://assets.babylonjs.com/environments/environmentSpecular.dds",_Scene);
_Scene.environmentTexture=texEnvironment;

...let us arrive to the correct behavior:

Great!!!

But of course I want to use local (off-line) assets. How can I do? ...and why are the rest of PBR and Standard textures working with a URL-path-style and not so the environment one?

[Guest]

The DDS is loaded with an XHR where the other images are loaded with a IMG object

So it has to do with XHR

can you do a profile to try to understand which url the XHR is trying to load?

[paleRider]

Hi Deltakosh:

This is my (spanish UI ) profiler screenshot you asked for:

Strange (under my opinion) is that image load shows HTTP status 200 (OK).

And indeed, going deeper the loaded image is this:

...what by the way has nothing to do with the DDS I specify to use (?).

Maybe I'm wrong but this kind of issue doesn't seems to have a fast solution, doesn't it?

Best regards.

 

P.S. By the way, I've found this related and interesting article.