RenderTargetTexture, aspectRatio and camera issues

[ceedee]

Hi
I was experimenting with RenderTargetTexture and have some problems.
Here is playground http://www.babylonjs-playground.com/#1OJL6I#7

Scene is rendered on texture using camera txcam.
And then same objects are rendered on main screen buffer.
Here are problems:

1) Aspect ratio and camera for RenderTargetTexture
When you drag the splitting line (the one betwean editor and scene) or just disable editor using "-Editor" button, the image on texture changes its aspect ratio, which is not good in my case.

 

So i added false ( true actually does nothing there)   for parameter doNotChangeAspectRatio in RenderTargetTexture constructor  ( line 38 in playground )
http://www.babylonjs-playground.com/#1OJL6I#8

Now the texture indeed do not changes aspect ratio but  texture completely ignores the camera which i chosen for it (txcam)  and uses the main camera used in scene.

2) When you look back on first playground ( http://www.babylonjs-playground.com/#1OJL6I#7 )
    and you move the camera around with mouse, specular light on rendered texture moves,  but should not move at all. That black ball is actually a light source ( it appears in main render as well) , its position is not moving so specular should not too.

I will appreciate if someone helps.  Maybe those are bugs, or I am do something in wrong way ?

regards
ceedee

[Nabroski]

chrome vs firefox

EDIT:
Oh im sorry it stooped working after 1-2 hours.

[ceedee]

On 5.02.2017 at 6:20 PM, Nabroski said:

chrome vs firefox

EDIT:
Oh im sorry it stooped working after 1-2 hours.

Your reply was confusing,  but the edit even more confusing. Could you explain?

[Nabroski]

Hello 

Yes, i also know that the playground is always a bit buggy for an inexperience user, you got a default renderloop with predef engine options (best quality) 
It looks like a bug, something wrog with the the call hierarchy  http://doc.babylonjs.com/classes/2.5/engine and (FrameBuffer->Texture)  will be ignored for some reason. 
So when you simple overload the RenderTextureAspectRatio=Engine.AspectRatio it should fix it probably. 

 

"chrome vs firefox

EDIT:
Oh i'm sorry it stooped working after 1-2 hours."

I just opened a window and start to rescale the window, - i was tired, it was late, i was hungry. And somehow it works. Later i return to this post and i stop working.
I know hacking the playground that firefox vs chrome has a slightly different layout management, when you push a layer to z-index:[1,2...n] they not behave the same. 
If i recall right, the split box is pushing the renderCanvas -webgl on a different z-index. But this is a long time ago. Maybe I am also wrong. 


 

[nikokoneko]

I was dealing with the similar problem, and found this post on how to handle PostProcess when resizing windows, this  may help you:

 

[Nabroski]

Here is a resize function. I was experimenting. And as i already mention: a playground has a predefined environment, i would move my project do a "localhost" -server, whatever, and do some test, (what 3th party lib you are using ? bootstrap, default css style etc.) if it fails, i would report a bug.

http://babylonjs-playground.com/#IDNAD#0

doNotChangeAspectRatio=true;
http://babylonjs-playground.com/#IDNAD#1

[ceedee]

On 7.02.2017 at 7:42 PM, Nabroski said:

Here is a resize function. I was experimenting. And as i already mention: a playground has a predefined environment, i would move my project do a "localhost" -server, whatever, and do some test, (what 3th party lib you are using ? bootstrap, default css style etc.) if it fails, i would report a bug.

http://babylonjs-playground.com/#IDNAD#0

doNotChangeAspectRatio=true;
http://babylonjs-playground.com/#IDNAD#1

Thanks, but unfortunately this does not help, image on the texture still changes its ratio.
Answering your question, i don't use any libs in my tests, only playground,  and files saved from playground using "Get .zip" button run locally.

I suppose the problem has something to do with this file: https://github.com/BabylonJS/Babylon.js/blob/master/src/Materials/Textures/babylon.renderTargetTexture.ts in line 333 and 344
which both are:

if (!this._doNotChangeAspectRatio) {
   scene.updateTransformMatrix(true);
}

Why i think that.
As far as i know those matrices have encoded both aspect ratio and camera position.  So when parameter doNotChangeAspectRatio=true it does not update the matrix so position of camera assigned to texture is completely ignored ( in my PG i linked earlier,  there are 2 cameras, one for scene and one for texture) and texture uses same camera position as main screen exactly like on screenshots I posted.
I think matrices should be updated in both cases of doNotChangeAspectRatio  parameter,  but in different way depending on its value ( updating both aspect ratio and camera position or only camera position )
Matrices may have something to with the second issue i described as well, which is strange behavior of specular reflections.

Does what I wrote make any sense?
 

Things get even stranger in this PG: http://www.babylonjs-playground.com/#1OJL6I#10
the difference in this PG i that it uses only one camera for both main render and texture render.
In this one when doNotChangeAspectRatio=true  the image on texture changes it's width like earlier, when using slider.
When doNotChangeAspectRatio=false aspect ratio of image rendered on texture does not change BUT main render do not resize anymore and is stretched when using slider.
Looks like doNotChangeAspectRatio in some way is "leaking" to main render from RenderTargetTexture constructor.
This behavior happens in both playground and locally saved fullscreen version without editor ( you only need to remove #canvasZone div  and leave the canvas itself intact to see it properly).

 

I do my testing on chromium 55.0.2883.87

[Nabroski]

@Vousk-prod. Carbon Copy you have a similar problem (maybe ) i think. 

 

 

Chrome 58 (64-dev) / Firefox 51

Engine Resize Rescale Everything Aspect ratio / seamlessly i don't see any errors. 
maybe
http://stackoverflow.com/questions/31646746/z-index-behaviour-is-different-in-chrome-to-firefox

 

@ceedee 

do not change the aspect = rescale the window will rescale the texture = true

Has less to do with a matrix.

The Raw WebGL is
viewport(0,0,canvas.width,canvas.height)
https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/viewport
https://webglfundamentals.org/webgl/lessons/webgl-resizing-the-canvas.html

and a renderTexture is the FrameBuffer same as Screenshoot. 

http://learningwebgl.com/blog/?p=1786

Sorry, i can help you any further. @Deltakosh 

 

 

[ceedee]

It is not easy to explain things properly, so to make things more clear I prepared some more screenshots.
They were made using PG from my first post in this topic ( http://www.babylonjs-playground.com/#1OJL6I#8 )
The behavior described happens in the playground and without playground as well. Without playground instead of moving slider one can resize the browser window to see that.

first pair for doNotChangeAspectRatio=true ,  in both slider positions:

 

Second one for doNotChangeAspectRatio=false

 

[GameMonetize]

What about this:

http://www.babylonjs-playground.com/#1OJL6I#11

[ceedee]

2 hours ago, Deltakosh said:

What about this:

http://www.babylonjs-playground.com/#1OJL6I#11

Thanks Deltakosh for the reply. Your solution  works fine in this case.

But it takes away from me the control of render which I have in case when i do manually renderTargetTexture.render().
The playground I made is simple with just one render on the texture because this way it was easier to explain the issue, and keep the question as clear as possible.

What i really want to achieve is something like that:

1) render scene on the texture, then
2) analyze the texture image ( reading directly rgb values or other method, does not matter yet at this stage of my project)
3) move texture camera a bit, maybe rotate it,  change some other settings, etc
4) repeat steps 1-3 few times, actual amount of repetitions may be different every time
5) saving information from analysis (step 2) in an array ( object, cache or whatever ) for later usage
6) render final scene in main render, optionally using information from analysis if it is usable.

Is it possible to do it in your way (  scene.customRenderTargets.push(renderTargetTexture); )   ? It seems to be hard as in this case rendering is executed automatically.

Or do it with manual executions of renderTargetTexture.render(),  but in way which gives proper/expected results ?     (As a side question It is interesting why it does not give currently expected results, is it a bug, or maybe I don't understand something.)

 

[GameMonetize]

Hello you can control when a rendertexture is rendered using renderTexture.refreshRate

The refreshRate property is set to 0 if you want the texture to only render once. If set to 1, it will render every frame, 2 every two frames, etc.

 

So basically set refreshRate to 0 anytime you want to update the render texture

[Nabroski]

2 hours ago, ceedee said:

 

1) render scene on the texture, then
2) analyze the texture image ( reading directly rgb values or other method, does not matter yet at this stage of my project)
3) move texture camera a bit, maybe rotate it,  change some other settings, etc
4) repeat steps 1-3 few times, actual amount of repetitions may be different every time
5) saving information from analysis (step 2) in an array ( object, cache or whatever ) for later usage
6) render final scene in main render, optionally using information from analysis if it is usable.

 

Nice 
This is a long Do To List 

keep on exploring

http://babylonjs-playground.com/#RZ83#0

[ceedee]

Hello

Thanks guys for help. I have made some progress.
Here is the  result: http://www.babylonjs-playground.com/#1OJL6I#68
It is something which we could call a "visibility probe",  something like a poor man's occlusion query system but for a bit different purpose.

It renders 6 textures of cube around main camera. Then reads textures using modified version of engine.readPixels and analyzes visibility of all meshes.
Meshes indexes are color coded into textures. Yo can see visible list of indexes at black bar at the top of the screen. Use mouse and arrow keys to fly between meshes.

On 10.02.2017 at 11:37 PM, Deltakosh said:

So basically set refreshRate to 0 anytime you want to update the render texture

This way you can do 1 render of texture per frame _at most_.
What i need is do arbitrary number of renderings of the same texture, per frame, before render main screen.
This is because i need to put on the scene arbitrary number of those visibility probes,  but it will be waste of texture memory, because each probe needs 6 textures.
So using just one probe used multiple times before main render is preffered.

 

On 11.02.2017 at 0:06 AM, Nabroski said:

http://babylonjs-playground.com/#RZ83#0

Your method is a bit slow, you use main screen rendering instead of render texture, and recreate texture every time.
But it gave me an idea. Instead of using DumpFramebuffer and recreating texture i looked into babylon source and found engine.readPixels, and i made custom version of it which does not allocates new TypedArray every time but uses same one. And used it in after render observable of renderTargetTexture
 

@Deltakosh  I will be doing that "visibility probe" again but as a proper class. (what I made in PG is just a messy but working prototype )
Are you interested in putting such component into the library ?

One more time thanks for helping.

[Nabroski]

Hello ceedee
Yes copying pixels i think this is the right track - it should work!

It was a messy stretchy playground yes -
Just a remainder to myself (why it is slow, i just put the framebuffer in the mainloop in the playground (a bad idea) you have to scope it out again in your real world project):
http://babylonjs-playground.com/#RZ83#1
var camera 

var scene

-> some objects
-> RendertargetTex_blendMode: {
        src: 'src alpha',
       dst: 'one minus src alpha

}

runRenderLoop->camera render some objects copy pixels rendertotexture

Man that looks interesting, i currently rushing from one project to an other.
I did a minimap -VR Cam a while ago
i dont know if it helps you. 
http://www.babylonjs-playground.com/#1VVORT#2

Your playground looks promising! 

 

 

 

 

 

[GameMonetize]

Quote

Are you interested in putting such component into the library ?

For sure as long as it is performant