How bind webGL texture current to shader on PIXI.Mesh?

[marcusfu 3]

Hi,

I want to create a mesh with shader, and through uniforms webGL textures to shader, 

I'm able use pixiapp.renderer.gl.createTexture, it works.

But other sprite in pixiapp(PIXI.application) stage also force to mesh's webGL Texture

is webGL texture binding index confusion? (gl.TEXTURE0)

maybe need to use pixi webgl abstraction?

does anyone have idea?

best regrads.

 

 

[ivan.popelyshev 1074]

Hello!

PixiJS binds textures automatically, you put a link to texture in shader uniform - and it'll work: https://pixijs.io/examples/#/mesh-and-shaders/triangle-textured.js

However, you want to upload texture for yourself. In that case you can use our texture resource API: https://pixijs.io/examples/#/textures/gradient-resource.js

You dont need to create gltexture nor to bind it. Just call texImage2D inside "upload" method. As for texture params, you can either set them in baseTexture wrapMode/scaleMode, either override "style" method of resource, it has the same signature as upload.

If you worried that PixiJS operations wont be optimal, you can snoop them through https://spector.babylonjs.com/ and see that texture params are exactly the same.

Welcome to the forums!

 

Edited February 12, 2020 by ivan.popelyshev

[ivan.popelyshev 1074]

However, if you do manual binds and other stuff, please call "renderer.reset()" method so pixi knows that someone else worked with that context. Yes, we store current boundTexture and active location top optimize our webgl commands sequence.

Its here: https://github.com/pixijs/pixi.js/blob/22c9cc00adaafbfd3a3f51dc74f3c9b4925686dd/packages/core/src/textures/TextureSystem.ts#L127

Edited February 12, 2020 by ivan.popelyshev

[marcusfu 3]

thanks for answer.

I try to use texture resource API: https://pixijs.io/examples/#/textures/gradient-resource.js 

It work like that: 

The texture resource only show one frame, upload function only run once.

How update gl.texImage2D by different data?

I try to bindTexture on custom data update function,

but other object(sprite, geometry) that add after mesh still force to mesh's texture.

[ivan.popelyshev 1074]

easy, just call "baseTexture.update()" , it will chang all updateID's , and TextureSystem wlll see that texture was changed on bind: 

https://github.com/pixijs/pixi.js/blob/1d09bfca1ae1a2fcdcca4c8a80ed59f605abcd76/packages/core/src/textures/BaseTexture.ts#L507

https://github.com/pixijs/pixi.js/blob/1d09bfca1ae1a2fcdcca4c8a80ed59f605abcd76/packages/core/src/textures/TextureSystem.ts#L141

When you call "baseTexture.setSize()" it changes ID's too.

Actually, if its only the format you've changed, you can just set "baseTexture.format" and use our regular BufferResource.

As for bind problem you encountered, I think I need to see your demo to solve that.

Edited February 14, 2020 by ivan.popelyshev

[marcusfu 3]

Thank for answer again!

Actually, I try to make video yuv to rgb in pixi.texture.

After decoded video data bind to three textures(yTexture, uTexture, vTexture),  pixi.shader trans to rgb values in pixi.geometry,

when video decoded data keep coming, that need to update textures(yTexture, uTexture, vTexture).

It's ok that run only one yuv to rgb geometry in canvas, but add other sprite, the textures confuse problem happened.

attach demo code.

pixi_YUVRGB_demo.zip

 

[ivan.popelyshev 1074]

I see you've tried many things

1. re-creating TextureSystem second time

2. resetting the renderer state, binding/unbinding resource to baesTexture

3. binding/unbinding resource to baseTexture on every upload

Unfortunately, that all should not affect behaviour. 

I dont know what is wrong with your case, I dont actually have websocket server with a video locally.

Even without video, I dont know what is supposed result of that demo and what you see is wrong with it.

Can you provide more information? I'm sure I can fix that and make your code much smaller if I at least see the bug with my eyes

However, you use pixi 5.2.0 - maybe 5.2.1 or dev version ( pixijs.download/dev/pixi.js ) will work better, I remember we fixed something related to texture bind

[marcusfu 3]

Thanks for help!

First, I try to replace pixi 5.2.1, the problem still happened.

Next, re-creating TextureSystem and resetting the renderer state really have no effect....

But, when I only try to binding baseTexture or "baseTexture.update()" or "baseTexture.setSize()" on every upload, It seem work!

 

Anyway, I try to present the bug and process in detail by video and picture.

It use Jsmpeg and FFmpeg trans RTMP stream to yuv data, and convert yuv to rgb texture.

Refer attach code above, I use PIXI.resources.Resource to call gl.texImage2D and baseTexture format, and try to update Texture of yuv on every upload.

Situation1:

No bind baseTexture and call gl.texImage2D every custom updateDataMethod(setUint8Array) , only call gl.texImage2D on upload once.

Result:

Mesh only show one frame texture data. can't follow video stream to update texture. 

Situation2: 

Call gl.texImage2D on upload once, bind baseTexture and call gl.texImage2D every custom updateDataMethod(setUint8Array).

Result:

Other sprite's texture confuse refer with mesh's texture.

Situation3:

Only Bind baseTexture every custom updateDataMethod(setUint8Array), and call gl.texImage2D on upload once.

Result:

It's look good, sprite's texture and mesh's texture correct.

 

Actually, I try to change "this.bind(this.baseTexture)" to "baseTexture.update()" or "baseTexture.setSize()", the result is same.

Just like you said, It will change all updateID's , and TextureSystem wlll see that texture was changed on bind.

It seem ok, thanks for help to solve that question!!

 

 

[ivan.popelyshev 1074]

I'm glad you solved it!

So your resource is basically BufferResource that can be updated.