Lightmap Masking Custom Filter Issue

[Roomy]

Hey, so, I have a relatively simple problem:

I want to cut the solid parts of a lightmap out of a darkness layer, usually one solid color. This can't be done without alpha compositing unless you use custom filters, so I decided to give it a crack. I got a version working in 3.0.0 PIXI (based off of an old lightmap filter), but when transitioning to 5.2.2, I'm experiencing all kinds of problems, namely that the lightmap, despite logging/extracting/etc. as the expected picture, seems to be treated as pure white by the filter.

You can find my demonstration here: https://codepen.io/roomyrooms/pen/mdOQggP?editors=1010

Edited March 15, 2021 by Roomy
Forgot to add my codepen link to the main post

[Roomy]

For those curious, here's a working version in 3.0.8: https://codepen.io/roomyrooms/pen/NWbmKLj?editors=0010

[ivan.popelyshev]

1. Wrong order of background and 

2. your lightmap stored in R channel, not in A, compared to version you gave me

3. you used different shader (old one)

https://codepen.io/ivanpopelyshev/pen/vYyqayo?editors=1010

I dont know what will you do next with it, but i recommend to look at https://pixijs.io/examples/#/plugin-layers/lighting.js , and my biggest layered example https://codesandbox.io/s/tender-franklin-iycmu

You have to understand how filters work exactly, and how renderTextures work, otherwise layering is just not possible. Also, blending and mesh-shaders.

That's the problem with all such projects - one demo looks easy but when you start modifying it, you cant go further without understanding.

Edited March 19, 2021 by ivan.popelyshev