How to read the pixel at a specific coordinate from color buffer?

[Leo5566]

Hello, 

Currently I am working on a map application which can let the users to interactive with huge amount of entities including points and polygons. I use leaflet map framework and a library called PixiOverlay to achieve that. The data are stored in particleContainer and also indexed by kd-tree. Therefore, we can select entities on the map by searching the coordinates of a click event in the tree.

Now I want to implement an onHover function that changes the appearance of cursor when the mouse pointer hovers on the entities. Soon I noticed that continually search in the tree takes too much time when the amount of entities is huge.

Although I am new to WebGL, I still tried hard to google for potential solutions. I found there is a color picking algorithm which includes the step "Read the pixel at location (mouse_x,mouse_y) from the color buffer which identifies the object." Is it also possible to read the color of a pixel directly in pixi.js?

Thanks.

Edited October 21, 2019 by Leo5566

[ivan.popelyshev]

Its good that you went to the source, you already know that all those things are done through readPixels. This operation is synchronous, it waits for data from GPU. Try not to call it more than once per frame.

webgl readPixels is wrapped by in WebGLExtract class, instance exists in "renderer.extract", find it in docs. I recommend to do "renderer.extract.pixels(x,y,1,1)" just after rendering is done in your gameloop. Gameloop explanation is here: https://github.com/pixijs/pixi.js/wiki/v5-Custom-Application-GameLoop Basically you need a handler with even lower priority than "render". If you call it in mouse event - there's no guarantee that you'll get that info.

Another approach is to render your scene in renderTexture, then you can call extract any time. However, dont just use "extract.pixels(myContainer)" because that will try to render whole container no matter how big its bounds are. You can look inside that code, its actually using "renderer.generateTexture()" - if you copy it you can supply your region in that method. Extract plugin has no magic, except flipY and post-divide, you can do all those things manually if it turns out pixi implementation is not good for your task.

If you want to make pixel-perfect interaction, it can be done without extract/readPixels, see https://github.com/pixijs/pixi.js/wiki/v5-Hacks#pixel-perfect-interaction , maybe parts of that code can help you too.

Welcome to the forums!

Edited October 21, 2019 by ivan.popelyshev

[otb]

Anyone implemented an webgl based efficient picking algorithm in pixijs such as decribed in:

https://webglfundamentals.org/webgl/lessons/webgl-picking.html

https://webgl2fundamentals.org/webgl/lessons/webgl-picking.html

Any hints or implementations would be very welcome.

Thanks