pixels() returns only 0

[DeusX]

Hello,

I'm currently working on some small project with pixi but with more complex shapes. Now I wanted to look at the image data of my sprites but the pixel values are all 0. I searched on google and a few suggestions were that the renderer hasn't rendered anything yet or that the sprites weren't loaded, but this is not the case because if I take another sprite (the cat from https://github.com/kittykatattack/learningPixi) my program suddenly works (I get an array with more than just all 0). Does this have something to do with my images? They are just .png files with a transparent background.

let Application = PIXI.Application;
let loader = PIXI.loader;
this.app = new Application({width: 556, height: 556});
this.cat = null;
document.body.appendChild(this.app.view);
loader
  .add("sprites/cat.png")
  .load((loader, resources) => {

    this.cat = new PIXI.Sprite(PIXI.loader.resources["sprites/cat.png"].texture);
    this.cat.x = 100;
    this.cat.y = 100;

    this.cat.anchor.x = 0.5;
    this.cat.anchor.y = 0.5;

    this.app.stage.addChild(this.cat);
    console.log(this.app.renderer.extract.pixels(this.cat));

    this.app.ticker.add(delta => this.mainLoop(delta));
});
//this works fine, but if I take something else than the cat.png it doesn't work anymore

Thanks

[ivan.popelyshev]

Cross-origin problems? Which web-server do you use?

[DeusX]

I'm using the electron framework to create a desktop application

[DeusX]

So, I just set up a plain canvas and tried it with the ctx.getImageData(...) function, this returns the same Array with just all zeroes. Any Ideas why this is happening?

[ivan.popelyshev]

Two reasons: 

1. Cross-Origin Ritual Sacrifice. make sure you read how to evade CORS on electron.
2. just after Application was created, something other created two or more webgl contexts and your original renderer context was lost. https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/isContextLost , try call "renderer.gl.isContextLost()" to determine that.

[DeusX]

I'm working on the first point, but I don't think this is the case here.

btw, renderer.gl.isContextLost()  always returns false so I think there are no other contexts are created.

 

[DeusX]

So, I checked it again and the problem was neither CORS nor the webgl context. First of all, I didn't really check the entire array because it was simply too large. I used the following to get the pixel values where I clicked. But this does only work if I don't resize the image. If I use e.g. this.sprite.width = 100, this.sprite.height = 50 it only logs zero. Is there a possibilty to adjust the size and keep the pixel values correct?

this.sprite.on("pointerdown", () => {

 

let px = Math.floor(this.app.renderer.plugins.interaction.mouse.global.x - this.sprite.x);

let py = Math.floor(this.app.renderer.plugins.interaction.mouse.global.y - this.sprite.y);

 

let index = (px+py * this.sprite.width) * 4;

 

console.log(this.app.renderer.extract.pixels(this.sprite)[index]);

)};

[ivan.popelyshev]

You didnt account for anchor and sprite transform. "var p = this.sprite.toLocal(mouse.global)" should give you local coords , then you floor them and shift by (anchor.x * width, anchor.y * height)

Also, `extract` works bad with elements that already have parents, there are no guarantees that this thing wont be shifted. It will also modify the sprite transform which is bad, because you are inside interaction call right now and all interaction with sprite children might go wrong. Use it on your own risk.

Pixi does not support per-pixel interaction, and there's no plugin for it.

1. generate bits from base texture that was loaded and look there, instead of `extract` info.

2. rewrite all pixi shaders, add `mouseCoords` uniform, make sure that shader writes object number there if alpha is not zero. Call `readPixels` on the end of each frame, get object number, find object in tree by that number.

Unfortunately, I dont have time to give you more information and guide through the process. Maybe someone else can add more ideas?

[ivan.popelyshev]

Suppose we go for bitmap. Take baseTexture, its "Image" source, draw it to temporary canvas, call "getImageData" on it, there's your bitmap, cache it somewhere. Don't forget to account for Texture shift (sprite.texture.frame!!!)

[DeusX]

I can't test it right now but from what I understand this should work? Where 100 and 50 are the size of the sprite.

And in my handler I keep my code(with addition of accounting for the things you listed) and work with the this.pixels array?

let canvas = this.app.renderer.plugins.extract.canvas();

let context = canvas.getContext("2d");

context.drawImage(this.sprite.texture.baseTexture.source, 0, 0, 100, 50 );

this.pixels = (context.getImageData(0, 0, 100, 50).data);

[ivan.popelyshev]

let canvas = document.createElement("canvas");
canvas.width = 100;canvas.height = 50;

and then your stuff. You dont need to use "extract". You can also store that temporary canvas somewhere in global or in the texture (cache it!). Also you dont need ALL pixels, just call getImageData(myX, myY, 1, 1);
 

Event better idea: cache whole array that returns getImageData, or only its Alpha component (every fourth elemtn starting from fourth). EVEN BETTER: STORE ALPHA BITS (int32 stuff, i hope you know bitwise operations), that'll give you only one byte of storage per 8 pixels - that's much better trade than 4 bytes per pixel

yeah, if "this.pixels" is your cache then its correct.

I like it when people are not afraid of difficulties

[DeusX]

I made a stationary and fast example right now and it seems to work, thanks. I will include that functionality in my game in the next days and will supply a solution to the problem here.

[ivan.popelyshev]

I'll try to make plugin based on your solution then