Way to build image pixel by pixel?

[Iacus]

Hello,

I'd like to visualize a simplex noise map by representing the values as shades of gray on a 2d image.

While using Processing/p5js I used to be able to access the pixels array and then set the values and update the image. Is there any similar way of achieving this in pixi, or should I look elsewhere?
If I had to build the image in p5, can I convert it to a pixi texture right away? (meaning, without intermediately saving it to disk or similar)

[ivan.popelyshev]

you need a BufferResource

"new PIXI.BaseTexture(new PIXI.resources.BufferResource(myUint8Array));" - here's baseTexture. when you change osmething, you have to do "baseTexture.resource.update(newBuffer);` or just update() if its the old one.

When time comes pixi will call resource upload() method https://github.com/pixijs/pixi.js/blob/dev/packages/core/src/textures/resources/BufferResource.ts#L55 which will put your buffer into webgl texture.

Yes, PixiJS is not focused on processing images like p5 does it, but the least we can do is to provide API to make manual uploads.

Getting info back from video memory is much harder- its "renderer.extract" plugin, that uses readPixels webgl function. If you do that -i really recommend to go through this WebGL function first, because pixi in that case is just a provider for webgl api's, we cant just make our own API's and cover underlying webgl, because those parts are performance-heavy and have many side-effects.

Edited May 7, 2020 by ivan.popelyshev

[Exca]

You can also use an external 2d canvas for a bit more of drawing api. You could even use p5js to draw that, then use that canvas as a source for basetexture.

[Iacus]

On 5/8/2020 at 10:56 AM, Exca said:

You can also use an external 2d canvas for a bit more of drawing api. You could even use p5js to draw that, then use that canvas as a source for basetexture.

How can I do this with an off-screen canvas? I don't really want to have a p5 sketch on the page as well...

[Exca]

const canvas = document.createElement("canvas");
//draw what you want to canvas..


//Build the basetexture, texture and sprite using the canvas
const canvasTexture = new PIXI.Texture( new PIXI.BaseTexture(canvas));
const sprite = new PIXI.Sprite(canvasTexture);
//Add the sprite where you want.

If the canvas changes, you need to call update to basetexture. (Or it might detect it automatically, cant remember right now).

And if you want to make it a bit faster you could use offScreenCanvas -element instead of canvas. Though that is not yet widely supported. https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas

[Iacus]

On 5/7/2020 at 2:37 PM, ivan.popelyshev said:

you need a BufferResource

"new PIXI.BaseTexture(new PIXI.resources.BufferResource(myUint8Array));" - here's baseTexture. when you change osmething, you have to do "baseTexture.resource.update(newBuffer);` or just update() if its the old one.

When time comes pixi will call resource upload() method https://github.com/pixijs/pixi.js/blob/dev/packages/core/src/textures/resources/BufferResource.ts#L55 which will put your buffer into webgl texture.

I'm trying this method but I'm not sure which range of values my array should have. Right now it has floats from 0.0 to 1.0.

I keep getting error core.es.js:822 WebGL: INVALID_OPERATION: texImage2D: no bound PIXEL_UNPACK_BUFFER
and the texture is a black rectangle.

It's also 1 value per pixel but I'm thinking it probably expects 3 or 4 values per pixel?

[ivan.popelyshev]

you have to pass Uint8Array (4 bytes per pixel) or Uint32Array (1 uint per pixel). Oh, gray.. either use different texture format, either use four numbers (clr, clr, clr, 255) for every pixel

Or try that one, 1 byte per pixel:

baseTexture.format = WebGLRenderingContext.LUMINANCE;

Welcome to webgl texture uploading science

Edited May 13, 2020 by ivan.popelyshev

[Iacus]

Hello again.

I'm coming back to this after a few days off...

So, I converted the values to grays with full alpha and packed them as a Uint32Array, and yet I get this error:
WebGL: INVALID_OPERATION: texImage2D: type UNSIGNED_BYTE but ArrayBufferView not Uint8Array or Uint8ClampedArray

This is an excerpt of the code I have. I only get a black square despite the values in the array looking ok

let options = {
        width: width,
        height: height,
    };
let colorValues = Uint32Array.from(colorHexValues);

let br = new PIXI.resources.BufferResource(colorValues, options);
let bt = new PIXI.BaseTexture(br);
let texture = new PIXI.Texture(bt);

let noiseImg = new PIXI.Sprite(texture);
noiseImg.position = { x: 100, y: 100 };
this.root.addChild(noiseImg);

 

[Martin Pabst]

Thank you all for this post. It helped a lot!

This worked for me:

let options = {
        width: width,
        height: height,
    };
let colorValues = Uint32Array.from(colorHexValues);

let u8 = Uint8Array(colorValues.buffer);             // This doesn't copy. It's just another view to same memory location

let br = new PIXI.resources.BufferResource(u8, options);   // constructor only works with Uint8Array
let bt = new PIXI.BaseTexture(br);
let texture = new PIXI.Texture(bt);

let noiseImg = new PIXI.Sprite(texture);
noiseImg.position = { x: 100, y: 100 };
this.root.addChild(noiseImg);

 

Edited May 10, 2021 by Martin Pabst