Can a StandardMaterial.diffusetexture be applied to a DynamicTexture canvas?

[Luis]

Hello there!

 

I need to generate a DynamicTexture with a bunch of compositing on the canvas.

One of the things will be to first draw an image and then, on top of it, write some text.

The only way I could find in docs and google was using the "new Image() and async loading" approach. But this is not very good for my use case (at least the way it is coded here now).

This is because, as the app starts, all the textures I will use are loaded in the loadingscreen, stored as StandardMaterials and then reused/relinked as the app needed.

So the question is:

After a StandardMaterial is created (and a diffusetexture is loaded and linked), then I start creating a DynamicTexture, is it possible to do something like this?

 

var dyntex = new DynamicTexture();

var ctx = dyntex.getContext();

ctx.drawImage( image_from_diffusetexture, 0, 0, width, height);

 

Sorry for the missing PG. I would not know how to do this in there.

[Gijs]

Hi! It was an interesting thing to figure out. Here is a solution:

CanvasRenderingContext2D.prototype.drawTexture = function(texture, x, y){
    let size = texture.getSize();
    let pixels = texture.readPixels();
    this.putImageData(new ImageData(new Uint8ClampedArray(pixels), size.width, size.height), x, y);
}

So then you would call drawTexture instead of drawImage

[Luis]

Hi Gijs,

It almost work.

The problem is that in this way, apparently, I can't shrink the texture being applied to the canvas  (to be drawn only in a defined constrained rectangle)

I tried this.scale() and manual scale but it seems to apply to the canvas but not to the texture being applied.

Also rotation seems not to work. (and I do rotate the canvas to write text in vertical and horizontal).

I will try my best to make a PG that reproduce the problem.

Thanks for the help!

 

[brianzinn]

Hi Luis - Welcome to the forum!

If you are getting into rotations and shrinking, you could apply an AdvancedDynamicTexture to the diffuseTexture of a  StandardMaterial:
https://doc.babylonjs.com/how_to/gui#advanceddynamictexture

On AdvancedDynamicTexture you can put an image with text over top using the 2D GUI controls like Rectangle, TextBlock and GuiImage.  You have rotations, padding, scaling all available and well tested.  You can see by default how it is applied here to a StandardMaterial:
https://github.com/BabylonJS/Babylon.js/blob/master/gui/src/2D/advancedDynamicTexture.ts#L727
The PG will be helpful to see if AdvancedDynamicTexture and GUI can be of use.  Cheers.

[Gijs]

14 hours ago, Luis said:

Hi Gijs,

It almost work.

The problem is that in this way, apparently, I can't shrink the texture being applied to the canvas  (to be drawn only in a defined constrained rectangle)

I tried this.scale() and manual scale but it seems to apply to the canvas but not to the texture being applied.

Also rotation seems not to work. (and I do rotate the canvas to write text in vertical and horizontal).

I will try my best to make a PG that reproduce the problem.

Thanks for the help!

 

You're welcome... and right of course, my answer wasn't exactly what you asked!

This is more like it:

function getTextureImage(texture, _canvas){
    let canvas = _canvas ? _canvas : document.createElement('canvas');
    let ctx = canvas.getContext('2d');
    let size = texture.getSize();
    let pixels = texture.readPixels();
    canvas.width = size.width;
    canvas.height = size.height;
    ctx.putImageData(new ImageData(new Uint8ClampedArray(pixels), size.width, size.height), 0, 0);
    return canvas;
}

That would be your image_from_diffusetexture

[Luis]

Hi brianzinn and Gijs,

I think I have finished my tests with success.
It was not what I wanted but was what I needed.
The path taken was accord brianzinn sugestion, through AdvancedDynamicTextures.

The good part was that I could achieve:
- Multi-texturing with color blending and alpha control
- Write texts on the object.
- Good control over UV coordinates
- Work with faces on individual basis
- Good set of built-in controls

The bad part was:
- The textures (in GUI.Image) needed to be loaded in runtime. It would be better if they could be loaded with AssetsManager, while LoadScreen was presented, and then reused/linked as we go. (that was the reason of my initial question in this thread)
- The TextBlock control lacks an important feature for my case, the fourth parameter of the function fillText() (that shrinks the text respecting a maximum width).
- I lost the ability to download the generated texture because ADT don't store the texture in the canvas (in getContext()). This was not a requirement per se, but i used this to see what was the underling generated texture. PG proved to be a very good tool in this regard and saved the day.
- The bigger itch was that after finished the ADT, I could only apply it to the emissiveTexture. This almost ruined the test but after disabling light to the mesh it was acceptable. Unfortunately the object now can't have shadowed and specular faces (not as beauty as using diffuseTexture).
- Working in right hand system as in my case was a little confusing (had to work in left and switch after all finished)

I'm glad that i could accomplish all the stuff i needed and the result was good. Just naming here the difficulties to give some feedback.
I read in more than one place in the forums people trying to achieve multi-texturing. Perhaps if we could use the ADT in diffuseTexture that could be another option?

Here is the PG:
http://www.babylonjs-playground.com/#YU58J3#8

line 22 and 23 is meant to be commented and uncommented to be able to change the view to the texture or the cube.

thank you two for the time and effort to help. Much appretiated.

-Luis

 

[brianzinn]

45 minutes ago, Luis said:

It would be better if they could be loaded with AssetsManager

I haven't tried, but is it possible to use the image task?

var imageTask = assetsManager.addImageTask("image task", "img.png");
imageTask.onSuccess = function(task) {
    console.log(task.image);
}

The src could then be "data:image/png;base64,..."?

 

48 minutes ago, Luis said:

The TextBlock control lacks an important feature for my case, the fourth parameter of the function fillText()

That's good feedback.  In the meantime, can you get away with using measureText() with different font sizes to get the right fontSize?  I have heard this before about auto-fitting text, so I think others would find something like that useful as well.

52 minutes ago, Luis said:

The bigger itch was that after finished the ADT, I could only apply it to the emissiveTexture.

If you look at how it is assigned to mesh, it does get applied to the diffuseTexture (by default it is applied to emissive and opacity, but for alpha blending, otherwise emissive and diffuse)
https://github.com/BabylonJS/Babylon.js/blob/master/gui/src/2D/advancedDynamicTexture.ts#L758

Glad you got it working.  Interested if these other ideas will work.  Cheers.