Is possible to load textures from a arraybufferview?

[CUCHO]

Hi. I am new to using Babylon.js and I've been wondering if is possible to load textures from a arraybufferview.

 

I used Three.js and there exists a function that allows to do.

 

DataTexture (data, width, height, format, type, mapping, wraps, wrapt, magFilter, minFilter, anisotropy)data - The data Should be arraybufferview. It contains the data of image based on the type and format.

Thanks.

[GameMonetize]

Hello and welcome,

 

 

you can use the dynamicTexture to display an arraybuffer:

var dynamicTexture = new BABYLON.DynamicTexture("dynamic texture", 512, scene, true);var textureContext = dynamicTexture.getContext();// textureContext is a Canvas2DContext which is a really powerful tool to use 2D API// For instance we can use the getImageData, putImagedata to paste arraybuffervar imageData = textureContext.getImageData(0, 0, 512, 512);var buf8 = new Uint8ClampedArray(data);imageData.data.set(buf8);textureContext.putImageData(imageData, 0, 0);dynamicTexture.update();

You can also use base64 data directly with a texture:

http://www.babylonjs-playground.com/#17YNKA

[julien-moreau]

var image = my_array_buffer;var texture = new BABYLON.Texture('data:my_image_name', scene, true,true, BABYLON.Texture.BILINEAR_SAMPLINGMODE, onLoad, onError, image, true);// also works with arraybuffers (used for DDS and TGA loading)

The problem is : if you change your arraybuffer, you'll have to reload your textures using this method

If it's a static buffer, you can use this method, or use the powerful dynamic texture as Deltakosh said if your arraybuffer is dynamic

[CUCHO]

Here's the thing. I want to display a DICOM Medical Image. The DICOM file provided to me an ArrayBuffer (Uint8Array) with pixeldata in LUMINANCE format or LUMINANCE ALPHA format.

For LUMINANCE format the ArrayBuffer comes in grayscale with one channel per pixel (L) and for LUMINANCE ALPHA format comes with two channels per pixel (L, Alpha).

 

For transformation to RGBA I have a shader. So, how could I pass this texture in LUMINANCE format or LUMINANCE ALPHA format to the shader to make the transformation to RGBA?

 

For example, in Three.js would be something like this:

// LUMINANCEvar texture = new THREE.DataTexture(    dicom.pixelData,     dicom.columns,     dicom.rows,    THREE.LuminanceFormat);var uniforms = {    uTexture: {type: 't', value: texture},    uWW: {type: 'f', value: dicom.windowWidth},    uWC: {type: 'f', value: dicom.windowCenter},    uAlpha: {type: 'f', value: 1}};var material = new THREE.ShaderMaterial({    uniforms: uniforms,     vertexShader: vShader,     fragmentShader: fShader});

or only in WebGL

// LUMINANCE_ALPHAgl.texImage2D(gl.TEXTURE_2D, 0, gl.LUMINANCE_ALPHA, dicom.Columns, dicom.Rows, 0, gl.LUMINANCE_ALPHA, gl.UNSIGNED_BYTE, dicom.PixelData);// LUMINANCEgl.texImage2D(gl.TEXTURE_2D, 0, gl.LUMINANCE, dicom.Columns, dicom.Rows, 0, gl.LUMINANCE, gl.UNSIGNED_BYTE, dicom.PixelData);

the shader for LUMINANCE:

precision highp float;varying vec2 vUV;uniform sampler2D uTexture;uniform highp float uWW;uniform highp float uWC;uniform highp float uAlpha;void main(void) {    highp vec4 texcolor = texture2D(uTexture, vUV);    highp float intensity = texcolor.r * 256.0;    highp float lower_bound = (uWW * -0.5) + uWC;    highp float upper_bound = (uWW *  0.5) + uWC;    intensity = (intensity - lower_bound) / (upper_bound - lower_bound);    gl_FragColor = vec4(intensity, intensity, intensity, uAlpha);}

I implemented this case in Three.js and WebGL, but the latest version of Three.js has compatibility issues with IE11 (extentions problems in GLSL or something like that) and WebGL is too complex for me. So I'm looking Babylon.js to see if I can implement this case.

 

I tried to implement Luaacro's solution, but bring me an error:

var texture = new BABYLON.Texture('data:texture', scene, true, true, BABYLON.Texture.BILINEAR_SAMPLINGMODE, null, null, dicom.PixelData);error: Uncaught TypeError: undefined is not a function

And Deltakosh's solution I think doesn't work for me. Since canvas2dcontext comes in RGBA format and I can't set another format without processing pixel by pixel and that hurts a lot of performance.

 

Please help me.

 

PD. Sorry for my English, I can read and understand English, but I suck at writing it

[julien-moreau]

Hum, embarrassing... ^^

 

Where exactly does the error happen ?

Can you give us the script's name and its error line ?

[GameMonetize]

There is always a way:)

 

Can you share your babylon.js code? or at least a dicom file? I will create a BABYLON.RawTexture object for you

[GameMonetize]

Annnnd tadammmm!!

 

Latest push introduced

Babylon.RawTexture(data: ArrayBufferView, width: number, height: number, format: number, scene: Scene, generateMipMaps: boolean = true, invertY: boolean = false, samplingMode: number = Texture.TRILINEAR_SAMPLINGMODE)

You can also use:

BABYLON.RawTexture.CreateLuminanceTexture(data: ArrayBufferView, width: number, height: number, scene: Scene, generateMipMaps: boolean = true, invertY: boolean = false, samplingMode: number = Texture.TRILINEAR_SAMPLINGMODE)BABYLON.RawTexture.CreateLuminanceAlphaTexture(data: ArrayBufferView, width: number, height: number, scene: Scene, generateMipMaps: boolean = true, invertY: boolean = false, samplingMode: number = Texture.TRILINEAR_SAMPLINGMODE)

[GameMonetize]

This is pretty alpha so please be indulgent

[CUCHO]

Thank you very much. I never thought that you could create the features I needed, this is pretty amazing. I am very grateful.

 

I've been testing the RawTexture function and I found two details.

 

The DICOM image usually don't come with power of 2 sizes (My sample image has width: 3480 and height: 4248), so bring me this error

Unable to create a BABYLON.RawTexture with specified resolution. You must define power of 2 size.

If I comment that error in the createRawTexture's function brings me this error

WebGL: drawElements: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering or is not 'texture complete'. Or the texture is Float or Half Float type with linear filtering while OES_float_linear or OES_half_float_linear extension is not enabled.

So I look at the setTexture function when TEXTURE_WRAP_S and TEXTURE_WRAP_T settings are configured. To display the DICOM image these parameters must be configured as:

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

It is possible that you make these changes?

 

Again, thank you very much for the help

[RaananW]

For a quick solution (without making changes) try using this:

var width = BABYLON.Tools.GetExponantOfTwo(imageWidth, 1024);var height = BABYLON.Tools.GetExponantOfTwo(imageHeight, 1024);

This will give you the largest exponent of 2 that the image width will fit in, with max value of 1024 (which you can change of course).

[CUCHO]

For a quick solution (without making changes) try using this:

var width = BABYLON.Tools.GetExponantOfTwo(imageWidth, 1024);var height = BABYLON.Tools.GetExponantOfTwo(imageHeight, 1024);

This will give you the largest exponent of 2 that the image width will fit in, with max value of 1024 (which you can change of course).

 

Hi and thank you for the help.

 

I tried your solution but I think it's not useful. The result is an image without form. I guess this is because the TEXTURE_WRAP_S and TEXTURE_WRAP_T are set to REPEAT, and because the sizes are very different from the real ones (3480 -> 2048, 4248 -> 4096).

[GameMonetize]

I'm always looking for good feature so when one comes I implement it

I pushed a new version that should work now

[CUCHO]

It works perfectly, thank you very much. It's a shame that IE11 doesn't work, bring me these errors.

error -> WEBGL11053: INVALID_OPERATION: texImage2D: invalid internal formaterror -> WEBGL11094: INVALID_OPERATION: texImage2D: The enumeration LUMINANCE is currently not supportedwarning -> WEBGL11098: drawElements: texture is a texture that is not a power of two or not a complete MIP map

According to the documentation of the WebGL API for Internet Explorer should have support for LUMINANCE textures and LUMINANCE ALPHA, but apparently not.

http://msdn.microsoft.com/en-us/library/ie/dn302435(v=vs.85).aspx

 

I don't know what else to do, maybe just wait for that support in the future.

 

Here is the DICOM image displayed with Babylon.js

http://www.biopacs.com/mpr/dicom_babylonjs_1.html

 

Thank you very much for all the help you gave me.

[GameMonetize]

I have no issue with IE. it is working well for me

[CUCHO]

I have no issue with IE. it is working well for me

What exact version of IE11 you currently have?

[GameMonetize]

v11.0.9879.0

[Subhasish]

@CUCHO the link(http://www.biopacs.com/mpr/dicom_babylonjs_1.html) that you provided is not loading :(.

Could you please share it here or give and updated url.

 

Thanks

[Jazvec]

Could anyone please advise. I have tried:

img1 = new Uint8Array(1024*512*4); // filled with 1024x512 jpg (RGBA)

materialSphere.diffuseTexture = new BABYLON.RawTexture(img1, 1024, 512, 0, scene);

and I receive error (chrome):

RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering.

Thank you!

[GameMonetize]

Hello can you please provide a PG reproducing the issue?

[Jazvec]

I have managed to fix my issue using the already mentioned code in the thread (new BABYLON.Texture('data:my_image_name', ...) , but I was wondering if there was a "more" direct approach to loading a binary texture (instead of converting it to base64). Let' say you get an image:

divContent.once('postcompose', function(event) {

      var canvas = event.context.canvas;

      var data = canvas.toDataURL('image/jpeg');

      var textureId = 'data:image_'+counter;

}

which works fine, you can directly insert it into the line of code below if your div matches the image dimensions and no additional image manipulation is needed. 

 

So my issue was when I tried a direct approach in JS

var img1 = new Uint8Array(1024*512*4); // filled with 1024x512 jpg (RGBA)

materialSphere.diffuseTexture = new BABYLON.RawTexture(img1, 1024, 512, 0, scene);

this seemed to not work, thus I am not sure if I was using it correctly.

 

Thanks

 

var image = my_array_buffer;var texture = new BABYLON.Texture('data:my_image_name', scene, true,true, BABYLON.Texture.BILINEAR_SAMPLINGMODE, onLoad, onError, image, true);

[Nabroski]

Hello i also interested in image manipulation,
so if you got some interesting ideas in a playground feel free to share

http://www.babylonjs-playground.com/#1VATKX#0

Best