• Content count

  • Joined

  • Last visited

  • Days Won


jonathanlurie last won the day on March 31

jonathanlurie had the most liked content!

About jonathanlurie

  • Rank

Contact Methods

  • Website URL
  • Twitter

Profile Information

  • Gender
  • Location
  • Interests
    Computer graphics, image processing, computational geometry, this kind of things...

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. jonathanlurie

    3D Textures and FLoat32

    Great! I'll try that tomorrow and I'll let you know!
  2. jonathanlurie

    3D Textures and FLoat32

    Hello all, I've been working a bit with 3D textures lately, to display MRI which are sometimes (more often than not) encoded in Float32. So far, I was converting the data into uint8 on a single channel (LUMINANCE) but then thought it would be nice to have my data sent to the GPU directly as Float32 (single channel: gl.R32F ). And I found a bug! Or rather something that I suspect to have been overlooked because not a lot of people uses 3D textures and even less want to have floating point precision. The guilty line of code lays in the Engine L5926 (link). When creating the 3D texture, something like that happens: this._gl.texImage3D(this._gl.TEXTURE_3D, 0, internalFormat, texture.width, texture.height, texture.depth, 0, internalFormat, this._gl.UNSIGNED_BYTE, data); and reading at the doc on MDN, we have a prototype like that (link😞 void gl.texImage3D(target, level, internalformat, width, height, depth, border, format, type, ImageData source); The mistake is in the confusion between 'internalFormat' and 'format' and also in the fact that the type in the Engine is forced to UNSIGNED_INT. Somewhere in the middle of this page, we can find this table: In the case of a RGB or LUMINANCE image in UNSIGNED_BYTE, the piece of code in the Engine would work fine (and it's how I used it until now), but this is only because the 'type' and 'internal type' are the same. In the case or single channel float 32, 'internal format' and 'format' will take different values, respectively gl.R32F and gl.RED, while the type becomes gl.FLOAT . So in my case, I replaced the line of code in the Engine by this dirty hardcoded version: this._gl.texImage3D(this._gl.TEXTURE_3D, 0, this._gl.R32F, texture.width, texture.height, texture.depth, 0, this._gl.RED, this._gl.FLOAT, data); As well as using that: let myDummyTexture = new BABYLON.RawTexture3D( new Float32Array(1),1,1,1, BABYLON.Engine.TEXTUREFORMAT_R32F, this._scene); And it works! (later on , I replaced that by an actual brain MRI texture, and it works too!) In the method Engine._getInternalFormat() , the internalFormat returned for Engine.TEXTUREFORMAT_R32F is gl.RED but the thing is, it should be gl.R32F and we should have another lookup method called for example Engine._getFormat() that returns gl.RED when Engine.TEXTUREFORMAT_R32F is given and a third lookup method (say Engine._getType() ) that would return gl.FLOAT when Engine.TEXTUREFORMAT_R32F is given. Then we would have every possible settings available! Do you think it's an update you could add to the core? Cheers
  3. jonathanlurie

    Quaternion and Euler Angle conversion issue

    Hey @Sebavan , thanks for the fix! You're right, using BABYLON.Quaternion.RotationYawPitchRoll(y, x, z) is a solid alternative! I've just updated to v3.2.0-rc.2 and now I can get my user to think in Euler angles (read/write) but have all my internal logic in quaternions!! Thanks for your reactivity on the updates and on providing playground examples, highly appreciated! Cheers.
  4. jonathanlurie

    Quaternion and Euler Angle conversion issue

    Thanks @Sebavan for the playground! It looks like both are still moving... Also, you made me realize I forgot to normalize my quat!
  5. jonathanlurie

    Quaternion and Euler Angle conversion issue

    I now know that it's the method 'Vector3.toQuaternion()' that is guilty! In my project, I just replaced it by a piece of code I borrowed from glMatrix that made more Babylony: function eulerAngleToQuaternion( eulerAngle={x:0, y:0, z:0}) { let toHalf = 0.5; let x = eulerAngle.x * toHalf; let y = eulerAngle.y * toHalf; let z = eulerAngle.z * toHalf; let sx = Math.sin(x); let cx = Math.cos(x); let sy = Math.sin(y); let cy = Math.cos(y); let sz = Math.sin(z); let cz = Math.cos(z); let quat = { x: sx * cy * cz - cx * sy * sz, y: cx * sy * cz + sx * cy * sz, z: cx * cy * sz - sx * sy * cz, w: cx * cy * cz + sx * sy * sz } return quat; } It now gives the same quaternion back, which is good enough for now. It also seems simpler than the one implemented in babylon.math.ts Cheers.
  6. jonathanlurie

    Quaternion and Euler Angle conversion issue

    Hey all, I've been playing a bit with quaternions to rotate meshes but I also need to use Euler angles. The quaternions are great for my internal cooking because (imo) they are easier to use and more versatile than Euler angles, but Euler angles are good for UI, it speaks to people (especially if we convert everything into degrees (the audience of the app is neuroscience researchers, so they would probably think quaternions are black magic). The BJS doc says "once you have used a quaternion, the rotation property of a mesh become null and basically no longer usable". Fortunately, we can have user think they set the rotation with a Euler angle but internally convert in into a quaternion. Unfortunately, I think I went into a bug. Here is what I have: // this._planeSystem is a mesh composed of 3 orthogonal planes let currentQuaternion = BABYLON.Quaternion.FromRotationMatrix( this._planeSystem.computeWorldMatrix(true) ) let eulerAngle = currentQuaternion.toEulerAngles() let someQuaternion = eulerAngle.toQuaternion() /* I expected currentQuaternion to be the same as someQuaternion (with possibly some epsylon) but instead, here is what I have: */ currentQuaternion { w: 0.9987502633525326, x: 0.04997916777685541, y: 0, z: -1.1097604563493715e-17 } eulerAngle { x: 0.09999999729822372, y: -1.1148677691588498e-18, z: -2.2278771887960178e-17 } someQuaternion { w: 0.9987502604624825, x: -5.56737237414314e-19, y: -2.786008172251712e-20, z: 0.04997916792147843 } /* I am not sure if it's toEulerAngles() or toEulerAngles() which is in fault but it looks like someQuaternion.z should have been someQuaternion.x */ I'm not sure, but looking at the source, it looks like you plan on giving the choice of the order. Do you confirm it's a bug? Cheers, Jonathan.
  7. jonathanlurie

    Building a RGB RawTexture

    Hello, I've been messing around with some colormaps I generate. I have a ShaderMaterial and would like to send some colormaps to the shader using ShaderMaterial.setTexture(). My colormaps are dynamically generated and result in a RGB image stored as a Uint8Array. To illustrate the test, I am using a 1x1 texture. I have no problem a RGBA image as uniform on the shader, like that: // this works well let cmTexture = new BABYLON.RawTexture( new Uint8Array([128, 128, 128, 255]), // data 1, // width 1, // height BABYLON.Engine.TEXTUREFORMAT_RGBA, // format this._scene, // scene false, // gen mipmaps false, // invertY BABYLON.Texture.TRILINEAR_SAMPLINGMODE, BABYLON.Engine.TEXTURETYPE_UNSIGNED_INT ) But, if I want to create a RGB texture (no alpha channel), it does not work: // JS lets me do it, but WebGL yells at me let cmTexture = new BABYLON.RawTexture( new Uint8Array([128, 128, 128]), // data 1, // width 1, // height BABYLON.Engine.TEXTUREFORMAT_RGB, // format this._scene, // scene false, // gen mipmaps false, // invertY BABYLON.Texture.TRILINEAR_SAMPLINGMODE, BABYLON.Engine.TEXTURETYPE_UNSIGNED_INT ) Then, I have the following error: The first warning ("WebGL: ...") occurs at the creation of the RawTexture, while the second one occurs when doing shaderMaterial.setTexture(...) I can still use RGBA so it's not a super big deal, but there is probably a little bug somewhere... Cheers.
  8. jonathanlurie

    Texture 3D, is it real?

    Hey @Deltakosh , I finally took the time to bake a more robust demo, it's here and the source is on Github and part of the list of Pixpipe's example. It's ok to share because all the scripts are on CDN and the volume data as well
  9. jonathanlurie

    Texture 3D, is it real?

    I am making a cleaner example that I can share but there is a thing. My 3D texture comes from an MRI (NIfTI file) and is encoded in Float32. For the sake of simplicity, I was so far converting the buffer from Float32Array into a Uint8Array and used BABYLON.Engine.TEXTUREFORMAT_LUMINANCE as a format. I'd rather use a flag that let me use the original float buffer, it would same time, memory and would allow more operation once on the shader. I thought of doing: let texture3D = new BABYLON.RawTexture3D( img3D._data, // <--- THIS IS a Float32Array dim0, dim1, dim2, BABYLON.Engine.TEXTUREFORMAT_LUMINANCE, scene ) // THIS IS THE LINE THAT MATTERS texture3D.textureType = BABYLON.Engine.TEXTURETYPE_FLOAT; Unfortunately, this does not work and I have a warning message saying "texImage3D: type UNSIGNED_BYTE but ArrayBufferView not Uint8Array". For the record, the float buffer I want to send has values from in [0, few thousands] (my point is, it's not a [0, 1] buffer). Is it something possible?
  10. jonathanlurie

    Texture 3D, is it real?

    Hi @Deltakosh , I am currently cleaning up this example a bit to make it more robust because for now, a lot of things are dirty and hardcoded. As soon as it's cleaner (hopefully this week), I'll let you know!! Thanks
  11. jonathanlurie

    Texture 3D, is it real?

    Well, it's already a custom shader and by discarding back, do you mean discarding the black surrounding? If so, then no, researchers and clinicians are used to keep the surrounding because getting rid of it (with a simple threshold) could mean discarding important signal.
  12. jonathanlurie

    Texture 3D, is it real?

    YEAH! it works perfectly! Thanks for adding that to the core! For loading the MRI data I use Pixpipe , which is a lib I am developing at the Montreal Neurological Hospital. I will share this example in a git page because I am not too sure how to deal with dependencies and remote data in a PG, and also the MRI has to be loaded in advance and use a callback that only then creates the scene -- I'm not sure we can do that in the playground. I will update this thread when I have a cleaner source to share! But in the meantime, here is a youtube capture of it in action: Cheers!
  13. jonathanlurie

    Texture 3D, is it real?

    Hi all, I am new here and new with BJS so I might ask for something obvious. Though, I looked well in the resources, how-tos and was digging pretty deeply into BJS codebase and couldn't find any answer. Also, in case of lack of satisfying answers, I'll also post my own solution to my problem (a hack). I want to load a 3D texture of a brain (MRI) and display it on 3 orthogonal planes, then i can move my plane-set around, and even spin it to display oblique slices. On my GLSL code, i need to load my 3D texture as a sampler3D and lookup for some world coordinates to feed gl_FragColor with the colors (actually LUMINANCE) from the texture (Look at the screenshot). And it works, but with a hack, because I could not find how to build a raw 3D texture object because the `RawTexture` class, even though it inherits `Texture`, will only call `engine.createRawTexture()` and never `engine.createRawTexture3D()`. At first, I thought I would just create my texture 3D object calling directly `engine.createRawTexture3D()` but this doesn't create an instance of the prototype/class `Texture`, only an `InternalTexture`, so when the texture validation time comes, calling methods like `myTexture.isReady()` would fail because `isReady()` (as well as other methods) are part of the `Texture` prototype/class. What is missing here is the 3D counterpart of `RawTexture`, so I made it myself and called it `RawTexture3D`, that I declared somewhere in my code where I have access to the BABYLON module: /* (In ES6 syntax) Here, 'data' is an Uint8Array and the rest is the usual things */ class RawTexture3D extends BABYLON.Texture { constructor(data, width, height, depth, format, scene, generateMipMaps = true, invertY = false, samplingMode = BABYLON.Texture.TRILINEAR_SAMPLINGMODE, type = BABYLON.Engine.TEXTURETYPE_UNSIGNED_INT) { super(null, scene, !generateMipMaps, invertY); this._texture = scene.getEngine().createRawTexture3D( data, width, height, depth, format, generateMipMaps, invertY, samplingMode ) this.is3D = true; } } This works well but it would be much better if it was part of the core, especially for maintenance. Say tomorrow `Texture.is3D` gets renamed into `Texture._is3D`, I would probably find out the hard way... If you have another solution to this hack, please share! We are not a lot having to work with 3D textures, so let's help each other! If you were looking for a hack, then this one is yours, and if you are one of the core dev of BabylonJS, then, could you add that to the core, pleeeeaaase? (or maybe i will do a PR at some point but I am not super familiar with TS) And also, thanks for this awesome lib! (I've started to use it yesterday so I don't even fully realize how awesome it is) Cheers.