Jump to content

Selecting GUI elements with scene.pickWithRay


Recommended Posts

My Problem:

I want to select a GUI-Element with scene.pickWithRay or would like to be able to use a ray to select a GUI-Element on an object somewhere inside a Scene.


Person in VR goes to a Panel inside a game/ scene and wants to interact with it.

In a VR-HMD he would be able to select it with his controllers.

If he doesn't have controllers a fallback to pick with your head-ray and interact with a press of a button would be nice.

If no controllers or buttons (like on simple Cell Phone), fall back to select when holding the ray on the GUI-Element is the plan.

Current Solution:

public getInteractiveButton (mesh:any) {
            var button = undefined;

// get one of different button types
            if (mesh) {
                if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but0')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but0')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but1')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but1')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but2')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but2')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but3')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but3')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but4')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but4')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName(name + '-playbutton')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName(name + '-playbutton')
                } else {
                    console.log('no button');
            return button;

        public vrinteract () {
            var pickInfo = scene.pickWithRay(Ray);

            if (pickInfo && pickInfo.hit && pickInfo.pickedMesh.name.includes('uiplane')) {
                if ((this.vrMouseOverMesh === undefined || this.vrMouseOverMesh === null) || (pickInfo.pickedMesh.uniqueId !== this.vrMouseOverMesh.uniqueId)) {
                    if (this.vrMouseOverMesh) {
                        let button2 = this.getInteractiveButton(this.vrMouseOverMesh);
                    let button = this.getInteractiveButton(pickInfo.pickedMesh);
                    this.vrMouseOverMesh = pickInfo.pickedMesh as BABYLON.Mesh;
            } else if (this.vrMouseOverMesh) {
                let button2 = this.getInteractiveButton(this.vrMouseOverMesh);
                this.vrMouseOverMesh = null;
            } else {
                console.log('no but');



What I would Like to know is, if there would be a better option.

Link to comment
Share on other sites

  • 2 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...