Selecting GUI elements with scene.pickWithRay

[rayFox]

My Problem:

I want to select a GUI-Element with scene.pickWithRay or would like to be able to use a ray to select a GUI-Element on an object somewhere inside a Scene.

Example:

Person in VR goes to a Panel inside a game/ scene and wants to interact with it.

In a VR-HMD he would be able to select it with his controllers.

If he doesn't have controllers a fallback to pick with your head-ray and interact with a press of a button would be nice.

If no controllers or buttons (like on simple Cell Phone), fall back to select when holding the ray on the GUI-Element is the plan.

Current Solution:

public getInteractiveButton (mesh:any) {
            var button = undefined;

// get one of different button types
            if (mesh) {
                if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but0')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but0')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but1')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but1')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but2')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but2')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but3')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but3')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but4')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but4')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName(name + '-playbutton')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName(name + '-playbutton')
                } else {
                    console.log('no button');
                }
            }
            return button;
        }

        public vrinteract () {
            var pickInfo = scene.pickWithRay(Ray);

            if (pickInfo && pickInfo.hit && pickInfo.pickedMesh.name.includes('uiplane')) {
                if ((this.vrMouseOverMesh === undefined || this.vrMouseOverMesh === null) || (pickInfo.pickedMesh.uniqueId !== this.vrMouseOverMesh.uniqueId)) {
                    if (this.vrMouseOverMesh) {
                        let button2 = this.getInteractiveButton(this.vrMouseOverMesh);
                        if(button2){
                            button2.onPointerOutObservable.notifyObservers();
                        }
                    }
                    let button = this.getInteractiveButton(pickInfo.pickedMesh);
                    if(button){
                        button.onPointerEnterObservable.notifyObservers();
                    }
                    this.vrMouseOverMesh = pickInfo.pickedMesh as BABYLON.Mesh;
                }
            } else if (this.vrMouseOverMesh) {
                let button2 = this.getInteractiveButton(this.vrMouseOverMesh);
                if(button2){
                    button2.onPointerOutObservable.notifyObservers();
                }
                this.vrMouseOverMesh = null;
            } else {
                console.log('no but');
            }
        }

 

 

What I would Like to know is, if there would be a better option.

[rayFox]

 

Here Is an Example of the working Code

https://www.babylonjs-playground.com/#TLE3BE

[Guest]

Please post your question on the new forum:
https://forum.babylonjs.com/