Sign in to follow this  

Selecting GUI elements with scene.pickWithRay

Recommended Posts

My Problem:

I want to select a GUI-Element with scene.pickWithRay or would like to be able to use a ray to select a GUI-Element on an object somewhere inside a Scene.


Person in VR goes to a Panel inside a game/ scene and wants to interact with it.

In a VR-HMD he would be able to select it with his controllers.

If he doesn't have controllers a fallback to pick with your head-ray and interact with a press of a button would be nice.

If no controllers or buttons (like on simple Cell Phone), fall back to select when holding the ray on the GUI-Element is the plan.

Current Solution:

public getInteractiveButton (mesh:any) {
            var button = undefined;

// get one of different button types
            if (mesh) {
                if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but0')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but0')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but1')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but1')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but2')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but2')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but3')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but3')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName('but4')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName('but4')
                } else if (mesh.material.emissiveTexture.getChildren()[0].getChildByName(name + '-playbutton')) {
                    button = mesh.material.emissiveTexture.getChildren()[0].getChildByName(name + '-playbutton')
                } else {
                    console.log('no button');
            return button;

        public vrinteract () {
            var pickInfo = scene.pickWithRay(Ray);

            if (pickInfo && pickInfo.hit &&'uiplane')) {
                if ((this.vrMouseOverMesh === undefined || this.vrMouseOverMesh === null) || (pickInfo.pickedMesh.uniqueId !== this.vrMouseOverMesh.uniqueId)) {
                    if (this.vrMouseOverMesh) {
                        let button2 = this.getInteractiveButton(this.vrMouseOverMesh);
                    let button = this.getInteractiveButton(pickInfo.pickedMesh);
                    this.vrMouseOverMesh = pickInfo.pickedMesh as BABYLON.Mesh;
            } else if (this.vrMouseOverMesh) {
                let button2 = this.getInteractiveButton(this.vrMouseOverMesh);
                this.vrMouseOverMesh = null;
            } else {
                console.log('no but');



What I would Like to know is, if there would be a better option.

Share this post

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.