2D UI Overlay

[Xeonzinc]

Hi All,

 

I've been spending some time trying to figure out the best way to add some decent 2D UI elements to a Babylon js game (i.e. alert boxes, status bars,clickable graphics on the edge of the screen). Ideally I'm trying to create the feel I've seen on the HTML5 polycraft game.

 

 

Currently my only options seems to be:

 

• Use a dynamic texture on a plane always facing the camera

          - I can't use existing components on here and it seems like it might be quite inefficient rendering all the 2D elements in 3D space.

 

• Use multiple canvas layers

         - The works and allows the use of pre-built HTML5 canvas components (such as zebkit components), but gets quite messy and your canvas layers have to be precisely sized as clicks don't propagate through layers.

 

 

 

What I think I probably really need is a way directly draw in 2D to the canvas as Babylon uses after the 3D rendering cycle completes, is this possible/efficient? Or is there some a simpler way to make a nice UI I have completely missed?

[Temechon]

Hello, 

 

You will need a new canvas to get the 2D context, as I think it is not possible to draw 3D and 2D for the same canvas.

The dynamic texture is not a bad idea, but not quite efficient.

 

Otherwise, you can still use HTML elements : div, input and so on.

[joshcamas]

I've been doing the "two layers approach", but have a problem: To allow the input to go through the GUI canvas, you have to set "pointer-events: none;". This of course makes it so you cant have buttons on the front canvas! D:

 

EDIT: A few minutes after typing this, I had an idea! For any input needed in the webgl canvas code, you could send the x+y the first canvas detected. Derp. xD

[Temechon]

However, I think the best way to handle this is to add HTML elements directly in the html page.

The only drawback I can see is if there are a lot of images, you will need to implement a custom loader, and add all images when there are all loaded.

[joshcamas]

I personally disagree with you on that. Having a canvas is great due to not having to mess with divs. Simply draw a image or text or anything on a canvas, and don't update it until you want it. Not slow at all, and you can access any clicks. Shiny!

To be fair i used HTML gui for my editor, and one very good reason to use HTML is for text boxes. This is all just my way ofdoing things of course.

[Temechon]

Maybe you are right Josh... But all click events seems like a pain in the ass to implement ^^

 

New idea : use sprites ! Babylon can do that easily, and I think you can easily implement action events on them. 

[joshcamas]

If you built a simple 2Dengine, click events would be fine. Every update check to see if there were any clicks on the top level, then set a variable.

Also for camera attachevents, simply attach the top canvas!

[Xeonzinc]

Thanks folks,

 

I've gone and had a go at implementing Josh's idea as I'm a fan of keeping it all canvas based.

 

Good News:

I now have a UI canvas completely covering the Babylon rendering canvas which dispatches mouse events to the Babylon canvas when it's alpha is 0 (i.e. there is nothing on the UI canvas). I also added a section allowing 'drags' (for my purposes arc-camera rotations) to continue working across elements on the UI canvas, which just feels a bit more natural. This works perfectly in Chrome.

 

Bad News:

IE doesn't play nice with duplicating and dispatching the mouse events, so this doesn't work on IE. I've tried a few variations but no luck and am giving up for tonight. Anyone have any ideas? Problem code:

new_event = new MouseEvent(evt.type, evt);			document.getElementById("babylon_canvas").dispatchEvent(new_event);

Anyway here is the full current code for non-IE use:

function transmitter() {	var self = this;	this.click_down = false;		this.transmit_mouse = function(evt) {			var tempx = evt.clientX;		var tempy = evt.clientY;		//Return the alpha channel of pixel below pointer;		cty = document.getElementById("UI_canvas").getContext("2d");		alpha = cty.getImageData(tempx,tempy,1,1).data[3];  			//Transmit when alpha is 0 or the mouse button is down and pointer is moving;		if (alpha==0 || (self.click_down == true && (evt.type == "pointermove" || evt.type == "pointerup"))) {			if (evt.type == "pointerdown") {				self.click_down	= true;			}						new_event = new MouseEvent(evt.type, evt);						document.getElementById("babylon_canvas").dispatchEvent(new_event);		}				if (evt.type == "pointerup") {			self.click_down	= false;		}	};	};	var transmit_event = new transmiter();	UI_canvas = document.getElementById("UI_canvas");UI_canvas.addEventListener("pointerup", transmit_event.transmit_mouse, false);		UI_canvas.addEventListener("pointerdown", transmit_event.transmit_mouse, false);		UI_canvas.addEventListener("pointerout", transmit_event.transmit_mouse, false);UI_canvas.addEventListener("pointerover", transmit_event.transmit_mouse, false);	UI_canvas.addEventListener("pointercancel", transmit_event.transmit_mouse, false);		UI_canvas.addEventListener("pointermove", transmit_event.transmit_mouse, false);UI_canvas.addEventListener("pointerenter", transmit_event.transmit_mouse, false);UI_canvas.addEventListener("pointerleave", transmit_event.transmit_mouse, false);

[Xeonzinc]

Ok, progress. I have now replaced the problem code:

new_event = new MouseEvent(evt.type, evt);			document.getElementById("babylon_canvas").dispatchEvent(new_event);

With this:

if (event.initMouseEvent) {     // all browsers except IE before version 9	pointerEvent = document.createEvent ("MouseEvent");	pointerEvent.initMouseEvent (evt.type, evt.bubbles, evt.cancelable, evt.view, evt.detail, evt.screenX, evt.screenY, evt.clientX,                                         evt.clientY, evt.ctrlKey, evt.altKey,evt.shiftKey, evt.metaKey, evt.button, evt.relatedTarget);	document.getElementById("babylon_canvas").dispatchEvent(pointerEvent);}

This now appears to work in both IE and Chrome, however IE 11 seems unable to handle the sheer number of event creation and initiations for 'pointermove' and slows down to a crawl. As far as I can tell this is the recommend method for event creation in IE, which makes me think the problem may be in the browser rather than my coding.

 

Unless anyone has any bright ideas I'm going to leave this one here with the knowledge that it isn't suitable for "pointermove" events on IE.

[Temechon]

Very nice work, thank you for contributing. I might use this for a next project

 

Cheers, 

[Junior]

Hi Guys,

 

I am trying to create a user interface and I have started a thread on the forum here on Wednesday, Dec. 3, 2014. So far, no one has responded to my request for help. I find your discussion very interesting, and I would like know if you found a solution that you could share with me. I am not a programmer but I use blender a lot for creating 3D models. The project that I am currently working on is the development of an interactive 3D car Model that will allow the user to customize it and then press a button to automatically generate an invoice of all the parts in the model. Are you able to give any assistance?

 

Cheers.  

[Stephen Andrews]

Hi Guys,

 

I am trying to create a user interface and I have started a thread on the forum here on Wednesday, Dec. 3, 2014. So far, no one has responded to my request for help. I find your discussion very interesting, and I would like know if you found a solution that you could share with me. I am not a programmer but I use blender a lot for creating 3D models. The project that I am currently working on is the development of an interactive 3D car Model that will allow the user to customize it and then press a button to automatically generate an invoice of all the parts in the model. Are you able to give any assistance?

 

Cheers.  

 

To be quite frank, I can't understand what you're saying.

 

You want the car to be customize-able, that is clear, but then you want a list of all the parts? Or you're trying to customize each part? Or you just need help making a button? 

 

What exactly is the issue that you need help with?

[Junior]

@ Stephen

 

There are 3 requests. I realize that they are not easy to achieve but I will list them in priority order.

1. Make 4 buttons so that users can view the car from 4 angles. Top, Front, Right, Left and Back 
2. Make 1 button so that users can see labels on the parts as illustrated  here . (The user should be able to click on the labels to see a description or annotation of the parts.)
3. Make 1 button so that a list of all the parts in the model can be generated and sent to a shopping cart for processing.

If you are able to assist with any of the requests, then I would greatly appreciate it.

 

Thanks

[JCPalmer]

Junior,

 

On point 1:  You will cut your work in half, by having the camera "track to constraint" the car, (select mesh, select camera, crtl-t) on the Blender side.  The exporter will convert that to camera.lockedTarget.  Then all you have to do is move the camera.  In Javascript,

 

camera.position.x =  ...;

camera.position.y =  ...;

camera.position.z =  ...;

 

or

camera.position = new BABYLON.VECTOR3(x, y, z)

[Junior]

@ JC Palmer,

 

Thanks for the tip. I will give it a try.

 

Cheers

[Junior]

Thanks, Triblade, I'll take you up on the offer soon.

 

Cheers.