Canvas 2D Joystick/Button Controls -move with rotation?

[webGLmmk]

Ok before I plunge into some crazy hair-brained attempt at a solution, I need to ask if there's an easier way to accomplish this. I created a set of camera/movement controls using Canvas 2D buttons. Look pretty good I think. Looks aren't everything.

The set on the left adjusts the POSITION of the camera along 3 axes.  The set of buttons on the right adjusts the ROTATION of the camera.

http://www.babylonjs-playground.com/#LYEU3#2

They don't work together at all. If i rotate the camera, then want to move forward in that direction, the position controls on the left move me along the original axes...Its like Im looking out the side window of a car thats moving forward. I want to be able to adjust the camera, then have a forward/backward movement action based on where the camera is pointing. The way it works when you are using keyboard/mouse. Except the buttons are to ensure all touchscreen users can adjust view/movement. I'm going to create more practical scenes with these controls.

position.x/y/z and cameraDirection.x/y/z  both do the same thing. I guess its obvious...the current button listeners are based on the static X/Y/Z axes of the scene. How do I get the camera to move according to the camera rotation? Or is my entire architecture here wrong... You need to be able to swivel the camera along X and Y... And then as long as I had forward and backwards, I could get rid of the other position buttons and have just one group of buttons. But I have to be able to do the forwards/backwards correctly first.

I was hoping someone would see something I dont, or offer a new perspective #pun  ...I've looked through FreeCamera, TargetCamera, and Camera classes...  

Confused by inputs Manager still...not at that level I don't think, unless at some point in the future someone could provide a more indepth non-typescripty breakdown :-D

I'm sure there are better ways to set up another Virtual Joystick. But I like the idea of having button control options...like a cockpit simulation or something...

Relevant code starts at line 333 in the Playground scene:

	forwardButton.pointerEventObservable.add(function (d, s) {
		camera.position.z += 20;
		//camera.cameraDirection.z += 20;
	}, BABYLON.PrimitivePointerInfo.PointerUp);
				
	backButton.pointerEventObservable.add(function (d, s) {
		camera.position.z -= 20;
	}, BABYLON.PrimitivePointerInfo.PointerUp);	

 

[Wingnut]

Hi again, W!

   Yep, I've been here before.  Rotate-around a bit, and suddenly "forward" and "backward" don't match the new rotation. 

http://www.babylonjs-playground.com/#LYEU3#5

Instead of setting .positions and .rotations, we can use the .translate and .rotate funcs.  Rotation-speaking, you have just entered the world of mesh.rotationQuaternion, which is another way to store mesh orientation.  The .rotationQuaternion property does not exist on most mesh... until needed.  When we called the .rotate function in the lines 377-397 area... BJS automatically created thing.rotationQuaternion.  At least I think that's what happened.  On newly-made mesh, console.log(mesh.rotationQuaternion) will return undefined.

You will see many places in the BJS source code... if (!mesh.rotationQuaternion) { install one }

Welcome to Quaternion hell.  heh.  Quaternions will give you a brain tumor, if you are not careful.    Local BJS superhero @RaananW loves them, and drives them great, but he's a robot, I suspect. 

Annnnyyyway... I put a ground in your "space" and I made a "thing" box.  Instead of moving the camera around... we will be moving "thing" around (for now).  Test your buttons, make sure your orientations are working the way you want.  I did some simple testing.  I think "thing" is moving correctly, and per your desires.

When it comes time to use the camera instead of the "thing" box... I would simply parent the camera to thing, and then set thing.visibility = 0.  Easy.  And remove that "testing ground", too, of course.

Pitch and yaw, but no roll? 

Yeah, free cameras don't "roll"... by default... unless... they are parented to a thing that allows z-axis roll.  Yours... would be so.   

You could roll (z-rotate) the free camera!  It's a rarely-seen thing.  You might want to invite the neighbors over for a "FreeCam Z-rot Party"!   Build up the suspense and anticipation, and then blow their minds with your delicious z-rolls!  Yum! 

What a goofball I am, huh?  Hope this helps.  Your scene is starting to look nice and work nice.  Well done!

You know what's coming up next, don't you?  Enabling a physics engine with zero gravity, and instead of .translate and .rotate... you applyImpulses.  Thrust.  That "thing" would drift along in space... just like a real spacecraft.  Put some particleSystem emitters on all 6 sides of thing, so that when you fire a thruster... it emits some "stuff".... maybe some smoke and fire and ice particles and soot.  Coooool.

Maybe... this post and linked ZIP... might be of interest.  It's controller.js file... controls 24 physics-active thruster ports, each having 4 particleSystems (not fleshed-out graphically, but ready for fire, smoke, soot, ice particles, all at the same time)... for a grand total of... 96 particleSystems.  Wow! 

I think you should grab that zip and give it a look-see.  All code... free to steal... even say that you wrote it... it's fine.

 

Fun side story:  The Wingnut Flyer was snagged on an issue for SIX MONTHS... I just COULD NOT figure-out how to fix it.  Wanna know what issue it was?  The exact same issue you just experienced... but in physics engine land.  Once the flyer rotated... left/right/up/down/forward/backward... were mis-oriented.  Truthfully, I shed a few tears when finally Deltakosh and I were able to communicate enough... for him to help me.  Look how large a post YOU needed... to explain the mis-orientation.  I didn't know HOW to ask Deltakosh what I needed. I kept asking him to "run my flyer", "run my flyer", "see what it's doing". 

Then one day... he tossed me a function, and I banged on it for awhile... and suddenly, life became wonderful again.  It's in the controller.js file. It's called doTransformPerFlyerQuat(), and I thanked Deltakosh heavily... in the source.  That simple (not-so-simple) 4-line function... made my entire year!

What a great story, eh?  (yawn)    Party on!

[webGLmmk]

On 9/15/2016 at 9:30 PM, Wingnut said:

Hi again, W!

   Yep, I've been here before.  Rotate-around a bit, and suddenly "forward" and "backward" don't match the new rotation. 

http://www.babylonjs-playground.com/#LYEU3#5

... I put a ground in your "space" and I made a "thing" box.  Instead of moving the camera around... we will be moving "thing" around (for now).  Test your buttons, make sure your orientations are working the way you want.  I did some simple testing.  I think "thing" is moving correctly, and per your desires.

When it comes time to use the camera instead of the "thing" box... I would simply parent the camera to thing, and then set thing.visibility = 0.  Easy.  And remove that "testing ground", too, of course.

Maybe... this post and linked ZIP... might be of interest.  It's controller.js file... controls 24 physics-active thruster ports, each having 4 particleSystems (not fleshed-out graphically, but ready for fire, smoke, soot, ice particles, all at the same time)... for a grand total of... 96 particleSystems.  Wow! 

I think you should grab that zip and give it a look-see.  All code... free to steal... even say that you wrote it... it's fine.

...

Excellent...I do recall having seen something about this in previous threads I've read, I just didn't connect why you'd need it.

So its actually making the mesh the parent to the camera:

	camera.parent = thing;
	thing.visibility = 0;

I almost thought the translate buttons weren't working...until I realized I just wasn't seeing the movement because it was too small...increased the posStep and its fine!

http://www.babylonjs-playground.com/#LYEU3#8

I swear, I gotta watch the whole "scale/view" thing in these scenes...Like too much lighting, movement too small to notice...where it's not the code thats the problem but the way it looks...I bet a good chunk of questions are coming down to things like that...going on my list of things to watch for along with scope.

Good idea with the physics engine and impulses. I can have it emit particles as the camera spins around making people dizzy...like WALL-E.    *edit*   VR anyone?

I want to get some randomized planets going so need to create a constructor that picks one of a few different images, flips it in a random direction, then applies it as a texture...although perhaps its time to look into whatever "procedural generation" is now.

Whoa so whats with the matrix thing? gnarly...Saving that for another day... nice work