Rapid camera change problem

[svatopluk]

Hello.

I'm developing application which gets filtered (low pass filter) pitch, roll and yaw data from a websocket connection. I use pitch and roll data to change the view angles of camera.

I'm using targetCamera,and I have attached the targetVector using the following code:

camera.lockedTarget = targetVector;

Each time the angle is recieved, the targetVector coordinates are calculated and changed and so the camera view angles are changed too. I recieve around 80 angle triplets per second, so there are 80 targetVector updates per second.

The problem is that the camera transitions are not smooth at all. The are very jittery and uneven. Camera is sometimes making a very small jumps. I have also tried to limit the number of targetVector changes to 10 changes per second but the problem is the same.

So I have two questions:

1. Is this a good way to update camera angles or there are some better ways to do it.
2. Is there a way to make camera transitions smoother

Thanks.

[Wingnut]

Hiya @svatopluk, welcome to the forum.  We need a playground for this one.

http://playground.babylonjs.com/#YINWN#2

Here, the torus is animating in lines 65-73.  We will use its .position to simulate inbound targetVectors.

In lines 57-62, I try setting camera.lockedTarget property every 1/10 second.  There is a test line there, too, for testing camera.setTarget() function.  No change in the issue seen, no matter setTarget() or .lockedTarget.

Up in lines 5-6, I test both targetCamera (line 5) and followCamera can be tested with line 6.  Both show the same issue.  I thought MAYBE... followCamera would be better because of its built-in easing.  But, no noticeable change in issue.

I think this is going to need an animation function.  Instead of force-feeding target/lockedTarget with values, you would feed the ending key-frame of a Vector3 animation.  The animation would run immediately upon seeing a new incoming targetVector, and instantly start a fast animation from currentTargetPosition (the beginning key frame)... to newTargetPosition (the ending key frame).   Therefore, the animation would smooth the "jump").

There may be troubles with an animation, too.  What if the animation finishes before the next targetVector arrives?  And what if a new targetVector arrives before the previous animation is complete?  The incoming targetVectors could easily have inconsistent/varying arrival speeds.  hmm.

Just thoughts.  Not necessarily wise.    Again, welcome.  Experts and idea-people are nearby.... and will surely have some (better) ideas.

[svatopluk]

Thanks for your answer. Sorry for late response but I didn't have the computer around.

I have managed to solve a part of the issue by getting smoother and better angle data. Now the transitions are somewhat smoother, and there are no jumps. But I must admit that it needs improvment. I get the similar result if alpha is set to 0.001. The situation gets bad when the targetVector is changing fast.

So if I understand this right the problem is that the targetVector position is changing too fast? The frequency of the angle data from server is 80 per second. Is that too slow?

[Wingnut]

Hi again, S!  Yeah, it's too slow if you are positioning and rotating large distances/degrees, and net transfers are not speed-consistent (as far as I know).

Can I ask why you want to "stream" camera movements? 

Why not send "packet types"... and one type of packet... is a camFly packet.  When its time to move a camera from one position/orientation to another, you send just one packet.

The client-side packetListener and packetParser... disassembles the packet, and if it's a packet requiring camera-fly service, it "installs" the values into a BABYLON animation, and then says "go".  (ie. request an action from the client-side camera-fly serviceHandler).

The animation's onAnimationEnd callback can report back to the server when the fly is complete (via the server-bound packetMaker service). 

Inside that camFly packet (could be xml), you could command how fast, what route, etc.  ALL the needed goods for the animation are inside the packet.  Let BABYLON/JS do the interpolation (in-beTWEENING) for you... once the packet arrives at the client.  It will do it smoothly, as it is no longer dependent upon internet speeds and targetVector value-differences (amount of difference between THIS targetVector and the previous one)  (step size) (stride). 

The interpolation (tweening/smooth-move-across-key-frames) is built-into the animation system, and 11 types of easing are easily available... between start and stop frames.  The BJS animation system is quite good-at flying things smoothly... easing-up to full speed as it leaves a target, easing-down to a stop at the new position/orientation.

Does this make sense?

Do the same when it is time to move/orient a mesh or light.  Send a changeMesh-type packet, and THAT packet could have instructions in it for a followCamera to be spawned and follow the mesh move/orient.  Or parent a cam to the mesh, first.  I just don't think you are going to like the results seen... when letting the server force-around the camera, directly. 

Maybe you will have 4 packetTypes/serviceHandlers on client side:
- meshChange service
- cameraChange service
- lightChange service
- environmentChange service

All those services could have choice options for animate-to-change or immediate-change.

But, maybe I'm wrong, and/or maybe a packetType parser, service-dispatcher system won't work for your project.  I just cannot foresee a solution that will smooth your direct-to-camera streaming-targetVector stuttering.  Maybe others will have different/better ideas or ways to fix your system.  hmm.  I'll keep thinking.

[svatopluk]

Hello and thank you for your answer.

Just to point out what I'm trying to do. The constraint is that i receive the angles only. I'm programming a airplane like game. Camera is set at some altitude and is looking directly on the ground(90 degrees). Also it is moving constantly. Angles are allowing the user to rotate the camera view from 0-180 pitch and roll.

I will use your suggestion and animate the targetVector movement to get a smooth change of the position. I'm not expecting to get more than 200 messages per second so I will limit the duration of the animation to 5ms and see what I'll get. The connection speed should not be a problem. If it lags too much it will be unplayable anyway so I cannot do anything about it.  This is I think a solid solution, so I'll try it out.

 

[Wingnut]

Ohhh, ok.  So, your server is intervening between user input control changes and camera moves? 

In other words, the user input (such as mouse drag) is going to the server first, and then the server sends the commands back to the client... to turn/position the camera?  The server is between the controller inputs and the camera?

Sorry, I don't have much experience with servers.  Thanks for the clarification and teaching... I appreciate it.  And maybe if I tell you enough wrong things, someone will jump in and correct me... and I'll become a better forum helper.  I'm not doing so well, here, so far. 

[svatopluk]

Controller -> Server -> BABYLON The server is between them, yes.

In this project I am developing all the components so I could choose where and how to prepare the data, either on the controller or on the BABYLON side. But the requirement is to send raw angles because this is a university project.

Thanks for your help. Right now I'm in the process of implementing the changes which we discussed. Cheers

[svatopluk]

Just an update.

I was updating the targetVector position in runRenderLoop function. Number of messages recieved by the server is greater than the number of runRenderLoop function call. Animating is not possible. So, maybe the position is changing to fast for camera transition to be smooth. I cannot conclude anything else.

[Wingnut]

*nod*  I'm still thinking, too.  I was thinking about rubber, believe it or not.  Let's pretend that we have a camera hanging-from surgical tubing... with one end held in our hand.  We could move our hand in jerky motions, but the rubber tubing sort-of refuses to transfer that jerk... to the camera hanging below. 

This is the same principle that a SteadyCam uses, with it's springs and weights.  If we were to use a physics system, we would put lots of mass on the camera, and pull it around with a physics "link" or two.  Physics links aren't very "rigid" by default.  They are a tiny bit stretchy... like rubber, especially when they pull on a mesh with high mass.

I'm not saying that you should move a "widget" that has a heavy camera hanging by some physics links, but, okay, that IS what I am saying.    I'm not promising anything.  There might be wiser ways to do this "rubber averaging system".  You'd still need to slow the targetVector speeds... to about 5 per frame.  The actual camera move... would run about .5 seconds behind the server.  This would allow time for an "averaging system" to analyze the previous 2-10 targetVectors (where the camera HAS BEEN)... and 2-10 of the most recent-arriving targetVectors.  The client is able to look into the future... because the actual camera moves are running .5 seconds behind the server.  During that .5 secs... an averager func does.... something.  I don't know what.  I would say it does an animation with interpolation... but... that's what you just tried.  It all would require MUCH slowing of inbound targetVectors.  Lots slower.

Hmm.  But still, you KNOW why the jerking is happening.  You are streaming targetVectors at the scene... faster than the renderLoop has time to update the cam.  The jerking you see, IS expected, because of what you are trying.  I think you have a good grasp of the issue.  Perhaps your project's design strategy will need to be adjusted.