solved DeviceOrientationCamera Buffer drift

[2x4b]

I have a POI app that uses the device orientation camera. While the camera works well, the compass in the device drifts. This means that the camera also drifts. I am trying to workout a way to insert a threshold of some sort that will compensate for it. 

This is the best I could do. Any Idea how to make it better?

var rotationBuffer = new BABYLON.Quaternion(0, 0, 0, 0);

var switchFlg = 0;

var rotBuf = BABYLON.Vector3.Zero();

scene.onBeforeRenderObservable.add(function () {

    if ((switchFlg % 2) == 0) {

        rotationBuffer = camera.rotationQuaternion;

        rotBuf = camera.rotationQuaternion.toEulerAngles().toDegrees();

    }

 

    ++switchFlg;

    var camE = camera.rotationQuaternion.toEulerAngles().toDegrees();

    var diff = Number(rotBuf.y).toFixed(10) - Number(camE.y).toFixed(10);

    if (Math.abs(diff) < 0.1 && diff != 0) {

        offsetRotation(camera, diff);

    }

});

 

 

BABYLON.Vector3.prototype.toDegrees=function() {

    return new BABYLON.Vector3(((this.x).toDegrees() + 360) % 360,((this.y).toDegrees()+ 360) % 360,((this.z).toDegrees()+ 360) % 360);

};

 

var offsetRotation = function (cam, deg) {

//can only work if this camera has a rotation quaternion already.

    if (!cam.rotationQuaternion) return;

 

    var cur = cam.rotationQuaternion.toEulerAngles().toDegrees();

 

    cur.y = -deg;

    cur.x=0;

    cur.z=0;

    var tmp = new BABYLON.Quaternion();

    BABYLON.Quaternion.RotationYawPitchRollToRef(cur.y.toRadians(), cur.x.toRadians(), cur.z.toRadians(), tmp);

 

    tmp.normalize();

    tmp.multiplyToRef(cam.rotationQuaternion, cam.rotationQuaternion);

}

 

[dbawel]

Hello @2x4b,

Welcome to the forum. Looking at your math skills, I assume you know this - but I would use Math.floor() as you already have enough information to set limits on the value returned dynamically. Just a little creative math should solve this in a few minutes. This method is one solution to limit drift.

Otherwise, there's always my favorite (as some know on this forum) - good old Linear Algebra. However, I don't see that you need to go there in this case.

DB

[2x4b]

You see math skills? I was not aware of having any math skills. creative copy paste skills; yes, math skills?; nope.

I don't see how math.floor() is useful here. I need to prevent fractions of degrees of movement. unless you propose I multiply all the values by say, 1000 before applying it. I Assume you mean i should apply it to the the threshold?

 

[RaananW]

Hi,

I would assume drift would be solved at a higher level! I don't remember ever experiencing such a drift in the browser, but I have experienced drifts of IMUs, so I know what you mean. 

The problem is not the fractions of degrees. The problem is the accumulation of such a drift. If you round, eventually it will "jump" in one direction instead of drifting. Also, without having the direct input of the gyro, you will not be able to know if the user is the one moving or that it is the drift you are experiencing. 

IMO - it is rather hard to implement drift compensation without having a direct interface to the native sensors - magnetometer, gyro and accelerometer. What device are you using that experiences such a serious drift that requires compensation?

[dbawel]

Usually camera drift is not an issue, as drift is generally an issue when you need to sync the sensors on more than 1 device to return a reliable value. We've dealt with this issue for years in the motion capture industry, and have come up with a few creative solutions. However, I was pointing towards gathering the data from the sensors in your device and unifying them with a specific bias using a math.floor value in a dynamic way. And yes, this obviously requires a common multiplication value to reach the level of detail you need to generate a smooth animation. 

However, I'm not certain why this is an issue if you're not making any scientific measurements or coordinating multiple device cameras as this is a very old problem with tons of solutions. So I'm curious what is the problem you're seeing?

Regardless, if you really need to reduce drift, then it's an issue of time / drift as I'm certain you know. And I've found the simplest solution is to either use a constant bias to unify all your sensor data over time (math.floor as an example), or if you want to up your accuracy, then select a single sensor where the drift is measurable (such as measuring translation and the acceleration separately, and use these known values as a common bias throughout your sensor array.)

Most cameras are already compensating for drift. All of the specs and the filters are posted, so look at the drift compensation for your specific device if you want to know more.

However, if you ever want to go there, then most people in this field are familiar with the Kalman filter due to it's complexity, then there is another filter which is far easier to implement called the "complimentary " filter. This is explained very well on the post below:

http://www.pieter-jan.com/node/11

Sorry I assumed there was a level of math skills used in the script form the original post. I often forget how much code I copy myself. Nice copying job, regardless. If you outline what specific use you are applying your sensor data to, then I assume I can point you to a solution. But do you really have a problem? Are you syncing your camera data in post to the image data from the camera sensor?

DB

[2x4b]

The problem I am having with the device (a samsung s6) is that the azmuth (i am assuming calculated from the magnetometer). As it is a Point of Interest app. the azimuth of the device is very important. Unfortunately, being a consumer device, it drifts. The BJS device orientation camera uses the device orientation to derive the relevant angle. While I can "calibrate" north, south etc, the 3d camera will move when the device is not being moved as a result of the crappy sensors (probably the magnetometer) in the device drifting. (at least, i assume that is the reason).

Best of all, the drift is not consistent. 

RaananW, i am unsure on how to get to the sensors native output. I am building on cordova. If i could get the data, i guess i would check change in azimuth against a 0(relative) value from the other sensors.

dbawel, thanks for the reading material. You lost me at bias. While i understand what you said, i have no understanding of how it is done. Looks like more reading for me!

appreciate the help!

 

 

[dbawel]

@2x4b-

OK, your last post helps. Since your using a Samsung Galaxy Sx phone - you're in luck. However every phone has a device debugger if you know how to look for them The filters to correct the noise by using a bias measurement and applying this back in the filter. Check out the link below:

http://www.analog.com/en/technical-articles/gyro-mechanical-performance.html

So, your's and other phones are most likely out of calibration, as the bias isn't dynamically corrected and applied over time. What does this mean to you - simply re-calibrate the sensors on your phone and it will work for a long time just as good as new with no drift! On the Samsung phones (everyone should do this) just dial *#0*# and this will send you to the very "secret" (yeah right) engineering menu. From there it's a fun playground (at least a good hour worth of play there), but you want to click on the sensor button. This will bring you to the screen to re-calibrate the Gyro. You'll find several buttons, but the two you're looking for are "Gyro Selftest" and "Gyro self test". to re-calibrate,  push the "Gyro Selftest" button first (important as this is the re-calibration), and you'll see the S6 run tests and re-calibrate. To commit this, press the "Gyro self test" button (remember these are two separate buttons) and this will run another test and make sure you're using the new bias.

That's it. All smartphones have a calibration feature, and all smartphone increase in drift over time. But the filters you're looking for are already in place, and just need to be updated. No need to re-write the wheel, which is why I didn't fully understand why you needed a filter in the first place. But I also had to learn this the hard way after owning a Note S4 and a GearVR for a while. But now you can mve forward to spread the word, and let everyone know that if you want performace from these devices, they require maintenance.

Cheers,

DB

[2x4b]

dbawel, I appreciate the extra reading material.

While i have been to the diag screen of the device many times, it would appear that I was not applying the calibration after running the "Gyro SelfTest". I was under the impression that once i got the blue line with a "3" on the compass, it was calibrated. That'll teach me to not read the manual that i don't have. 

I shall perform the steps you indicated as soon as the boss give me the device back.

I wonder how much it will swing due to magnetic interference...

Thanks again!

[2x4b]

Ah.. it didn't work.

So instead I wrote an input handler for the camera. This works with one minor flaw.... it flips the camera 180 on Y when gamma changes from neg to pos (though not every time).

any ideas?

 

private _lastPosition:any;

private _deviceOrientation = (evt: DeviceOrientationEvent) => {

            if (!this._camera._initialQuaternion) {

                this._camera._initialQuaternion = new Quaternion();

            }

  if (this._rotationRate != 0 &&

                this._rotationRate.alpha != null) {

 

                if (Math.abs(this._rotationRate.alpha) > 0.08) {

 

                    // camera position is the last position + the difference to the new position - the drift

                    this._alpha = (evt.alpha !== null ? evt.alpha : 0) - (this._drift);

                    if ((evt.gamma !== null ? evt.gamma : 0) < 0)

                    this._alpha += 180;

                    this._alpha = this._alpha%360;

                    // add to cache

                    this._lastPosition = this._alpha;

 

                } else {

                    // drift is the difference between the sensor input and the camera position

                    this._drift = ((evt.alpha !== null ? evt.alpha : 0) - this._lastPosition) % 360;

 

                    // this._alpha = this._lastPosition;

                }

 

                // this._alpha = evt.alpha !== null ? evt.alpha : 0;

                this._beta = evt.beta !== null ? evt.beta : 0;

                this._gamma = evt.gamma !== null ? evt.gamma : 0;

}

 

[2x4b]

I seem to have solved the issue. This works in landscape mode.

 

        private _deviceOrientation = (evt: DeviceOrientationEvent) => {

            var drift: any = 0;

            this._beta = evt.beta !== null ? evt.beta : 0;

            this._gamma = evt.gamma !== null ? evt.gamma : 0;

 

            if (this._rotationRate && this._rotationRate !== 0) {

 

                if (Math.abs(this._rotationRate.alpha) < 0.08) {

                    // drift is the difference between the sensor input and the camera position

                    drift = ((evt.alpha !== null ? evt.alpha : 0) - this._alpha);

                }

            }

 

            if (Math.abs(this._drift - drift) > 160 &&

                Math.abs(this._drift - drift) < 270) {

                drift += 180;

            }

 

            this._alpha = (((evt.alpha !== null ? evt.alpha : 0) - drift) + 360) % 360;

 

            this._drift = drift;

        }

[GameMonetize]

great job