Precise head tracking is an absolute necessity in virtual reality and mixed reality applications. In XR applications, everything is based on sight. If a user’s brain “sees” something different than what other senses are reporting, the user will become disoriented. It takes away from the authenticity and enjoyment of the experience, or worse – it causes motion sickness for users
Actually achieving proper head tracking is a complicated task; there are different factors you need to account for in order to get it just right for your device or application. With a strong understanding of these factors combined with key knowledge of the sensors at hand, true-to-life performance is virtually within reach.
Why Head Tracking is Needed
While the actual experience is quite different between AR and VR applications, both require precise head tracking. In augmented or mixed reality, discrepancies in motion jump out more clearly since the virtual world doesn’t move in the same way we know the physical world does. Imagine a physical desk with a virtual cube sitting flat on it. Without proper head tracking, we would notice the instant a cube corner rotates into the table that it’s wrong. The virtual is not abiding by the same physical motion that we know was made.
In VR applications, discrepancies will still quickly cause motion sickness or, at the very least, a negative user experience. Although we’re seeing a completely virtual environment, our brains can still sense small differences between what we see and hear and how we moved our heads. And even if it doesn’t affect us immediately, we are bound to get motion sick.
Similar problems can occur in 3D audio applications. If the sound doesn’t move with the motion and direction of the user’s head, they may notice jumps or lag in the sound.
Critical Factors in Head Tracking
There are 3 critical factors to account for in order to deliver a strong UX:
- Latency: This is the delay between action and reaction. There will always be some degree of latency, but the key is to minimize it so that this delay isn’t detected by the user.
- Accuracy: This refers to the precision in tracking motion or positioning in an XR environment. If the sensors (and their algorithms) are not accurate, you might be able to track head motion in real time but it isn’t going to reflect accurately in the virtual environment.
- Smoothness: You want to create an XR experience that’s free from jumps. Output that changes abruptly breaks immersion and, in the case of games, can even result in death.
Even when these factors are optimized, errors can occur. One common issue is called “jitter”. Jitter usually falls into different categories:
- Timing jitter: This occurs when the answer “arrives” early or late, causing a jump in motion in the XR environment that doesn’t track with the user’s movements.
- Noise: This is visible when the truth value is still, but yet the output is in motion. This is the more critical type of jitter and is usually the cause of motion sickness; it disorients the user because the XR environment moves even though they’re staying still.
So how, exactly, can you design a system for XR that shows users a true-to-life experience? These are its basic components:
- Accurate head tracking
- High sensor output rate
- Low latency
- Stabilized output free from jumps
Achieving Accurate Head Tracking in XR
Accurate head tracking might seem pretty straightforward, but it’s actually quite complicated; there isn’t one right answer for what you can do to achieve it. But there are some things you need to consider.
A Level Environment
You want to make sure your virtual or mixed reality environment is level, so that everything is properly calibrated. Compensating for an accelerometer’s Zero G Offset (ZGO) over time is a good way of keeping a system level with reality and offers stable long-term heading performance.
Using a Magnetometer
At a bare minimum, you’ll need an accelerometer and gyroscope to have accurate head tracking, but you also need to figure out whether or not you should use a magnetometer. If a magnetic environment is unstable, it will produce a lot of different answers for its magnetic north, further complicated by user motion and the environment. But if you have a stable magnetic environment, you get the benefits of a consistent orientation output over time.
Without the use of a magnetometer, you will likely need a system for geo-referencing, to help you determine where you are in the world, like which direction you’re facing and where you are linearly. This requires some outside input.
Accounting for Gyroscope Errors
The gyroscope determines orientation very quickly and isn’t affected by the magnetic environment. However, for all its splendor, consumer-grade gyroscopes naturally drift over time. And if you don’t consider and solve this issue, it can greatly affect UX.
The magnetometer could alleviate this issue by giving stable absolute orientation readings, but 6-axis sensor systems do not have the support of a magnetometer. Drift can be especially problematic when you have a head and hand controllers that are slowly moving away in different directions.
Long-term data from the accelerometer and magnetometer may show orientation information different from the (drifting) gyroscope output. If XR controllers are being moved more quickly than the gyroscope can track, then that data will be lost without algorithmic help. Implementing algorithms to correct this is critical to addressing sensor error.
Running Things Smoothly
Sometimes the answers from truth and the gyroscope disagree (remember, the gyroscope is intended more for the short term, while the accel and magnetometer are better for long-term information). Stabilization is the idea of making sure that “catching up” to the right answer isn’t jarring to the user. Having algorithms and sensors fill in the blanks more cleanly is important for a smooth experience.
Outside of these external choices and tricky errors, the XR system needs its data fast. That’s where the important factors of sensor output rate and latency come into play. A high sensor rate keeps the virtual world in pace with the real world. But it doesn’t matter how accurate your data is if high latency gets it late to the user.
The bottom line? Head tracking is complicated, but it’s the most crucial piece of the entire XR experience. If you don’t account for it in your design, you’re going to create an inauthentic, uncomfortable experience for your users, regardless of the virtual application.
Fortunately, CEVA understands how to solve the problems that are at the crux of delivering true-to-life XR experiences. Our BNO085 has accurate head tracking, all at high sensor rate (1kHz), low latency (coupled with predictive algorithms to overcome with system latency), and stabilization algorithms to keep experiences running smoothly.
We can help you streamline your design process and integrate crucial features for your XR application to save you the headache. To learn more, just contact our team today.
Ceva