Sensor Fusion and Filtering: Combining Data in Robotic Perception Systems

This article is the second in the “How to Build a Robot?” ”intermediate-level” series for those interested. You can find our first article at the link below:

Sensor fusion is the process of mathematically combining data from multiple sensors to produce a state estimate that is more reliable and accurate than what any single sensor could provide on its own. Because real-world sensors have flaws such as measurement noise, bias, and limited update rates, using raw sensor data directly generally leads to unstable robot behavior.

In this article, we cover the filtering and sensor fusion techniques commonly used in robotics in technical detail.

Why Can’t a Single Sensor Be Trusted?

While every sensor type produces reliable data under certain conditions, it can have significant errors under others. For example, wheel encoders can measure position changes with high precision in the short term, but they accumulate error (drift) over time due to wheel slip.

GPS modules, on the other hand, provide absolute position information over the long term, but they have a low update rate and can experience signal loss indoors or between tall buildings. For this reason, rather than relying on a single sensor, robotic systems favor fusion algorithms that statistically combine data from complementary sensors.

The Kalman Filter: Optimal Estimation for Linear Systems

The Kalman filter is the most widely used fusion algorithm in robotics, producing the most likely state estimate from noisy measurements in linear dynamic systems. The filter operates through an iterative loop made up of two basic steps. In the prediction step, the system’s previous state and its known motion model are used to predict what the next state will be.

In the update step, this prediction is corrected by comparing it against the newly arriving sensor measurement; the weight of this correction is automatically adjusted based on the reliability (covariance) of both the system model and the sensor. The core assumption of the Kalman filter is that the system dynamics are linear and the noise follows a Gaussian distribution; when these assumptions hold, the filter produces a statistically optimal estimate.

The Extended Kalman Filter (EKF): Nonlinear Systems

The vast majority of real robotic systems have nonlinear motion or measurement models; for example, the relationship between a mobile robot’s orientation angle and its change in position involves trigonometric functions.

A standard Kalman filter cannot be applied directly to such systems. The Extended Kalman Filter (EKF) adapts nonlinear system equations to the standard Kalman filter framework by linearizing them around the current state at each step (by computing a Jacobian matrix).

EKF is widely used in mobile robot localization and simultaneous localization and mapping (SLAM) applications, but it should be kept in mind that the linearization approach can increase the margin of error in highly nonlinear systems.

The Complementary Filter: A Low-Computation-Cost Alternative

The complementary filter is a simple but effective technique that can produce results similar to the Kalman filter, particularly for processing IMU data, at a much lower computational cost.

This method takes advantage of the fact that accelerometer data is reliable over the long term but noisy against vibrations, while gyroscope data is precise in the short term but tends to drift over time.

The complementary filter applies a high-pass filter to the gyroscope data and a low-pass filter to the accelerometer data, then takes a weighted average of the two, balancing out the weaknesses of each sensor with the other’s strengths. In microcontroller-based projects with limited processing power, the complementary filter is often the practical choice over EKF.

The Particle Filter: Nonlinear and Multimodal Situations

The particle filter is a Monte Carlo method that represents a robot’s possible state not with a single Gaussian distribution, but with a large number of weighted “particles” (samples).

This approach offers a more flexible solution than EKF in situations where multiple possible hypotheses about the robot’s position must be evaluated simultaneously (for example, if a robot is uncertain which direction it is heading in a symmetric corridor).

In each iteration, the particles are updated according to the motion model, then reweighted based on how well they match the new sensor measurement, with low-weight particles eliminated and replaced by copies of high-weight particles (resampling).

Since the computational cost of the particle filter increases proportionally with the number of particles, real-time applications require a balance between particle count and accuracy.

A Practical Approach to Multi-Sensor Fusion: Combining Sensors with the Extended Kalman Filter

In modern mobile robot and unmanned aerial vehicle projects, it is common practice to combine multiple sensor sources — such as IMU, wheel encoder, GPS, and sometimes visual odometry — within the same EKF framework, rather than relying on a single sensor pair.

In this approach, each sensor is fed into the filter as an “update” at its own update rate; for example, the IMU might provide updates a hundred times per second, while the GPS provides one update per second. By taking into account each sensor’s measurement noise covariance, the filter ensures that a less reliable sensor has less influence on the final estimate.

This multi-source fusion approach is offered as ready-made modules in open-source robotics software frameworks (for example, the robot_localization package in the ROS ecosystem).

Criteria to Consider When Choosing a Filter

Which filtering method to use in a robotics project should be determined based on the system’s degree of linearity, the available processing power, and real-time operation requirements.

A standard Kalman filter is sufficient for simple linear systems, while most real-world robotics applications require EKF. In embedded systems with limited processing power, the complementary filter offers a practical alternative. In uncertain environments where multiple possible hypotheses exist about the robot’s position, the particle filter can be a more suitable option.

Conclusion

Sensor fusion and filtering form the mathematical framework for producing a reliable state estimate from noisy and incomplete sensor data. While the Kalman filter and the extended Kalman filter are the most widely used methods in robotics, alternatives such as the complementary filter and the particle filter can offer more suitable solutions under certain constraints or types of uncertainty.

In the next installment of the series, we will cover how these fusion and control algorithms are organized within a modular software architecture — that is, the use of ROS (Robot Operating System).

---

Frequently Asked Questions