Event-based Visual Odometry and Inertial Odometry

1. Event-based Visual Odometry (VO)

Event-based Visual Odometry (Event VO) is a paradigm that leverages asynchronous event cameras to estimate motion. Unlike traditional frame-based cameras, event cameras output brightness changes at pixel-level with microsecond resolution.

Sensor Used: Event VO commonly uses Dynamic Vision Sensors (DVS) or DAVIS cameras.

Limitations

• Requires motion to generate events
• Sparse data in static environments
• Challenging outdoors due to noise and illumination variation
• Works better indoors due to controlled lighting

2. Neuromorphic Sensor: DAVIS 240C

The DAVIS 240C sensor combines a standard frame-based camera with an event-based sensor.

Key Characteristics

• Resolution: \(240 \times 180\)
• Outputs both frames and events
• High temporal resolution (events)
• APS grayscale frames available

3. UZH DAVIS 240C Dataset

The DAVIS 240C dataset from UZH provides synchronized multi-modal data for visual-inertial odometry.

Available Data

• RGB Images – Frame-based grayscale images
• events.txt – Asynchronous event stream \((u,v,t,p)\)
• groundtruth.txt – 6-DoF poses
• camera_calib.txt – Intrinsic calibration parameters
• imu.txt – IMU measurements (accelerometer + gyroscope)

The ground truth is represented as: \(\text{Pose} = (t_x, t_y, t_z, q_x, q_y, q_z, q_w)\). This dataset enables both Event VO and Event-based Visual-Inertial Odometry (VIO).

4. Experiment 1: Understanding Event Sensor Readings

Events are accumulated over fixed time windows (30ms) to generate event frames.

Pipeline

1. Read events from 0–30ms (Frame 1)
2. Read events from 30–60ms (Frame 2)
3. Create image grid of size \(H' \times W'\)
4. Accumulate events based on polarity

5. Experiment 2: Event-driven VO

Pipeline

1. Read events.txt
2. Generate time surfaces (Frame 1 and Frame 2)
3. Track features using SSD patches
4. Normalize using camera intrinsics
5. Estimate Essential matrix using 8-point method
6. Recover pose
7. Compare with ground truth (RRE, RTE)

Table 1: Experiment 2: Event VO Performance (Frame Spacing ~30ms)

Pair	Features Tracked	Min RRE (°)	Min RTE (°)
0 → 1	42	3.6075	70.2578
1 → 2	22	1.1908	75.7862
2 → 3	35	14.8443	88.1986
3 → 4	93	1.9156	10.1369
4 → 5	87	0.5051	61.3960
Mean	--	4.4127	61.1551

6. Experiment 3: Event VO with Increased Temporal Window (100ms)

In this experiment, the temporal window for event accumulation is increased from 30ms to 100ms to improve feature correspondences.

Pipeline

1. Read events from events.txt
2. Accumulate events in two windows: Frame 1: \(t \in [0, 100]\) ms and Frame 2: \(t \in [100, 200]\) ms
3. Generate time surfaces for both frames
4. Detect features on Frame 1 (e.g., corners or high gradient regions)
5. Track features to Frame 2 using SSD-based patch matching
6. Normalize matched keypoints using camera intrinsics: \(x_n = K^{-1} x\)
7. Estimate Essential Matrix using 8-point algorithm
8. Recover pose \((R, t)\) from Essential matrix
9. Compare with ground truth to compute RRE and RTE

Table 2: Experiment 3: Event VO Performance (Frame Spacing ~100ms)

Pair	Features Tracked	Min RRE (°)	Min RTE (°)
0 → 1	64	2.8065	68.8572
1 → 2	42	0.9701	71.2876
2 → 3	55	10.5619	84.6981
3 → 4	106	1.1219	8.9631
4 → 5	98	0.38412	63.0639
Mean	--	3.0917	59.3739

7. Experiment 4: Robust Estimation using RANSAC and MAGSAC++

This experiment introduces robust estimation techniques to handle outliers in feature correspondences.

Pipeline

1. Generate time surface images (same as Experiment 3)
2. Detect and match features using SSD-based patch matching
3. Normalize matched keypoints using camera intrinsics
4. Apply robust estimation: RANSAC with \(k = 100\) iterations and MAGSAC++ with adaptive thresholding
5. Estimate Essential Matrix for inlier correspondences
6. Recover relative pose \((R, t)\)
7. Compute RRE and RTE using ground truth

Table 3: Experiment 4: RANSAC vs MAGSAC++ Performance (100 Iterations)

Pair	Total Features	RANSAC (RRE / RTE)	MAGSAC++ (RRE / RTE)
0 → 1	42	1.88° / 83.27°	1.92° / 80.19°
1 → 2	22	3.44° / 77.41°	2.82° / 89.97°
2 → 3	35	1.03° / 64.44°	2.85° / 75.86°
3 → 4	93	4.13° / 14.60°	2.42° / 18.97°
4 → 5	87	0.43° / 63.82°	1.17° / 81.56°

8. Experiment 5: Hough Transform-based Corner Detection

This experiment replaces standard feature detection with Hough-based corner detection tailored for event frames.

Pipeline

1. Generate time surface images (100ms window)
2. Apply Hough Transform to detect corner-like structures
3. Extract keypoints from Hough space
4. Track features between frames using SSD patches
5. Normalize matched keypoints
6. Estimate Essential Matrix (8-point method)
7. Recover pose \((R, t)\)
8. Compute RRE and RTE

Table 4: Experiment 5: Hough-based Feature Matching Performance

Pair	Hough Corners Matched	RRE (°)	RTE (°)
0 → 1	17	171.11	71.24
1 → 2	17	45.60	71.28
2 → 3	38	4.35	61.20
3 → 4	30	4.54	16.32
4 → 5	26	32.39	67.36

9. Experiment 6: Direct Event-based Visual Odometry

Instead of relying on discrete features, this method directly uses intensity variations from event frames.

Pipeline

1. Generate time surfaces for consecutive windows
2. Initialize pose estimate \((R, t)\)
3. Define photometric error between warped event frames
4. Optimize pose by minimizing: \(\sum || I_1(u,v) - I_2(\text{warp}(u,v, R, t)) ||^2\)
5. Iterate optimization using gradient-based methods
6. Output optimized pose
7. Compute RRE and RTE

Table 5: Experiment 6: Direct Event VO (Frame-wise Errors)

Frame Pair	RRE (°)	RTE (°)
0 → 1	33.26	87.10
1 → 2	4.37	78.14
2 → 3	113.81	46.45
3 → 4	33.42	88.32
4 → 5	6.50	125.43

Table 6: Experiment 6: Absolute Trajectory Evaluation (Sim(3) Aligned)

Metric	Value
ATE (RMSE)	0.0328 m
APE Translation (Mean)	0.0307 m
APE Rotation Vector (RMSE)	145.9184°

10. Experiment 7: Direct Event-based Visual-Inertial Odometry

This experiment integrates IMU data with direct event VO for improved robustness.

Pipeline

1. Read IMU data (angular velocity, acceleration)
2. Integrate IMU to obtain motion prior
3. Generate time surface images
4. Use IMU prior to initialize pose
5. Minimize photometric error with IMU regularization: \(E = E_{\text{photometric}} + \lambda E_{\text{IMU}}\)
6. Optimize pose \((R, t)\)
7. Compute RRE and RTE

Visualization Results

Conclusion

Event-based Visual Odometry demonstrates strong performance in high-speed and high dynamic range scenarios. While feature-based methods suffer from sparse correspondences, increasing temporal windows improves robustness. Direct methods and sensor fusion (VIO) further enhance accuracy. However, challenges remain in translation estimation due to limited epipolar constraints and noise sensitivity.