Visual Odometry Integrated Semantic Constraints towards Autonomous Driving 2022-01-7095

Robust data association is a core problem of visual odometry, where image-to-image correspondences provide constraints for camera pose and map estimation. Current state-of-the-art visual semantic odometry uses local map points semantics, building semantic residuals associated with all classes to realize medium-term tracking of points. Considering the problem of inefficient semantic data associations and redundant semantic observation likelihood model in the visual semantic odometry, we propose a visual odometry, Local Semantic Odometry (LVSO), which is integrated with medium-term semantic constraints based on local nearest neighbor distance model. Firstly, the ERFNet model is introduced to predict semantic information, and the local nearest neighbor distance function based on breadth-first search in adaptive distance is used to establish a semantic observation likelihood model to determine the functional form of the semantic residual; then according to features of medium-term constraints, we establish a medium-term sliding window to manage the semantic map points which is used to build data associations with the latest key frame. Considering semantic segmentation reliability and under-constrained semantic residuals, a graph optimization model integrating semantic residuals and reprojection residuals is established. The algorithm is verified on the KITTI dataset. The results show that the absolute pose estimation error of LVSO is reduced by 11.37% and the relative pose estimation error is reduced by 1.67%, which improves the pose estimation accuracy of the semantic odometry and reduces drift in the pose.