Outdoor visual navigation aid for the blind in dynamic environments

This thesis proposes a visual navigation aid for the blind. Our goal is to develop a wearable system to help the Visually Impaired (VI) navigate in highly dynamic outdoor environments. The proposed solution uses both visual sensing and an existing map available online. Our work focuses on position estimation, which consists of two parts: Visual Odometry (VO) and localization. We propose different VO methods to compute user motion even in cluttered environments using either a consumer-grade wearable stereo camera or a smartphone. For the case of the stereo camera, instead of computing egomotion from 3D point correspondences in consecutive frames, we propose finding the ground plane, then decomposing the six Degrees of Freedom (6-DoF) egomotion into a motion of the ground plane, and a 3-DoF planar motion on the ground plane. The ground plane is estimated at each frame by analysis of the disparity array. To decelerate the accumulation of VO error, the Inertial Measurement Unit (IMU) readings are used to approximate the ground plane orientation when walking along flat terrain. VO is extended to a monocular system so that the proposed framework is applicable to smartphone platforms with small pocketsize form factors. In addition, mobile devices are more accessible to VI users. ❧ The VO output is the latest camera pose given in a local coordinate system defined with respect to the first camera frame. To localize the user in global coordinates, the proposed system combines the VO output with the semantic information available in an existing map downloaded from a free map service. The Monte Carlo Localization (MCL) framework is used to match the local motion trajectory of the user with the shape of the street network found in the map. The framework allows the system to estimate both the global position and orientation of the user continuously. Our system is validated on real scenarios of hours of walking in both open terrain and urban environment in Downtown Los Angeles. Experiment results show that our method corrects the cumulative drifting error of position estimation computed from either a stereo camera or smartphone. It also localizes users without Global Positioning System (GPS) even in areas with equiangular intersections and equilength streets. ❧ To accelerate the localization process, we model GPS noise with Gaussian distribution and incorporate GPS measurements as prior knowledge to reduce the search area. Experiment results show that GPS readings improve localization speed even when the signal is noisy in an urban area. In the case of temporary loss, which often occurs in dynamic environments, our system improves re-localization speed by initializing the localization locally with the last known user’s location. ❧ We also focus on choosing appropriate sensors for the position estimation. We explore a variety of possible modalities and different combination between wearable stereo camera, IMU, GPS, and integrated sensors in mobile phones. The lightweight wearable stereo camera not only measures depth, but also provides a natural appearance to the VI user. The disadvantage is that it is composed of two low quality image sensors with an extremely short baseline. On the other hand, the smartphone camera captures images with higher resolution, but lacks depth measurement. By analyzing the experimental results, we found that, surprisingly, the monocular device provides lower position error in urban environments compared with the stereo camera, which is only more accurate in open areas.