In order to realize the stereo vision navigation of the autonomous robot, the accuracy and rapidity of image matching become the hotspot and difficulty of the research. From the study of the work environment of the mobile robot, the relative position invariance of image matching was put forward, and based on this principle, the matching algorithm based on the nearest neighbor and the second-nearest neighbor SIFT feature point was improved. All features of the pair matching images (the before and after frame images) were sorted by pixel coordinate value of theYaxis. Then, the SIFT feature points were searched from the local area of corresponding relation. If the ratio of the nearest and second nearest neighbor meets certain thresholdT, the point is the pair matching point. Finally, the error matching points were eliminated by the relative position invariance. The improved algorithm avoids searching the nearest neighbor and the second-nearest neighbor SIFT within the global area, so the real-time performance is improved greatly and the error matching points are eliminated mainly. The experiment result showed that the matching speed and accuracy of the present matching algorithm are greatly improved.