Due to the weakness of match information and influence of noise, the calculation precision of depth cannot be guaranteed. Therefore fusion of multiple depth maps is a typical technique for multi-view stereo (MVS) reconstruction. This paper introduced an antinoise fusion method that took advantage of the confidence of 3D points. This method first performed a refinement process on every depth map to enforce consistency over its neighbors, which could remove most errors and fill many holes simultaneously. After refinement, it deleted redundancies of every point by retaining the point that its confidence was maximal in its neighbors. Finally, it obtained a point cloud by merging all depth maps and used an iterative least square algorithm to further eliminate the noise points. The quality performance of the proposed method was evaluated on several data sets and the comparison with other algorithm was also given in the paper.