Aiming at the problem of the edge error caused by the traditional empirical mode decomposition method, an improved empirical mode decomposition method is proposed for eliminating the decomposition error in the edge zones of the signals in this paper, in which, a mirror extension method and Gerchberg extrapolation iteration is introduced to eliminate the decomposition error at edges position, respectively. The improved method can effectively suppress the mode decomposition error caused by the signal edge and improve the decomposition accuracy of the EMD. It is also applied in the analysis of the deformed fringe pattern for eliminating the background components in 3D optical measurement, by which a better reconstructed result of 3D surface can be obtained.