The impact of the system nonlinear on measuring accuracy of three-step phase-shifting measuring?profilometry with the traditional sinusoidal frequency modulated grating projection is discussed. A binary encoded sinusoidal frequency modulated grating is used to improve the accuracy of absolute phase calculation in three-step phase-shifting phase measuring algorithm. A comparison of accuracy of absolute phase in phase-shifting measuring?profilometry with the traditional frequency modulated grating projection and with binary encoded frequency modulated grating projection based on Floyd-Steinberg is completed as well. These results show that the three-step phase-shifting algorithm based on sinusoidal frequency modulated grating template is sensitive to the nonlinear of the measuring system, however, the application of the binary encoded frequency modulated grating template not only maintains the advantage of calculating absolute phase from a single set of fringe patterns, but also avoids the impact from the nonlinear of the system. The new method greatly improves the measurement accuracy of phase-shifting measuring profilometry based on frequency modulated grating projection. Computer simulation and experiment verified the effectiveness of the proposed method.