The tooth surface coordinates are difficult to be got about HFT hypoid gear. For the problem, a mathematical modeling method of tooth surface based on MATLAB is presented in this paper, by which the tooth surface coordinates can be calculated automatically. Firstly, according to the Gleason machine, the coordinate transformation matrix is established from the machine tool to the workpiece. Secondly, the equation of tooth surface is established based on the gear machining parameters and the tooth germ parameters. Then, the estimated value of the gear meshing mean points is obtained based on the gear machining theory, and other points’ coordinates of the tooth surface can be solved by taking this point as the initial value of the function. Finally, SURFACE, the solution programmer of tooth surface coordinates of HFT hypoid gear, is established. To validate the method presented, taken the hypoid gear as an example, 3D models of the gear sets is established through importing tooth surface coordinates into CATIA Software. There is a little difference between the tooth surface of the established model and the actual gear, so this method can be applied to solve this kind of gears.