Metal structures often process defects caused by stress, corrosion and fatigue. Among kinds of non-destructive testing technologies, alternating current potential drop technique (ACPD) as a valid method has been widely used in pitting corrosion and crack measurement. When ACPD is used to test different kind of defects, due to the various geometry shapes, the approach to solve defect depth is different as well. Therefore, it is essential to classify and identify the measured defects in the test region. The purpose of this paper was to find an approach which can classify different defects accurately. In this paper, we studied on two kinds of typical defect based on an eigenvector which consists of 4 adjacent drag factors. After large quantities of simulation, the pitting and crack eigenvector datasets were built. Moreover, a support vector machine (SVM) optimized by genetic algorithm and trained by eigenvector datasets was obtained. Simulation test data showed that the trained and optimized SVM model has a high classification accuracy, and the metal plate experiment also indicated that the model has a good precision in actual defect classification. All the experiments showed that the proposed approach used for defect classification had high precision in pitting and crack classification.