To effectively reduce color artifacts and blurring of the CFA interpolation images, a support vector machines (SVM) based interpolation scheme is proposed, in which support vector regression (SVR) is used to estimate the color difference between the two color channels with applying spectral correlation of the R, G, B channels. The neighbor training sample models are selected on the color difference plane with considering spatial correlation, and the unknown color difference between two color channels is estimated by the trained SVM and input pattern, then the missing color values at each pixel can be obtained. Simulation results indicate that the proposed scheme produces visually pleasing full-color images and obtains higher PSNR and smaller NCD results than other conventional CFA interpolation algorithms.