To solve the problems on high nonlinearity in spectral features in hyperspectral image classification, a classification algorithm, based on multi layer perceptron convolutional layer and batch normalization layer improved convolutional neural network in spectral domain processing, is proposed to improve the nonlinear feature extraction ability. By constructing a seven layer network structure, the algorithm implements a multi layer local sensing structure, analyzes the spectral information pixel by pixel, distinguishes the spectral information of different targets, takes the full spectrum segment set as input, discards the spatial information, and uses the momentum gradient descent training. The algorithm trains multi layer local perceptual convolutional neural networks to realize the extraction and classification of spectral features of different target objects. In the experiment, two sets of hyperspectral remote sensing images are used for comparative analysis. Taking the Pavia University data set as an example, in the case of 3 600 training samples, the test set is 1 800 samples, the accuracy of the proposed method is 9023% and the accuracy of the LeNet 5 and Linear SVM method are 8794% and 9000% respectively. In the case of 21 000 training samples, the test set is all samples, the accuracy is 9723%, 9664% and 9240% respectively for the proposed method, LeNet 5 and Linear SVM method. The experimental results show that the proposed method is superior to the traditional neural network in the case of small training set, which can effectively extract the data features, and is superior to SVM algorithm for the small sample classification in terms of accuracy and computational cost. In the large scale training set, this method shows good learning ability and is superior to LeNet 5 in classification accuracy. The multi layer local perceptual network structure proposed in this paper enhances the learning ability of nonlinear features, it can utilize hyperspectral images much more effectively than traditional SVM and general deep learning networks, both in small sample classification and large sample classification. The spectral domain information of the pixel by pixel point can effectively improve the classification accuracy.