期刊导航

论文摘要

基于串并行卷积门阀循环神经网络的短文本特征提取与分类

Short Text Feature Extraction and Classification Based on Serial-Parallel Convolutional Gated Recurrent Neural Network

作者:唐贤伦(重庆邮电大学 自动化学院, 重庆 400065);林文星(重庆邮电大学 自动化学院, 重庆 400065);杜一铭(重庆邮电大学 计算机学院, 重庆 400065);王婷(重庆邮电大学 自动化学院, 重庆 400065)

Author:TANG Xianlun(School of Automation, Chongqing Univ. of Posts and Telecommunications, Chongqing 400065, China);LIN Wenxing(School of Automation, Chongqing Univ. of Posts and Telecommunications, Chongqing 400065, China);DU Yiming(School of Computer Sci. and Technol., Chongqing Univ. of Posts and Telecommunications, Chongqing 400065, China);WANG Ting(School of Automation, Chongqing Univ. of Posts and Telecommunications, Chongqing 400065, China)

收稿日期:2018-10-20          年卷(期)页码:2019,51(4):125-132

期刊名称:工程科学与技术

Journal Name:Advanced Engineering Sciences

关键字:特征表示;短文本分类;循环神经网络;门阀循环单元

Key words:feature representation;short text classification;recurrent neural network;gated recurrent unit

基金项目:国家自然科学基金项目(61673079);重庆市基础科学与前沿技术研究项目(cstc2016jcyjA1919)

中文摘要

针对短文本数据特征少、提供信息有限,以及传统卷积神经网络(convolutional neural network,CNN)和循环神经网络(recurrent neural network,RNN)对短文本特征表示不充分的问题,提出基于串并行卷积门阀循环神经网络的文本分类模型,处理句子特征表示与短文本分类。该网络在卷积层中去除池化操作,保留文本数据的时序结构和位置信息,以串并行的卷积结构提取词语的多元特征组合,并提取局部上下文信息作为RNN的输入;以门阀循环单元(gated recurrent unit,GRU)作为RNN的组成结构,利用文本的时序信息生成句子的向量表示,输入带有附加边缘距离的分类器中,引导网络学习出具有区分性的特征,实现短文本的分类。实验中采用TREC、MR、Subj短文本分类数据集进行测试,对网络超参数选择和卷积层结构对分类准确率的影响进行仿真分析,并与常见的文本分类模型进行了对比实验。实验结果表明:去掉池化操作、采用较小的卷积核进行串并行卷积,能够提升文本数据在多元特征表示下的分类准确率。相较于相同参数规模的GRU模型,所提出模型的分类准确率在3个数据集中分别提升了2.00%、1.23%、1.08%;相较于相同参数规模的CNN模型,所提出模型的分类准确率在3个数据集中分别提升了1.60%、1.57%、0.80%。与Text-CNN、G-Dropout、F-Dropout等常见模型相比,所提出模型的分类准确率也保持最优。因此,实验表明所提出模型可改善分类准确率,可实际应用于短文本分类场景。

英文摘要

In order to address the problems that the features and information is limited in short text, the short text features are not fully expressed by traditional convolutional neural network (CNN) and recurrent neural network (RNN), a text classification model named convolutional gated recurrent neural network was proposed to represent sentence feature vector and classify short texts. The pooling operation was removed in convolution layerof the model to retain sequential structure and location information in text data. Series-parallel convolution structure was used to extract multi-feature combination of words and local context information as the input of RNN. Then, the gated recurrent unit (GRU) was used as the structure of RNN to represent the sentence features based on the sequential information of text. The features were input to the classifier with additive margin to guide network to learn distinguishing features and realize short text classification. The short text classification data set TREC, MR, and Subj were applied for testing. The influence of network hyper-parameters selection and convolution layer structures on classification accuracy were simulated and analyzed, and common text classification models were compared in experiments. Experimental results demonstrated that the classification accuracy of text data was improved by removing the pooling operation and using smaller convolution kernels for series-parallel convolution in the multi-feature representation. Compared with the GRU with the same number of parameters, the classification accuracy of the proposed model was increased by 2.00%, 1.23% and 1.08% in three datasets respectively. Compared with the CNN with the same number of parameters, the classification accuracy of the proposed model was increased by 1.60%, 1.57% and 0.80% in three datasets respectively. Compared with Text-CNN, G-Dropout, F-Dropout and other common models, the classification results also kept best. Therefore, experiments showed that the classification accuracy was effectively improved by the proposed model, which could be applied to short text classification scenarios.

关闭

Copyright © 2020四川大学期刊社 版权所有.

地址:成都市一环路南一段24号

邮编:610065