基于SCAD-ESN的时间序列预测模型
Time-series Prediction Model Based on SCAD-ESN
作者：张各各(河南科技大学 网络与通信技术研究所, 河南 洛阳 471023;河南科技大学 网络信息中心, 河南 洛阳 471023)；徐珍(河南科技大学 数学与统计学院, 河南 洛阳 471023)；曾波(河南科技大学 网络与通信技术研究所, 河南 洛阳 471023;河南科技大学 网络信息中心, 河南 洛阳 471023)；陈祥涛(河南科技大学 网络信息中心, 河南 洛阳 471023)
Author：Zhang Gege(Network and Communication Research Inst., Henan Univ. of Sci. and Technol., Luoyang 471023, China;Network and Info. Center, Henan Univ. of Sci. and Technol., Luoyang 471023, China)；Xu Zhen(School of Mathematics and Statistics, Henan Univ. of Sci. and Technol., Luoyang 471023, China)；Zeng Bo(Network and Communication Research Inst., Henan Univ. of Sci. and Technol., Luoyang 471023, China;Network and Info. Center, Henan Univ. of Sci. and Technol., Luoyang 471023, China)；Chen Xiangtao(Network and Info. Center, Henan Univ. of Sci. and Technol., Luoyang 471023, China)
收稿日期：2017-01-23 年卷（期）页码：2017,49(6):129-134
期刊名称：工程科学与技术
Journal Name：Advanced Engineering Sciences
关键字：混沌时间序列预测;回声状态网络;平滑消边绝对偏离罚函数;粒子群算法
Key words：chaotic time series prediction;echo state network;smoothly clipped absolute deviation penalty;particle swarm optimization
基金项目：国家自然科学基金资助项目（11501067）；赛尔网络下一代互联网技术创新项目资助（NGII20150508）
中文摘要
回声状态网络（ESN）是一种重要的时间序列预测方法，但在训练数据存在噪声或野点情况下，ESN将会出现过拟合问题。针对该问题，提出基于平滑消边绝对偏离罚函数的回声状态网络（SCAD-ESN）模型。不同于在模型中加入岭回归、L1范数罚函数及小波降噪等常规方法，该模型利用SCAD罚函数对变量进行选择，将小变量置为零以满足变量稀疏性，将大变量直接置为常数，从而能够很好地解决ESN过拟合问题并满足近似无偏估计。对于SCAD罚函数的非凸函数优化问题，提出基于局部二次近似（LQA）的求解方法，将最小角回归（LQR）方法用于SCAD罚函数求解，避免了计算量巨大的问题。使用基于粒子群优化（PSO）的超参数选取方法快速确定平滑消边绝对偏离-回声状态网络模型的超参数，克服利用经验选取超参数时存在的盲目性较大且难以确定整体最优的超参数问题。混沌系统数值仿真和网络流量仿真结果表明，相对于常规模型，该模型能有效地降低测试误差，从而克服过拟合问题。
英文摘要
Echo state network (ESN) is an important method for time series prediction.However,the overfitting problem is likely to occur when the training data contain noise or outliers.To solve this problem,an ESN model based on smoothly clipped absolute deviation (SCAD) penalty function was proposed in this paper.Different from the traditional methods,such as ridge regression,L1norm penalty,wavelet denoising and other methods added into the ESN model,the SCAD penalty function was used to select the variables of the ESN model.Specially,to meet the variable sparseness,the small coefficients are set to zero.And the large coefficients are taken as constants,which can well solve the over-fitting problem of ESN and satisfy approximate unbiased estimation.For the nonconvex optimization problem of SCAD penalty function,the local quadratic approximation (LQA) solution was presented in the paper,and the enormous computational complexity of the least angle regression (LQR) method for solving the SCAD penalty function was overcome.Then,the particle swarm optimization (PSO) is used to quickly determine the hyperparameters selection of smoothly clipped absolute deviation-echo state network (SCAD-ESN) model.The proposed method overcame the blindness of the conventional methods using the experience to select the hyperparameters,which is blind and difficult to determine the global optimum.Finally,the chaotic system simulation and network traffic simulation showed that,compared with the conventional models,the model can effectively reduce the test error and overcome overfitting problem.
【关闭】
