JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE) ›› 2016, Vol. 46 ›› Issue (4): 21-27.doi: 10.6040/j.issn.1672-3961.1.2016.078

Previous Articles     Next Articles

Anomaly detection in network traffic based on online feature selection

MO Xiaoyong, PAN Zhisong*, QIU Junyang, YU Yajun, JIANG Mingchu   

  1. College of Command Information System, PLA University of Science and Technology, Nanjing 210007, Jiangsu, China
  • Received:2016-03-01 Online:2016-08-20 Published:2016-03-01

PDF (PC)

1066

Abstract: Traditional batch feature selection methods had the limitations in time and space when dealing large-scale backbone network traffic. A method based on online feature selection detection was proposed to address the limitations, which integrated the idea of online learning into the linear classification model. When selecting the features, the classifier was first updated by online gradient descent and projected to a L1 ball to ensure that the norm of the classifier is bounded, and then the truncate function was used to control the quantity of features. The analysis results showed that the proposed method could make a good use of the time-sequence property of traffic, reduce the time of anomaly detection and hold the similar accuracy when comparing with the batch methods, and meet the real-time demand of network traffic anomaly detection. The proposed method provided a new idea for the network traffic anomaly detection.

Key words: network traffic, anomaly detection, time-sequence, online feature selection, batch learning

CLC Number: 

  • TP181
[1] 杨龙琪. 网络安全态势感知关键技术研究[D]. 南京: 中国人民解放军理工大学, 2015. YANG Longqi. Key techniques of network security situation awareness[D]. Nanjing: PLA University of Science and Technology, 2015.
[2] MOORE A, ZUEV D, CROGAN M. Discriminators for use in flow-based classification[R]. UK: Computer Science Department, Queen Mary University of London, 2005.
[3] LI Wei, MOORE A. A machine learning approach for efficient traffic classification[C] //Proceedings of 15th International Symposium on MASCOTS'07. Istanbul, Turkey: IEEE Press, 2007:310-317.
[4] MOORE A, ZUEV D. Internet traffic classification using bayesian analysis techniques[J]. Acm Sigmetrics Performance Evaluation Review, 2005, 33(1):50-60.
[5] KIM H, CLAFFY K, FOMENKOV M, et al. Internet traffic classification demystified: myths, caveats, and the best practices[C] //Proceedings of the 2008 ACM CoNEXT Conference. Madrid, Spain: ACM Press, 2008:1-12.
[6] NGUYEN T, ARMITAGE G. A survey of techniques for internet traffic classification using machine learning[J]. Communications Surveys & Tutorials, 2008, 10(4):56-76.
[7] ZHAO Zheng, MORSTATTER F, SHARMA S, et al. Advancing feature selection research[R]. USA:School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, 2010.
[8] KATAKIS I, TSOUMAKAS G, VLAHAVAS I. On the utility of incremental feature selection for the classification of textual data streams[C] // Proceedings of the 10th Panhellenic Conference on Informatics. Volos, Greece: Springer Berlin Heidelberg Press, 2005:338-348.
[9] WENERSTROM B, GIRAUD-CARRIER C. Temporal data mining in dynamic feature spaces[C] // Proceedings of the Sixth ICDM'06. Hong Kong, China: IEEE Computer Society Press, 2006:1141-1145.
[10] MASUD M, CHEN Q, GAO J, et al. Classification and novel class detection of data streams in a dynamic feature space[C] // Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases. Barcelona, Spain: Springer Berlin Heidelberg Press, 2010:337-352.
[11] YANG Longqi, HU Guyu, LI Dong, et al. Anomaly detection based on efficient Euclidean projection[J]. Security and Communication Networks, 2015, 8(17):3229-3237.
[12] WIDROW B, HOFF M E. Adaptive switching circuits[C] // Proceedings of the 1960 IRE WESCON Convention Record. Los Angeles, USA: Institute of Radio Engineers Press, 1960:96-104.
[13] ROSENBLATT F. The perceptron: a probabilistic model for information storage and organization in the brain[J]. Psychological Review, 1958, 65(6):386-408.
[14] FREUND Y, SCHAPIRE R E. Large margin classification using the perceptron algorithm[J]. Machine Learning, 1999, 37(3):277-296.
[15] WANG Jialei, ZHAO Peilin, HOI S C H, et al. Online feature selection and its applications[J]. Knowledge and Data Engineering, 2014, 26(3):698-710.
[16] ABERNETHY J, BARTLETT P, RAKHLIN A. Multitask learning with expert advice[C] // Proceedings of the 2007 COLT. San Diego, USA: Springer Berlin Heidelberg Press, 2007:484-498.
[17] LUGOSI G, PAPASPILIOPOULOS O, STOLTZ G. Online multi-task learning with hard constraints[C] // Proceedings of the COLT'09. Montreal, Canada: ACL Press, 2009:315-320.
[18] WARMUTH M K, KUZMIN D. Online variance minimization[J]. Machine Learning, 2012, 87(1):514-528.
[19] DEKEL O, GILAD-BACHRACH R, SHAMIR O, et al. Optimal distributed online prediction using mini-batches[J]. The Journal of Machine Learning Research, 2012, 13(1):165-202.
[20] JAIN P, KULIS B, DHILLON I S, et al. Online metric learning and fast similarity search[C] //Proceedings of the NIPS'09. Vancouver, Canada: NIPS Foundation Press, 2009:761-768.
[21] BORDES A, ERTEKIN S, WESTON J, et al. Fast kernel classifiers with online and active learning[J]. The Journal of Machine Learning Research, 2012, 6(3):1579-1619.
[22] DONOHO D L. Compressed sensing[J]. Information Theory, 2006, 52(4):1289-1306.
[23] FONTUGNE R, BORGNAT P, ABRY P, et al. Mawilab: combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking[C] //Proceedings of the 2010 ACM CoNEXT conference. Philadelphia, USA: ACM Press, 2010:1-12.
[1] YANG Ai-min1, ZHOU Yong-mei1, DENG He2, ZHOU Jian-feng3. Method of feature generation and selection for network traffic classification [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 1-7.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[2] YUE Yuan-Zheng. Relaxation in glasses far from equilibrium[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 1 -20 .
[3] CHENG Daizhan, LI Zhiqiang. A survey on linearization of nonlinear systems[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 26 -36 .
[4] WANG Yong, XIE Yudong. Gas control technology of largeflow pipe[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 70 -74 .
[5] LIU Xin 1, SONG Sili 1, WANG Xinhong 2. [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 98 -100 .
[6] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 104 -107 .
[7] CHEN Huaxin, CHEN Shuanfa, WANG Binggang. The aging behavior and mechanism of base asphalts[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 125 -130 .
[8] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 131 -136 .
[9] LI Shijin, WANG Shengte, HUANG Leping. Change detection with remote sensing images based on forward-backward heterogenicity[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 1 -9 .
[10] ZHAO Ke-Jun, WANG Xin-Jun, LIU Xiang, CHOU Yi-Hong. Algorithms of continuous top-k join query over structured overlay networks[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 32 -37 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd