Please wait a minute...
Tsinghua Science and Technology  2021, Vol. 26 Issue (4): 484-495    doi: 10.26599/TST.2020.9010022
    
Intrusion Detection System Using Voting-Based Neural Network
Mohammad Hashem Haghighat*(),Jun Li*()
Department of Automation, Tsinghua University, Beijing 100084, China.
Download: PDF (2444 KB)      HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

Several security solutions have been proposed to detect network abnormal behavior. However, successful attacks is still a big concern in computer society. Lots of security breaches, like Distributed Denial of Service (DDoS), botnets, spam, phishing, and so on, are reported every day, while the number of attacks are still increasing. In this paper, a novel voting-based deep learning framework, called VNN, is proposed to take the advantage of any kinds of deep learning structures. Considering several models created by different aspects of data and various deep learning structures, VNN provides the ability to aggregate the best models in order to create more accurate and robust results. Therefore, VNN helps the security specialists to detect more complicated attacks. Experimental results over KDDCUP’99 and CTU-13, as two well known and more widely employed datasets in computer network area, revealed the voting procedure was highly effective to increase the system performance, where the false alarms were reduced up to 75% in comparison with the original deep learning models, including Deep Neural Network (DNN), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU).



Key wordsdeep learning      Voting-based Neural Network (VNN)      network security      Pearson correlation coefficient     
Received: 24 March 2020      Published: 12 January 2021
Fund:  National Natural Science Foundation of China(61872212);National Key Research and Development Program of China(2016YFB1000102)
Corresponding Authors: Mohammad Hashem Haghighat,Jun Li     E-mail: l-a16@mails.tsinghua.edu.cn;junl@ tsinghua.edu.cn
About author: Mohammad Hashem Haghighat received the BS degree in computer engineering from Shiraz Azad University, Shiraz, Iran in 2008, and the MS degree in computer engineering from Sharif University of Technology, Tehran, Iran in 2010. He is currently a PhD candidate at Tsinghua University, Beijing, China. His research interests include network security, intrusion detection systems, deep learning, and information forensics.|Jun Li received the PhD degree from New Jersey Institute of Technology (NJIT) in 1997, and the MEng and BEng degrees in automation from Tsinghua University in 1998 and 1985, respectively. He is currently a professor at the Department of Automation, Tsinghua University, and his research interests include network security and network automation.
Cite this article:

Mohammad Hashem Haghighat,Jun Li. Intrusion Detection System Using Voting-Based Neural Network. Tsinghua Science and Technology, 2021, 26(4): 484-495.

URL:

http://tst.tsinghuajournals.com/10.26599/TST.2020.9010022     OR     http://tst.tsinghuajournals.com/Y2021/V26/I4/484

15].
">
Fig. 1 RNN architecture[15].
Fig. 2 SAE architecture.
AcronymExpression
VNNVoting-based Neural Network
DDoSDistributed Denial of Service
ANNArtificial Neural Network
DNNDeep Neural Network
CNNConvolutional Neural Network
RNNRecurrent Neural Network
LSTMLong Short-Term Memory
GRUGated Recurrent Unit
BMBoltzmann Machine
SAEStacked Auto-Encoder
SVMSupport Vector Machine
P2PPoint to Point
Table 1 Acronyms used through the paper.
Fig. 3 VNN architecture.
Fig. 4 Evolution of NSLKDD dataset.
Hyper parameterValue
Train size90%
Test size10%
Dropout0.5
Batch inputOn
Activation functionRelu
Layers number of CNN4
Layers number of LSTM2
Layers number of CNN-LSTM4
Layers number of DNN2
Layers number of GRU2
Number of input attributes37
Number of input subsets38
OutputBinary and five-class
UMT0.7
TU0.5
Table 2 Hyper parameters used to test KDDCUP’99.
Fig. 5 Normalized form of model accuracy (The blue dashed lines show UMT).
MethodNumber of errorsNumber of correctionsCorrection rate (%)
BinaryDNN777293.73
CNN8729711.12
LSTM155155135.53
CNN-LSTM99314814.90
GRU180470839.25
Five-classDNN205 43925 49712.41
CNN205 30674633.64
LSTM208 84981 26338.90
CNN-LSTM85 06863 67574.85
GRU208 51328 37413.61
Table 3 KDDCUP’99 error correction.
Fig. 6 System accuracy: voting-based vs. normal-based using KDDCUP’99 dataset.
Predicted
NormalMaliciousTotal
ActualNormal301 031203301 234
Malicious488188 121188 609
Total301 519188 324489 843
Table 4 KDDCUP’99 binary classification confusion matrix.
Predicted
NormalDoSR2LU2RProbingTotal
ActualNormal277 26921920 60800298 096
DoS490188 107570188 609
R2L0623060003122
U2R010001
Probing01410015
Total277 759188 40323 67470489 843
Table 5 KDDCUP’99 five-class classification confusion matrix.
FPRFNRAccuracyPrecisionRecallF_Score
Binary classification0.00110.00160.99860.99930.99840.9989
Five-class classification0.09820.00210.95630.93020.99790.9628
Table 6 Measurement result of KDDCUP’99 study.
Fig. 7 VNN vs. other deep learning architectures.
DayNumber of flows (million)Botnet (%)Normal (%)Command and control (%)Background (%)
12.821.411.070.03097.47
21.811.040.500.11098.33
34.710.562.480.00196.94
41.210.152.250.00497.58
50.130.533.61.15095.70
60.560.791.340.03097.83
70.110.031.470.02098.47
82.950.172.462.40097.32
92.756.501.570.18091.70
101.318.111.200.00290.67
110.117.602.530.00289.85
120.330.652.340.00796.99
131.932.011.650.06096.26
Table 7 CTU13 label distribution.
Fig. 8 SAWANT window-based feature extraction procedure.
Fig. 9 SAWANT architecture.
Hyper parameterValue
Train size10%
Test size90%
Dropout0.2
Batch inputOn
Activation functionRelu
Number of CNN layers4
Number of LSTM layers2
Number of DNN layers2
Number of GRU layers2
Number of input attributes72
Number of input subsets73
OutputMalicious rate
UMT0.8
TU0.5
Table 8 Hyper parameters used to test CTU-13.
Fig. 10 Model accuracy reported by the system during the training phase.
MethodNumber of errorsNumber of correctionsCorrection rate (%)
DNN17 11212 41872.57
CNN76 523825110.78
LSTM668 597272 50740.74
GRU630 54190 90214.42
Table 9 CTU-13 error correction.
Fig. 11 System accuracy: voting-based vs. normal-based using CTU-13 dataset.
Predicted
NormalMaliciousTotal
ActualNormal2 103 0582542 103 312
Malicious767145 921146 688
Total2 103 825146 1752 250 000
Table 10 CTU-13 confusion matrix.
FPRFNRAccuracyPrecisionRecallF_Score
0.00170.00040.99950.99990.99960.9998
Table 11 Measurement result of CTU-13 study.
[1]   Sophos 2020 threat report, , 2020.
[2]   McAfee labs threats report, , 2019.
[3]   Behal S., Kumar K., and Sachdeva M., D-FACE: An anomaly-based distributed approach for early detection of DDoS attacks and flash events, Journal of Network and Computer Applications, vol. 111, pp.49-63, 2018.
[4]   Elejla O., Belaton B., Anbar M., and Alnajjar A., Intrusion detection systems of ICMPv6-based DDoS attacks, Neural Computing and Applications, vol. 30, no. 1, pp. 45-56, 2018.
[5]   Haghighat M. H. and Li J., Edmund: Entropy based attack detection and mitigation engine using netflow Data, in Proc. of 8th International Conference on Communication and Network Security, Chengdu, China, 2018, pp. 1-6.
[6]   Idhammad M., Afdel K., and Belouch M., Semi-supervised machine learning approach for DDoS detection, Applied Intelligence, vol. 48, no. 10, pp. 3193-3208, 2018.
[7]   Terzi D. S., Terzi R., and Sagiroglu S., Big data analytics for network anomaly detection from netflow data, in Proc. of 2017 International Conference on Computer Science and Engineering, Antalya, Turkey, 2017, pp. 592-597.
[8]   Vidal J. M., Orozco A. L. S., and Villalba L. J. G., Adaptive artificial immune networks for mitigating DoS flooding attacks, Swarm and Evolutionary Computation, vol. 38, pp. 94-108, 2018.
[9]   Wang R., Jia Z., and Ju L., An entropy-based distributed DDoS detection mechanism in software-defined networking, in Proc. of 2015 IEEE Trustcom/BigDataSE/ISPA, Helsinki, Finland, 2015, pp. 310-317.
[10]   Aceto G., Ciuonzo D., Montieri A., and Pescapé A., Multi-classification approaches for classifying mobile app traffic, Journal of Network and Computer Applications, vol. 103, pp. 131-145, 2018.
[11]   Lotfollahi M., Siavoshani M. J., Hosseinzade R. S., and Saberian M. S., Deep packet: A novel approach for encrypted traffic classification using deep learning, Soft Computing, vol. 24, no. 3, pp. 1999-2012, 2020.
[12]   Aceto G., Ciuonzo D., Montieri A., and Pescapè A., MIMETIC: Mobile encrypted traffic classification using multimodal deep learning, Computer Networks, vol. 165, pp. 1186-1191, 2019.
[13]   Mansouri N. and Fathi M., Simple counting rule for optimal data fusion, in Proc. of 2003 IEEE Conference on Control Applications, Istanbul, Turkey, 2003, pp. 1186-1191.
[14]   Ciuonzo D., De Maio A., and Rossi P. S., A systematic framework for composite hypothesis testing of independent Bernoulli trials, IEEE Signal Processing Letters, vol. 22, no. 9, pp. 1249-1253, 2015.
[15]   Khan A. and Zhang F., Using recurrent neural networks (RNNs) as planners for bio-inspired robotic motion, in Proc. of 2017 IEEE Conference on Control Technology and Applications, Mauna Lani, HI, USA, 2017, pp. 1025-1030.
[16]   Kim J. and Kim H., Applying recurrent neural network to intrusion detection with hessian free optimization, in Proc. of 2015 International Workshop on Information Security Applications, Jeju Island, Korea, 2015, pp. 357-369.
[17]   Kim J., Kim J., Thu H. L. T., and Kim H., Long short term memory recurrent neural network classifier for intrusion detection, in Proc. of 2016 International Conference on Platform Technology and Service, Jeju South, Korea, 2016, pp. 1-5.
[18]   Yin C., Zhu Y., Fei J., and He X., A deep learning approach for intrusion detection using recurrent neural networks, IEEE Access, vol. 5, pp. 21 954-21 961, 2017.
[19]   Althubiti S., Nick W., Mason J., Yuan X., and Esterline A., Applying long short-term memory recurrent neural network for intrusion detection, in Proc. of IEEE Southeast Conference 2018, St. Petersburg, FL, USA, 2018, pp. 1-5.
[20]   Tang T. A., Mhamdi L., McLernon D., Zaidi S. A. R., and Ghogho M., Deep recurrent neural network for intrusion detection in SDN-based networks, in Proc. of 2018 4th IEEE Conference on Network Softwarization and Workshops, Montreal, Canada, 2018, pp. 202-206.
[21]   Yao Y., Wei Y., Gao F., and Yu G., Anomaly intrusion detection approach using hybrid MLP/CNN neural network, in Proc. of Sixth International Conference on Intelligent Systems Design and Applications, Jinan, China, 2006, pp. 1095-1102.
[22]   Wu K., Chen Z., and Li W., A novel intrusion detection model for a massive network using convolutional neural networks, IEEE Access, vol. 6, pp. 50 850-50 859, 2018.
[23]   Aminanto M. E. and Kim K., Deep learning-based feature selection for intrusion detection system in transport layer, in Proc. of Summer Conference of Korea Information Security Society, Busan, Korea, 2016, pp. 535-538.
[24]   Javaid A., Niyaz Q., Sun W., and Alam M., A deep learning approach for network intrusion detection system, in Proc. of 9th EAI International Conference on Bio-inspired Information and Communications Technologies, Brussels, Belgium, 2016, pp. 21-26.
[25]   Farahnakian F. and Heikkonen J., A deep auto-encoder-based approach for intrusion detection system, in Proc. of 2018 20th International Conference on Advanced Communication Technology, Chuncheon-si Gangwon-do, South Korea, 2018, pp. 178-183.
[26]   Salakhutdinov R. and Hinton G., Deep boltzmann machines, in Proc. of Twelfth International Conference on Artificial Intelligence and Statistics, Clearwater, FL, USA, 2009, pp. 448-455.
[27]   Gao N., Gao L., Gao Q., and Wang H., An intrusion detection model based on deep belief networks, in Proc. of IEEE 2014 Second International Conference on Advanced Cloud and Big Data, Huangshan, China, 2014, pp. 247-252.
[28]   Zhang X. and Chen J., Deep learning-based intelligent intrusion detection, in Proc. of 2017 IEEE 9th International Conference on Communication Software and Networks, Guangzhou, China, 2017, pp. 1133-1137.
[29]   Alrawashdeh K. and Purdy C., Toward an online anomaly intrusion detection system based on deep learning, in Proc. of 2016 15th IEEE International Conference on Machine Learning and Applications, Anaheim, CA, USA, 2016, pp. 195-200.
[30]   Vinayakumar R., Soman K. P., and Poornachandran P., A comparative analysis of deep learning approaches for network intrusion detection systems (N-IDSs): Deep learning for N-IDSs, International Journal of Digital Crime and Forensics, vol. 11, no. 3, pp. 65-89, 2019.
[31]   Vinayakumar R., Alazab M., Soman K. P., Poornachandran P., Al-Nemrat A., and Venkatraman S., Deep learning approach for intelligent intrusion detection system, IEEE Access, vol. 7, pp. 41 525-41 550, 2019.
[32]   Vinayakumar R., Soman K. P., and Poornachandran P., Evaluation of recurrent neural network and its variants for intrusion detection system (IDS), International Journal of Information System Modeling and Design, vol. 8, no. 3, pp. 43-63, 2017.
[33]   Vinayakumar R., Soman K. P., and Poornachandran P., Evaluating effectiveness of shallow and deep networks to intrusion detection system, in Proc. of 2017 International Conference on Advances in Computing, Communications and Informatics, Manipal, India, 2017, pp. 1282-1289.
[34]   Vinayakumar R., Soman K. P. and Poornachandran P., Applying convolutional neural network for network intrusion detection, in Proc. of 2017 International Conference on Advances in Computing, Communications and Informatics, Manipal, India, 2017, pp. 1222-1228.
[35]   Haghighat M. H., Abtahi Foroushani Z., and Li J., SAWANT: Smart window-based anomaly detection using netflow traffic, in Proc. of 2019 IEEE 19th International Conference on Communication Technology, Xi’an, China, 2019, pp. 1396-1402.
[36]   KDD CUP 1999 dataset, , 1999.
[37]   Janarthanan T. and Zargari S., Feature selection in UNSW-NB15 and KDDCUP’99 datasets, in Proc. of 2017 IEEE 26th International Symposium on Industrial Electronics, Edinburgh, UK, 2017, pp. 1881-1886.
[38]   Lippmann R. P., Fried D. J., Graf I., Haines J. W., Kendall K. R., McClung D., Weber D., Webster S. E., Wyschogrod D., Cunningham R. K., et al., Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation, in Proc. of DARPA Information Survivability Conference and Exposition, Hilton Head, SC, USA, 2000, pp. 12-26.
[39]   Tavallaee M., Bagheri E., Lu W., and Ghorbani A., A detailed analysis of the KDDCUP’99 dataset, in Proc. of 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, Canada, 2009, pp. 1-6.
[40]   zgür A. and Erdem H., A review of KDD’99 dataset usage in intrusion detection and machine learning between 2010 and 2015, .
doi: 10.7287/PEERJ.PREPRINTS.1954
[41]   Finney S. J. and DiStefano C., Non-normal and categorical data in structural equation modeling. Structural Equation Modeling: A Second Course, no. 10, vol. 6, pp. 269-314, 2006.
[42]   CTU-13 botnet traffic dataset, , 2011.
[1] Qinchen Cao,Weilin Zhang,Yonghua Zhu. Deep Learning-Based Classification of the Polar Emotions of "Moe" -Style Cartoon Pictures[J]. Tsinghua Science and Technology, 2021, 26(3): 275-286.
[2] Yuzhu Cheng, Weiping Wang, Jianxin Wang, Haodong Wang. FPC: A New Approach to Firewall Policies Compression[J]. Tsinghua Science and Technology, 2019, 24(1): 65-76.
[3] Weiwei Jiang, Lin Zhang. Geospatial Data to Images: A Deep-Learning Framework for Traffic Forecasting[J]. Tsinghua Science and Technology, 2019, 24(1): 52-64.
[4] Feng Wang, Huaping Liu, Fuchun Sun, Haihong Pan. Fabric Recognition Using Zero-Shot Learning[J]. Tsinghua Science and Technology, 2019, 24(06): 645-653.
[5] Qi Dang, Jianqin Yin, Bin Wang, Wenqing Zheng. Deep Learning Based 2D Human Pose Estimation: A Survey[J]. Tsinghua Science and Technology, 2019, 24(06): 663-676.
[6] Yuling Tian, Xiangyu Liu. A Deep Adaptive Learning Method for Rolling Bearing Fault Diagnosis Using Immunity[J]. Tsinghua Science and Technology, 2019, 24(06): 750-762.
[7] Kaiming Nan, Sicong Liu, Junzhao Du, Hui Liu. Deep Model Compression for Mobile Platforms: A Survey[J]. Tsinghua Science and Technology, 2019, 24(06): 677-693.
[8] Zebang Shen, Binbin Yong, Gaofeng Zhang, Rui Zhou, Qingguo Zhou. A Deep Learning Method for Chinese Singer Identification[J]. Tsinghua Science and Technology, 2019, 24(04): 371-378.
[9] Xiaocheng Feng,Lifu Huang,Bing Qin,Ying Lin,Heng Ji,Ting Liu. Multi-Level Cross-Lingual Attentive Neural Architecture for Low Resource Name Tagging[J]. Tsinghua Science and Technology, 2017, 22(6): 633-645.
[10] Mengmeng Wang,Jianwei Liu,Jian Mao,Haosu Cheng,Jie Chen,Chan Qi. RouteGuardian: Constructing Secure Routing Paths in Software-Defined Networking[J]. Tsinghua Science and Technology, 2017, 22(4): 400-412.
[11] Donglai Fu,Xinguang Peng. TPM-Based Remote Attestation for Wireless Sensor Networks[J]. Tsinghua Science and Technology, 2016, 21(3): 312-321.
[12] Zhenlong Yuan,Yongqiang Lu,Yibo Xue. DroidDetector: Android Malware Characterization and Detection Using Deep Learning[J]. Tsinghua Science and Technology, 2016, 21(1): 114-123.
[13] Zhen Chen,Yuhao Wen,Junwei Cao,Wenxun Zheng,Jiahui Chang,Yinjun Wu,Ge Ma,Mourad Hakmaoui,Guodong Peng. A Survey of Bitmap Index Compression Algorithms for Big Data[J]. Tsinghua Science and Technology, 2015, 20(1): 100-115.
[14] Chen Lin, Chen Xingshu, Jiang Junfang, Yin Xueyuan, Shao Guolin. Research and Practice of Dynamic Network Security Architecture for IaaS Platforms[J]. Tsinghua Science and Technology, 2014, 19(5): 496-507.
[15] Zhen Chen,Wenyu Dong,Hang Li,Peng Zhang,Xinming Chen,Junwei Cao. Collaborative Network Security in Multi-Tenant Data Center for Cloud Computing[J]. Tsinghua Science and Technology, 2014, 19(1): 82-94.