A Comparative Study of Classical and Deep Learning Models for Edge-Oriented Network Anomaly Detection

Introduction

An intrusion detection system (IDS) monitors traffic or host activity for patterns associated with malicious behavior. In anomaly-based IDS research, machine-learning models are trained from labeled traffic so that detection is not limited to a manually maintained signature set . The resulting classifier still has to operate somewhere in the network. For an edge deployment, the practical question is therefore not only whether the model detects attacks, but also whether the required computation is reasonable at the point where the traffic is observed. Edge computing moves part of the processing path closer to the source of the data . In IoT and fog systems, this can reduce the amount of traffic sent to a remote cloud and can shorten the response path, although an edge node usually has fewer compute and storage resources than a cloud server . It is not sufficient to evaluate an IDS operating at the edge just on its ability to distinguish between legitimate and malicious traffic. While the memory and processing needed for inference must remain within the bounds of the edge node, its false-alarm behavior must continue to be acceptable for the intended usage. Even if a detector has good statistical performance, it could not be appropriate for the target platform if its execution costs are too high. Neural and classical models take distinct approaches to this issue. Neural networks are able to capture nonlinear connections among the input variables, whereas linear classifiers and support vector machines employ relatively simple decision functions . IoT-oriented intrusion detection has also examined autoencoders and lightweight support vector machines . The model family alone cannot be used to interpret their performance. The distribution of the test traffic, the amount of retained characteristics, the chosen decision threshold, and the representation given to the classifier can all have an impact on the observed outcome. A portion of the edge inference cost can be reduced by reducing the input space. Less telemetry has to be gathered and analyzed when fewer traffic fields are needed. The quantity of inputs, as well as well as well as well as well. Ambusaidi et al. reported that informative feature subsets might eliminate superfluous dimensions without sacrificing effective intrusion-detection capabilities. However, when the feature set is decreased, some information is unavoidably lost, which might change the ratio of recall to precision. Because of this, accuracy alone should not be used to evaluate the impact of feature reduction. What may be logically inferred from a comparison is also influenced by the dataset selection. UNSW-NB15, Bot-IoT, TON_IoT, Edge-IIoTset, and CICIoT2023 were built for more current network or IoT environments . The current IoT protocols and the variety of network attacks are not replicated by NSL-KDD, which is significantly older. It is used in this study for a different reason. The dataset was created to reduce the duplicate-record bias related to KDD Cup 99 , and its development and official test files offer a fixed benchmark. Therefore, rather than being proof of present IoT threat coverage, the results are taken as evidence concerning model behavior under a controlled feature-budget experiment.

In this paper, two linear classifiers are compared with a compact neural network and the evaluation method is kept the same. All models are trained and tested using the same preprocessing process, validation partition and official NSL-KDD test set. The input representation is then manipulated so that the impact of feature budget on the results can be studied isolated from the changes in the evaluation protocol. Feature ranking is performed only on the training partition. We report accuracy and F1-score together with balanced accuracy, Matthews correlation coefficient (MCC), false-alarm rate (FAR), and receiver operating characteristic area under the curve (ROC-AUC). Parameter count, estimated FP32 storage, and a structural multiplication count are included as reproducible proxies for edge-side model cost.

The paper has four specific contributions.

• Linear regression, a linear support vector machine (SVM), and a compact deep neural network (DNN) are evaluated under one binary anomaly-detection protocol.

• The encoded representation is then reduced from 118 features to 20 and 10 features with a Fisher ranking fitted on training data only.

• Detection results are reported together with model size and per-sample structural computation.

• Finally, the analysis separates threshold-independent discrimination from thresholded behavior, which allows the ROC-AUC result to be interpreted alongside F1-score and FAR rather than as a substitute for them.

The organization of the paper is as follow: Section 2 reviews the IDS and edge-security work most closely related to this comparison. Section 3 defines the dataset, preprocessing, feature ranking, classifiers, and evaluation measures, and Section 4 records the experimental scenario. Section 5 reports the results and examines the detection-resource trade-offs. Limitations are discussed in Section 6, followed by the conclusion and future work in Section 7.

Complete Article

The complete article, including all figures, tables, equations and algorithms, is available in the official publication PDF.

View Full PDF

Conclusion

We compared logistic regression, a linear SVM, and a compact DNN for binary network anomaly detection using three feature budgets. Preprocessing and feature ranking were fitted without access to the official test set, and a held-out validation partition was used to choose decision thresholds. Final results were obtained on the official NSL-KDD test file. The comparison reports detection metrics beside parameter storage and a structural computation proxy to keep the edge-side cost visible. No single configuration was strongest on every measure. With all 118 encoded inputs, the compact DNN had the highest F1-score and ROC-AUC within the full-feature condition. After the Fisher ranking reduced the input to 20 features, logistic regression reached F1 0.8054, balanced accuracy 0.8113, and MCC 0.6186 with 21 parameters and about 0.082 KB of FP32 parameter storage. The 20-feature DNN instead produced the highest ROC-AUC, 0.9230. At 10 features, false-alarm rates declined, but attack recall declined as well. For this benchmark, model depth does not remove the need to examine the input representation and operating threshold. A compact linear classifier can perform well when the selected features suit its decision boundary, while the small DNN can offer better score ranking at a higher structural cost.

Future work will repeat the same protocol on Edge-IIoTset, TON_IoT, and CICIoT2023, extend the task to multiclass attack recognition, evaluate repeated training seeds, and run the frozen models on physical edge hardware.

References

1. A. L. Buczak and E. Guven, “A survey of data mining and machine learning methods for cyber security intrusion detection, ” IEEE Communications Surveys & Tutorials, vol. 18, no. 2, pp. 1153–1176, 2016, doi: 10.1109/COMST.2015.2494502
2. H. Liu and B. Lang, “Machine learning and deep learning methods for intrusion detection systems: A survey, ” Applied Sciences, vol. 9, no. 20, p. 4396, 2019, doi: 10.3390/app9204396
3. M. Satyanarayanan, “The emergence of edge computing, ” Computer, vol. 50, no. 1, pp. 30–39, 2017, doi: 10.1109/MC.2017.9
4. M. Burhan, H. Alam, A. Arsalan, R. A. Rehman, M. Anwar, M. Faheem, and M. W. Ashraf, “A comprehensive survey on the cooperation of fog computing paradigm-based IoT applications: Layered architecture, real-time security issues, and solutions, ” IEEE Access, vol. 11, pp. 73303–73329, 2023, doi: 10.1109/ACCESS.2023.3294479
5. A. A. Diro and N. K. Chilamkurti, “Distributed attack detection scheme using deep learning approach for Internet of Things, ” Future Generation Computer Systems, vol. 82, pp. 761–768, 2018, doi: 10.1016/j.future.2017.08.043
6. R. Vinayakumar, M. Alazab, K. P. Soman, P. Poornachandran, A. Al-Nemrat, and S. Venkatraman, “Deep learning approach for intelligent intrusion detection system, ” IEEE Access, vol. 7, pp. 41525–41550, 2019, doi: 10.1109/ACCESS.2019.2895334
7. Y. Meidan, M. Bohadana, Y. Mathov, Y. Mirsky, D. Breitenbacher, A. Shabtai, and Y. Elovici, “N-BaIoT: Network-based detection of IoT botnet attacks using deep autoencoders, ” IEEE Pervasive Computing, vol. 17, no. 3, pp. 12–22, 2018, doi: 10.1109/MPRV.2018.03367731
8. S. U. Jan, S. Ahmed, V. Shakhov, and I. Koo, “Toward a lightweight intrusion detection system for the Internet of Things, ” IEEE Access, vol. 7, pp. 42450–42471, 2019, doi: 10.1109/ACCESS.2019.2907965
9. M. A. Ambusaidi, X. He, P. Nanda, and Z. Tan, “Building an intrusion detection system using a filter-based feature selection algorithm, ” IEEE Transactions on Computers, vol. 65, no. 10, pp. 2986–2998, 2016, doi: 10.1109/TC.2016.2519914
10. N. Moustafa and J. Slay, “UNSW-NB15: A comprehensive data set for network intrusion detection systems, ” in 2015 Military Communications and Information Systems Conference (MilCIS), 2015, pp. 1–6, doi: 10.1109/MilCIS.2015.7348942
11. N. Koroniotis, N. Moustafa, E. Sitnikova, and B. Turnbull, “Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset, ” Future Generation Computer Systems, vol. 100, pp. 779–796, 2019, doi: 10.1016/j.future.2019.05.041
12. A. Alsaedi, N. Moustafa, Z. Tari, A. N. Mahmood, and A. Anwar, “TON_IoT telemetry dataset: A new generation dataset of IoT and IIoT for data-driven intrusion detection systems, ” IEEE Access, vol. 8, pp. 165130–165150, 2020, doi: 10.1109/ACCESS.2020.3022862
13. M. A. Ferrag, O. Friha, D. Hamouda, L. Maglaras, and H. Janicke, “Edge-IIoTset: A new comprehensive realistic cyber security dataset of IoT and IIoT applications for centralized and federated learning, ” IEEE Access, vol. 10, pp. 40281–40306, 2022, doi: 10.1109/ACCESS.2022.3165809
14. E. C. P. Neto, S. Dadkhah, R. Ferreira, A. Zohourian, R. Lu, and A. A. Ghorbani, “CICIoT2023: A real-time dataset and benchmark for large-scale attacks in IoT environment, ” Sensors, vol. 23, no. 13, p. 5941, 2023, doi: 10.3390/s23135941
15. M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set, ” in 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009, pp. 1–6, doi: 10.1109/CISDA.2009.5356528
16. R. Sommer and V. Paxson, “Outside the closed world: On using machine learning for network intrusion detection, ” in 2010 IEEE Symposium on Security and Privacy, 2010, pp. 305–316, doi: 10.1109/SP.2010.25
17. M. Al-Qatf, Y. Lasheng, M. Al-Habib, and K. Al-Sabahi, “Deep learning approach combining sparse autoencoder with SVM for network intrusion detection, ” IEEE Access, vol. 6, pp. 52843–52856, 2018, doi: 10.1109/ACCESS.2018.2869577
18. N. Chaabouni, M. Mosbah, A. Zemmari, C. Sauvignac, and P. Faruki, “Network intrusion detection for IoT security based on learning techniques, ” IEEE Communications Surveys & Tutorials, vol. 21, no. 3, pp. 2671–2701, 2019, doi: 10.1109/COMST.2019.2896380
19. H. Bangui and B. Buhnova, “Lightweight intrusion detection for edge computing networks using deep forest and bio-inspired algorithms, ” Computers & Electrical Engineering, vol. 100, p. 107901, 2022, doi: 10.1016/j.compeleceng.2022.107901
20. P. Mahadevappa, R. K. Murugesan, R. Al-Amri, R. Thabit, A. H. Al-Ghushami, and G. Alkawsi, “A secure edge computing model using machine learning and IDS to detect and isolate intruders, ” MethodsX, vol. 12, p. 102597, 2024, doi: 10.1016/j.mex.2024.102597
21. I. Sharafaldin, A. Habibi Lashkari, and A. A. Ghorbani, “Toward generating a new intrusion detection dataset and intrusion traffic characterization, ” in Proceedings of the 4th International Conference on Information Systems Security and Privacy, 2018, pp. 108–116, doi: 10.5220/0006639801080116