Abstract
Keywords
Introduction
Fatigue at the wheel is estimated to play a part in 10% to 20% of road accidents every year [1], which corresponds to roughly 100,000 crashes, 50,000 injuries and 800 deaths annually [2]. Set against a global toll that the World Health Organization (WHO) puts at around 1.35 million road deaths per year [4], within which driver fatigue is a recognized and preventable risk factor [1], [3], the need for reliable fatigue detection is clear. Monitoring the eyes is one of the most direct ways of meeting it [3]. Eye closure can be observed without any contact with the driver, and it reveals brief episodes of microsleep. These are particularly dangerous at high speed: at 100 km/h a car travels almost 28 m every second, so even a short lapse leaves the vehicle uncontrolled over a long distance. Progress in Machine Learning (ML) and Deep Learning (DL) over the past decade has made such measurements much more precise, to the point where a DSDS that operates in real time can realistically be built around them.
Convolutional Neural Networks (CNNs) have become the standard tool for image classification, and they are well suited to recognizing sleepiness from facial images. Of all facial cues, the eyes are the ones most often used in ML-based fatigue detectors [5], since knowing whether they are open or closed is useful for sleep monitoring and eye-strain assessment as well as for driver fatigue detection [6]. Transfer Learning (TL) has made CNNs practical for narrow tasks of this kind. A network pretrained on a large, general dataset already carries reusable features, so a comparatively small set of eye images is enough to fine-tune it [7]. VGG-16 and Inception-V3 are two of the architectures most frequently adapted in this way, largely because of the quality of the features they extract [8]. VGG-16, from the Visual Geometry Group at Oxford, is a plain but deep stack of 16 weight layers [9], whereas Inception-V3, developed at Google, arranges its layers into inception modules so that a deeper network can be computed more economically [10]. The present study compares VGG-16, Inception-V3 and ResNet-50 as eye-state classifiers for driver sleepiness detection, using Accuracy (ACC) and related error metrics as the main criteria and considering their suitability for real-time use.
Sleepiness slows reactions, narrows attention and impairs judgement, and each of these effects raises the risk of a crash. A DSDS that observes the driver continuously and issues a warning as the eyes begin to close could protect passengers and other road users as well as the driver. Transport operators would also benefit, since crashes bring vehicle damage, medical and insurance claims and lost working time, and long-haul drivers, who are the most exposed to fatigue, stand to gain most. Advances in Artificial Intelligence (AI), DL, computer vision and TL have made camera-based driver monitoring far more dependable, yet published systems still vary widely in accuracy, and it is not obvious which pretrained backbone should be chosen. The motivation for this work is therefore practical: to establish how VGG-16, Inception-V3 and ResNet-50 compare when all three are fine-tuned for the same eye-state task under one training protocol.
The contributions of this paper can be summarized as follows:
A controlled, single-protocol empirical comparison in which three ImageNet-pretrained architectures, VGG-16, Inception-V3 and ResNet-50, are fine-tuned under identical settings to classify eye state (open versus closed) as the perception stage of a driver drowsiness-monitoring pipeline, rather than a new architecture;
A common, fully specified training protocol (ImageNet-pretrained weights fine-tuned end to end, Adam optimizer, learning rate of 0.00001, batch size of 32, cross-entropy loss, and a fixed author-defined partition of 2,840 training and 1,200 test images), so that differences in performance can be attributed mainly to the architecture rather than to the training settings;
An evaluation that goes beyond ACC to include the Misclassification Rate (MCR), precision, sensitivity, specificity, F1-score, confusion matrices and training/testing loss and accuracy curves, with particular attention to closed-eye errors, which are the errors that matter most for safety; and
A safety-oriented error analysis in which the three models are compared primarily by their closed-eye detection rate (specificity) rather than by overall accuracy alone, with VGG-16 reaching a testing ACC of 99.67%, against 96.08% for Inception-V3 and 96.42% for ResNet-50; because the earlier studies used different datasets and protocols, these values indicate where the present models stand rather than a controlled ranking (see Section 5.4).
The rest of the paper is organized as follow: Section 2 reviews earlier DL- and TL-based work on driver sleepiness detection. Section 3 presents the dataset, the overall framework and the three CNN architectures, and Section 4 describes the experimental setup and evaluation metrics. The results are reported and discussed in Section 5, together with a comparison against previous studies, and Section 6 gives the conclusions and directions for future work.
Complete Article
The complete article, including all figures, tables, equations and algorithms, is available in the official publication PDF.
Conclusion
This study fine-tuned three well-established DL architectures, VGG-16, Inception-V3 and ResNet-50, to classify eye state for driver sleepiness detection. VGG-16 performed best, with a testing ACC of 0.9967 and only four of 600 closed-eye images misclassified, while Inception-V3 and ResNet-50 reached 0.9608 and 0.9642, respectively. Because all three models recognized every open eye, the comparison was decided by how well they detected closed eyes, which is also the error that matters most for safety. One caveat concerns model size. VGG-16 has far more parameters than either of the other two networks, and on constrained in-vehicle hardware a lighter model may still be the better choice; because inference cost was not measured in this study, that trade-off has yet to be quantified. On balance, the findings suggest that a TL-based eye-state classifier can serve as the core of a practical DSDS, as long as its present limitations are dealt with. Only one public dataset of static eye images was used, and the system has not yet been exposed to real driving conditions, in which illumination changes, drivers may wear eyeglasses and the head moves. The reported accuracy may also be optimistic if images of the same person appear in both the training and the test sets. Future work will accordingly focus on subject-independent evaluation, real-time implementation and in-vehicle testing, robustness to real-world variation, measurement of inference latency on embedded hardware, and the integration of multimodal data to raise the ACC and reliability of the system further.
Data Availability
10.5281/zenodo.23360724
References
- National Academies of Sciences, Engineering, and Medicine, Commercial Motor Vehicle Driver Fatigue, Long-Term Health, and Highway Safety: Research Needs. Washington, DC, USA: The National Academies Press, 2016.
- H. A. Khalil, S. A. Hammad, H. E. Abd El Munim, and S. A. Maged, “Low-cost driver monitoring system using deep learning, ” IEEE Access, vol. 13, pp. 14151–14164, 2025.
- M. S. Al-Quraishi, S. S. A. Ali, A. Q. Muhammad, T. B. Tang, and S. Elferik, “Technologies for detecting and monitoring drivers' states: A systematic review, ” Heliyon, vol. 10, no. 20, 2024.
- World Health Organization, Global Status Report on Road Safety 2018. Geneva, Switzerland: World Health Organization, 2018. [Online]. Available: https://www.who.int/publications/i/item/9789241565684
- L. Kong, K. Xie, K. Niu, J. He, and W. Zhang, “Remote photoplethysmography and motion tracking convolutional neural network with bidirectional long short-term memory: Non-invasive fatigue detection method based on multi-modal fusion, ” Sensors, vol. 24, no. 2, 2024.
- Y. Dong, Z. Hu, K. Uchimura, and N. Murayama, “Driver inattention monitoring system for intelligent vehicles: A review, ” IEEE Transactions on Intelligent Transportation Systems, vol. 12, no. 2, pp. 596–614, 2010.
- S. Saurav, P. Gidde, R. Saini, and S. Singh, “Real-time eye state recognition using dual convolutional neural network ensemble, ” Journal of Real-Time Image Processing, vol. 19, no. 3, pp. 607–622, 2022.
- B. Sowmiya, K. Saminathan, and M. C. Devi, “An ensemble of transfer learning based InceptionV3 and VGG16 models for paddy leaf disease classification, ” ECTI Transactions on Computer and Information Technology, vol. 18, pp. 89–100, 2024.
- D. Hong et al., “Genetic syndromes screening by facial recognition technology: VGG-16 screening model construction and evaluation, ” Orphanet Journal of Rare Diseases, vol. 16, no. 1, Art. no. 344, 2021.
- E. Bazgir, E. Haque, M. Maniruzzaman, and R. Hoque, “Skin cancer classification using Inception network, ” World Journal of Advanced Research and Reviews, vol. 21, no. 2, pp. 839–849, 2024.
- B. R. Sri, Y. Akanksha, R. Puthali, and T. Anuradha, “Early driver drowsiness detection using convolution neural networks, ” in Proceedings of the 2nd International Conference on Electronics and Sustainable Communication Systems (ICESC), 2021, pp. 1779–1784.
- B. Jain, A. Yadav, and R. Gill, “Comparative study of CNN architectures for driver drowsiness, ” in Proceedings of the 14th International Conference on Computing Communication and Networking Technologies (ICCCNT), Jul. 2023, pp. 1–7.
- M. I. B. Ahmed et al., “A deep-learning approach to driver drowsiness detection, ” Safety, vol. 9, no. 3, Art. no. 65, 2023.
- A. Narayanan, V. Aiswaryaa, A. T. Anand, and N. Kadiresan, “Real-time detection of distracted drivers using a deep neural network and multi-threading, ” in Proceedings of the International Conference on Artificial Intelligence and Data Engineering. Singapore: Springer Nature Singapore, May 2019, pp. 89–100.
- A. Turki, O. Kahouli, S. Albadran, M. Ksantini, A. Aloui, and M. B. Amara, “A sophisticated drowsiness detection system via deep transfer learning for real time scenarios, ” AIMS Mathematics, vol. 9, no. 2, pp. 3211–3234, 2024.
- I. D. B. Rozaimi, F. H. K. Zaman, S. A. C. Abdullah, H. Z. Abidin, and L. Mazalan, “Driver drowsiness detection using deep learning models based on different camera positions, ” in Proceedings of the 11th International Conference on Information and Communication Technology (ICoICT), Aug. 2023, pp. 611–616.
- P. Patil, “Drowsiness detection dataset, ” Kaggle. [Online]. Available: https://www.kaggle.com/datasets/prasadvpatil/mrl-dataset
- Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition, ” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998, doi: 10.1109/5.726791
- Y. Chen, H. Jiang, C. Li, X. Jia, and P. Ghamisi, “Deep feature extraction and classification of hyperspectral images based on convolutional neural networks, ” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 10, pp. 6232–6251, 2016.
- I. Jahan, K. M. A. Uddin, S. A. Murad, M. S. U. Miah, T. Z. Khan, M. Masud, S. Aljahdali, and A. K. Bairagi, “4D: A real-time driver drowsiness detector using deep learning, ” Electronics, vol. 12, no. 1, Art. no. 235, 2023, doi: 10.3390/electronics12010235
- S. Tammina, “Transfer learning using VGG-16 with deep convolutional neural network for classifying images, ” International Journal of Scientific and Research Publications, vol. 9, no. 10, 2019, doi: 10.29322/IJSRP.9.10.2019.p9420
- H. Zheng, Z. Yang, W. Liu, J. Liang, and Y. Li, “Improving deep neural networks using softplus units, ” in Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2015, doi: 10.1109/IJCNN.2015.7280459
- S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift, ” in Proceedings of the 32nd International Conference on Machine Learning (ICML), vol. 37, 2015, pp. 448–456.
- S. Park and N. Kwak, “Analysis on the dropout effect in convolutional neural networks, ” in Computer Vision–ACCV 2016 Workshops. Cham, Switzerland: Springer, 2017, pp. 189–204, doi: 10.1007/978-3-319-54184-6_12
- L. R. Al-Khazraji, A. R. Abbas, and A. S. Jamil, “The effect of changing targeted layers of the deep dream technique using VGG-16 model, ” International Journal of Online and Biomedical Engineering, vol. 19, no. 3, 2023.
- Z. Liu, J. Peng, X. Guo, S. Chen, and L. Liu, “Breast cancer classification method based on improved VGG16 using mammography images, ” Journal of Radiation Research and Applied Sciences, vol. 17, no. 2, Art. no. 100885, 2024.
- T. N. Abu-Jamie and S. S. Abu-Naser, “Classification of sign-language using VGG16, ” 2022.
- O. Russakovsky et al., “ImageNet large scale visual recognition challenge, ” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015, doi: 10.1007/s11263-015-0816-y
- M. A. S. Al Husaini, M. H. Habaebi, T. S. Gunawan, M. R. Islam, E. A. A. Elsheikh, and F. M. Suliman, “Thermal-based early breast cancer detection using Inception V3, Inception V4 and modified Inception MV4, ” Neural Computing and Applications, vol. 34, no. 1, pp. 333–348, 2022, doi: 10.1007/s00521-021-06372-1
- C. Szegedy et al., “Going deeper with convolutions, ” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1–9, doi: 10.1109/CVPR.2015.7298594
- L. Emma, “Building and training neural networks: Architectures, optimization, and challenges, ” Mar. 2025.
- R. Khanam, M. Hussain, R. Hill, and P. Allen, “A comprehensive review of convolutional neural networks for defect detection in industrial applications, ” IEEE Access, vol. 12, pp. 94250–94295, 2024.
- X. Wang et al., “A recognition method of ancient architectures based on the improved Inception V3 model, ” Symmetry, vol. 14, no. 12, Art. no. 2679, 2022.
- O. F. Hassan, A. F. Ibrahim, A. Gomaa, M. A. Makhlouf, and B. Hafiz, “Real-time driver drowsiness detection using transformer architectures: A novel deep learning approach, ” Scientific Reports, vol. 15, no. 1, Art. no. 17493, 2025.
- L. Sharara et al., “A real-time automotive safety system based on advanced AI facial detection algorithms, ” IEEE Transactions on Intelligent Vehicles, vol. 9, no. 6, pp. 5080–5100, 2023.
- S. E. Abdallah, W. M. Elmessery, M. Y. Shams, N. S. A. Al-Sattary, A. A. Abohany, and M. Thabet, “Deep learning model based on ResNet-50 for beef quality classification, ” Optimization, vol. 30, p. 31, 2023.
- V. Y. Cambay, P. D. Barua, A. Hafeez Baig, S. Dogan, M. Baygin, T. Tuncer, and U. R. Acharya, “Automated detection of gastrointestinal diseases using ResNet50*-based explainable deep feature engineering model with endoscopy images, ” Sensors, vol. 24, no. 23, Art. no. 7710, 2024, doi: 10.3390/s24237710
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition, ” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778, doi: 10.1109/CVPR.2016.90