Abstract
Artificial Intelligence (AI) has significantly transformed phishing detection by enabling organisations to identify complex and evolving cyber threats through automated analysis of emails, URLs, websites, and online behaviour. Although AI-based systems can improve threat detection, their effectiveness often depends on collecting and processing large amounts of data, including personal and sensitive information. This creates important concerns regarding privacy, data protection, individual rights, and responsible data use. The growing use of AI in cybersecurity creates a challenging balance between protecting organisations and individuals from cyber threats and respecting principles such as transparency, accountability, and personal autonomy. Machine learning and generative AI systems may introduce bias, generate false alarms, or produce decisions that are difficult to understand and explain. AI-enabled monitoring can also increase surveillance risks, create uncertainty regarding data retention, and raise questions about responsibility when automated systems make incorrect decisions. This paper examines the legal and ethical challenges associated with AI-driven phishing detection, particularly focusing on the collection and use of personal data and the ability of existing legal frameworks to address emerging AI-related risks. It adopts a comparative approach by examining Indian data protection regulations alongside those of the European Union. The study also considers ethical issues related to digital monitoring and automated threat assessment. It emphasises data minimisation, purpose limitation, transparency, explainability, human oversight, proportionality, and responsible governance as essential principles for achieving effective cybersecurity while protecting privacy, individual rights, and responsible use of AI.
Keywords
Introduction
The increased digitalisation of our world has changed the ways in which people and various institutions whether governmental, business-related or financial communicate, exchange information, and make financial and other transactions. Nevertheless, the increased usage of digital technology is also accompanied by the growing sophistication of cyber threats, including phishing attacks. The latter refers to malicious and fraudulent actions that are carried out by means of deceptive communications or websites, aimed at inducing people to provide their personal confidential information, including passwords, credit card details, personal data, etc. Phishing is quite effective since it does not depend on the technical vulnerabilities of the victim, but uses the human behaviour for achieving its malicious goals [1-2]. Additionally, attackers are now taking increasingly clever advantage of social engineering methods, and the traditional protection tools are no longer effective. Hence, it is crucial to find more efficient ways of detecting such attacks [3-5]. With the recent emergence of AI, there are some possibilities to address this problem. Phishing detection systems can use machine learning and deep learning to analyse different types of emails, URLs, websites, and communications, looking for any suspicious indications of malicious activity. Moreover, owing to the advanced progress in natural language processing and generative AI, the systems may analyse linguistic patterns of suspicious communications [5-6]. Hence, AI is one of the essential weapons in safeguarding businesses against phishing attacks [7-8]. However, with the growing adoption of AI in the fight against cyber threats emerges another pressing issue. Confidential, private and sensitive information of users can also be included in the data that aids the detection of phishing by AI algorithms. In this case, the analysis of emails by the AI algorithm can mean the analysis of their content, sender and recipient information, URLs, metadata, attachments, and user activities, etc. These things, of course, are essential for the security of the user but in reality they are an infringement on his privacy [9-11].
So we have a paradox here which arises in the context of applying AI. How can the organisations use the power of AI, yet without compromising the privacy and rights of the user? This problem becomes even more complicated by the fact that AI algorithms act in a very automated and large-scale manner, making it hard to comprehend the decision that was made by the AI algorithm and to question it. AI can determine that the message is suspicious and dangerous, block the message, label a user as suspicious, identify that some behaviours are suspicious, but not explain how the AI made this determination. Moreover, mistakes made in such systems can result in serious problems. For instance, if there’s a false positive, the legitimate communication will be blocked, and vice versa. This is why the accuracy of the algorithm does not imply the suitability of the implementation of an AI system. The effectiveness of the AI system is not the only issue to be addressed in the discussion of transparency, explainability, fairness, accountability, proportionality and human supervision [2-3, 9].
Moreover, there is a legal aspect of AI-based detection of phishing attempts. If data is processed by organisations, for example for cyber security, the conformity with data protection rules and other principles should be considered. Key points like the legal framework for data processing, data minimization, purpose limitation, transparency, retention, security, rights of individuals and accountability are particularly crucial if the AI systems are continuously analyzing digital communications. In India, the Digital Personal Data Protection Act, 2023 is an important legal framework in the context of analysing the duties related to the processing and protection of digital personal data. The international standards are comparative and are based on the principles of lawfulness, fairness, transparency, purpose limitation and data minimisation, in the form of the EU General Data Protection Regulation (GDPR). The elaboration of AI-specific regulations also confirms the increasing necessity of appropriate governance and accountability of the automated systems [10-11]. However, there are many questions regarding the ethics of AI phishing detection. Organisations might also have a legitimate need to monitor communications to make sure that their systems are safeguarded from the attacks. Meanwhile, over-monitoring can be a problem of digital surveillance, well beyond the demands of the cybersecurity activities. Employees and users of the system are unlikely to know the extent of their communications being monitored and analyzed by the automated system as well as how this information is collected and processed. Further, algorithmic bias, explainability, over-collection of data, and lack of accountability for automated decision-making issues are of concern [2-3, 10-11].
In the current technology-driven world, phishing is becoming a significant issue, as attackers can also use AI to create convincing and personalized phishing content. Using the generative AI, cyber criminals can produce grammatically correct and contextually appropriate messages that will be more difficult to spot using the traditional indicators. A technological battle seems to be unfolding where the AI technology is used as a cybersecurity defense and attack. If the phishing attacks become more sophisticated, organisations will be encouraged to collect and process even more information, so they can detect the attacks. So, the dilemma of balancing the enhanced security and privacy becomes even more real [5-6, 8].
Hence, this research is aimed at analysing the legal and ethical implications of AI-driven phishing detection as well as its interaction with data protection. It will try to overcome the focus on the technological issues of whether or not AI is able to perform the detection and consider the issues of whether it is possible to implement this kind of detection lawfully, ethically, in a proportional way and respect individuals’ rights. It will explore the privacy implications of the collection and processing of the data by the automated detection systems, look at relevant principles, and discuss ethical concerns regarding the monitoring and decision making of automated detection systems. In addition, the concept of ensuring responsible use of AI-based cybersecurity and the role of principles such as data minimisation, privacy by design, transparency, explainability, human supervision and accountability will be assessed [2-3, 10-11].
Complete Article
The complete article, including all figures, tables, equations and algorithms, is available in the official publication PDF.
Conclusion
Artificial Intelligence is now a tool to combat phishing attacks. It assists organisations to consider many emails, URLs, websites and people’s actions. This enables them to discover activities quicker than more traditional techniques that depend on rules. Using machine learning, deep learning and natural language processing techniques can aid in identifying phishing attempts. These include messages that are extremely personal and even AI-generated messages. These tools are most effective if they are data rich. This poses a dilemma between safety and privacy and data rights. The study indicates that the ethical issues come into play when using AI to detect phishing. When you review emails, the details of emails, how people are acting, and other information can help you determine if the data is being processed in an illegal manner. It also brings up issues about keeping the needed data, limiting why data is used, being clear about what happens to data, deciding how long to keep it and who is responsible. Also, there might be mistakes in automatic checks. May state something is safe when it is not or may not cite something that is not safe. AI can sometimes be hard to explain, which means that people might not understand why a decision was made. They may not be able to ask questions about this. Being watched, being free, being fair and being reasonable are other issues that can arise when watching all the time. The accuracy of an AI system should therefore not be used to assess it. It should also advance privacy and operate in an open and responsible manner. A good way to handle this is to include cybersecurity, data protection and ethical AI from the start. Only organisations need to take just the data that is necessary for phishing detection. They should explain clearly their reasons for taking it and for how long they are willing to do so. They should be able to store data securely. Be open and honest about what they are doing. They also need to ensure there is some human oversight of AI decisions. Ensuring good and trusted systems can be achieved through Privacy-by-design and Explainable AI. The protection of data rules should also be taken into account in India when tackling and preventing cyber threats. In other regions, such as GDPR, ideas are provided on the approaches to be taken with data management in order to be responsible. Finally, it should not be considered a compromise between security and privacy when using AI for phishing. The ultimate objective is to develop systems that can do both. With the evolution of phishing techniques and the incorporation of AI to create defensive messages, AI will become even more critical. There should be more stringent regulations in place at the time to ensure that the use of AI in security does not result in excessive surveillance or misuse of personal information. The future of AI with phishing is not about better algorithms, but about rules that are legal, ethical, private, and hold people accountable. These rules must safeguard security and people’s rights.
References
- H. Chauhan, S. Degadwala, and D. Soni, “Robust intrusion detection systems with explainable artificial intelligence: A review, ” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 12, no. 1, pp. 20-24, 2026, doi: 10.32628/CSEIT26123
- F. Charmet, H. C. Tanuwidjaja, S. Ayoubi, P.-F. Gimenez, Y. Han, H. Jmila, G. Blanc, T. Takahashi, and Z. Zhang, “Explainable artificial intelligence for cybersecurity: A literature survey, ” Annals of Telecommunications, vol. 77, pp. 789-812, 2022, doi: 10.1007/s12243-022-00926-7
- F. Yan, S. Wen, S. Nepal, C. Paris, and Y. Xiang, “Explainable machine learning in cybersecurity: A survey, ” International Journal of Intelligent Systems, vol. 37, no. 12, pp. 12305-12334, 2022, doi: 10.1002/int.23088
- L. Tang and Q. H. Mahmoud, “A survey of machine learning-based solutions for phishing website detection, ” Machine Learning and Knowledge Extraction, vol. 3, no. 3, pp. 672-694, 2021, doi: 10.3390/make3030034
- C. Catal, G. Giray, B. Tekinerdogan, S. Kumar, and S. Shukla, “Applications of deep learning for phishing detection: A systematic literature review, ” Knowledge and Information Systems, vol. 64, no. 6, pp. 1457-1500, 2022, doi: 10.1007/s10115-022-01672-x
- S. A. Salloum, T. Gaber, S. Vadera, and K. Shaalan, “Phishing email detection using natural language processing techniques: A literature survey, ” Procedia Computer Science, vol. 189, pp. 19-28, 2021, doi: 10.1016/j.procs.2021.05.077
- M. Rastogi, A. Chhetri, D. K. Singh, and G. Rajan V, “Survey on detection and prevention of phishing websites using machine learning, ” in Proc. 2021 International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), pp. 78-82, 2021, doi: 10.1109/ICACITE51222.2021.9404714
- E. Kytidou, T. Tsikriki, G. Drosatos, and K. Rantos, “Machine learning techniques for phishing detection: A review of methods, challenges, and future directions, ” Intelligent Decision Technologies, vol. 19, no. 6, pp. 4356-4379, 2025, doi: 10.1177/18724981251366763
- P. W. C. Prasad, M. S. Sayeed, D.-M. Nguyen, D. P. Hutabarat, and G. M. Mohiuddin, “Explainable AI: Enhancing decision-making in the detection of cyber threats, ” Frontiers in Computer Science, vol. 8, Art. no. 1762332, 2026, doi: 10.3389/fcomp.2026.1762332
- Government of India, The Digital Personal Data Protection Act, 2023, Act No. 22 of 2023, Ministry of Electronics and Information Technology, Aug. 11, 2023.
- European Parliament and Council of the European Union, “Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 (General Data Protection Regulation), ” Official Journal of the European Union, L 119, pp. 1-88, 2016.
