The rise of generative AI is always seen in the context of content generation, and the impact it will have on writing and the arts. Yet AI also poses a significant threat in the realm of cyberattacks. This is particularly pronounced for phishing attacks, where the difference between a user believing or not believing a phishing email or message is often the contents of that message.

While it’s important to educate people on all of the signs of a phishing attack, it’s also important to contextualise these attacks within the new AI environment. The evolving nature of cyberattacks as a result of generative AI emphasises the need for robust cybersecurity measures: from better education to improved threat detection, backups, and recovery solutions.

Phishing for funds

Traditional phishing attacks are generally conducted through emails, direct messages, and malicious websites, all of which are designed to imitate a legitimate business. A common phishing scam is to imitate an email from a major bank, asking you to login to address a security threat, or make a change to your account. If you have an account with that bank, you might mistake the email for a genuine communication, and visit the site to login. By ‘logging in’ or filling in a form, you’ll then give them your account details, which they can use to hack into your bank account.

All phishing scams hinge on this ability to trick people into giving away personal information, often for financial gain. The aim is to make the fake communication look as convincing as possible, and snare as many people as possible, in order to increase their chances of success. Other popular methods for phishing scams include impersonating Netflix or other popular subscription services, and impersonating online shopping and delivery companies, claiming that a delivery has been missed.

There are a few ways users can differentiate a fake email from a real one. First, while the name of the sender will usually imitate the business, their email address will not be from the business’ real domain. Second, most businesses will never send you a link asking you to login to your account, and will instead direct you to visit the website yourself. Third, many phishing emails and texts contain spelling and grammatical mistakes, unusual language or strange formatting compared to real messages.

The impact of AI on phishing

It’s in this last area that phishing scams are starting to benefit from generative AI. Its ability to write accurate copy to a brief, eliminate errors and adopt a professional writing style is – to the untrained eye at least – bringing phishing messages much closer to real business communications. Large Language Models (LLMs) can heighten this, using real-time information to incorporate current details from the news and corporate websites to make phishing messages more convincing and urgent.

AI-driven chatbots are also contributing to the acceleration of phishing campaigns. Where phishing websites once seemed static and sterile, separated from any live information or social media, chatbots can make them feel more dynamic. By engaging users with a chatbot on a phishing site, they will feel as though they are talking to a real person or professional service, and may be more easily tricked into giving away their personal details.

Another growing phenomenon is spear phishing, where individuals are targeted using information gathered from their social media feeds, or data breaches that have leaked their details. AI-generated emails using this data become even more insidious, with specific details about individuals that may make them feel as though only a legitimate service could know so much about them. The combination of generative AI and targeted social engineering amplifies the effectiveness and impact of phishing attacks, and poses a grave threat to cybersecurity.

The future of phishing and AI

One of the key perils lies in the growing sophistication of AI-generated spear phishing emails. With the ability to analyse vast datasets and mimic our styles and patterns of speech, AI can craft messages that are not only convincing, but also tailored to an individual’s preferences and expectations. This not only makes them extremely difficult to discern from legitimate communications, but also increases the likelihood of recipients falling victim to these deceptive tactics.

AI  can also enhance the scalability and efficiency of spear phishing campaigns. Automated processes allow attackers to target multiple individuals simultaneously, while still applying personal information scraped from social media, or sourced from leaked data. AI-driven spear phishing can operate at a speed and scale that surpasses traditional manual methods, where the generic content of messages make success rates much lower. This increase in customisation at scale presents perhaps the biggest challenge for cybersecurity defences, and increases the value of training.

The dynamic nature of AI further compounds this danger. Machine learning algorithms already allow these attacks to evolve and adapt based on the responses they receive, and these are only improving. This adaptability is comparable to a real-world virus, which evolves to work its way around defences, and increase its survivability. This makes it difficult for static security measures to keep pace, as AI-powered spear phishing continually refines its strategies to bypass traditional defences – and demands that we apply ourselves to keep pace.

–

As organisations and individuals navigate an evolving digital landscape, understanding the increasing dangers of AI-driven phishing attacks becomes imperative to working safely online. Implementing robust cybersecurity measures, including advanced threat detection systems and user education programs, is crucial to mitigating the risks associated with this sophisticated form of cyber threat.

The convergence of AI and spear phishing underscores the need for a proactive and adaptive cybersecurity stance in order to effectively protect against these evolving and personalised cyber threats. Sota has a whole wealth of cybersecurity knowledge across our expert team, including a range of cybersecurity solutions. To learn more and find out how we can help your organisation, get in touch with us today.