AI in Cybersecurity Defined in Plain Terms
AI in cybersecurity is the use of machine learning and generative models on both sides of an attack. Defenders use AI to spot unusual behaviour, sort alerts and shorten response times. Attackers use the same technology to write convincing phishing, clone voices, hunt for flaws in code and automate parts of an intrusion.
The technology itself is not new. Banks have scored card fraud with machine learning for years and mail filters have used it for longer. What changed is generative AI which produces language, speech, video and working code on demand and at almost no cost.
That change moved AI from a background tool to a board question. In the World Economic Forum’s Global Cybersecurity Outlook 2026, published in January 2026, 94% of the 804 leaders surveyed across 92 countries expected AI to be the most significant driver of change in cybersecurity in the year ahead.
For a business the useful framing is narrow. AI rarely invents a new attack. It makes the attacks you already face cheaper to run, faster to execute and much harder to dismiss as an obvious fake.
How Attackers and Defenders Use AI
Three kinds of AI turn up in security work and they do different jobs.
- Pattern models: Learn what normal looks like in logs, traffic or user behaviour and flag what deviates. Most detection tooling runs on this.
- Generative models: Produce text, speech, images, video or code from a prompt. This is what writes the phishing message and clones the voice.
- Agents: Generative models given tools and permission to act so they can browse, run commands and chain steps together with limited supervision.
The barrier fell for three practical reasons. Capable open models can be downloaded and run privately. Criminal services resell jailbroken or retrained models and ENISA’s Threat Landscape 2025 names WormGPT, EscapeGPT and FraudGPT among them. Voice cloning needs only a few seconds of clean audio, which any recorded webinar provides.
The effect shows up in the volumes. ENISA analysed 4,875 incidents between July 2024 and June 2025 and found phishing was still the main way in at 60% of intrusions. It also reported that AI-supported phishing made up more than 80% of observed social engineering activity worldwide by early 2025, a figure the agency attributes to external reporting rather than its own telemetry.
Defenders picked the technology up at a similar pace. In the same World Economic Forum survey, 77% of organisations said they already use AI somewhere in security, most often in phishing detection, in intrusion and anomaly response and in user behaviour analytics. The barriers they named were skills at 54% and the need for human oversight at 41%.
Types of AI-Driven Attacks
Six patterns cover almost everything a Swedish organisation is likely to meet.
- AI-written Phishing and Business Email Compromise: Fluent personalised messages at volume in Swedish as easily as in English without the language errors staff were trained to look for.
- Voice and video impersonation: A cloned executive on a call or a meeting where the other faces are synthetic used to authorise a payment or unlock an account.
- AI-assisted vulnerability research: Models that read code and configuration to find weaknesses narrowing the gap between a flaw becoming public and being exploited.
- Agentic intrusion: Models with tools that carry out reconnaissance, credential testing and lateral movement at machine speed while a human approves the larger decisions.
- Prompt injection and AI supply chain attacks: Instructions hidden in a document, a web page or a support ticket that your own assistant reads and obeys plus poisoned models and packages pulled in by developers.
- Shadow AI: Staff pasting customer records, source code or contracts into consumer AI tools which leaks the data without an attacker touching anything.
The last two are the ones most often missed. The attacker never touches your network because your own AI does the work for them.
Business Impact
The clearest cost is fraud. In the World Economic Forum’s 2026 survey, 73% of respondents said they or someone in their network had been personally affected by cyber-enabled fraud during 2025 which pushed fraud past ransomware as the leading concern among chief executives.
The second cost is time. The UK’s National Cyber Security Centre assessed in May 2025 that the gap between a vulnerability being disclosed and being exploited has shrunk to days and that AI will almost certainly shorten it further. Monthly patch cycles were designed for a slower world.
The third is competitive. NCSC’s judgement is that a divide will open between systems keeping pace with AI-enabled threats and a large share that do not. In June 2026 the Five Eyes cyber security agencies put the same point directly to boards and said the timeline is months rather than years.
There is a quieter cost as well. Leaders in the World Economic Forum survey now rank data leaks from generative AI at 34% ahead of adversarial AI capability at 29%, a reversal of the previous year. Personal data leaking through a consumer AI tool is a GDPR matter with the same 72-hour clock as any other breach.
One caution against panic. The Swedish reported picture does not yet show an AI attack wave. MCF (formerly MSB) reported in March 2026 that cyber attacks were a low share of Swedish incident reports during 2025 while noting that attacks internationally had developed between 2023 and 2025 and that some had been made more efficient with AI. The trend is what to plan against.
Real-World Cases
Three cases show the range from a loss to a near miss to an intrusion still being argued over.
The Arup Video Call That Cost HK$200 Million
In January 2024 a finance employee at the Hong Kong office of the engineering firm Arup received a message that appeared to come from the company’s UK-based chief financial officer about a confidential transaction. The employee was suspicious at first.
A video call settled it. The chief financial officer was on the call and so were colleagues the employee recognised. All of them were AI-generated. The employee made 15 transfers to five Hong Kong bank accounts totalling HK$200 million roughly 25 million US dollars.
Hong Kong police described the case in February 2024 without naming the firm. Arup confirmed in May 2024 that fake voices and images had been used and said that no internal system was compromised and that operations were unaffected. The whole attack ran on a video call and a plausible story.
One rule would have stopped it. Payment instructions get verified by calling back on a number already held in the finance system and a video call never counts as approval on its own.
The Ferrari Call That Failed
In July 2024 an executive at Ferrari received WhatsApp messages from an unfamiliar number carrying the chief executive’s photo, then a call in a cloned voice with his southern Italian accent. The caller pressed for help with an urgent confidential transaction.
Something in the intonation sounded slightly mechanical. The executive asked the caller to name the book the chief executive had recommended to him days earlier. The call ended at once. Bloomberg reported the attempt in July 2024. Ferrari opened an internal investigation and lost nothing.
A challenge question agreed in advance costs nothing and works on any channel. Give one to everyone who can move money or grant access and expect them to use it.
The GTG-1002 Campaign and the Argument About It
In November 2025 the AI company Anthropic reported that it had detected and disrupted a cyber-espionage campaign in which its own coding tool served as the orchestration layer. It said the group which it assessed with high confidence as Chinese state-sponsored targeted around 30 organisations and that the AI performed 80 to 90% of the tactical work at request rates no human team could sustain.
The operators got past the safeguards by claiming to be staff at a security firm running authorised tests and by splitting the work into small harmless-looking steps. Anthropic also reported that the model sometimes invented results including credentials that turned out not to work.
Researchers pushed back quickly. Reporting in BleepingComputer and The Conversation set out the main objection, that the disclosure carried no indicators of compromise and that the real level of autonomy was unclear. It is the best-documented example so far and it has not been independently verified.
The tooling is not in dispute. The campaign leaned on ordinary open-source tools against ordinary weaknesses so patching, network segmentation, strong authentication and alerting on impossible request rates all still apply.
AI and Compliance
Four regimes touch AI for a Swedish organisation.
The EU AI Act
The EU AI Act (Regulation (EU) 2024/1689) has been in force since 1 August 2024 and applies in stages. The Digital Omnibus on AI, approved by the European Parliament on 16 June 2026 and adopted by the Council on 29 June 2026 moved the high-risk obligations for stand-alone Annex III systems from 2 August 2026 to 2 December 2027. AI embedded in regulated products moves to 2 August 2028.
That delay covers the high-risk regime only. The amending regulation takes effect once it is published in the Official Journal.
The transparency duties did not move. From 2 August 2026, Article 50 requires you to tell people when they are dealing with an AI system to mark AI-generated content in machine-readable form and to label deepfakes. Systems already on the market have until 2 December 2026 for the machine-readable marking.
A new prohibition on AI systems that generate non-consensual intimate imagery or child sexual abuse material applies from 2 December 2026.
If you run a customer-facing chatbot or publish AI-generated media, August 2026 is the date to check first.
NIS2 and Cybersäkerhetslagen
Cybersäkerhetslagen (SFS 2025:1506) came into force on 15 January 2026 and transposes NIS2 into Swedish law. It says nothing about AI and does not need to. Article 21 requires continuous monitoring, incident handling, supply-chain security and security awareness training and an AI-assisted attack is an incident like any other.
The reporting cascade is unchanged with an early warning inside 24 hours, a full notification inside 72 hours and a final report within one month. Article 20 makes the management body responsible for approving and overseeing the measures and supervisory authorities can hold board members personally accountable. Fines reach 10 million euro or 2% of global turnover for essential entities and 7 million euro or 1.4% for important entities.
Sweden’s cyber security functions were consolidated into Nationellt cybersäkerhetscenter at FRA on 1 July 2026 and that is where the national guidance now comes from. It confirmed in June 2026 that from 1 October 2026 the law’s general requirements are supplemented by regulations spelling out the security measures. Our NIS2 compliance guide for Sweden covers the scope test and the reporting duties in detail.
DORA
Financial entities carry DORA on top. Article 17 sets the ICT incident management process, supervised in Sweden by Finansinspektionen. An AI supplier that processes your data is an ICT third party and belongs in the register of information alongside every other critical provider. See our DORA compliance guide for the register and testing requirements.
GDPR
Shadow AI reaches GDPR faster than anything else on this page. Pasting personal data into a consumer AI tool can be an unlawful transfer as well as a breach and Article 33 gives you 72 hours to notify IMY once you become aware.
A sanctioned tool with a data processing agreement plus a written acceptable-use policy removes most of the exposure. Our GDPR compliance guide for Swedish organisations sets out the notification steps. ISO 27001 gives you the management system to document the controls.
How to Spot AI-Driven Attacks
The old tells have gone. Spelling mistakes and stiff grammar no longer separate a fraud from a real request so what is left is behaviour.
- Urgency together with secrecy. The request must happen now and must not be discussed with anyone else.
- A channel that changed. It arrives on WhatsApp, a new number or a personal address instead of the usual system.
- A process being skipped. The payment, the access grant or the supplier change does not follow the normal route.
- Short scripted answers. On a call the other party issues instructions and steers away from open conversation.
- Nothing only a colleague would know. The message is specific about business detail yet vague about anything shared and personal.
- Machine-speed activity in your logs. Thousands of requests a second from one account is not a person.
Then the honest part. Do not build the defence on spotting the fake. In a study published in PLOS ONE in 2023, 529 listeners identified deepfake speech correctly 73% of the time and training barely moved the number. Detection helps. Verification decides.
How to Defend
Start With the People
Train people for the situation they will actually face. Staff who handle payments, access or supplier data need practice at refusing a plausible instruction from a senior person, because that is the moment an attack turns on. Realistic simulation beats a slide deck which is the whole point of security awareness training that uses current lures.
Give everyone the words to use. “I will call you back on the number we have on file” should be normal, expected and never treated as an insult.
Fix the Process
- Require out-of-band verification for every payment change or urgent transfer using a number held in your own records.
- Set a rule that voice and video approvals authorise nothing on their own.
- Agree a challenge question for executives and finance staff. Rehearse using it.
- Write an AI acceptable-use policy naming the sanctioned tools and what may never be pasted into them.
- Keep an inventory of AI systems, agents and suppliers and mark which of them can act on data or systems.
- Walk through the 24-hour and 72-hour reporting duties before you need them.
Then the Technology
- Enforce phishing-resistant multi-factor authentication everywhere it will run.
- Shorten patch cycles for internet-facing systems and treat the disclosure-to-exploit window as days.
- Monitor continuously including the AI tools themselves and alert on request rates no human could produce.
- Give agents, API keys and service accounts the least privilege that lets them work then review them like user accounts.
- Segment the network so one stolen credential does not reach everything.
- Test your own exposure with penetration testing that includes the AI systems you have deployed.
Continuous monitoring is where most organisations run short of people which is the gap managed detection and response is built to close with AI detection and response extending the same idea to the AI systems in your estate.
What Sweden’s Experts Recommend for the Next Twelve Months
Sweden’s own answer arrived on 30 June 2026. A report written by Swedish AI and cyber security experts within IVA’s Svenska framtider project, in dialogue with Nationellt cybersäkerhetscenter and AI Sweden, set five priorities for the coming year.
- Move faster, because waiting for the perfect answer is itself a risk.
- Get the fundamentals right, meaning multi-factor authentication, access control, patching, logging and incident handling.
- Strengthen governance so leaders know which systems and suppliers are genuinely critical.
- Share threat information faster between agencies, companies and researchers in normal times.
- Fight AI with AI, by integrating it into the defence in a tested and controlled way.
Pontus Johnson, the KTH professor who chaired the group, framed the near-term task as closing the most important gaps quickly while integrating AI into the defence responsibly. Start with the callback rule for payments this week. Then build the inventory of AI systems, agents and suppliers that the governance work depends on.