Phishing Simulation for Staff: How to Run One That Works

A phishing simulation reduces risk when staff experience it as practice, not as a trap: tell people a programme exists, use lures that look like the attacks you actually receive, never name or shame anyone, teach the moment someone clicks, and measure reports as well as clicks. In Microsoft 365, Attack simulation training in Microsoft Defender for Office 365 handles the sending, teaching and reporting, if you hold the licence Microsoft requires. Our security awareness training page describes how we run ours.

Should you tell staff about phishing simulations in advance?

Tell them the programme exists, not when each message will be sent. The UK National Cyber Security Centre (NCSC) warns that punishing people for clicking on emails you sent starts to resemble entrapment, and advises checking with your HR department before running any phishing simulation. Announcing the programme, its purpose and how results are used removes the feeling of being tricked by your own IT team.

A short announcement from senior management should cover:

• Why the programme exists: to build the habit of reporting, not to catch people out.
• That nobody will be disciplined for clicking a simulation.
• How to report a suspicious message, and which button to use.
• Who sees individual results, and that managers see team trends only.

The NCSC also says the message that people will not get in trouble for reporting needs buy-in across all departments, including HR, support and senior management. Agree the rules with HR before the first send.

What makes a simulated lure realistic but fair?

Realistic means it resembles what reaches your inboxes. Attack simulation training offers several social engineering techniques: credential harvest, malware attachment, link in attachment, link to malware, drive-by URL and OAuth consent grant, plus a how-to guide that teaches users, for example, how to report phishing. Links can be a URL or a QR code. Choose the technique that matches what your mail filters and recent incidents show.

Do not train people to hunt for spelling mistakes. In a real-world example in its phishing guidance, the NCSC describes a malware email that claimed to be an invoice needing urgent attention, was relevant to the roles of some recipients, and was well written, with good spelling and grammar. Teach the signs that still hold:

• Pressure to act quickly.
• Requests for passwords, payment changes or sensitive files.
• A request that breaks the normal way a task is done, such as a new bank account for a known supplier.
• Sender addresses or links that are almost, but not quite, right.

The NCSC also recommends making sure staff know the normal ways of working for key tasks, such as how payments are made, so unusual requests stand out. Test those processes in your simulations.

Fair means it tests judgement without exploiting fear. Agree with HR which themes are off limits, for example pay cuts, redundancies, medical results or family emergencies. Microsoft rates each payload's complexity by the number of warning signs it contains, and more signs make a payload easier to identify. Start with lower-complexity payloads and raise the difficulty as reporting improves.

Why should nobody be named or shamed for clicking?

Because it makes the real problem worse. The NCSC's guidance is direct: blaming users for clicking on links does not work, people click for a range of reasons, including being busy and stressed, and employees who are afraid for their jobs will not report mistakes. Someone who clicks a real phish and reports it within minutes gives you an early warning; someone who stays quiet gives the attacker time.

• No leaderboards, no names in all-staff emails, no simulation results in performance reviews.
• Individual results go to the person and the security lead only.
• Use repeat results for support, not discipline. Attack simulation training labels someone a "repeat offender" when they are compromised in consecutive simulations, two by default, and you can change that threshold. Treat the label as a cue for a short, private conversation and targeted training.

What should happen right after someone clicks?

Teach while the message is still on screen. In Attack simulation training:

• Choose a landing page, the page people see if they open the payload, that explains what just happened. Add payload indicators to the simulated email so people can see which details gave it away. Microsoft notes that payload indicators are not available for the malware attachment and link to malware techniques.
• Assign training with every simulation. Microsoft recommends this, saying employees who go through training are less susceptible to similar attacks. The training due date defaults to 30 days after the simulation ends; choose 7 days so the lesson stays close to the event.
• Thank the people who report. A positive reinforcement notification can be delivered immediately after someone reports the simulated phish.

The NCSC makes the same point for real reports: quickly tell people what action was taken, and make it clear their reports make a difference.

Which numbers show the programme is working?

Click rate alone tells you little, because it moves with the difficulty of the lure you picked. The NCSC suggests that as well as measuring how many people clicked on a phishing email, you focus on how many people reported it. Track these for each simulation and as a trend:

• Reported: the share of recipients who reported the message. This should rise.
• Compromised: users the simulation counts as compromised. Microsoft notes the signal varies by technique; in a credential harvest it is supplying credentials. Attack simulation training compares your actual compromise rate with a predicted rate based on historical Microsoft 365 data, which puts the result in context.
• Speed: the report shows when the first link was clicked and the first credentials were entered. Compare that with when the first report reached your reporting mailbox.
• Training completion: who finished the assigned training by the due date.

Check a surprising click before you talk to anyone about it. Microsoft notes that automated inspection of the payload link by security services, non-Microsoft filtering services or email forwarding can lead to false positives.

Two practical catches. First, Microsoft states that simulation messages reported through non-Microsoft tools are not captured in attack simulation reports, so give staff the built-in Report button in Outlook. CISA likewise encourages organisations to use the reporting features built into Outlook and other cloud email platforms. Second, Microsoft lists the UAE among the latest countries for Attack simulation training and says all features except reported email telemetry are available there for now. If your tenant's data is stored in the UAE and report counts do not appear, count reports in your reporting mailbox instead.

How often should you run phishing simulations?

Choose a rhythm you can sustain and review every round. CISA's joint phishing guidance asks organisations to regularly educate users on spotting suspicious emails and on reporting when they have opened one. We prefer smaller, regular simulations sent to different groups over one large annual send that colleagues can warn each other about. Our own programme sends simulated phishing every month and tracks click rate, report rate and credential-disclosure rate.

• Vary the technique and the theme each round.
• Send during working hours. Attack simulation training can deliver messages to each user during their regional working hours.
• Cover everyone over the year. One targeting option is users not targeted by a simulation in the last three months.
• Give new joiners the basics before their first simulation.

Simulations are not the only method. The NCSC notes that some companies instead ask staff to write their own phishing emails, which gives them a richer view of the influence techniques used and can avoid an "us vs them" feeling.

How do you set up Attack simulation training in Microsoft 365?

1. Check the licence. Microsoft states that Attack simulation training requires a Microsoft 365 E5 or Microsoft Defender for Office 365 Plan 2 licence, and Plan 2 is available as an add-on or included in subscriptions such as Microsoft 365 E5. E3 customers get only a limited trial subset: a credential harvest payload and two training experiences. Our Defender for Office 365 page compares the plans side by side.
2. Assign roles with least privilege. Give the person running campaigns the Attack Simulation Administrator role rather than Global Administrator, which Microsoft says to limit to emergency scenarios or cases where no other role will do.
3. Set up reporting first. Use the built-in Report button in Outlook. If reports go to a reporting mailbox, configure it as a SecOps mailbox; Microsoft warns that otherwise a reported message might trigger a training assignment from the simulation product.
4. Build the first simulation. In the Microsoft Defender portal, go to Email and collaboration, then Attack simulation training. Choose the technique and payload, the target users and exclusions, the training, the landing page, the notifications and the launch dates.
5. Review and adjust. Read the simulation report with the security lead, follow up individually and privately, and choose the next technique.

Simulations test the human layer only. They do not replace the technical controls, such as the anti-phishing policies whose impersonation protection covers the senders and domains you name. For the wider set of attacks to model your lures on, see email threats in the UAE and the controls that stop them.

When a real phishing email does get through, Defender alerts on our managed IT service are triaged by our Dubai-based SOC in business hours, with P1 incidents covered 24/7. To plan a simulation programme your staff will accept, contact us.