Cybersecurity decisions can be enhanced by obtaining a LLM, yet this may not apply to all individuals.
In a groundbreaking research, a group from the Max Planck Institute for Human Development is examining the influence of Large Language Models (LLMs) on decision-making in cybersecurity. The study, which focuses on areas like phishing detection, password management, and incident response, is based on CompTIA Security+ concepts.
The research involves two focus groups: one working on security tasks without AI support, and the other using an LLM. The participants are master's students with backgrounds in cybersecurity.
The findings suggest that while the group using an LLM showed more accuracy on routine tasks, such as spotting phishing attempts and ranking password strength, they may be at risk of over-reliance on models, reduced independent thinking, and a loss of diversity in how problems are approached.
In harder tasks, LLM users sometimes followed incorrect model suggestions, particularly when matching defense strategies to complex threats. To mitigate this, governance policies such as allow lists, dependency approval workflows, and gating can help enforce discipline and reduce blind trust in LLM recommendations.
Bar Lanyado, Lead Researcher at Lasso Security, recommends organizations to establish a human-in-the-loop structure to prevent blind trust in LLMs. Additionally, implementing baseline training on LLM failure modes, such as common model errors, hallucinations, outdated knowledge, and prompt injection risks, is crucial.
High-resilience individuals performed well with or without LLM support and were better at using AI guidance without becoming over-reliant on it. On the other hand, low-resilience participants did not gain as much from LLMs; in some cases, their performance did not improve or even declined.
To build resilience, the study suggests open-ended suggestions for high-resilience individuals, while lower-resilience users might need guidance, confidence indicators, or prompts that encourage them to consider alternative viewpoints. Pairing less experienced or lower-resilience teams with those skilled in analysis and questioning model recommendations could also help bridge this gap.
Organizations cannot assume adding an LLM will raise everyone's performance equally; designing AI systems that adapt to the user is recommended. To ensure safety, teams should always confirm that suggested outputs exist in terms of packages, check repository activity and security vulnerabilities, and run scans before adoption.
Lastly, to prevent automation misinformation and bias, organizations should validate LLM outputs against logs, captures, or other ground truth before taking any action. Enable continuous feedback loops: encourage teams to document when LLM outputs helped or misled them to build a culture of reflection and safe usage.
Security leaders need to plan for these differences when building teams and training programs. By understanding the impact of LLMs on cybersecurity decision-making, organizations can make informed decisions about integrating AI into their security strategies.
