Outsmarting Attackers: Adaptive Cyber Deception for Critical Infrastructure

Author: Denis Avetisyan

A new framework leverages artificial intelligence to dynamically defend industrial control systems against evolving cyber threats.

A novel reinforcement learning agent leverages large language models to dynamically construct cyber deceptions, probing system vulnerabilities with adaptive strategies.

This review details the integration of reinforcement learning and large language models to enhance security in operational technology environments like those using the DNP3 protocol.

Despite advancements in cybersecurity, defending critical infrastructure against sophisticated attacks remains a persistent challenge. This paper, ‘Network- and Device-Level Cyber Deception for Contested Environments Using RL and LLMs’, explores an innovative approach to cyber defense through the dynamic deployment of deception strategies within operational technology (OT) networks. Specifically, we investigate the fusion of reinforcement learning (RL) and large language models (LLMs) to create adaptive cyber deception frameworks capable of increasing attacker costs and enhancing the security of industrial control systems-such as those employing the DNP3 protocol. Could this integration of AI-driven deception ultimately shift the advantage to defenders in contested cyber environments?

The Cracks in the Fortress: Exposing Operational Technology Vulnerabilities

The backbone of modern society – critical infrastructure – is experiencing a surge in cyberattacks targeting Operational Technology (OT) systems. These systems, which control industrial processes like power grids, water treatment, and manufacturing, are increasingly the focus of both state-sponsored actors and financially motivated criminals. Adversaries are deploying increasingly sophisticated techniques, moving beyond simple disruption to actively manipulate processes and potentially cause physical damage. This escalation represents a significant departure from traditional cybersecurity threats, demanding a proactive and layered defense strategy capable of addressing vulnerabilities unique to these often-legacy systems. The potential consequences extend far beyond data breaches, posing risks to public safety, economic stability, and national security, necessitating urgent attention and investment in OT cybersecurity.

The escalating frequency and sophistication of cyberattacks targeting operational technology (OT) systems reveal a critical inadequacy in conventional security approaches. Historically, IT security models – firewalls, antivirus software, and perimeter-based defenses – are proving largely ineffective when applied to the unique characteristics of OT environments. These systems, designed for reliability and control rather than inherent security, often lack the patching capabilities and segmentation necessary to withstand modern threats. Consequently, a shift towards proactive, adaptive defense strategies is paramount. This includes employing threat intelligence tailored to industrial protocols, leveraging anomaly detection based on process behavior, and implementing robust network segmentation to limit the blast radius of potential compromises. Furthermore, a move beyond signature-based detection toward behavioral analysis and machine learning is essential to anticipate and neutralize attacks that exploit previously unknown vulnerabilities within these critical infrastructure systems.

Operational Technology (OT) systems, the backbone of critical infrastructure, frequently rely on communication protocols developed decades ago, such as DNP3, Modbus, and Profibus. These protocols, while functional for their original purpose, now present significant vulnerabilities in the face of modern cyber threats. Their legacy design often lacks robust authentication, encryption, and intrusion detection capabilities, making them easily exploitable. Compounding this issue is their pervasive deployment across essential sectors-power grids, water treatment facilities, and transportation networks-meaning a successful attack on even a single system could have cascading and widespread consequences. The sheer scale of these deployments, coupled with the difficulty of retrofitting security measures onto aging infrastructure, creates a substantial and growing risk profile for organizations responsible for maintaining these vital services.

Phenix, a GUI-based orchestration tool utilizing Minimega virtualization, visualizes the cyber network topology of the contested environment.

Illusions of Control: The Art of Cyber Deception

Cyber deception operates by altering an attacker’s understanding of a target environment, introducing false information and strategically placed decoys to influence their actions. This manipulation aims to disrupt the typical attack sequence, causing delays, misdirection, and potentially leading the adversary to incorrect conclusions about system vulnerabilities and available resources. Deceptive elements can include fabricated data, simulated systems, and misleading network configurations, all designed to interact with the attacker and generate a distorted perception of reality. Successful implementation requires careful planning to ensure the deception aligns with attacker Tactics, Techniques, and Procedures (TTPs) and avoids immediately revealing its artificial nature, thus maximizing the disruption to their offensive operations.

Cyber deception, unlike traditional honeypots which passively wait for exploitation, involves the proactive construction of a manipulated reality for an attacker within the target Cyber-Physical System (CPS). This extends beyond simply mirroring legitimate assets; deception mechanisms actively influence the adversary’s reconnaissance, decision-making, and ultimately, their attack path. By presenting false information about system configurations, data criticality, or network topology, the defense can steer the attacker towards fabricated targets or delay their progress. This shaping of the attacker’s understanding is achieved through the deployment of decoys, mirages, and moving target defenses, all designed to create a distorted perception of the CPS environment and hinder accurate threat assessment.

Successful cyber deception necessitates responses that evolve with attacker actions and convincingly mimic legitimate system behavior. Static deception techniques are readily identifiable and offer limited disruption; therefore, systems must dynamically adjust decoys, data presentations, and network interactions based on observed probing and exploitation attempts. This requires automated decision-making capabilities, including real-time analysis of attack vectors, intelligent resource allocation to maintain believable illusions, and the capacity to learn from interactions to refine deception strategies. The complexity of these adaptations is further increased by the need to maintain operational functionality while simultaneously deceiving the adversary, demanding a sophisticated understanding of both attack methodologies and normal system processes.

The agent's deceptive behavior is governed by a personality file outlining its specific roles and responsibilities. — The agent’s deceptive behavior is governed by a personality file outlining its specific roles and responsibilities.

Learning to Mislead: Reinforcement Learning for Adaptive Defense

Reinforcement Learning (RL) enables the automation and optimization of network and host deception techniques through iterative learning within a simulated environment. This approach bypasses the need for manually defined deception rules by allowing an agent to discover optimal actions through trial and error. The agent receives feedback in the form of rewards or penalties based on its actions, progressively refining its strategy to maximize cumulative reward. This learning process occurs entirely within the simulation, allowing for safe exploration of various deception tactics and adaptation to evolving adversarial behaviors before deployment in a live network. The use of RL facilitates dynamic deception, where strategies are not pre-programmed but rather emerge from the agent’s interaction with the simulated environment, potentially improving resilience against sophisticated attacks.

Network-level deception, when employing Reinforcement Learning (RL), centers on dynamically altering network characteristics such as traffic patterns and routing tables to mislead adversaries. This can involve creating false network services or redirecting malicious traffic to isolated environments. In contrast, host-level deception, also driven by RL, concentrates on simulating realistic system behaviors and responses. This includes crafting believable file systems, processes, and services to convince an attacker they have compromised a legitimate system, thereby delaying detection and gathering intelligence. Both approaches leverage RL agents to learn optimal deception strategies based on observed attacker actions and environment feedback, but differ in the scope of manipulation – network infrastructure versus individual system emulation.

The application of Reinforcement Learning to network and host deception relies fundamentally on the Markov Decision Process (MDP). An MDP defines an environment where an agent transitions between states based on actions, with each transition having an associated probability and reward. In this context, the ‘state’ represents the network or host configuration and the attacker’s actions, the ‘action’ is the deception strategy employed, and the ‘reward’ quantifies the success of the deception – for example, increased attacker dwell time or resource consumption. Effective Reward Engineering, specifically the careful design of the reward function, is crucial; it guides the RL agent towards optimal deception policies. By accurately reflecting the desired outcomes – such as maximizing detection avoidance or increasing the cost of attack – a well-defined reward function not only improves the realism of the deception but also enables potentially faster convergence of the learning algorithm towards an effective and adaptive deception strategy.

The Illusion of Authenticity: Enhancing Realism with Language Models

Host-level deception increasingly leverages the capabilities of Large Language Models (LLMs) to construct convincingly realistic responses, effectively mimicking legitimate system behavior. These models don’t simply parrot pre-scripted answers; instead, they dynamically generate text tailored to the specific context of an interaction, creating a far more nuanced and believable facade. By analyzing incoming requests and formulating replies that align with expected system functions, LLMs can convincingly emulate a genuine host, diverting attackers and concealing critical assets. This approach moves beyond simple pattern matching, enabling the creation of deceptive environments that respond intelligently to probing and reconnaissance attempts, ultimately enhancing the resilience of cyber-physical systems against sophisticated threats.

Retrieval-Augmented Generation, or RAG, significantly elevates the performance of Large Language Models in deceptive environments by grounding their responses in factual, external knowledge. Rather than relying solely on the parameters learned during training, RAG allows the LLM to first retrieve relevant information from a designated knowledge source – such as system logs, network data, or cybersecurity threat intelligence – and then utilize this retrieved context to formulate a more accurate and pertinent response. This process combats the tendency of LLMs to “hallucinate” or generate plausible but incorrect information, a critical flaw when aiming for realistic system mimicry. By dynamically incorporating up-to-date and specific details, RAG enables the LLM to craft responses that not only sound authentic but are also demonstrably relevant to the ongoing cyber-physical scenario, greatly enhancing the overall effectiveness of the deception strategy.

Rigorous evaluation of generated responses is central to the effectiveness of large language model integration within deception strategies. Metrics such as Perplexity are employed to assess the quality and coherence of the LLM’s output, and crucially, these scores are directly incorporated into the reinforcement learning (RL) reward function. This feedback loop incentivizes the LLM to produce increasingly realistic and convincing responses, significantly enhancing the overall deception capability. Studies demonstrate that this approach leads to a reduction in episode length during training compared to systems relying solely on cyber-physical (CP) interactions, and also facilitates earlier stabilization of the learning process – a result of the modulation introduced by the deception probability $P_{dec}$ . The ability to rapidly refine deceptive behaviors, guided by quantifiable linguistic quality, represents a substantial advancement in creating truly convincing and adaptive host-level deception systems.

Repeatedly updating the honeypot's personality file yields a more convincingly human-like response. — Repeatedly updating the honeypot’s personality file yields a more convincingly human-like response.

Bridging the Divide: Real-World Deployment and Future Horizons

Successfully transitioning Reinforcement Learning (RL) agents from controlled simulations to real-world cybersecurity operations demands robust Sim-To-Real transfer techniques. These methods address the inherent discrepancies between the simulated environment and the complexities of live networks, such as variations in network traffic, system latency, and attacker behaviors. Researchers are exploring domain randomization, where agents are trained on a diverse range of simulated scenarios, and domain adaptation, which fine-tunes agents using limited real-world data. Effective Sim-To-Real transfer not only enhances the agent’s ability to generalize its learned policies but also minimizes the risks associated with deploying unproven strategies in critical infrastructure, ultimately ensuring that these agents maintain effectiveness and resilience when facing novel and unpredictable cyber threats.

The future of cybersecurity hinges on proactive defense, and advancements in Multi-Agent Reinforcement Learning (MARL) promise a significant leap forward in this domain. Rather than relying on static defenses or isolated responses, MARL allows for the creation of networks of autonomous agents capable of learning and coordinating complex deception strategies. These agents can dynamically adapt to attacker behaviors, creating illusions, misdirections, and honeypots across an entire system – effectively raising the cost and difficulty of successful breaches. By enabling agents to learn from each other’s experiences and coordinate their actions, MARL moves beyond simple reactive measures to establish a continuously evolving and resilient defensive posture capable of anticipating and neutralizing emerging cyber threats. This coordinated approach represents a paradigm shift, transforming cybersecurity from a game of detection and response to one of strategic deception and proactive control.

Cybersecurity increasingly demands proactive defenses, and the future of threat mitigation lies in sophisticated deception technologies that dynamically evolve alongside attacker strategies. Current static defenses are quickly overcome by adaptive adversaries; therefore, ongoing research focuses on systems capable of continuous learning and adaptation. These systems employ techniques like reinforcement learning to analyze attacker behaviors, refine deceptive environments, and anticipate future attack vectors. The ability to automatically generate and deploy realistic, yet subtly flawed, systems – honeypots, deceptive data, and manipulated network services – offers a powerful means of diverting, delaying, and ultimately defeating increasingly sophisticated cyber threats. This continuous cycle of learning and adaptation is not merely about reacting to attacks, but about proactively shaping the cyber landscape to the defender’s advantage, creating an environment where attackers face escalating costs and diminishing returns.

The pursuit detailed within this research echoes a fundamental tenet of understanding any complex system: probing its boundaries to reveal its inner workings. This framework, integrating reinforcement learning and large language models to dynamically generate deceptive elements within OT networks, isn’t merely about security; it’s about exploiting comprehension of the DNP3 protocol and the inherent assumptions within industrial control systems. As Ada Lovelace observed, “The Analytical Engine has no pretensions whatever to originate anything.” This paper doesn’t seek to create security, but rather to skillfully manipulate existing vulnerabilities – to reverse-engineer the attacker’s perspective and, through carefully constructed illusions, expose the limits of their understanding. The adaptive deception isn’t a destination, but a continuous, intelligent test of the system’s resilience.

Beyond the Mirage

The presented framework, while demonstrating the potential of integrating reinforcement learning and large language models for OT deception, inevitably highlights the gaps in current understanding. Security, after all, isn’t about erecting impenetrable walls, but about understanding the predictable patterns of those who attempt to breach them. This research correctly identifies the need for adaptive deception, yet the true contest lies in anticipating the adversary’s adaptation to the deception itself-a recursive problem demanding increasingly sophisticated adversarial training. The reliance on DNP3, while practical for focused testing, implicitly acknowledges the limitations of generalizing these techniques across the vast, heterogeneous landscape of industrial protocols.

Future work should not shy away from embracing controlled chaos. Rather than striving for perfect emulation of normal system behavior, a more fruitful avenue lies in deliberately introducing subtle anomalies – carefully calibrated inconsistencies that reveal attacker intent without triggering widespread disruption. The cost of maintaining a convincing illusion is high; the benefit of revealing the attacker’s probing, far greater.

Ultimately, this line of inquiry forces a re-evaluation of the fundamental security paradigm. The goal shouldn’t be prevention, a demonstrably losing game, but rather intelligent detection through engagement. The system isn’t secured when it remains unbreached; it’s secured when it learns faster from each attempt than the attacker.

Original article: https://arxiv.org/pdf/2603.17272.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/