The Reality Gap in AI Code Security

Author: Denis Avetisyan

New research reveals a significant performance drop-off when applying deep learning and large language models to detect vulnerabilities in real-world code.

A deployment-focused framework facilitates the evaluation of vulnerability detection models, emphasizing a holistic approach to assessing system security.

A practical evaluation demonstrates that current AI models struggle with dataset bias, code representation, and generalization to unseen projects, hindering their effectiveness in deployment-oriented vulnerability detection.

Despite promising performance on curated benchmarks, the real-world efficacy of deep learning (DL) for vulnerability detection remains largely unproven. This study, ‘From Lab to Reality: A Practical Evaluation of Deep Learning Models and LLMs for Vulnerability Detection’, systematically evaluates both DL models (ReVeal, LineVul) and pretrained large language models (LLMs) on a newly constructed, time-sensitive dataset of Linux kernel vulnerabilities, revealing a substantial drop in performance when transitioning from controlled lab settings to realistic deployment scenarios. Our findings demonstrate that current approaches struggle with code representation, dataset generalization, and ultimately, reliable vulnerability detection in the wild. Can we bridge the gap between academic progress and practical security, and develop more robust and adaptable models for safeguarding software systems?

The Expanding Threat Landscape: Vulnerabilities in Modern Codebases

Contemporary software development routinely incorporates numerous third-party components – libraries, frameworks, and modules created by external developers. While this practice accelerates development and reduces costs, it simultaneously expands the potential attack surface exponentially. Each integrated component represents a new point of vulnerability, as flaws within that code are now directly accessible to those targeting the larger application. This reliance introduces risks beyond the control of the original software author, demanding constant vigilance and robust security testing not only of the core application, but also of every external dependency. The complexity is further compounded by supply chain attacks, where malicious code is injected into a seemingly legitimate third-party component, propagating vulnerabilities across countless applications that utilize it. Consequently, modern software security requires a shift in focus from solely securing internally developed code to comprehensively assessing the integrity and security of the entire ecosystem of external dependencies.

Contemporary software development prioritizes rapid iteration and deployment, a pace that increasingly challenges established vulnerability detection techniques. Historically, these methods – relying on manual code reviews, penetration testing, and signature-based scanning – proved effective but are now hampered by the sheer velocity of code changes and the escalating scale of modern applications. The proliferation of microservices, containerization, and continuous integration/continuous delivery (CI/CD) pipelines means that codebases are in a perpetual state of flux, rendering traditional, time-consuming analyses obsolete before completion. Moreover, the intricate dependencies within modern applications, often incorporating countless third-party libraries and frameworks, create a complex web of potential vulnerabilities that are difficult to map and assess using conventional tools. Consequently, a significant gap exists between the rate of vulnerability introduction and the capacity of existing detection methods to identify and mitigate these risks, demanding more dynamic and automated approaches to secure software.

The escalating complexity of modern software necessitates a shift towards automated vulnerability detection. Contemporary codebases often comprise millions of lines of code, frequently incorporating numerous third-party libraries and dependencies; manual inspection simply cannot scale to address this immensity. Consequently, researchers are actively developing and refining automated tools leveraging techniques like static and dynamic analysis, machine learning, and fuzzing to proactively identify potential weaknesses. These solutions aim to scan code for patterns indicative of vulnerabilities, simulate attacks to assess system resilience, and learn from past exploits to predict future threats. The focus is no longer solely on finding vulnerabilities, but on establishing continuous, automated systems that can keep pace with the rapid development cycles and evolving threat landscape, ensuring software remains secure throughout its lifecycle.

Maintaining software integrity through robust vulnerability detection is paramount in the face of increasingly sophisticated cyber threats. A compromised codebase can lead to data breaches, financial loss, and reputational damage, impacting individuals and organizations alike. Proactive identification and remediation of weaknesses – before malicious actors can exploit them – is no longer simply a best practice, but a necessity. This requires a shift towards continuous security assessment, integrating automated tools and techniques throughout the software development lifecycle. The potential consequences of neglecting vulnerability detection extend beyond immediate financial costs; long-term damage to trust and operational stability can be devastating, highlighting the critical role of preventative security measures in a connected world.

VentiVul CVE fixes demonstrate a range of code modifications to address vulnerabilities.

Representing Code for Machine Learning: A Foundational Step

Effective application of machine learning to source code necessitates transforming it into a numerical format while preserving crucial structural and semantic information. Source code, inherently textual, is not directly consumable by most machine learning algorithms; therefore, preprocessing is required. Structural properties encompass the code’s syntactic arrangement – the relationships between keywords, operators, and identifiers – while semantic properties relate to the code’s meaning and behavior. Ignoring either aspect can lead to a loss of vital information, negatively impacting model performance. For example, the nesting of control flow statements or the relationships between variables and functions represent structural elements, while the purpose of a function or the data dependencies within a block of code constitute semantic elements. The chosen representation method must therefore carefully balance the preservation of these properties to facilitate accurate learning and analysis.

Token-based representation converts source code into a sequence of discrete tokens, such as keywords, identifiers, and operators, effectively linearizing the code for processing by sequence models like recurrent neural networks. This approach focuses on lexical information and immediate context. Conversely, graph-based representation models code as a graph structure, where nodes represent code elements (e.g., variables, functions, control flow statements) and edges represent relationships between them, like data dependencies or control flow. This allows models, particularly graph neural networks (GNNs), to capture complex structural relationships and dependencies within the code that are lost in token-based methods. Both techniques aim to distill essential code characteristics into a format suitable for machine learning algorithms, but differ significantly in how they encode the code’s underlying structure.

Graph-based representation of source code leverages graph neural networks (GNNs) to model code as a graph structure, where nodes represent code elements – such as variables, operators, and literals – and edges define relationships between them, including data flow, control flow, and syntactic dependencies. This approach contrasts with linear representations like token sequences by explicitly capturing the complex interconnections inherent in code. GNNs can then operate on this graph, learning node embeddings that encode both the characteristics of individual code elements and their contextual relationships within the broader program structure. The resulting embeddings provide a richer and more nuanced representation of the code, enabling machine learning models to better understand its semantics and structural properties, ultimately improving performance in tasks like vulnerability detection and code similarity analysis.

The efficacy of vulnerability detection models is directly correlated with the chosen code representation technique. Models utilizing token-based representations may struggle with complex code structures and long-range dependencies, potentially leading to decreased accuracy and increased false positives. Conversely, graph-based representations, which explicitly model relationships between code elements such as data flow and control flow, consistently demonstrate improved performance in identifying vulnerabilities, particularly in cases where the vulnerability’s manifestation depends on these structural characteristics. Specifically, the ability to capture non-local dependencies and contextual information within the code graph contributes to a more robust and accurate vulnerability assessment, as evidenced by benchmark results on datasets like SARD and Juliet.

Comparing feature space visualizations and cluster analysis across four datasets reveals distinct representations generated by GNN and CodeBERT Tokenizer.

Deep Learning Models for Vulnerability Detection: Benchmarking and Datasets

Deep learning models are gaining prominence in automated vulnerability detection due to their capacity for both scalability and improved accuracy compared to traditional methods. These models utilize techniques like neural networks to analyze source code and identify patterns indicative of security flaws. The scalability arises from the ability of these models to process large codebases efficiently, automating a traditionally manual and time-consuming process. Improvements in accuracy stem from the model’s capacity to learn complex relationships within the code, leading to fewer false positives and negatives in vulnerability identification. This is particularly valuable in modern software development environments where code volume and complexity are constantly increasing, and rapid vulnerability assessment is critical.

ReVeal and LineVul are deep learning models designed for automated vulnerability detection that utilize code representations learned from source code. ReVeal employs a convolutional neural network (CNN) to analyze code as an image, identifying patterns indicative of vulnerabilities. LineVul, conversely, focuses on learning embeddings of code lines, enabling it to detect vulnerabilities based on semantic similarity to known flawed code. Both models require substantial training data to effectively learn these representations and accurately classify vulnerable code segments, with performance dependent on the quality and diversity of the training dataset used to generate the initial embeddings.

Datasets BigVul, Devign, Juliet, and ICVul are essential resources for developing and assessing deep learning models designed for vulnerability detection. BigVul focuses on Java code, offering a substantial volume of vulnerable samples. Devign comprises C/C++ code with a focus on real-world vulnerabilities. Juliet, developed by NIST, provides a large collection of C/C++ code with intentionally inserted vulnerabilities, serving as a common benchmark. ICVul is a more recent dataset concentrating on inter-procedural C vulnerabilities. However, the utility of these datasets is directly tied to data quality; inconsistencies, inaccurate labeling, or a lack of diversity in vulnerability types can significantly hinder model performance and generalization capabilities. Careful dataset curation, including thorough validation and cleaning, is therefore critical for reliable model training and evaluation.

Performance evaluations of deep learning models for vulnerability detection reveal substantial generalization challenges. Specifically, models trained on one dataset often exhibit significantly reduced accuracy when evaluated on different datasets. For instance, the LineVul model achieved an F1-score of only 38.77 when trained on the Juliet dataset and tested on ICVul. Similarly, ReVeal demonstrated an F1-score of 0.5 when trained on BigVul and subsequently tested on Devign. These results highlight a critical limitation: models appear to learn dataset-specific patterns rather than generalizable vulnerability characteristics, necessitating careful consideration of training and testing data distributions and the need for more robust generalization techniques.

Analysis of ICVul and Juliet datasets reveals that the top five vulnerability types exhibit significantly different centroid distances compared to neutral samples, indicating a potential for improved vulnerability detection through distance-based methods.

Beyond Synthetic Benchmarks: Evaluating Real-World Performance

Conventional vulnerability detection often leans heavily on synthetic datasets – collections of code specifically crafted to exhibit certain flaws. However, these artificially generated examples frequently fail to capture the complexity and nuance of vulnerabilities found in real-world software. The very process of creating such datasets can inadvertently introduce biases or oversimplify the conditions that lead to exploitable flaws, leading to inflated performance metrics that don’t translate to practical security. Consequently, models trained and evaluated solely on synthetic data may demonstrate high accuracy in a lab setting but prove surprisingly ineffective when confronted with the messy, unpredictable codebases encountered in actual software development and deployment. This disconnect highlights a critical need for evaluation methodologies that prioritize realism and reflect the genuine challenges of identifying vulnerabilities in production environments.

Traditional vulnerability assessments frequently employ synthetic datasets that, while convenient, often fail to capture the complexities of real-world codebases. Consequently, researchers are increasingly turning to deployment-oriented evaluation techniques to obtain a more realistic understanding of security tool performance. This approach moves beyond isolated code snippets, instead analyzing entire files – reflecting how tools encounter code in practical scenarios – and employing function-pair comparisons. By presenting tools with both vulnerable and patched versions of code, these comparisons directly assess a system’s ability to discern semantic changes and identify true vulnerabilities, rather than simply flagging code that appears suspicious. Such rigorous testing provides a more nuanced and trustworthy measure of a tool’s effectiveness in a live deployment environment, revealing limitations not apparent in simplified, synthetic benchmarks.

The VentiVul dataset represents a significant advancement in vulnerability detection evaluation by intentionally focusing on the challenge of out-of-distribution generalization. Existing datasets often contain samples similar to those used during model training, leading to inflated performance metrics that don’t translate to real-world scenarios where codebases and vulnerability patterns can differ substantially. VentiVul addresses this limitation by incorporating a diverse range of vulnerabilities sourced from multiple projects and introducing variations in coding styles and vulnerability contexts. This deliberate design forces models to move beyond memorization and demonstrate a genuine ability to identify vulnerabilities in unseen code, providing a more realistic and robust assessment of their practical effectiveness. The dataset’s construction prioritizes testing a model’s capacity to adapt to novel situations, mimicking the dynamic and unpredictable nature of software security challenges.

Despite demonstrating over 92% accuracy when tested against the VentiVul dataset, large language models revealed a significant performance disparity when identifying actual vulnerabilities. This high accuracy was largely driven by a strong bias towards predicting code as non-vulnerable, resulting in exceptionally low F1-scores ranging from 0 to 4.9. This metric highlights the models’ inability to correctly identify positive cases-true vulnerabilities-amidst a sea of safe code. Further analysis exposed a limited capacity to understand the semantic impact of code patches; models could only correctly differentiate approximately 6 out of every 100 pairs of patched code segments, suggesting a superficial understanding of code functionality and a reliance on statistical correlations rather than genuine comprehension of the changes introduced by security fixes. This disconnect between overall accuracy and precise vulnerability detection underscores the critical need for evaluation metrics that prioritize identifying true positives and assessing a model’s capacity to reason about code semantics.

The model accurately predicts function behavior both before and after applying the VentiVul fix.

Towards Automated Code Repair and Enhanced Security

Effective code repair hinges not simply on patching symptoms, but on a deep comprehension of vulnerability origins. Researchers are increasingly focused on identifying the root causes – be it flawed logic, improper input validation, or memory management errors – that allow exploits to occur. By analyzing the precise mechanisms that introduce weaknesses, developers can move beyond reactive fixes to create more robust and preventative solutions. This approach involves techniques like static and dynamic analysis, coupled with the study of common vulnerability patterns, to pinpoint the precise lines of code responsible and understand the conditions leading to their exploitation. Ultimately, addressing these underlying causes significantly reduces the likelihood of future vulnerabilities and fosters a more secure software ecosystem, rather than perpetually chasing emergent flaws.

The systematic examination of code fixes implemented to address software vulnerabilities reveals recurring patterns in the types of errors developers commonly encounter. Research indicates that a disproportionate number of security flaws stem from a relatively small set of coding mistakes, such as improper input validation, buffer overflows, and incorrect handling of resource allocation. By meticulously analyzing the specific changes made to rectify these issues – the lines of code added, modified, or deleted – researchers can build a deeper understanding of the root causes of security vulnerabilities. This knowledge then informs the development of more effective static analysis tools, automated testing frameworks, and even educational resources designed to prevent these errors from being introduced in the first place. Essentially, each code fix acts as a case study, offering valuable data points in the ongoing effort to proactively improve software security and build more resilient systems.

The convergence of vulnerability detection and automated code repair represents a paradigm shift in software security, moving beyond reactive patching towards a proactive defense. Traditionally, vulnerabilities are identified through testing or post-deployment discovery, necessitating manual intervention to develop and apply fixes. However, integrating detection mechanisms with automated repair systems allows for immediate correction of flaws as they are identified, potentially before they can be exploited. This approach leverages advancements in static and dynamic analysis to pinpoint weaknesses, then employs techniques like program synthesis or machine learning to generate and implement corrective code changes. The promise lies not just in faster remediation, but in reducing the window of opportunity for attackers and ultimately building software that actively defends against threats, fostering a more secure and resilient digital ecosystem.

The pursuit of truly resilient and secure software necessitates ongoing investigation into automated code repair techniques. Current approaches, while promising, often address symptoms rather than root causes, leaving systems vulnerable to novel attacks and increasingly sophisticated exploits. Future research must prioritize the development of systems capable of not only identifying and patching vulnerabilities, but also of learning from past errors to proactively prevent similar flaws from emerging in new code. This includes exploring advanced machine learning models, formal verification methods, and techniques for automatically generating and testing security patches. Ultimately, sustained effort in this domain represents a critical investment in the long-term stability and trustworthiness of the digital infrastructure upon which modern society depends, moving beyond reactive security measures toward a future of self-healing and inherently secure software.

The study meticulously details how seemingly robust deep learning and large language models falter when transitioning from curated benchmarks to the complexities of real-world codebases. This echoes G. H. Hardy’s assertion: “A mathematician, like a painter or a poet, is a maker of patterns.” The ‘patterns’ these models learn are often brittle, exquisitely tuned to the specific characteristics of training data, and fail to generalize when confronted with the inevitable variations found in production code. The research underscores that performance metrics are merely snapshots; true evaluation requires understanding how a system behaves over time as it encounters unforeseen inputs – a testament to the idea that architecture dictates behavior. The limitations in cross-dataset generalization reveal that the underlying ‘architecture’ of these models is not yet sufficiently adaptable to handle the nuances of diverse code representations and dataset biases.

Beyond the Benchmark

The pursuit of automated vulnerability detection, as evidenced by this work, reveals a recurring pattern: performance in contrived settings bears little resemblance to efficacy in the wild. Each reported advance, each novel architecture, introduces a new dependency – on curated datasets, specific code representations, or carefully tuned hyperparameters. Every new dependency is the hidden cost of freedom, and the system’s structural limitations become glaringly apparent when confronted with the messiness of real-world codebases. The observed failures in cross-dataset generalization are not merely statistical anomalies; they are symptoms of a deeper problem – a reliance on superficial patterns rather than a genuine understanding of code semantics.

Future progress will necessitate a shift in focus, moving beyond the pursuit of incremental gains in benchmark scores. The field must grapple with the challenges of data heterogeneity, embracing techniques that allow models to learn from imperfect, unlabeled data. A more holistic approach is needed, one that considers not only the detection of vulnerabilities but also their root causes – the systemic flaws in software development processes that give rise to them.

Ultimately, the true measure of success will not be the number of vulnerabilities identified, but the reduction in their prevalence. This demands a re-evaluation of the entire paradigm, acknowledging that automated tools are merely one component of a much larger, more complex system. The structure, after all, dictates the behavior.

Original article: https://arxiv.org/pdf/2512.10485.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/