Modeling Complexity: When Agents Meet Neural Networks

by

Author: Denis Avetisyan

A new framework bridges the gap between agent-based simulations and deep learning to create more interpretable and reliable models of complex systems.

The agent-based model-informed neural network successfully captures the trajectories of the susceptible-infected-recovered (SIR) dynamics-defined by parameters $n=100$, $p=0.05$, $\beta=0.4$, $\gamma=0.2$, and a time step of $dt=0.1$-while simultaneously preserving the integrity of the compartmental structure inherent to the system.
The agent-based model-informed neural network successfully captures the trajectories of the susceptible-infected-recovered (SIR) dynamics-defined by parameters $n=100$, $p=0.05$, $\beta=0.4$, $\gamma=0.2$, and a time step of $dt=0.1$-while simultaneously preserving the integrity of the compartmental structure inherent to the system.

This review explores the integration of agent-based modeling principles with neural networks, including Neural ODEs and Graph Neural Networks, for improved structural consistency and counterfactual analysis in fields like epidemiology and macroeconomics.

Modeling complex systems often requires balancing data-driven flexibility with the enforcement of known physical or behavioral constraints. This challenge motivates the work ‘Towards agent-based-model informed neural networks’, which introduces a novel framework integrating principles from agent-based modeling with neural networks to learn interpretable, structure-preserving dynamics. By leveraging restricted graph neural networks and hierarchical decomposition, the authors demonstrate improved performance in forecasting and parameter recovery across diverse applications, including epidemiology and macroeconomics. Could this approach pave the way for more robust and insightful models of complex phenomena where underlying mechanisms are partially understood?

The Inevitable Complexity of Dynamic Systems

Real-world systems, be they the progression of a pandemic, the fluctuations of financial markets, or even the behavior of ant colonies, rarely adhere to simple, linear patterns. These systems are characterized by a multitude of interacting components, each influencing and being influenced by others, creating feedback loops and emergent behaviors. This inherent complexity arises from the dynamic interplay between these elements, where initial conditions have disproportionate effects – often referred to as the “butterfly effect” – and where predicting long-term outcomes becomes exceedingly difficult. Unlike simplified models that assume stability or predictable change, these systems are constantly evolving, adapting to internal and external pressures, and exhibiting non-linear responses that defy traditional analytical techniques. The sheer number of variables and their intricate relationships necessitate novel approaches to understanding and, crucially, anticipating their behavior.

Historically, predictive models in fields like epidemiology and economics have frequently depended on streamlining complex realities into manageable, yet often inaccurate, representations. These techniques, while computationally efficient, typically achieve tractability by imposing strong assumptions – linearity, homogeneity, or independence – that rarely hold true in genuine dynamic systems. For example, assuming a constant rate of infection ignores the influence of behavioral changes or varying immunity levels, diminishing the model’s capacity to forecast outbreaks effectively. This reliance on simplification frequently results in a trade-off between mathematical convenience and predictive power, leading to models that may capture broad trends but fail to anticipate critical shifts or accurately reflect the nuances of the phenomenon under investigation. Consequently, researchers are increasingly recognizing the limitations of these approaches and seeking methods that embrace, rather than suppress, the inherent complexity of real-world processes.

Understanding how complex systems evolve necessitates moving beyond equations that describe aggregate behavior and instead focusing on agent-based modeling. This approach simulates the actions of autonomous ‘agents’ – be they individual organisms, people, or even molecules – within a defined environment and allows researchers to observe emergent patterns arising from local interactions. Unlike traditional top-down modeling, which often averages out crucial details, agent-based models capture the heterogeneity and feedback loops inherent in real-world phenomena. By defining rules for each agent’s behavior and allowing them to interact, simulations can reveal how global trends emerge from countless individual decisions, offering insights unattainable through conventional analytical methods. This shift enables the exploration of ‘what-if’ scenarios and a deeper comprehension of system resilience, adaptation, and potential tipping points, proving vital in fields ranging from epidemiology to social science and economics.

ABM-NN outperforms a Graph Convolutional Network baseline in predicting system trajectories, especially when generalizing to unseen scenarios.
ABM-NN outperforms a Graph Convolutional Network baseline in predicting system trajectories, especially when generalizing to unseen scenarios.

Synergistic Modeling: Bridging Agent Behavior and Neural Networks

Agent-Based Models (ABMs) simulate systems by defining individual agents with specified attributes and behavioral rules. These agents interact within a defined environment according to these rules, and the emergent, aggregate behavior of the system is then observed. This bottom-up approach allows for the modeling of complex phenomena arising from local interactions, without requiring a centralized control mechanism or global equation. ABMs are particularly useful in scenarios where individual heterogeneity and decentralized decision-making are key characteristics of the system, such as modeling social dynamics, epidemic spread, or market behavior. The simulation proceeds by iteratively updating the state of each agent based on its rules and interactions, providing a dynamic representation of the system’s evolution.

Agent-based models, while capable of representing complex system dynamics, present significant computational challenges. The time required for simulation scales with the number of agents and the complexity of their interactions, often necessitating high-performance computing resources. Furthermore, defining appropriate agent behaviors-the rules governing individual actions and responses-requires substantial effort and careful calibration against empirical data. Incorrectly specified rules can lead to inaccurate or unstable simulations, demanding iterative refinement and validation. This calibration process is often non-trivial, as the optimal parameter values may not be readily apparent and can be sensitive to initial conditions and model assumptions.

Agent-Based Model informed Neural Networks (ABM-NNs) represent a hybrid approach to modeling complex systems, leveraging the benefits of both agent-based modeling and neural networks. Traditional ABMs excel at incorporating domain knowledge and mechanistic understanding through explicitly defined agent behaviors, but can suffer from computational limitations and difficulties in scaling to large, complex environments. Neural networks, conversely, demonstrate flexibility and generalization capabilities but often lack inherent mechanistic interpretability. ABM-NNs address these limitations by using data generated from ABMs to train neural networks, effectively transferring mechanistic insights into the learning process. This allows the neural network to generalize to unseen graph structures – configurations not present in the original training data – while retaining some of the mechanistic plausibility derived from the underlying ABM. The result is a model capable of predicting system behavior on novel graphs with improved out-of-sample performance compared to purely data-driven neural networks or traditional ABMs.

Unlike a standard feed-forward network, a physics-informed Hamiltonian neural network accurately preserves energy and generates physically consistent trajectories when simulating a spring system, as demonstrated by the color gradient representing time progression.
Unlike a standard feed-forward network, a physics-informed Hamiltonian neural network accurately preserves energy and generates physically consistent trajectories when simulating a spring system, as demonstrated by the color gradient representing time progression.

Dissecting the Mechanism: How ABM-NNs Capture System Evolution

Agent-Based Model Neural Networks (ABM-NNs) utilize Neural Ordinary Differential Equations (Neural ODEs) to model the change in agent states over time. Unlike discrete time-step approaches, Neural ODEs define dynamics as a continuous process, represented by the equation $ \frac{dz}{dt} = f(z(t), t) $, where $z(t)$ represents the agent’s state at time $t$ and $f$ is a neural network determining the rate of change. This continuous formulation allows for variable-length simulations and avoids the limitations imposed by fixed time steps, enabling more flexible and realistic modeling of complex system dynamics. The continuous-time representation is particularly advantageous for systems where events occur at irregular intervals or where precise timing is critical to the observed behavior.

Restricted Graph Neural Networks (RGNNs) improve the modeling of local interactions in Agent-Based Models (ABMs) by limiting the scope of message passing between agents. Unlike standard Graph Neural Networks (GNNs) which can consider the entire graph, RGNNs define a restricted neighborhood – typically based on physical proximity or defined interaction ranges – for each agent. This restriction reduces computational complexity and focuses the network’s attention on the most relevant interactions. Specifically, message passing occurs only between agents within this defined neighborhood, allowing the network to learn and represent localized dynamics more effectively. The resulting architecture is particularly well-suited for ABMs where interactions are inherently local, such as in simulations of physical systems or social networks with limited communication ranges.

FiLM (Feature-wise Linear Modulation) Adapters are incorporated into the ABM-NN architecture to conditionally scale and shift the activations of neural network layers based on input features. This process allows the network to adapt its behavior without altering the learned weights, effectively modulating the network’s response to different scenarios or input conditions. Specifically, each input feature is transformed into two vectors: a scale vector and a shift vector. These vectors are then applied element-wise to the activations, allowing the network to emphasize or suppress specific features based on the current context. This conditional modulation improves the network’s flexibility and ability to generalize across diverse simulation conditions, as the same underlying network can exhibit different behaviors depending on the input features provided to the FiLM adapters.

ABM-NNs are designed with the capability to enforce Conservation Laws within their simulations, which constrains the network’s learned dynamics to adhere to predefined physical principles. This constraint is achieved through specific network architectures and loss functions that penalize violations of these laws, resulting in more realistic and plausible agent-based model behavior. Empirical evaluation demonstrates that ABM-NNs incorporating conservation constraints consistently achieve lower error rates in out-of-sample generalization tests compared to Graph Convolutional Networks (GCN), Graph SAGE, and Graph Transformers. This improved performance indicates that preserving fundamental physical laws enhances the network’s ability to accurately predict system evolution in unseen scenarios.

Despite increasing observational noise, the macro SIR learner effectively tracks underlying dynamics, maintaining separation between in-sample training data and out-of-sample forecasts.
Despite increasing observational noise, the macro SIR learner effectively tracks underlying dynamics, maintaining separation between in-sample training data and out-of-sample forecasts.

Expanding the Horizon: Advanced Neural Dynamics and Future Directions

Neural Continuous-time Dynamical Systems (CDEs) represent a significant advancement over Neural Ordinary Differential Equations (ODEs) in the realm of time series modeling, particularly when dealing with data that isn’t uniformly sampled. While Neural ODEs require consistent time intervals between data points, Neural CDEs can effectively process data with varying and irregular time steps. This is achieved by framing the dynamics as a continuous process and utilizing a learned representation of the rate of change, rather than discrete transitions. Consequently, Neural CDEs demonstrate enhanced robustness to missing or unevenly spaced data, a common characteristic of real-world time series. This flexibility allows for more accurate modeling of complex systems where data acquisition isn’t perfectly controlled, and it broadens the applicability of neural networks to a wider range of scientific and engineering challenges, including financial modeling, climate science, and epidemiological forecasting.

Hamiltonian Neural Networks represent a significant advancement in neural network design by incorporating the principles of Hamiltonian mechanics, a cornerstone of physics. Traditional neural networks often lack inherent constraints on their behavior, potentially leading to unrealistic or unstable simulations, particularly when modeling physical systems. These networks, however, are constructed to explicitly preserve energy, a fundamental quantity in physics, throughout their computations. This is achieved by formulating the network’s dynamics as a Hamiltonian system, ensuring that the total energy – the sum of kinetic and potential energies – remains constant over time. By enforcing this conservation law, the models demonstrate improved stability, accuracy, and physical plausibility in simulations of dynamic systems, from molecular dynamics to celestial mechanics. The framework allows for more reliable long-term predictions and a reduction in the need for extensive hyperparameter tuning, making them particularly well-suited for applications where physical fidelity is paramount.

The capacity to model interactions within complex networks is significantly enhanced through the integration of graph-based approaches. These methods move beyond traditional time-series analysis by explicitly representing relationships between entities as nodes and edges, allowing for a more nuanced understanding of systemic behavior. In social systems, this translates to modeling information diffusion or the spread of behaviors, while in ecological contexts, it enables the simulation of predator-prey dynamics or disease transmission. By leveraging graph neural networks and related techniques, researchers can capture dependencies and feedback loops that would otherwise be obscured, providing insights into emergent phenomena and improving predictive accuracy across diverse interconnected systems. This approach is particularly valuable when dealing with data where relationships are as important as, or even more important than, the individual data points themselves.

Recent advancements in neural dynamics are demonstrably expanding the scope of solvable problems across multiple disciplines. These techniques aren’t simply improving existing models; they are enabling accurate simulations of complex systems previously considered beyond reach. Specifically, models leveraging these innovations successfully reproduce aggregate epidemic curves, mirroring real-world observations with high fidelity – crucially, they also closely match ground truth data when evaluating the impact of time-limited social distancing interventions. Beyond epidemiology, the same underlying framework exhibits strong generalization capabilities, accurately forecasting macroeconomic trajectories using data from 2021-2024, a period deliberately held out from training. This suggests a powerful ability to capture underlying systemic dynamics and extrapolate beyond the specific conditions of the training data, promising significant breakthroughs in fields ranging from public health to economic forecasting.

Training neural networks with decoupled optimization and curriculum learning-using separate learning rates for network parameters and functional components-yields smooth gradients and ensures the learned functions conserve mass.
Training neural networks with decoupled optimization and curriculum learning-using separate learning rates for network parameters and functional components-yields smooth gradients and ensures the learned functions conserve mass.

The pursuit of structurally-consistent models, as detailed in this work, echoes a fundamental tenet of systems understanding. It acknowledges that complex systems aren’t merely collections of variables, but entities governed by underlying rules and interactions. Andrey Kolmogorov observed, “The most important things are the ones we don’t know.” This sentiment applies directly to the challenge of modeling complex systems; the framework presented attempts to illuminate those ‘unknowns’ by embedding agent-based modeling’s explicit rules within the learning process of neural networks. By prioritizing conservation laws and structural consistency, the research strives to move beyond purely data-driven approaches, acknowledging that true understanding necessitates an appreciation for the governing principles, even those initially obscured.

What Lies Ahead?

The convergence of agent-based modeling and neural networks, as explored in this work, doesn’t represent a destination, but rather an acceptance of inherent systemic imperfection. The pursuit of structurally-consistent models, guided by conservation laws, is less about achieving predictive accuracy-an illusion in truly complex systems-and more about mapping the boundaries of inevitable error. Each incident, each deviation from expectation, is not a failure of the model, but a step toward understanding the system’s natural modes of decay and adaptation. The framework presented offers a means to trace these trajectories, to observe how systems fail, rather than striving for the unrealistic goal of preventing failure altogether.

Future work will undoubtedly grapple with the challenge of scalability. While the initial demonstrations in epidemiology and macroeconomics are promising, extending this approach to systems with substantially larger agent populations or more intricate interactions will necessitate innovative computational strategies. More importantly, the field must confront the epistemological implications of counterfactual analysis. Establishing causal relationships within complex systems is fraught with difficulty, and relying solely on neural network-derived inferences risks mistaking correlation for genuine mechanistic understanding.

Ultimately, the value of this approach may lie not in its ability to predict the future, but in its capacity to create robust, interpretable models that age gracefully. Time, after all, is not a metric to be optimized, but the medium in which all systems erode-and the rate of erosion is, perhaps, the most informative metric of all.

Original article: https://arxiv.org/pdf/2512.05764.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2025-12-08 10:08