When Machine Learning Meets Soil: A Reality Check for Geotechnical Engineering

by

Author: Denis Avetisyan

A new review assesses the current capabilities of deep learning techniques, specifically Physics-Informed Neural Networks and Operator Learning, for solving complex problems in soil mechanics and foundation design.

The Deep Operator Network architecture learns a functional mapping from input loads to structural deflections by encoding both the applied load <span class="katex-eq" data-katex-display="false"> p(x) </span> and query locations <span class="katex-eq" data-katex-display="false"> x </span> into latent representations, subsequently combining these via a dot product to rapidly predict deflections <span class="katex-eq" data-katex-display="false"> w(x) </span> for arbitrary load patterns following offline training.
The Deep Operator Network architecture learns a functional mapping from input loads to structural deflections by encoding both the applied load p(x) and query locations x into latent representations, subsequently combining these via a dot product to rapidly predict deflections w(x) for arbitrary load patterns following offline training.

Despite recent advances, traditional numerical methods currently outperform deep learning approaches in most geotechnical applications, particularly regarding accuracy, speed, and reliable extrapolation.

Despite increasing enthusiasm for data-driven solutions, many geotechnical engineering challenges remain poorly suited to emerging deep learning techniques. This is the central argument of ‘Deep Learning in Geotechnical Engineering: A Critical Assessment of PINNs and Operator Learning’, which rigorously benchmarks physics-informed neural networks (PINNs), deep operator networks (DeepONet), and graph network simulators against established numerical methods. The study reveals that while these approaches show theoretical promise, they are often orders of magnitude slower and less accurate than traditional solvers, particularly when extrapolating beyond training data-a common requirement in real-world geotechnical prediction. Given these limitations, what criteria should guide the responsible application of deep learning within this traditionally physics-based field?

The Limits of Traditional Modeling in Geotechnical Systems

Geotechnical engineering, concerned with the behavior of earth materials, fundamentally depends on numerical modeling to predict how soils and rocks will react to various loads and environmental conditions. The Finite Difference Method, a cornerstone of this practice, approximates continuous soil behavior by dividing the ground into a discrete mesh and calculating interactions at each point. This allows engineers to simulate phenomena like slope stability, foundation settlement, and earthquake-induced ground motion. While conceptually straightforward, accurately capturing the intricate, often nonlinear, responses of real-world soils requires increasingly complex models and substantial computational resources. These simulations are crucial for ensuring the safety and longevity of infrastructure projects, from bridges and tunnels to dams and buildings, making robust and reliable numerical techniques indispensable to the field.

Established geotechnical engineering techniques, such as the Finite Difference Method, frequently encounter challenges when modeling realistic soil behavior due to their computational demands and inherent difficulties with nonlinearity. Complex soil responses – those exhibiting disproportionate reactions to applied stress – require increasingly refined meshes and smaller time steps, escalating processing time and resource consumption. This computational burden is particularly acute when simulating large-scale phenomena or investigating long-term behavior. Consequently, the predictive power of these methods is often limited, especially in scenarios involving intricate geological formations or extreme loading conditions, as capturing the full spectrum of nonlinearities becomes computationally prohibitive and introduces potential inaccuracies in the modeled results.

Geotechnical modeling with established numerical techniques, while valuable, presents challenges regarding adaptability and efficiency. These traditional approaches frequently demand extensive manual calibration to accurately reflect real-world soil behavior, a process that is both time-consuming and potentially subjective. More critically, models calibrated for specific conditions often struggle to reliably predict outcomes when faced with previously unseen scenarios, hindering their broader applicability. Despite these limitations, finite difference solvers maintain a substantial performance advantage; studies reveal they can resolve complex wave propagation problems in mere seconds, a stark contrast to the approximately 90,000 times slower computation times observed with Physics-Informed Neural Networks (PINNs) for the same tasks.

To avoid artificially inflated accuracy due to spatial leakage, geotechnical data should be split based on site location rather than randomly, as soil properties typically exhibit correlation lengths of 50-100 meters.
To avoid artificially inflated accuracy due to spatial leakage, geotechnical data should be split based on site location rather than randomly, as soil properties typically exhibit correlation lengths of 50-100 meters.

The Allure and Perils of Function Approximation

Multi-Layer Perceptrons (MLPs) represent a class of feedforward artificial neural networks capable of approximating any continuous function, given sufficient complexity-specifically, a sufficient number of neurons in each layer-and appropriate training data. This functional approximation capability stems from the universal approximation theorem, which mathematically establishes this potential. In practical applications, MLPs learn to map input variables to output variables by adjusting the weights and biases within their interconnected layers during a training process. This learning is typically achieved through optimization algorithms like stochastic gradient descent, minimizing the difference between the network’s predictions and the known, correct outputs provided in the training dataset. The ability to learn complex, non-linear relationships from data makes MLPs a potentially valuable alternative to traditional numerical methods in scenarios where defining an explicit functional form is challenging or impossible.

Extrapolation failure in Multi-Layer Perceptrons (MLPs) represents a significant limitation when applying these networks to geotechnical prediction. This phenomenon occurs because MLPs, while effective at interpolating within the bounds of their training dataset, demonstrate reduced accuracy when predicting outcomes for input values that fall outside of that range. In geotechnical engineering, this is particularly problematic as site characterization often requires predictions for conditions beyond those directly observed during data collection – for instance, estimating soil behavior under loads exceeding those used in the training dataset. The network essentially learns a mapping of inputs to outputs within the training domain and does not inherently generalize to unseen conditions, leading to potentially unreliable predictions and requiring careful consideration of the limitations when interpreting results.

The selection of activation functions – ReLU, tanh, or Sigmoid – within Multi-Layer Perceptrons (MLPs) impacts model performance characteristics, but does not address the fundamental issue of extrapolation failure. While Physics-Informed Neural Networks (PINNs) represent a capable approach, they currently achieve an error rate of approximately 10-3. This level of accuracy is substantially lower than that obtained through traditional finite difference methods, which consistently demonstrate an error rate of 10-15, highlighting a significant performance gap in complex modeling scenarios.

Despite achieving high training accuracy on short-term consolidation settlement, multi-layer perceptrons using ReLU, tanh, and sigmoid activations all exhibit catastrophic extrapolation failure at 10 years, with ReLU demonstrating unbounded growth and the others prematurely saturating instead of approaching the true asymptotic solution of 100 mm.
Despite achieving high training accuracy on short-term consolidation settlement, multi-layer perceptrons using ReLU, tanh, and sigmoid activations all exhibit catastrophic extrapolation failure at 10 years, with ReLU demonstrating unbounded growth and the others prematurely saturating instead of approaching the true asymptotic solution of 100 mm.

Bridging the Gap: Physics-Informed Neural Networks as a Hybrid Approach

Physics-Informed Neural Networks (PINNs) represent a hybrid approach to solving complex problems by integrating the capabilities of neural networks with established physical laws. Unlike traditional machine learning models trained solely on data, PINNs incorporate governing equations – such as the \frac{\partial^2 u}{\partial t^2} = c^2 \frac{\partial^2 u}{\partial x^2} Wave Equation – directly into the model’s loss function. This constraint forces the neural network’s predictions to adhere to known physical principles, leading to improved accuracy, especially in scenarios with limited training data or where extrapolation is required. By minimizing both the discrepancy between predicted and observed data and the residual of the governing equations, PINNs enhance generalization capabilities and provide physically plausible solutions.

Automatic Differentiation (AD) is a core component of Physics-Informed Neural Networks (PINNs) enabling the efficient computation of derivatives necessary for both data-driven and physics-based loss function optimization. Unlike numerical differentiation which relies on finite difference approximations, AD leverages the chain rule to compute derivatives with machine precision. This is critical for evaluating the residual of the governing physical equation – such as \nabla^2 u = f – within the loss function. By automatically calculating these derivatives, PINNs can effectively backpropagate gradients through the physics-based terms alongside the data-driven terms, allowing the neural network to learn solutions that simultaneously satisfy both the observed data and the underlying physical laws without explicit symbolic differentiation.

Despite their potential benefits, Physics-Informed Neural Networks (PINNs) can exhibit significant computational costs when compared to traditional numerical methods. Benchmarking indicates that for a dataset of 10,000 cases, the DeepONet architecture, a type of PINN, requires 258 seconds to complete, whereas a direct finite difference solution achieves the same result in only 1.1 seconds. This represents a 235x performance disadvantage for the neural network approach, highlighting the need for careful evaluation of implementation details and consideration of alternative methods when computational efficiency is paramount.

Although inverse PINNs can recover velocity profiles from waveform data with reasonable accuracy (<span class="katex-eq" data-katex-display="false">RMSE = 3.18 \times 10^{-2}</span>), automatic differentiation consistently converges faster and more reliably for solving inverse problems, making it the preferred initial approach.
Although inverse PINNs can recover velocity profiles from waveform data with reasonable accuracy (RMSE = 3.18 \times 10^{-2}), automatic differentiation consistently converges faster and more reliably for solving inverse problems, making it the preferred initial approach.

Toward a More Complete Understanding: Deep Operators and Graph Networks

Deep Operator Networks (DONs) represent a paradigm shift in modeling complex systems by moving beyond predicting specific values and instead learning the entire functional mapping between inputs and outputs. Traditional machine learning often focuses on point predictions, struggling with the intricacies of dynamic processes where the evolution of a system is crucial. DONs, however, directly approximate operators – mathematical entities that transform functions into other functions – enabling them to capture the underlying physics governing a system’s behavior. This approach is particularly advantageous when dealing with scenarios where the input isn’t a set of discrete data points, but rather a continuous function representing a field or a time series. By learning this functional relationship, DONs can effectively extrapolate beyond the training data and accurately simulate the system’s response to novel inputs, offering a powerful tool for tasks like predicting the deformation of materials or modeling fluid dynamics with greater fidelity than conventional methods.

Integrating Deep Operator Networks (DONs) with established, simplified physical models-such as the Winkler Foundation, a beam theory commonly used in civil engineering-represents a significant step towards practical application and enhanced learning efficiency. This coupling strategy addresses a core challenge in applying machine learning to complex physical systems: the need for vast datasets. By leveraging the foundational understanding encoded within the simplified model, the DON requires substantially less training data to accurately predict system behavior. This not only accelerates the learning process but also dramatically improves the model’s ability to generalize to scenarios outside the immediate training set, fostering robustness and reliability in predicting the dynamics of complex phenomena. The combination provides a pathway to create data-efficient and physically plausible simulations, promising advancements in fields ranging from structural mechanics to material science.

Recent progress in simulating granular materials leverages particle-based methods, notably the Material Point Method, enhanced by the capabilities of Graph Network Simulators. This combination creates a powerful pathway for investigating complex interaction laws governing these materials, effectively modeling their behavior through discrete particle interactions. Critically, the accuracy of this approach is validated by the performance of automatic differentiation techniques used to recover underlying parameters; studies demonstrate a high degree of precision, with relative errors consistently below 7% – specifically, 0.56% and 6.8% – confirming the method’s ability to not only simulate but also accurately quantify the governing physics of granular systems.

The model demonstrates robust generalization to unseen boundary conditions during granular column collapse, accurately predicting flow dynamics with varying barrier configurations due to its learning of local interaction laws rather than global mappings.
The model demonstrates robust generalization to unseen boundary conditions during granular column collapse, accurately predicting flow dynamics with varying barrier configurations due to its learning of local interaction laws rather than global mappings.

The assessment highlights a critical tension within the application of machine learning to established fields. While Physics-Informed Neural Networks (PINNs) and operator learning offer innovative approaches, the study demonstrates their limitations when faced with the demands of extrapolation – a frequent necessity in geotechnical engineering. This echoes Igor Tamm’s sentiment: “The more you study, the more you discover there is to know.” The pursuit of elegant solutions, like those offered by these networks, must be tempered by a pragmatic understanding of their scope. Every simplification – the reduced computational cost, perhaps – has a cost, and a comprehensive view of the problem, as the research suggests, remains paramount to reliable results. The article implicitly advocates for a holistic approach, mirroring Tamm’s emphasis on continuous learning and acknowledging the inherent complexity of natural phenomena.

Where Do We Go From Here?

The pursuit of elegant solutions in geotechnical engineering appears, for the moment, to have circled back to a familiar truth: simplicity often outlasts complexity. The enthusiasm for physics-informed neural networks and operator learning, while justified by their theoretical potential, has encountered the pragmatic limitations of data dependency and extrapolation. A system’s predictive power is only as good as its understanding of the underlying physics, and a purely data-driven approach, however sophisticated, cannot fully compensate for a lack of fundamental insight.

The challenge, then, isn’t to simply amass more data or build larger networks. Rather, it lies in fostering a more symbiotic relationship between these emerging techniques and established numerical methods. The future likely resides not in replacing finite difference or finite element analysis, but in augmenting them – using machine learning to intelligently guide mesh refinement, accelerate convergence, or identify critical parameters. Scalability will be determined not by computational power, but by clarity of thought.

Ultimately, the ecosystem of geotechnical modeling demands a holistic perspective. Every component-from constitutive models to data acquisition-influences the whole. Progress will hinge on acknowledging the inherent limitations of each tool and integrating them thoughtfully, recognizing that a robust solution is not merely a technical achievement, but a testament to a clear and comprehensive understanding of the earth’s behavior.

Original article: https://arxiv.org/pdf/2512.24365.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-01-02 07:46