Seeing What’s Changed: Predicting Anomalies in Satellite Views

by

Author: Denis Avetisyan

A new deep learning approach frames change detection in satellite imagery as a problem of predicting future surface reflectance, identifying anomalies through discrepancies between prediction and observation.

The model leverages a multi-temporal, multi-spectral approach-processing sequences of Sentinel-2 acquisitions alongside partially masked target frames-to predict complete scenes, with anomalies identified through discrepancies between prediction and observation, achieved via a Swin Transformer backbone, FPN decoder, and a training strategy employing random masking and a combined loss function of L1 reconstruction, structural similarity, and high-frequency detail preservation.
The model leverages a multi-temporal, multi-spectral approach-processing sequences of Sentinel-2 acquisitions alongside partially masked target frames-to predict complete scenes, with anomalies identified through discrepancies between prediction and observation, achieved via a Swin Transformer backbone, FPN decoder, and a training strategy employing random masking and a combined loss function of L1 reconstruction, structural similarity, and high-frequency detail preservation.

This review details a temporal inpainting method for anomaly detection in satellite imagery, leveraging foundation models and deep learning for improved change detection capabilities.

Detecting subtle surface changes from satellite imagery remains a persistent challenge due to noise and natural variations. This is addressed in ‘Anomaly detection in satellite imagery through temporal inpainting’, which introduces a deep learning approach framing change detection as a temporal prediction problem. By training an inpainting model to reconstruct past satellite observations, the method identifies anomalies as discrepancies between predicted and actual reflectance. Could this approach unlock automated, global-scale monitoring of dynamic surface processes using freely available data?

Subtle Shifts & The Limits of Baseline Detection

Conventional anomaly detection techniques, designed to flag drastic deviations from established norms, frequently fail when confronted with the gradual shifts observable in satellite imagery. These methods often rely on fixed thresholds or simple before-and-after comparisons, proving insufficient to capture the nuanced alterations indicative of developing crises or long-term environmental trends. Subtle changes – a creeping landslide, the slow spread of deforestation, or the initial stages of urban encroachment – can easily fall beneath the radar, leading to delayed responses and potentially exacerbating negative consequences. The inherent challenge lies in distinguishing between normal fluctuations and the beginnings of significant, yet initially imperceptible, events, demanding more sophisticated analytical approaches capable of tracking evolving patterns rather than static anomalies.

The ability to detect alterations on the Earth’s surface holds immense practical value across numerous disciplines. In the immediate aftermath of disasters – such as earthquakes, floods, or wildfires – rapid identification of impacted areas is essential for effective resource allocation and rescue operations. Beyond emergency response, consistent monitoring of surface changes plays a vital role in infrastructure management, enabling early detection of subsidence, landslides, or structural weaknesses in critical assets like bridges, pipelines, and buildings. Furthermore, understanding these dynamic processes is fundamental to broader environmental studies; tracking deforestation, glacial retreat, or the expansion of urban areas provides crucial data for modeling climate change, managing natural resources, and predicting future environmental risks. Consequently, advancements in surface change detection directly contribute to both immediate safety concerns and long-term planetary health.

Traditional methods of surface change detection frequently falter when confronted with the nuanced complexities of real-world environments. The assumption that significant change manifests as a simple exceedance of a pre-defined threshold proves unreliable; natural variability, atmospheric conditions, and gradual shifts often blur the lines of what constitutes a genuine anomaly. Static comparisons between images, while computationally efficient, fail to account for evolving landscapes, seasonal cycles, or the interplay of multiple factors influencing surface reflectance. Consequently, critical events – such as slow-developing landslides, subtle infrastructure degradation, or the early stages of ecological stress – can be easily overlooked, highlighting the need for more adaptive and sophisticated analytical approaches that embrace the inherent dynamism of Earth’s surface.

Temporal inpainting consistently outperforms RX and temporal median baselines in detecting synthetic anomalies, achieving ROC-AUC of 0.949±0.104 and PR-AUC of 0.854±0.233, and maintains high sensitivity even with subtle anomalies, as demonstrated by its superior performance across varying anomaly intensities and visual reconstruction accuracy.
Temporal inpainting consistently outperforms RX and temporal median baselines in detecting synthetic anomalies, achieving ROC-AUC of 0.949±0.104 and PR-AUC of 0.854±0.233, and maintains high sensitivity even with subtle anomalies, as demonstrated by its superior performance across varying anomaly intensities and visual reconstruction accuracy.

Predicting the Future: A Baseline for Anomaly Detection

The system operates on the principle of forecasting future video frames using a learned representation of preceding frames, thereby creating a predictive baseline. This baseline is not intended to perfectly replicate reality, but rather to establish an expectation of normal temporal progression. Deviations between the predicted future frame and the actual observed frame are then quantified and used as an anomaly score. The magnitude of this difference indicates the likelihood of an anomalous event; larger discrepancies suggest a higher probability of an anomaly occurring within the video sequence. This approach allows for the detection of unusual or unexpected events by comparing predicted and observed data, rather than relying on pre-defined anomaly signatures.

The SATLAS Foundation Model employed for temporal prediction leverages a Swin Transformer architecture, chosen for its demonstrated efficacy in capturing long-range dependencies within sequential data. This transformer-based approach enables the model to encode historical image frames into a high-dimensional latent space, effectively representing temporal patterns. Subsequent extrapolation within this latent space allows for the generation of predictions regarding future frames. The Swin Transformer’s hierarchical structure and shifted windowing scheme contribute to efficient processing of image data and improved performance in modeling complex temporal dynamics compared to standard transformer implementations.

Image inpainting techniques are employed to generate predicted future frames by filling in missing or extrapolated visual information. This process leverages contextual understanding of previously observed frames to synthesize plausible content for areas where future data is unavailable. The reconstructed frames are then directly compared to actual observed frames; discrepancies between the predicted and observed content indicate potential anomalies. The robustness of this comparison is enhanced by the inpainting algorithms’ ability to handle occlusions and partial visibility, providing a stable basis for anomaly scoring even in complex scenes. Multiple inpainting strategies are evaluated to optimize reconstruction accuracy and minimize false positive anomaly detections.

Analysis of Sentinel-2 imagery using a novel anomaly detection method successfully identified the <span class="katex-eq" data-katex-display="false">2023</span> Tepehan surface rupture along the East Anatolian Fault, exhibiting a clear pre-earthquake baseline and minimal background noise compared to alternative approaches.
Analysis of Sentinel-2 imagery using a novel anomaly detection method successfully identified the 2023 Tepehan surface rupture along the East Anatolian Fault, exhibiting a clear pre-earthquake baseline and minimal background noise compared to alternative approaches.

Layering the Details: Architecture and Loss Functions

The SATLAS model utilizes a Feature Pyramid Network (FPN) to address the challenge of capturing temporal dynamics at varying scales. The FPN constructs a feature pyramid from the input data by combining low-resolution, semantically strong features with high-resolution, semantically weak features through a top-down pathway and lateral connections. This progressive upsampling process allows the model to detect anomalies that manifest at different temporal frequencies and spatial scales; for example, subtle, short-duration events are captured by the high-resolution layers, while longer-duration, broader-scale patterns are detected by the lower-resolution layers. The resulting multi-scale feature representation improves the model’s ability to analyze and predict time-series data by providing a comprehensive understanding of temporal changes across different scales.

Laplacian filtering is integrated into the loss calculation to enhance the preservation of high-frequency details during reconstruction. This technique operates by computing the discrete Laplacian of the image, which approximates the second spatial derivative and effectively highlights edges and fine structures. By including the Laplacian in the loss function, the model is penalized for smoothing or removing these critical details, thereby improving its ability to accurately reconstruct the input and detect subtle changes indicative of anomalies. The Laplacian is calculated using a ( \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} ) operator, applied via a discrete kernel, ensuring sensitivity to rapid intensity variations within the input data.

The SATLAS model utilizes Masked L1 Reconstruction Error as a primary loss function to optimize anomaly detection. This loss focuses reconstruction efforts on pertinent regions of the input data, effectively disregarding irrelevant background information and concentrating learning on potential anomalies. The masking process involves identifying and excluding areas deemed non-critical, thereby reducing the impact of noise and improving the model’s sensitivity to subtle, anomalous changes. Minimizing the L1 norm – the sum of absolute differences – between the reconstructed and original masked data promotes accurate reconstruction of relevant features and strengthens the model’s ability to differentiate between normal and anomalous patterns. This targeted approach results in a more robust and efficient anomaly detection system.

Multi-Scale Structural Similarity (MS-SSIM) is implemented as a supplementary loss function to the L1 reconstruction error, addressing limitations in pixel-wise comparisons. MS-SSIM evaluates perceptual similarity by considering luminance, contrast, and structure across multiple scales; it computes similarity maps based on local statistics and aggregates these to produce a single similarity score. This metric is more robust to minor variations and noise than the L1 loss, and its inclusion guides the model to generate predictions that are more visually consistent with the input data, particularly for subtle anomalies that may not significantly impact the L1 error but are perceptually important. The MS-SSIM component therefore improves the overall quality and reliability of the anomaly detection process by enhancing the model’s sensitivity to structural distortions.

Beyond Baseline: Performance and Validation

Evaluations reveal a substantial performance advantage for this novel anomaly detection method when contrasted with established techniques like the RX Anomaly Detector and the Temporal Median Predictor. Rigorous testing demonstrates consistently higher accuracy in identifying deviations from normal conditions within Sentinel-2 data. This improved performance isn’t merely incremental; the system consistently surpasses baseline detectors in metrics critical for real-world applications, indicating a more robust and reliable approach to environmental monitoring and disaster assessment. The ability to reliably differentiate between expected variations and genuine anomalies is paramount, and this method demonstrably exceeds the capabilities of currently available alternatives.

To rigorously assess the anomaly detection system’s capabilities, a suite of synthetic anomalies was intentionally introduced into Sentinel-2 datasets. This controlled experimentation allowed researchers to move beyond reliance on naturally occurring, unpredictable events and instead evaluate the model’s sensitivity and robustness under precisely defined conditions. By varying the characteristics – magnitude, spatial extent, and temporal evolution – of these artificial anomalies, the system’s ability to consistently identify subtle deviations from normal patterns was quantified. The resulting data provided a benchmark for performance, revealing the model’s capacity to generalize and avoid false positives, ultimately ensuring its reliability in real-world disaster response scenarios where accurate and timely detection is paramount.

The system excels at identifying nuanced alterations within Sentinel-2 data, a capability of paramount importance for accelerating disaster response. Traditional methods often struggle with the subtle indicators preceding large-scale events-a slight vegetation stress before drought, early signs of inundation before flooding, or minute changes in land surface temperature preceding wildfires. This approach, however, is specifically designed to capture these delicate shifts, allowing for earlier detection and more effective resource allocation. By focusing on the granular details within the satellite imagery, the system provides a critical advantage in situations where time is of the essence, potentially mitigating damage and saving lives through proactive intervention and informed decision-making.

The anomaly detection system achieves heightened reliability and accuracy through a focus on prediction error, moving beyond traditional methods that rely on absolute values. This approach allows the system to identify anomalies as significant deviations from predicted values, rather than simply flagging unusual data points. Rigorous testing demonstrates strong performance, with a mean Receiver Operating Characteristic Area Under the Curve (ROC-AUC) of 0.949, indicating excellent discrimination between anomalous and normal instances. Further validation is provided by a Precision-Recall Area Under the Curve (PR-AUC) of 0.854 and an F1 Score of 0.849, confirming a robust balance between precision and recall – crucial for minimizing both false positives and false negatives in real-world applications.

The pursuit of elegant change detection, as presented in this work, feels… familiar. It’s a temporal prediction problem disguised as deep learning, which inevitably means someone will call it AI and raise funding. This paper frames anomalies as discrepancies between predicted and observed reflectance – a sophisticated way of saying ‘what we expected didn’t happen.’ Yann LeCun once stated, “Everything is machine learning at some point.” And he’s right, of course. They’ll start with a simple bash script to flag changes, then layer on complexity until the documentation lies again and they’re debugging someone else’s inscrutable network. It used to be a simple bash script, honestly. Now it’s foundation models and temporal inpainting, but the core problem-spotting what’s different-remains stubbornly, frustratingly, the same. Tech debt is just emotional debt with commits, after all.

What’s Next?

The framing of change detection as a temporal prediction task, while elegant, merely shifts the problem. Discrepancies will inevitably arise not from a failure to predict change, but from the inherent ambiguity of ‘normal’ reflectance. Every surface, after all, is a compromise between ideal models and atmospheric noise. The current reliance on image inpainting, a technique born of artistic reconstruction, exposes a fundamental tension: are anomalies truly ‘missing information’, or simply states the model hasn’t yet encountered?

Future iterations will likely wrestle with the cost of generalization. Foundation models, powerful as they are, demand data – and satellite archives, while vast, are still finite. The inevitable overfitting to common events will necessitate a renewed focus on anomaly characterization – not just detection. A pixel flagged as unusual is only marginally useful; understanding how it deviates, and whether that deviation is physically plausible, is the true challenge.

The pursuit of ‘perfect’ prediction is a familiar loop. Everything optimized will one day be optimized back, as production systems reveal edge cases unforeseen in controlled environments. The field doesn’t need more algorithms, but more robust logging-a detailed record not of what was predicted, but of why the prediction failed. The architecture isn’t a diagram; it’s a compromise that survived deployment-for now.

Original article: https://arxiv.org/pdf/2512.23986.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-01-01 23:27