Decoding Sri Lankan Markets: A Data-Driven Approach

by

Author: Denis Avetisyan

New research combines natural language processing, macroeconomic data, and advanced forecasting to unlock predictive signals in Sri Lanka’s stock market.

This study presents a quantitative financial modeling framework integrating ESG sentiment analysis, unsupervised clustering, and time-series forecasting for the Sri Lankan stock market.

Predicting market behavior in emerging economies remains challenging due to data scarcity and unique local dynamics. This research, ‘Quantitative Financial Modeling for Sri Lankan Markets: Approach Combining NLP, Clustering and Time-Series Forecasting’, addresses this gap by introducing a novel quantitative framework that integrates Environmental, Social, and Governance (ESG) sentiment-extracted via FinBERT-with macroeconomic indicators and advanced time-series models to forecast Sri Lankan stock market signals. The resulting architecture achieves high accuracy in regime identification and directional forecasting, leveraging correlations with global markets like the S&P 500. Could this holistic approach offer a scalable solution for navigating complex and volatile markets beyond Sri Lanka?

The Inherent Flaws of Conventional Valuation

Conventional financial models, historically reliant on quantitative metrics like revenue and profit, frequently underestimate the significant influence of Environmental, Social, and Governance (ESG) factors on market dynamics. These non-financial considerations, encompassing a company’s sustainability practices, social responsibility, and ethical leadership, are increasingly recognized as drivers of long-term value and risk. Research demonstrates that strong ESG performance can correlate with reduced cost of capital, improved operational efficiency, and enhanced brand reputation, ultimately impacting stock performance and investor confidence. Ignoring these elements creates an incomplete picture of a company’s true value, potentially leading to mispriced assets and inaccurate investment decisions. The omission of ESG data represents a systematic bias in traditional analysis, hindering the ability to accurately predict market trends and assess the overall health of financial systems.

Analyzing Environmental, Social, and Governance (ESG) data for market insights demands more than simple data aggregation; it requires discerning the sentiment embedded within vast quantities of textual information. Unlike numerical data, ESG reports, news articles, and social media posts convey opinions and perceptions that are nuanced and often ambiguous. Consequently, researchers employ sophisticated Natural Language Processing (NLP) techniques – including contextualized word embeddings, sentiment lexicons, and increasingly, large language models – to automatically identify and quantify the emotional tone expressed in these texts. These methods go beyond basic positive/negative classifications, attempting to capture subtleties like irony, sarcasm, and the specific context surrounding ESG-related claims. The accuracy of these NLP models is paramount, as misinterpreting sentiment can lead to flawed analyses and inaccurate predictions regarding corporate performance and market trends. Successfully extracting meaningful sentiment from textual ESG data unlocks a powerful new dimension for investors and analysts seeking a comprehensive understanding of risk and opportunity.

Accurate financial forecasting increasingly demands a nuanced understanding of how macroeconomic indicators interact with environmental, social, and governance (ESG) sentiment. Traditional economic models often treat these factors in isolation, yet shifts in ESG perception – driven by events like climate disasters or social justice movements – can significantly amplify or dampen the effects of conventional economic signals. For example, positive ESG sentiment around a company adopting sustainable practices might bolster investor confidence even amidst broader market downturns, while negative sentiment related to ethical lapses could exacerbate losses. Consequently, integrating macroeconomic data – such as inflation rates, GDP growth, and unemployment figures – with sophisticated analyses of ESG-related textual data is vital. This approach allows for a more comprehensive assessment of risk and opportunity, potentially uncovering predictive patterns that would remain hidden when relying solely on traditional financial metrics.

Achieving truly robust financial predictions demands a departure from siloed analysis, instead embracing the integration of diverse data streams and analytical methodologies. Traditional financial models, often reliant on historical pricing and quantitative indicators, benefit significantly from the inclusion of non-traditional data – encompassing ESG reports, news articles, social media feeds, and even satellite imagery. However, simply collecting this data is insufficient; advanced techniques like Natural Language Processing, machine learning, and time-series analysis are essential to extract meaningful signals and identify complex relationships. Furthermore, a holistic approach necessitates combining these insights with conventional macroeconomic indicators, allowing for a more nuanced understanding of market drivers and ultimately, more accurate forecasting of asset performance and systemic risk. This convergence of data and analytical power promises a future where financial models are not just predictive, but also responsive to the evolving landscape of sustainability and responsible investing.

Revealing Latent Structure Through Unsupervised Methods

Unsupervised learning techniques, specifically HDBSCAN and UMAP, are employed to identify distinct market regimes within Environmental, Social, and Governance (ESG) data. HDBSCAN, a density-based clustering algorithm, excels at discovering clusters of varying shapes and densities without requiring pre-defined cluster counts. UMAP, a dimensionality reduction technique, projects high-dimensional ESG data into a lower-dimensional space while preserving global structure, facilitating more effective clustering. By combining these methods, the analysis reveals previously unobserved groupings of ESG data points, which are interpreted as latent market regimes characterized by shared characteristics in underlying economic conditions and sentiment.

Identified ESG regimes demonstrate consistent correlations with definable economic conditions. Analysis reveals that each regime is characterized by a unique profile of key performance indicators, including but not limited to volatility, credit spreads, and macroeconomic factors. Furthermore, sentiment analysis of news articles and social media data associated with each cluster consistently reflects specific attitudes towards risk, growth, and sustainability. These distinct indicator and sentiment combinations allow for the categorization of market states, enabling a more nuanced understanding of prevailing economic environments and their potential impact on asset performance. The consistency of these profiles across different time periods supports the hypothesis that these regimes are not random fluctuations but represent recurring patterns in market behavior.

Principal Component Analysis (PCA) is utilized post-clustering to confirm the validity and meaningfulness of the regimes identified by HDBSCAN. PCA reduces the dimensionality of the ESG dataset while preserving variance, allowing for visualization and assessment of cluster separation in a lower-dimensional space. Specifically, the first two principal components are examined to determine if the identified clusters exhibit clear differentiation, indicated by minimal overlap and distinct groupings. This process confirms that the observed regimes are not artifacts of the clustering algorithm, but rather reflect inherent structure within the ESG data, thereby enhancing the interpretability of the resulting market regime classifications.

The application of unsupervised learning techniques to Environmental, Social, and Governance (ESG) data facilitates the reduction of dimensionality in complex market analysis. By clustering data points with similar ESG characteristics, a more manageable and interpretable representation of market dynamics is achieved. This simplification allows for the identification of underlying patterns and relationships that may be obscured by the high-dimensionality and noise inherent in raw ESG datasets. The resulting clusters effectively serve as proxies for distinct market states, enabling analysts to focus on a reduced set of representative conditions rather than individual data points, improving the efficiency of market monitoring and regime identification.

Empirical Validation: Mapping Regimes to Economic Reality

A dense neural network was implemented to establish a relationship between identified Environmental, Social, and Governance (ESG) regimes and prevailing economic conditions. Training of this network yielded an accuracy of 84.04% on the training dataset. Performance was further evaluated using a separate validation dataset, achieving an accuracy of 82.0%. This indicates a capacity for generalization beyond the initial training data, suggesting the model can reasonably predict economic conditions given a defined ESG regime, though further testing is necessary to determine real-world efficacy.

Following the initial mapping of ESG regimes to economic conditions via a dense neural network, a gradient boosting classifier was implemented to improve predictive performance. This secondary model operates by sequentially combining weak prediction models, typically decision trees, to correct errors made by prior models in the sequence. The gradient boosting process iteratively learns from residuals, weighting misclassified instances to refine subsequent predictions and minimize overall error. This refinement strategy demonstrably enhances the model’s ability to accurately predict economic conditions based on identified ESG regimes, building upon the foundation established by the neural network.

Multiple time-series forecasting techniques were implemented to predict the performance of the S&P SL20 and ASPI stock market indices. Evaluated models included Simple Recurrent Neural Networks (SRNN), Multi-Layer Perceptrons (MLP), Long Short-Term Memory networks (LSTM), and Gated Recurrent Units (GRU). Performance was quantified using the $R^2$ coefficient of determination; specifically, the GRU model, when trained on intraday ASPI data, achieved an $R^2$ value of 0.801, indicating a strong predictive capability of the model for that dataset.

Correlation analysis was performed utilizing the S&P 500 as a benchmark to assess the relationship with the S&P SL20 index. Results indicate a statistically significant Pearson correlation coefficient of 0.72, with a p-value less than 0.001. This demonstrates a strong positive linear association between the two indices, suggesting that movements in the S&P 500 are reliably associated with corresponding movements in the S&P SL20. The calculated p-value confirms the statistical significance of this correlation, minimizing the likelihood that the observed relationship is due to random chance.

Synthesizing Insight: A Holistic Market Signal Generator

The core of this market signal generator lies in a carefully constructed system of rule-based fusion logic, designed to synthesize distinct analytical approaches. Specifically, insights derived from ESG sentiment analysis – gauging public perception of a company’s environmental, social, and governance practices – are systematically combined with predictions generated through time-series forecasting, which analyzes historical market data to anticipate future trends. This isn’t simply an averaging of results; the system employs predefined rules to prioritize signals based on contextual relevance and predictive power. For instance, a strongly negative ESG sentiment coinciding with a downturn in forecasted earnings might trigger a more pronounced sell signal than either indicator alone. By intelligently integrating these diverse data streams, the generator aims to move beyond the limitations of single-factor analysis, creating a more robust and nuanced assessment of investment potential.

The convergence of disparate data streams into a unified market signal represents a significant advancement in investment analysis. By synthesizing insights from both qualitative sources, like environmental, social, and governance (ESG) sentiment, and quantitative time-series forecasting, a more holistic picture of potential investment opportunities emerges. This integrated approach transcends the limitations of relying on single data types, mitigating risks associated with incomplete information and offering investors a more nuanced understanding of market dynamics. The resulting signal isn’t merely a compilation of data; it’s a refined indicator, capable of identifying subtle shifts and previously unseen connections that might otherwise influence asset valuation and overall portfolio performance. Ultimately, this holistic view empowers more informed decision-making, allowing for strategic allocation of capital based on a broader and more reliable foundation of market intelligence.

The analytical engine at the heart of this market signal generator relies heavily on FinBERT, a specialized transformer model designed for financial text analysis. This model doesn’t simply assess whether an ESG-related text is positive or negative; instead, it dissects the nuances of language to identify specific sentiments related to environmental, social, and governance factors. By processing large volumes of textual data – including company reports, news articles, and social media feeds – FinBERT pinpoints subtle indicators of risk and opportunity that might otherwise go unnoticed. The resulting sentiment scores, derived from FinBERT’s understanding of context and financial terminology, then become a crucial input for the overall signal generation process, providing a data-driven assessment of a company’s ESG performance and potential market impact. This allows for a more precise and informative evaluation than traditional methods, ultimately enhancing the reliability of the investment signals.

The culmination of this integrated system lies in its capacity to empower investors with refined decision-making tools. By synthesizing insights from both natural language processing of Environmental, Social, and Governance data and quantitative time-series predictions, a more nuanced market signal emerges. This signal isn’t merely a prediction, but a confluence of factors offering a holistic view of asset potential. Investors can leverage these signals to recalibrate portfolio allocations, identify emerging opportunities, and potentially mitigate risks more effectively. Ultimately, the system aims to move beyond traditional analysis, facilitating strategies designed to enhance portfolio performance and align investments with a broader understanding of market dynamics and sustainability factors.

The research meticulously constructs a predictive framework, demanding a rigorous foundation akin to mathematical proof. It’s not sufficient for models to simply perform on historical data; their internal logic must be demonstrably sound, especially when incorporating the nuanced data of ESG sentiment. As Blaise Pascal observed, “The eloquence of angels is no more than the sweetness of geometry.” This sentiment perfectly mirrors the approach taken here – a commitment to the precision of quantitative methods, ensuring the identified signals are not merely correlations, but reflections of underlying economic realities. The integration of time-series forecasting, clustering, and sentiment analysis seeks to establish a provable relationship between indicators and market behavior, rather than relying on conjecture.

What’s Next?

The presented methodology, while demonstrating predictive capability within the Sri Lankan market, merely scratches the surface of a fundamental challenge: extracting signal from inherently noisy data. The integration of NLP-derived ESG sentiment, macroeconomic factors, and time-series forecasting represents a step toward rigorous modeling, yet relies on the problematic assumption that these proxies perfectly encapsulate the underlying economic realities. Future work must confront the limitations of such approximations – the unavoidable gap between mathematical representation and lived experience.

A critical area for advancement lies in formalizing uncertainty. Current implementations treat point estimates as truth, ignoring the probabilistic nature of both macroeconomic forecasts and sentiment analysis. Bayesian frameworks, coupled with robust sensitivity analysis, offer a path toward more defensible and reliable predictions. Furthermore, the unsupervised clustering approach, while effective for dimensionality reduction, requires validation beyond observed market behavior. Theoretical justification for the chosen clustering algorithms, grounded in information theory or dynamical systems, is essential.

Ultimately, the pursuit of predictive power in financial markets is a Sisyphean task. However, in the chaos of data, only mathematical discipline endures. The true value of this research – and its successors – will not be measured solely by forecasting accuracy, but by the degree to which it refines the underlying mathematical framework, pushing the boundaries of what can be rigorously known, even within an inherently unpredictable system.

Original article: https://arxiv.org/pdf/2512.20216.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2025-12-24 09:56