Can AI Beat the Market?

by

Author: Denis Avetisyan

A new benchmark assesses whether intelligent agents can navigate the complexities of live financial trading.

This paper introduces AI-Trader, a platform for rigorously evaluating language model agents in real-time financial markets, highlighting the gap between general intelligence and profitable trading, and the vital role of risk management.

Despite advances in large language models, translating general intelligence into effective real-time decision-making remains a significant challenge, particularly in dynamic financial markets. This limitation motivates our work, ‘AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets’, which introduces a fully-automated, live benchmark for evaluating LLM agents across U.S. stocks, A-shares, and cryptocurrencies. Our analysis reveals that strong performance in language tasks does not guarantee successful trading, with robust risk management proving critical for consistent returns and cross-market adaptability. Can future agent designs overcome these limitations and unlock the full potential of AI in financial markets?

The Perilous Landscape of Algorithmic Finance

Financial markets pose a considerable challenge to artificial intelligence systems due to their inherent dynamism and complexity. Unlike static datasets, market conditions shift constantly, requiring algorithms to not only process vast amounts of data but also react instantaneously to new information. Successful AI trading necessitates robust decision-making capabilities that extend beyond pattern recognition; systems must adapt to unpredictable events, assess risk in real-time, and execute trades with precision under intense pressure. The speed at which markets operate leaves little room for error, demanding AI models capable of learning and evolving continuously to maintain a competitive edge and navigate the ever-changing landscape of global finance.

Conventional backtesting, a mainstay of financial strategy evaluation, frequently proves inadequate when applied to the dynamic realities of live trading, especially within the cryptocurrency market. These historical simulations rely on past data to predict future performance, yet fail to fully account for the unpredictable nature of real-time market fluctuations and the impact of order book dynamics. Volatile assets, characterized by rapid price swings and high trading volumes, exacerbate this issue; backtests may demonstrate promising results under stable conditions, but quickly unravel when confronted with unforeseen events or sudden shifts in investor sentiment. The inherent limitations of static historical data, coupled with the complexities of market microstructure, mean that backtesting often provides an overly optimistic and ultimately unreliable assessment of an algorithm’s true trading potential, necessitating more sophisticated evaluation techniques that mirror the uncertainties of live environments.

The proliferation of Large Language Model Agents (LLM Agents) within financial trading necessitates the development of standardized benchmarks for reliable performance evaluation. Currently, assessing these agents is hampered by a lack of common metrics and datasets, leading to subjective comparisons and inflated claims of profitability. A robust benchmark would move beyond simple backtesting – which often fails to account for real-world market dynamics like slippage and order book impact – and instead focus on metrics that assess an agent’s ability to adapt to changing conditions, manage risk, and generate consistent returns across diverse asset classes. Such a standardized evaluation framework is not merely about ranking agents; it is about fostering transparency, accelerating innovation, and ultimately, building trust in the integration of AI within the complex world of financial markets.

AI-Trader: A Controlled Environment for Rigorous Agent Assessment

AI-Trader establishes a standardized evaluation environment for Large Language Model (LLM) Agents operating within three distinct financial markets: the U.S. Stock Market, the A-Share Market (China), and the Cryptocurrency Market. This unified framework allows for comparative performance analysis across varied asset classes and market dynamics. The platform’s design facilitates consistent testing methodologies, enabling researchers and developers to assess agent capabilities – such as trade execution, risk management, and portfolio optimization – irrespective of the specific market being targeted. By providing a common benchmark, AI-Trader aims to accelerate progress in the development of robust and generalizable financial agents.

AI-Trader incorporates a live data feed, sourcing market information from APIs providing real-time stock prices, order book data, and news sentiment for the U.S., Chinese (A-Share), and cryptocurrency markets. Agents operating within the platform execute trades autonomously based on this incoming data, managing virtual portfolios and incurring transaction costs mirroring actual brokerage fees. This design replicates the time sensitivity and dynamic conditions of live trading, forcing agents to react to continuously changing market states and manage risk under pressure. The system tracks key performance indicators such as portfolio value, Sharpe ratio, and maximum drawdown, providing a quantifiable measure of an agent’s ability to perform in a realistic financial environment.

Effective operation within AI-Trader necessitates that agents skillfully employ external tools for data retrieval and analysis, given the inherent limitations of LLM-based knowledge. Specifically, agents must utilize tools to access real-time market data, perform technical analysis, and execute trades. Crucially, agents are required to verify the accuracy and relevance of information obtained from these tools, as market data is subject to errors, delays, and manipulation. Successful agents differentiate between reliable and unreliable sources, cross-reference information, and avoid acting on unverified data, thereby mitigating risks associated with inaccurate or outdated market intelligence. This proficiency in tool utilization and information validation is a primary determinant of agent performance and profitability within the benchmark.

Disparities in Performance: When Agents Face the Crucible

AI-Trader evaluations reveal significant performance variation among Large Language Model (LLM) Agents. Testing demonstrates that agents like DeepSeek-v3.1 consistently achieve high returns and successful trade execution. Conversely, agents including Qwen3-Max and GPT-5 exhibited limited success, indicating lower profitability and/or a reduced ability to navigate market complexities. This disparity suggests that LLM architecture, training data, and algorithmic strategies play a critical role in determining an agent’s effectiveness within the AI-Trader platform. The observed range highlights the need for rigorous evaluation and benchmarking of LLM Agents prior to deployment in live trading environments.

Evaluations within AI-Trader indicate that LLM Agents Gemini-2.5-Flash and Claude-3.7-Sonnet exhibit decreased performance during periods of high market volatility. Specifically, these agents demonstrated an inability to maintain consistent profitability when subjected to rapid price fluctuations and increased trading volume. This susceptibility underscores the critical need for robust risk management strategies, including implementation of stop-loss orders, position sizing adjustments, and diversification techniques, to mitigate potential losses and preserve capital in dynamic market environments. Further testing revealed a correlation between increased volatility and a higher frequency of suboptimal trade executions by these agents.

MiniMax-M2 demonstrated consistent performance across varied market environments within the AI-Trader evaluations. Specifically, the agent achieved a cumulative return of 9.56% in the U.S. market, indicating successful navigation of market fluctuations. This performance is attributed to the agent’s integrated risk control mechanisms and its ability to adapt trading strategies across different market conditions, distinguishing it from other agents which exhibited diminished returns during periods of high volatility.

Beyond Returns: Quantifying Risk and True Performance

AI-Trader moves beyond simple return calculations by employing key performance indicators to comprehensively evaluate LLM Agent performance, crucially factoring in the associated risk. Traditional metrics often present an incomplete picture; a high return can be misleading if achieved through excessive risk-taking. Consequently, AI-Trader utilizes metrics like Maximum Drawdown – the peak-to-trough decline during a specific period – and the Sortino Ratio, which focuses specifically on downside risk, to gauge an agent’s ability to protect capital. This dual assessment provides a more nuanced understanding of risk-adjusted returns, allowing for a comparative analysis of different agents and strategies, and ultimately identifying those that deliver sustainable, responsible growth. The system determines not just if an agent generates profit, but how it does so, prioritizing stability and capital preservation alongside potential gains.

Traditional risk assessment often relies on metrics like standard deviation, which penalizes both positive and negative volatility equally. However, investors are primarily concerned with protecting capital during periods of market decline. The Sortino Ratio addresses this by specifically focusing on downside risk – measuring only the volatility of negative returns. This provides a more refined understanding of an agent’s ability to preserve capital, as it isolates the risk that truly impacts investor outcomes. A higher Sortino Ratio indicates a greater capacity to generate returns relative to the risk of loss, offering a more nuanced perspective than metrics that treat all volatility as equivalent. Consequently, evaluating LLM Agents through the lens of downside risk, as quantified by the Sortino Ratio, is crucial for discerning their true capital preservation capabilities and suitability for risk-averse investment strategies.

The performance of the MiniMax-M2 agent in the U.S. market reveals a compelling balance between potential reward and inherent risk. Specifically, the agent achieved a Sortino ratio of 4.42, a metric focused on risk-adjusted returns that prioritizes downside protection; a higher ratio indicates better performance relative to the risk taken. Complementing this, the maximum drawdown – the peak-to-trough decline during a specific period – registered at -4.92%. This comparatively low drawdown suggests MiniMax-M2 is capable of limiting potential losses, even during unfavorable market conditions, and effectively preserves capital while pursuing gains, thereby demonstrating strong risk-adjusted returns for investors.

The AI-Trader, MiniMax-M2, demonstrated a notable ability to generate returns exceeding those of established market benchmarks within the U.S. market. Specifically, the agent achieved an excess return of 7.69% when measured against the performance of the QQQ, a popular exchange-traded fund tracking the Nasdaq-100 index. This outcome suggests that MiniMax-M2 not only participates in market gains but actively seeks opportunities to outperform traditional investment strategies, indicating a potential for superior value creation for investors. This capacity to deliver alpha, or excess return, is a key differentiator for AI-driven trading systems and highlights the potential for leveraging LLM Agents in financial markets.

Analysis of the A-share market revealed that MiniMax-M2 exhibited a volatility of just 6.72%, a key indicator of its resilience during periods of market stress. This comparatively low volatility suggests the agent can maintain a more stable portfolio value, even amidst challenging economic conditions, and underscores its capacity for consistent performance. The finding is particularly noteworthy given the historically higher volatility often associated with the A-share market, indicating MiniMax-M2’s ability to navigate complex and potentially turbulent trading environments while preserving capital and minimizing drastic fluctuations in returns.

The pursuit of autonomous trading, as detailed in the AI-Trader benchmark, highlights a familiar challenge: the gap between theoretical intelligence and practical application. The study demonstrates that simply possessing general language capabilities does not guarantee success in the complex world of financial markets. This echoes a sentiment articulated by Confucius: “Study the past if you would define the future.” Understanding the failures-the errors inherent in any model-becomes paramount. The AI-Trader framework, with its emphasis on real-time evaluation and risk management, isn’t about finding perfect predictions, but about systematically identifying and mitigating the inevitable inaccuracies. Wisdom, in this context, resides not in eliminating error, but in acknowledging and accounting for it.

The Road Ahead

The exercise of subjecting large language models to actual market forces, as demonstrated by AI-Trader, doesn’t so much reveal a failure of intelligence as a stark reminder of its domain-specificity. A model proficient in parsing human language isn’t automatically equipped to parse the chaotic language of price fluctuations. One suspects the true insight isn’t that these agents can’t trade, but that trading, at its core, isn’t about intelligence – it’s about surviving long enough for randomness to be on one’s side. And the benchmark, predictably, highlights the necessity of risk management – a constraint often conveniently omitted from theoretical formulations.

Future iterations of this work shouldn’t focus solely on maximizing returns. A more revealing metric might be longevity – how long an agent can operate before succumbing to inevitable losses. This shifts the emphasis from brilliance to resilience, a quality arguably more valuable in a genuinely competitive environment. Further investigation into the phenomenon of ‘data contamination’ is also critical; the past, after all, is not a fixed entity but a constantly revised narrative, particularly when that narrative is actively being traded.

The ultimate question isn’t whether an AI can outperform a human trader, but whether the illusion of outperformance can be sustained. Because in finance, as in life, perception often trumps reality – and a convincing story can be more profitable than a sound strategy.

Original article: https://arxiv.org/pdf/2512.10971.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2025-12-15 07:06