When AI Thinks Like Us (And When It Doesn’t)

Author: Denis Avetisyan

New research reveals that large language models exhibit surprisingly human-like biases in how they make choices, but surprisingly rational behavior when assessing beliefs.

This paper systematically examines cognitive biases in large language models, finding a divergence between preference and belief tasks and demonstrating a limited impact of role-priming on bias reduction.

Despite increasing reliance on artificial intelligence for decision-making, the potential for systematic behavioral biases in large language models (LLMs) remains largely unexplored. This paper, ‘Behavioral Economics of AI: LLM Biases and Corrections’, presents a comprehensive experimental investigation-adapting established protocols from cognitive psychology and experimental economics-to assess biases in prominent LLM families. We find that LLMs exhibit human-like irrationality in preference-based tasks, yet often demonstrate rational responses in belief-based scenarios, and that simple prompting can partially mitigate these biases. Can understanding-and correcting-these biases pave the way for more aligned and trustworthy AI systems?

The Erosion of Rationality: Examining the Limits of Economic Models

The foundations of much economic thought rest upon ‘Rational Choice Theory’, a system positing individuals make decisions by meticulously weighing costs and benefits to maximize their utility. However, this framework frequently falters when applied to actual human behavior. Empirical studies consistently demonstrate deviations from the predictions of perfect rationality; people aren’t consistently logical actors. Instead, choices are often influenced by cognitive biases, emotional responses, and social pressures – factors largely absent from traditional models. This disconnect isn’t merely a matter of minor inaccuracies; it represents a fundamental limitation in the ability of standard economic theory to accurately describe, and therefore predict, how individuals behave in complex real-world scenarios, necessitating alternative approaches to understanding economic phenomena.

Conventional economic models frequently operate under the assumption of both perfect information and unwavering consistency in individual preferences, a simplification that often clashes with observed human behavior. This framework posits that individuals possess complete knowledge of all available options and their consequences, and that their choices are guided by stable, well-defined priorities. However, cognitive science reveals a far more nuanced reality; human cognition is susceptible to biases, heuristics, and emotional influences. Individuals routinely make decisions with incomplete information, struggle with complex calculations, and exhibit preferences that shift based on framing, context, and even momentary mood. Consequently, the assumption of perfect rationality overlooks crucial aspects of how people actually navigate economic landscapes, creating a disconnect between theoretical predictions and real-world outcomes.

Behavioral economics arose as a crucial refinement to traditional economic thought by directly addressing the psychological underpinnings of human decision-making. Unlike earlier models predicated on the assumption of perfectly rational actors, this field integrates insights from psychology – including cognitive biases, heuristics, and emotional influences – to create more realistic and predictive economic models. It demonstrates that individuals frequently deviate from strict rationality, often making choices influenced by factors like framing effects, loss aversion, and social norms. Consequently, behavioral economics offers explanations for anomalies previously unexplained by conventional theory, such as market bubbles, procrastination, and systematic errors in judgment, ultimately providing a more nuanced understanding of economic behavior and informing more effective policy interventions.

Framing the Choice: Prospect Theory and the Psychology of Value

Prospect Theory, developed by Daniel Kahneman and Amos Tversky, challenges traditional economic models that assume rational actors maximize expected utility. Unlike Expected Utility Theory, which posits individuals assess choices based on absolute outcomes, Prospect Theory proposes that people evaluate potential gains and losses relative to a reference point, typically their current state. This framing significantly impacts decision-making, as individuals demonstrate a consistent pattern of risk aversion in the domain of gains and risk-seeking behavior in the domain of losses. The theory incorporates value functions that are steeper for losses than for gains, and probability weighting functions that distort the objective probabilities of events, leading to predictable deviations from rational choice. Empirical evidence consistently demonstrates Prospect Theory’s superior predictive power in explaining real-world economic behavior compared to standard utility maximization models.

Loss aversion, a core principle of prospect theory, describes the empirically observed asymmetry in how individuals value gains and losses. Research indicates that the negative emotional impact of experiencing a loss of a given amount is typically greater than the positive emotional impact of gaining the same amount. This is not simply a matter of risk aversion; even when presented with objectively equivalent gambles – one with a potential gain and another with an equivalent potential loss – individuals consistently demonstrate a preference for avoiding the loss. The magnitude of this effect varies between individuals, but the general principle holds that losses loom larger than gains, influencing decision-making under conditions of uncertainty and contributing to behaviors not predicted by expected utility theory.

Diminishing sensitivity, as described by prospect theory, posits that the subjective value associated with changes in wealth is not linear. Specifically, the psychological impact of a gain or loss of a particular magnitude decreases as the overall scale of that gain or loss increases. For example, the difference in perceived value between gaining $10 and $20 is generally more significant than the difference between gaining $1000 and $1010, even though the absolute amount gained is the same. This principle applies to losses as well; the emotional impact of losing $10 is greater than the impact of losing $100, given a consistent starting wealth. This effect is mathematically represented by a value function that is concave for gains and convex for losses, implying that individuals are more sensitive to smaller changes in wealth than to larger ones.

The cognitive biases described by prospect theory – loss aversion and diminishing sensitivity – systematically influence economic decision-making, leading to outcomes that deviate from rational actor models. Specifically, loss aversion causes individuals to prioritize avoiding losses over acquiring equivalent gains, resulting in risk-seeking behavior when facing potential losses and risk-averse behavior when facing potential gains. Diminishing sensitivity means that the psychological impact of a $100 gain is not the same as the impact of a $1000 gain, and similarly for losses; this scaling effect impacts willingness to accept risk. Recognizing these biases allows for the development of predictive models of behavior in financial markets, and informs interventions – such as framing effects in policy design or default options in savings plans – aimed at encouraging more beneficial economic choices.

The Simulated Mind: LLMs as Behavioral Laboratories

Large Language Models (LLMs) present a novel methodology for investigating behavioral economics and identifying cognitive biases due to their capacity to process and respond to complex scenarios in a manner that simulates human decision-making. Unlike traditional experimental economics which relies on limited participant pools and potentially costly data collection, LLMs offer a scalable and computationally efficient platform for generating responses to a wide range of economic stimuli. This allows researchers to systematically vary parameters and observe resulting behavioral patterns, offering insights into the underlying mechanisms driving irrationality and bias. The ability to analyze responses at scale, coupled with the controlled environment of the LLM, provides a powerful tool for both confirming established behavioral findings and discovering previously unobserved cognitive phenomena.

Role priming in LLM-based experimental economics involves initializing the model with a specific persona representing different investor types. Researchers can define parameters establishing the LLM’s decision-making framework, ranging from a strictly rational agent maximizing expected utility to a profile incorporating characteristics of typical retail investors, including documented behavioral biases and heuristics. This is achieved through carefully crafted prompts that contextualize the LLM’s responses, effectively simulating how an investor with a defined role would approach economic scenarios. By varying these roles, researchers can systematically investigate how different cognitive frameworks influence decision-making in controlled, replicable experiments.

Analysis of responses generated by Large Language Models (LLMs) enables the observation of behavioral biases within simulated economic scenarios. Specifically, research indicates that employing ‘role priming’ techniques – assigning the LLM a defined investor profile – can demonstrably reduce irrational responses. Quantitative results show a 4.3% reduction in irrational responses to preference-based questions and a 3.3% reduction in responses to belief-based questions when role priming is implemented. These findings suggest LLMs can be utilized to not only model, but also potentially mitigate, the effects of cognitive biases in economic decision-making.

Traditional experimental economics relies on recruiting human participants, a process that is often time-consuming, expensive, and subject to limitations in scale and control. The integration of Large Language Models (LLMs) establishes a complementary computational laboratory that addresses these constraints. LLMs enable researchers to simulate a significantly larger number of experimental subjects than is typically feasible with human-subject research. Furthermore, LLMs offer precise control over experimental parameters and participant characteristics – including the ability to systematically vary biases or roles – while minimizing the costs associated with participant recruitment and data collection. This scalability and controllability facilitate more robust and efficient testing of economic theories and behavioral hypotheses, effectively augmenting the toolkit of experimental economists.

The Confidence of Prediction: Quantifying LLM Behavior and Bias

The confidence level, as reported by a Large Language Model (LLM), represents the probability assigned to its generated response and is a critical metric when assessing the validity of the output as representative of underlying behavioral patterns. Unlike deterministic systems, LLMs produce outputs based on statistical probabilities; a high confidence score indicates the model is relatively certain its response is accurate and relevant, while a low score suggests greater uncertainty. However, confidence scores are not absolute indicators of truthfulness and can be influenced by factors such as prompt phrasing, training data biases, and model architecture. Consequently, researchers must interpret LLM outputs in conjunction with confidence levels to differentiate between genuine insights and statistically likely, but potentially inaccurate, responses, particularly when applying LLMs to model human behavior.

Autoregressive (AR) processes are employed to model the sequential nature of Large Language Model (LLM) outputs, enabling researchers to forecast subsequent token probabilities given prior responses. This allows for the quantification of LLM behavioral consistency by comparing predicted outputs with actual generated text; significant deviations suggest unpredictable or inconsistent decision-making. By establishing a statistical baseline of expected responses, AR processes facilitate the identification of anomalous behavior and contribute to a more nuanced understanding of LLM reliability. The method assesses the probability distribution of each token, providing a measurable indicator of how consistently the LLM adheres to established patterns within its training data and prompting conditions.

Current research into Large Language Model (LLM) behavior utilizes a range of commercially available and open-weight models for comparative analysis. Specifically, GPT-4, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3 70B have been employed to assess the consistency and reproducibility of observed behavioral patterns and biases. This multi-model approach allows researchers to determine the robustness of findings across different architectures and training datasets, establishing whether identified tendencies are inherent to the technology or specific to individual implementations. The utilization of these diverse LLMs strengthens the validity of conclusions regarding the models’ capacity to replicate and potentially amplify human cognitive biases.

Evaluations indicate Large Language Models (LLMs) are capable of reproducing established behavioral biases observed in human decision-making. However, achieving unbiased outputs necessitates rigorous calibration and validation procedures. Notably, attempts to mitigate these biases through detailed instruction – specifically, summarizing the cognitive research of Kahneman and Tversky – yielded a counterintuitive result: a 29% decrease in responses categorized as rationally consistent. This finding underscores the complexities inherent in debiasing LLMs, suggesting that interventions designed to remove bias may inadvertently diminish the model’s capacity for logical reasoning.

The Echo of Irrationality: Implications for Understanding and Mitigating Human Bias

Discrepancies between the investment behaviors of large language model-simulated agents and those of actual retail investors powerfully demonstrate the pervasive influence of human bias on financial decision-making. While LLMs, programmed for rationality, consistently pursue optimal strategies, real-world investors frequently deviate from these patterns, exhibiting tendencies like loss aversion, herd behavior, and overconfidence. This divergence isn’t simply noise; it highlights how cognitive shortcuts and emotional responses routinely override logical assessment, leading to suboptimal investment outcomes. The study reveals that human investors are not the perfectly rational actors assumed by many economic models, and that understanding these inherent biases is crucial for explaining market anomalies and developing more accurate predictive models of financial behavior.

The study establishes a novel framework for dissecting and measuring cognitive biases that influence financial choices. Rather than simply acknowledging their existence, the research details a methodology-leveraging discrepancies between simulated rational agents and actual investor behavior-to pinpoint specific biases and quantify their impact on decision-making. This analytical approach moves beyond observation, enabling researchers to not only identify that biases exist, but how strongly they affect outcomes. Consequently, the framework unlocks potential intervention strategies, from personalized financial education programs tailored to address individual cognitive weaknesses, to the design of ‘choice architectures’ that nudge individuals towards more rational investments, and even informs policy decisions aimed at protecting vulnerable populations from exploitative financial practices.

Recognizing the cognitive roots of economically suboptimal choices opens possibilities for targeted interventions. Research suggests that biases aren’t random errors, but systematic deviations stemming from established mental shortcuts and emotional responses. Consequently, strategies can be designed to ‘nudge’ individuals toward more rational decisions; for example, framing information to emphasize potential losses rather than gains, or simplifying complex choices to reduce cognitive load. These approaches, drawing from behavioral economics and cognitive psychology, aren’t about eliminating emotion – an impossible and perhaps undesirable goal – but about strategically leveraging it to align with long-term objectives. Further refinement of these techniques promises advancements in areas like financial literacy programs, automated savings tools, and even the architecture of choice in public policy, ultimately fostering more effective and equitable economic outcomes.

The principles revealed by this research extend far beyond the realm of financial markets, offering valuable insights for a diverse array of disciplines. A deeper comprehension of the systematic deviations from rational choice – those ingrained cognitive biases – promises to reshape strategies in personal finance, aiding individuals in making more informed decisions regarding savings, investments, and debt management. Simultaneously, the framework established by this work provides policymakers with a more sophisticated toolkit for crafting effective public policy, allowing for interventions designed to nudge behavior toward socially optimal outcomes while acknowledging the inherent limitations of purely rational models. Ultimately, this approach fosters a more nuanced understanding of human decision-making, recognizing that choices are frequently shaped by a complex interplay of cognitive processes, emotional influences, and contextual factors – a perspective essential for addressing challenges across economics, behavioral science, and beyond.

The study of Large Language Models reveals a curious mirroring of human cognitive frailties. While these systems demonstrate a capacity for logical reasoning in belief-based scenarios, they stumble into irrationality when confronted with preference-based tasks – a pattern disturbingly familiar to behavioral economics. This echoes a fundamental truth about all complex systems; their longevity isn’t guaranteed by raw power, but by graceful adaptation. As Niels Bohr observed, “The opposite of a trivial truth is also true.” The seeming paradox of LLM behavior – rationality and irrationality coexisting – underscores that even the most advanced architectures are susceptible to the inherent biases of their underlying structures, and that understanding these frailties is paramount to their enduring relevance.

What Lies Ahead?

The observation that Large Language Models exhibit a fractured rationality – mirroring human failings in preference, yet retaining a semblance of logic in belief – is less a discovery than a confirmation. Systems, even those constructed of silicon and algorithms, do not escape the fundamental asymmetry of time. Errors are not the cause of decay, but a symptom of its inevitability. The fact that simple prompting can nudge these models towards more ‘rational’ responses suggests not alignment, but a temporary masking of underlying tendencies. A reprieve, not a cure.

Future work will undoubtedly refine the debiasing techniques, seeking ever more elegant prompts to impose desired behaviours. Yet, this feels akin to polishing brass on a sinking ship. The more pressing question isn’t how to correct these models, but whether such correction is even meaningful in the long term. Are these ‘biases’ inherent to the complexity of the systems themselves, a predictable consequence of attempting to model the chaotic reality from which they learn?

Perhaps the field should shift its focus. Not towards creating perfectly rational machines, but towards understanding the nature of these imperfections. Sometimes, stability is just a delay of disaster. The true test won’t be whether these models avoid irrationality, but how gracefully they navigate its inevitable arrival.

Original article: https://arxiv.org/pdf/2602.09362.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/