Author: Denis Avetisyan
A new index reveals a concerning trend: despite growing power, major AI companies are becoming less transparent about the models shaping our future.
The 2025 Foundation Model Transparency Index provides a rigorous evaluation of transparency practices across leading AI developers, assessing metrics related to open weights, safety documentation, and regulatory compliance.
Despite growing recognition of the need for responsible AI development, transparency among foundation model developers is demonstrably declining. The 2025 Foundation Model Transparency Index offers a rigorous, quantitative assessment of leading AI companies’ practices regarding data sourcing, model usage, and post-deployment monitoring. Our analysis reveals a significant decrease in overall transparency-falling from 58 to 40 out of 100-with substantial opacity surrounding training data and model impact. Given increasing regulatory pressure for greater accountability, this index highlights critical information gaps and poses the question of whether current policy interventions will be sufficient to drive meaningful progress towards responsible AI development.
The Accountability Deficit: Foundation Models and the Erosion of Trust
The rapid advancement of foundation models – the large AI systems powering increasingly complex applications – is shadowed by a critical lack of standardized transparency. While these models demonstrate remarkable capabilities across diverse tasks, the opacity surrounding their development and operation introduces substantial risks for societal impact. Without clear documentation regarding training data, model architecture, and potential biases, it becomes exceedingly difficult to anticipate and mitigate harms ranging from discriminatory outcomes to the spread of misinformation. This absence of accountability isn’t merely a technical challenge; it represents a fundamental impediment to responsible AI deployment, hindering public trust and potentially exacerbating existing inequalities as these powerful technologies become further integrated into daily life. The growing capabilities of these models demand a parallel increase in scrutiny and openness to ensure their benefits are widely shared and their risks effectively managed.
Assessments of foundation models, despite their growing influence, remain fragmented and fail to consistently examine vital AI development practices. Recent findings from the 2025 Foundation Model Transparency Index (FMTI) demonstrate a considerable accountability gap, with models achieving an average score of just 40.69 out of 100. This score, a notable decrease from 58 in the prior year, suggests a worrying trend wherein the ability to scrutinize these powerful systems is not keeping pace with their advancement. The lack of standardized, comprehensive evaluations hinders efforts to understand potential risks and ensure responsible development, creating challenges for those seeking to mitigate harms and promote trustworthy AI.
The capacity to effectively evaluate and address potential harms stemming from foundation models is fundamentally hampered by a lack of transparency surrounding both the data used in their creation and the processes guiding their development. Recent assessments, such as the 2025 Foundation Model Transparency Index, reveal a troubling decline in openness, with average scores decreasing from 58 in 2024 to 40.69 out of 100. This downward trend suggests an increasing difficulty in understanding how these powerful AI systems function and, crucially, in proactively identifying and mitigating risks related to bias, misinformation, or unintended consequences – ultimately hindering responsible innovation and eroding public trust.
Standardized Indicators: Mapping the Terrain of Transparency
Robust indicator definition is fundamental to achieving consistency and comparability in transparency assessments. Without clearly defined metrics, evaluations become subjective and lack the granularity needed for meaningful comparison across different systems or organizations. This necessitates specifying precise measurement criteria for each indicator, detailing data requirements, calculation methods, and acceptable ranges. A standardized approach to indicator definition enables automated evaluation processes, reduces ambiguity, and allows for the tracking of progress over time, ultimately moving beyond qualitative judgments to data-driven insights into transparency levels.
Comprehensive transparency indicators require detailed information across three core areas: data sourcing, model training, and intended use cases. Data sourcing documentation should specify the origin, collection methods, and any preprocessing steps applied to the training data. Model training details must include the algorithms used, hyperparameter settings, validation procedures, and performance metrics. Finally, indicators must clearly define the model’s intended use cases, including specific applications, limitations, and potential biases. Capturing these elements provides a complete picture of the AI system, enabling thorough evaluation and responsible deployment.
The implementation of standardized indicators enables the automated evaluation of transparency practices, shifting assessments from subjective, qualitative reviews to objective, quantifiable metrics. This automation allows for longitudinal tracking of progress and consistent comparisons between different organizations or models. The 2025 Fairness, Accountability, and Transparency Index (FMTI) results demonstrate the value of this approach, with IBM achieving the highest score of 95/100, indicating that robust, indicator-driven transparency initiatives can be effectively measured and demonstrably improve performance.
Automated Auditing: A System for Measuring the Immeasurable
Automated Transparency Reporting is facilitated by the deployment of an ‘Automated Evaluation Agent’-a software system designed to collect and process large volumes of data related to AI practices. This agent operates by autonomously accessing and analyzing publicly available documentation, including model specifications and technical reports, to create a quantifiable assessment. The system’s scalability allows for the consistent evaluation of multiple AI companies and models, overcoming the limitations of manual review processes. Data gathered is then used to generate standardized reports, providing a consistent metric for comparison and identification of areas requiring improvement in AI transparency.
The Automated Evaluation Agent functions by applying a pre-defined set of indicators – encompassing areas such as data governance, model limitations, and safety testing – to assess the practices of AI developers. These indicators are quantitatively scored based on information provided in documentation like Model Cards and Technical Reports. The agent then aggregates these scores to produce standardized reports, allowing for consistent and comparable evaluations across different AI companies. This standardized reporting facilitates benchmarking and identifies areas where companies fall short of established transparency benchmarks, as demonstrated by the results of the 2025 FMTI, where scores varied significantly.
Automated transparency reporting systems depend on standardized documentation for verification of AI developer claims; specifically, ‘Model Cards’, ‘Technical Reports’, and ‘System Cards’ serve as primary data sources for evaluation. Analysis of the 2025 Fairness, Accountability, and Transparency Index (FMTI) revealed significant discrepancies in reported transparency practices, with both Midjourney and xAI receiving the lowest scores of 14/100. This result indicates a substantial need for improved documentation and data provision from these companies to accurately assess their adherence to transparency standards and facilitate independent evaluation of their AI systems.
The Long View: Toward Responsible AI and a Future Built on Trust
Greater transparency in the development and deployment of artificial intelligence is fundamentally linked to building public trust and ensuring responsible innovation. When the processes behind AI systems – including data sources, model architecture, and potential limitations – are openly communicated, it allows for external scrutiny and identification of potential harms before they manifest as negative societal impacts. This proactive approach fosters accountability, enabling developers to address biases, correct errors, and ultimately build AI that aligns with human values. Without such openness, the ‘black box’ nature of many AI systems can erode public confidence and hinder the widespread adoption of technologies with the potential to benefit society, creating a justifiable concern about unintended consequences and reinforcing the need for clear, accessible information regarding AI’s inner workings.
A recent evaluation of foundation model developers demonstrated that diligent scrutiny of data usage is critical for identifying and addressing potential biases and ethical shortcomings. The assessment revealed a concerning trend: while two companies, AI21 Labs and Writer, notably improved their transparency regarding data practices, four others experienced declines in their scores. This disparity underscores the varying levels of commitment to responsible AI development within the industry. Systematic data assessment allows organizations to proactively pinpoint problematic patterns, mitigate unfair or discriminatory outcomes, and ultimately build more trustworthy and ethical AI systems. The findings suggest that ongoing monitoring and a commitment to transparency are not universally adopted, highlighting a crucial area for improvement and reinforcing the need for standardized benchmarks and greater accountability.
The recently developed Foundation Model Transparency Index offers a crucial yardstick for evaluating progress in responsible AI development, functioning as both a diagnostic tool and a driver for industry-wide improvements. This index assesses the clarity and accessibility of information regarding training data, model capabilities, and potential limitations – essential components for building trustworthy AI systems. The current average score of 40.69, however, underscores a significant gap between aspiration and reality, indicating that most organizations still have considerable work to do in openly communicating about their foundational models. While a few companies demonstrate increasing commitment to transparency, the overall result serves as a clear call to action for the entire field to prioritize openness and accountability as core tenets of AI innovation, ultimately fostering greater public trust and responsible deployment.
The pursuit of transparency in foundation models, as detailed in the 2025 Foundation Model Transparency Index, isn’t about achieving a perfect blueprint, but cultivating a healthy ecosystem. The index reveals a concerning trend – a decline in openness from AI companies – suggesting that these systems are becoming less like gardens tended with care and more like overgrown landscapes. Barbara Liskov observed, “It’s one of the really tough things about systems. It’s not just about correctness, it’s about how well they handle failure.” This resonates deeply; the index isn’t merely measuring what companies say about their models, but assessing their ability to gracefully manage the inevitable imperfections and potential harms inherent in these complex creations. Resilience, therefore, isn’t found in preventing all failures, but in designing for forgiveness – a principle desperately needed in the rapidly evolving world of AI.
What Lies Ahead?
The 2025 Foundation Model Transparency Index doesn’t reveal a lack of progress, but the predictable consequence of complex systems achieving scale. Long stability is the sign of a hidden disaster; the measured decline in transparency isn’t a failure of intent, but the inevitable erosion of initial conditions. Each reported metric, each quantified disclosure, represents a snapshot – a momentary equilibrium before the model’s internal logic reasserts itself. The architecture wasn’t built to be transparent, it was built to appear so, and appearances are fleeting.
Future iterations of this Index will undoubtedly track the proliferation of obfuscation, the increasing difficulty of tracing causal pathways within these models. The question isn’t whether transparency can be achieved, but whether the illusion of transparency can be maintained long enough to forestall the most immediate consequences. The focus must shift from evaluating individual companies to mapping the emergent properties of the entire ecosystem – recognizing that regulatory compliance isn’t a solution, but a temporary pressure on a fundamentally unstable system.
The true metric for success won’t be a higher score on the Index, but the ability to accurately predict the shape of the inevitable failures. Systems don’t fail – they evolve into unexpected shapes. And those shapes, while unpredictable in detail, are rarely entirely surprising to those who understand the underlying dynamics of growth and decay.
Original article: https://arxiv.org/pdf/2512.10169.pdf
Contact the author: https://www.linkedin.com/in/avetisyan/
See also:
- Fed’s Rate Stasis and Crypto’s Unseen Dance
- Blake Lively-Justin Baldoni’s Deposition Postponed to THIS Date Amid Ongoing Legal Battle, Here’s Why
- Gold Rate Forecast
- Ridley Scott Reveals He Turned Down $20 Million to Direct TERMINATOR 3
- Top 10 Coolest Things About Indiana Jones
- Global-e Online: A Portfolio Manager’s Take on Tariffs and Triumphs
- BTC Dumps to $90K, HYPE Crashes 9%-What’s Next? 🚀💥
- Dogecoin’s Decline and the Fed’s Shadow
- Northside Capital’s Great EOG Fire Sale: $6.1M Goes Poof!
- The VIX Drop: A Contrarian’s Guide to Market Myths
2025-12-15 00:19