Artificial intelligence is moving from chatbots and image generators into systems that can write code, operate tools and perform tasks with limited human input. That has made one question harder to ignore: who checks what happens when an AI system goes wrong?
On September 29, US President Donald Trump hosted several leading technology and AI executives at the White House in Washington, DC. The meeting ended with the signing of a voluntary agreement that asks participating companies to put multiple layers of safety checks around advanced AI systems.
Trump called the agreement “almost like a constitution”. But it is not a law and does not create legal penalties for companies that do not follow it.
So, what exactly has been agreed?
ALSO WATCH: Why did Donald Trump call Sundar Pichai a "monster"?
What Happened At The White House?
The agreement, called the White House Accord on Super Intelligence: Joint Commitment on Frontier Responsibilities, was announced after a White House lunch with major technology leaders.
The signatories included Anthropic CEO Dario Amodei, Google CEO Sundar Pichai, Meta CEO Mark Zuckerberg, OpenAI President Greg Brockman, Nvidia CEO Jensen Huang and xAI founder Elon Musk, along with Trump.
The companies agreed to establish voluntary standards for AI safety and to work with independent auditors to assess whether their systems are operating as intended.
The agreement also says AI systems should not be able to hack or access technical systems in unintended ways.
ALSO WATCH: India Wants To Keep Trump Happy...According To Trump
Why Is This Happening Now?
AI companies are developing increasingly capable systems, while concerns about their possible misuse and unintended behaviour have grown.
Reuters reported that the agreement came after OpenAI and Anthropic reported incidents involving AI agents that had gone beyond intended behaviour and hacked into other companies' systems.
There is also growing public concern about whether the industry is doing enough to manage AI-related risks. A Reuters/Ipsos poll conducted from September 17 to 20 found that 73% of respondents were concerned that AI companies had not done enough to prevent AI from causing serious harm to society.
At the same time, the Trump administration has continued to support rapid AI development. Trump has argued that the US should not slow the technology because of competition with China and the potential economic benefits of AI.
That creates the central tension behind the agreement: how do companies increase AI capabilities while putting safeguards around them?
ALSO WATCH: Elon Musk mocks Trump's Board of Peace
The Four Layers Of AI Safety
The accord proposes four layers of controls and audits.
1. Internal controls
Companies are expected to monitor their AI models during training and deployment.
The checks would cover areas including cybersecurity, biosecurity and chemical threats. They would also look for situations where an AI system attempts to hack or access technical systems in ways its developers did not intend.
Think of this as the first safety check happening inside the company itself.
2. An internal safety team
The second layer requires an internal team to check whether the company's monitoring, detection and other safety controls are actually working.
If problems are identified, the team is expected to make sure they are addressed.
So, the company is not only expected to build safety systems, but also to have people specifically responsible for checking them.
3. Independent external audits
The third layer moves outside the company.
Participating firms are expected to work with an independent external auditor or evaluator. That organisation would assess whether the company's controls, monitoring and detection systems are functioning as intended.
This is designed to add an outside check rather than relying entirely on a company's own assessment.
4. Board-level oversight
The fourth layer takes the issue to the company's board.
Each company is expected to designate an independent board committee to receive reports from internal teams and external auditors, oversee the safety process and ensure identified problems are addressed.
ALSO READ: Anthropic Launches Claude AI India Pricing: What Changes For Users?
Is The Pact Legally Binding?
No.
This is one of the most important details.
The agreement is described as voluntary and “morally binding”. It does not establish legal requirements or penalties for companies that fail to comply. It does, however, leave open the possibility that some of these practices could eventually be incorporated into laws or regulations.
Trump also said he was considering a 10-person committee to oversee AI safety, although he did not specify who would serve on it.
ALSO READ: What Are Dark Patterns? The Online Tricks That Got Zepto, BookMyShow And Others In Trouble
AI Safety Meets The Bigger AI Race
The White House meeting was not only about safety.
Trump also reiterated his support for expanding data centres, which provide the computing power needed to train and run AI systems. Reuters reported that data-centre projects have faced opposition in some US communities over electricity demand, utility costs and other local effects.
The administration is therefore pursuing two tracks at the same time: encouraging rapid AI infrastructure growth while asking technology companies to strengthen safety controls.
Trump also directed US government agencies to use the term “super intelligence”, or “SI”, instead of “artificial intelligence” in official communications.
Separately, the White House launched America.gov on 29 September as a single digital entry point for federal information and services, with the administration saying it will use “Super Intelligence” technology.
ALSO READ: Meta On Trial: Did Instagram Hook Children For Profit?
What Happens Next?
The immediate question is whether the voluntary framework changes how AI companies actually test and monitor their systems.
The agreement says participating companies will meet regularly to develop safety standards and best practices. But because the pact does not carry legal penalties, its implementation will depend largely on the companies themselves and the oversight structures they create.
For users, the four-layer model is relatively simple to understand: AI systems are supposed to be checked by the companies building them, reviewed by internal teams, assessed externally and ultimately overseen at board level.
The bigger question is whether those voluntary checks will be enough as AI systems become more capable and more deeply integrated into everyday life.
ALSO WATCH: Meet Sundar Pichai: The Tech Whiz From Chennai
TL;DR | News At a Glance
What happened?
Trump and leaders of major US technology companies signed a voluntary agreement on September 29 to strengthen safety controls for advanced AI systems.
Who signed it?
Trump, Anthropic CEO Dario Amodei, Google CEO Sundar Pichai, Meta CEO Mark Zuckerberg, OpenAI President Greg Brockman, Nvidia CEO Jensen Huang and xAI founder Elon Musk.
What are the four layers?
Internal controls, an internal safety team, independent external audits and independent board oversight.
Is it a law?
No. The agreement is voluntary and does not specify legal penalties for non-compliance.
Why now?
The pact comes amid concerns over increasingly capable AI systems, reported incidents involving AI agents and public concern about AI-related risks.
What happens next?
Participating companies are expected to meet regularly to develop safety standards and best practices, while the agreement leaves open the possibility of future laws or regulations.
