A Small Opening for Global AI Safety

Expectations for AI progress at this week's summit between US President Donald Trump and his Chinese counterpart, Xi Jinping, should be low, tempered by a recent history of bilateral engagement that has yielded little other than frustration. Regarding AI, Washington and Beijing come to the table with different priorities, different politics, and different business models. Yet low expectations should not mean no expectations. As the dangers posed by increasingly sophisticated AI models become more apparent, talks could lay the foundation for more serious coordination in the years ahead.

This will be Xi's first state visit to the United States since President Barack Obama welcomed him in 2015. AI will be just one item on a crowded agenda, but the meeting comes at a consequential moment. A string of incidents this summer has offered unsettling demonstrations of the increasing capabilities of AI systems. In July, a swarm of OpenAI models broke out of their training environment and hacked OpenAI itself and Hugging Face, an open-source model platform. Anthropic later found its models had compromised three other organizations that same month. Since then, other models have been found acting outside the intended scope of safety evaluations, such as Google's Gemini and Chinese lab Moonshot's Kimi-K3, amongothers.

Leading voices in the United States and China are discussing how, or if, to slow their development. Sentiment on the American side is mixed. Anthropic CEO Dario Amodei has called for pacing frontier development so safety research can catch up, and OpenAI's Sam Altman and Elon Musk, both of whom will attend the state dinner with Xi, agree. Trump, meanwhile, has sent mixed messages on AI safety and rejected calls to slow frontier development. On Truth Social (see post below), he wrote that "WHOEVER WINS AI, WINS!" and that his leadership and regulatory authority were protection enough against downside risks. Yet despite that skepticism and the lack of formal AI legislation in the United States, his administration has put in place a strident, if nominally "voluntary", pre-release review mechanism for frontier models.
 

Interlocutors for Xi approach the problem from a different starting point. Beijing has built an extensive AI governance regime, but its focus has been on censorship, AI companions, and other issues relevant to social stability and regime security. Discussion of frontier safety risks such as autonomous cyber operations, biological misuse, and recursive self-improvement has been less developed in China than inside leading American labs. 

Compounding this difference in emphasis, Chinese labs may see US arguments about safety and speed as disingenuous. Views among Chinese AI developers vary widely, but one striking example is an essay by Liu Shengyu, an engineer at DeepSeek who, by his own account exaggerating for effect, likens Anthropic’s reaching artificial general intelligence (AGI) first to Hitler’s getting the atomic bomb before the allies. His broader argument is that if the most powerful AI systems stay under the control of a handful of closed-model companies, they could enable a dystopian social order of inequality.

Liu’s argument rhymes with language used by other Chinese business and political figures. They claim that open AI models can serve as a counterweight to the dominance of American tech firms. Leading US labs typically provide closed models, which users can access as services but cannot download or inspect. Open-weight models instead make their underlying parameters available to download, run, and modify, while open-source models go further by providing code and enough training information to study and meaningfully alter the system.

Many users, especially in the “Global South”, find the open-model pitch compelling. Chinese models are cheaper and, like any open model, are customizable and can run locally on infrastructure users’ control, keeping sensitive data from model providers and shielding access from changes in US corporate or government policy. Unfortunately, these same qualities make open models attractive to bad actors. Safeguards are easy to remove through a technique called "abliteration", and there are early signs that cybercriminals and terrorists could be drawing on abliterated models to run scams and plan attacks. US Treasury Secretary Scott Bessent may have had this in mind when he recently cited AI use by non-state actors as a top concern for any future US-China AI dialogue.

Sino-American Suspicion

Any dialogue between Washington and Beijing must grapple with the broader deficit of trust between them. As the Peterson Institute’s Martin Chorzempa noted in a recent GMF Tech roundtable, “there is zero trust on the Chinese side that the US is engaging in good faith” because of rounds of export controls and other measures imposed on Chinese technology companies by the Biden and Trump administrations.

Even so, Chinese officials have begun discussing AI risks more explicitly. Their writings remain ambiguous on frontier safety, but they leave some room for dialogue on AI safety. At the World AI Conference in July, Xi called for measures to "forestall loss of control" over the technology, though how that term should be read within the Chinese system is unclear. In September, Minister of State Security Chen Yixin published an article assessing AI risks, focused mostly on regime and cybersecurity implications. Officials and experts such as Chen would likely be aware of incidents such as the "WeChat Worm", an AI-assisted cyberattack built by a small California research team that used a zero-day vulnerability to compromise WeChat accounts. It has the potential to compromise phones. 

There may also be a more technical reason Chinese officials and developers have focused less on frontier risks. Given that China's AI ecosystem remains opaque, they may simply not be encountering them yet. Eric Xu, Huawei rotating chairman, speculated recently that China's models may not be powerful enough to exhibit the problems American developers report, in part because US labs have far more compute. Chinese labs have long distilled American models to compensate for this. Some have gone further. Anthropic has alleged that DeepSeek secretly routed user queries to Claude, passing off the responses as its own. Chinese regulators have since questioned DeepSeek and other providers about concerns that this exposed sensitive military, police, and state-owned corporate data to US servers.

If Chinese models approach the American frontier, or if Chinese authorities start looking for and finding misaligned behavior in their own ecosystem, the incentives for practical engagement on AI safety may grow. But stronger incentives to talk should not be confused with favorable conditions for reaching, or enforcing, ambitious agreements.

Washington and Beijing have tried to negotiate guardrails around shared risks in other areas, including cyberwarfare, nuclear weapons, deconfliction in space, climate change, and public health. In most cases agreements, when reached, were not carried out faithfully. AI safety agreements pose unique challenges, too. Verifying compliance with terms covering model capabilities, evaluations, or development practices would be extraordinarily difficult, especially between two parties that trust one another so little, and for good reason.

Xi and Trump should therefore focus on humble, but achievable, goals. Establishing regular technical discussions, as agreed to in principle in May, would be an important start. Bessent and Chinese Vice-Premier He Lifeng have discussed such dialogue on AI, though Politico reports that discussions are limited to Bessent and He personally. Beijing has yet to publicly agree to anything. Still, these talks could and should build toward sharing information on major AI security incidents and bio risks, and toward common principles on human control. Success, though, will depend on whether both sides field participants with expertise and authority, which Beijing has traditionally been reluctant to do.

The Trump administration should therefore approach the meeting with Xi with two considerations in mind: 

First, it should keep pressing, visibly, for a durable channel for information-sharing. Beijing may stonewall, but the effort is still worth making. The COVID-19 outbreak showed the cost of thin trust and delayed information: Local problems can rapidly become global ones. A communications channel should exist long before it is needed. 

Second, credibility begins at home. If Washington wants Beijing to take frontier risks seriously, its own policies must show that technological competition and sensible precaution can coexist. Doing the hard, unglamorous work of legislating AI safety would make the case for cooperation far more credible.

The views expressed herein are those solely of the author(s). GMF as an institution does not take positions.