In my two years as attorney general, I spent more time working on national security than any other issue. The range and gravity of the threats posed to our nation are multifold, from terrorism, armed conflicts and nuclear proliferation, to the impacts of financial insecurity and the climate crisis. All profound challenges, yet ones our superb intelligence and defence communities are well placed to counter. The risk from AI is of an altogether different magnitude, not only in scale, but also because there is a limited time window to address it. The risk is difficult to overestimate: it poses an existential risk to humanity. The lights are flashing red and we must act now before it’s too late.

Although the extraordinary threats posed by AI have long been flagged by those closely involved in its creation, it was not until very recently, after the disclosure of the Hugging Face hack and the resignation of Jacob Coxon from Anthropic, that the issue at last started to gain traction in the public discourse. This moment is an opportunity, and to capitalise on it we desperately need increased urgency in international coordination to regulate frontier AI. There must also be meaningful public engagement with how AI should be shaped to benefit humanity, rather than entrench the power of a few. We cannot leave this to China, the US and big tech – we must all demand a voice in the debate.

AI has enormous potential benefits for us all. It will be the source of mind-blowing advances in science and medicine, and offers hope for empowerment of some of the world’s most dispossessed people. Yet the risks we are now facing are truly staggering. I would put them in two very broad categories. The first is existential risk: quite literally the possibility that AI could kill billions, either because it enables terrible people to do terrible things – such as design and deploy biological weapons – or because AI itself determines it is more efficient to dispense with humans.

The second category is existential challenge – that, left unchecked, the development of AI will deprive billions of us of meaningful agency over our lives, with utterly profound consequences for economies, national security and human happiness. As Bill Gates recently wrote, the AI era will herald “one of the most turbulent times in human history”, noting that it will either be the greatest equaliser ever invented or the worst source of injustice. He is right. On its current trajectory, it risks simultaneously revolutionising the meaning and availability of productive work, while placing unimaginable power in the hands of not only two states (the US and China) but also a small number of individuals who control the tech giants.

Read More: When AI is subject to less regulation than your fridge, something is startlingly wrong