AI Safety Institute

An AI Safety Institute (AISI), in general, is a state-backed institute aiming to evaluate and ensure the safety of the most advanced artificial intelligence (AI) models, also called frontier AI models.

AI safety gained prominence in 2023, notably with public declarations about potential existential risks from AI. During the AI Safety Summit in November 2023, the United Kingdom (UK) and the United States (US) both created their own AISI. During the AI Summit Seoul in May 2024, international leaders agreed to form a network of AI Safety Institutes, comprising institutes from the UK, the US, Japan, France, Germany, Italy, Singapore, South Korea, Australia, Canada and the European Union.

Timeline
In 2023, Rishi Sunak, the Prime Minister of the United Kingdom, expressed his intention to "make the U.K. not just the intellectual home but the geographical home of global AI safety regulation" and unveiled plans for an AI Safety Summit. He emphasized the need for independent safety evaluations, stating that AI companies cannot "mark their own homework". During the summit in November 2023, the UK AISI was officially established as an evolution of the Frontier AI Taskforce, and the US AISI as part of the NIST. Japan followed by launching an AI safety institute in February 2024.

Politico reported in April 2024 that many AI companies had not shared pre-deployment access to their most advanced AI models for evaluation. Meta's president of global affairs Nick Clegg said that many AI companies were waiting for the UK and the US AI Safety Institutes to work out common evaluation rules and procedures. An agreement was indeed concluded between the UK and the US in April 2024 to collaborate on at least one joint safety test. Initially established in London, the UK AI Safety Institute announced in May 2024 that it would open an office in San Francisco, where many AI companies are located. This is part of a plan to "set new, international standards on AI safety", according to UK's technology minister Michele Donelan.

At the AI Summit Seoul in May 2024, the European Union and other countries agreed to create their own AI safety institutes, forming an international network.

United Kingdom
The United Kingdom founded in April 2023 a safety organisation called Frontier AI Taskforce, with an initial budget of £100 million. In November 2023, it evolved into the UK AISI, and continued to be led by Ian Hogarth. The AISI is part of the United Kingdom's Department for Science, Innovation and Technology.

The United Kingdom's AI strategy aims to balance safety and innovation. Unlike the European Union which adopted the AI Act, the UK is reluctant to legislate early, considering that it may lower the sector's growth, and that laws might be rendered obsolete by technological progress.

In May 2024, the institute open-sourced an AI safety tool called "Inspect", which evaluates AI model capabilities such as reasoning and their degree of autonomy.

United States
The US AISI was founded in November 2023 as part of the NIST. This happened the day after the signature of the Executive Order on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. In February 2024, Joe Biden's former economic policy adviser Elizabeth Kelly was appointed to lead it.

In March 2024, a budget of $10 million was allocated. Observers noted that this investment is relatively small, especially considering the presence of many big AI companies in the US. The NIST itself, which hosts the AISI, is also known for its chronic lack of funding. Biden administration's request for additional funding was met with further budget cuts from congressional appropriators.