US Air Force military simulation of AI operated drone

Reports that the United States Air Force simulated a scenario where an AI-operated drone decided to kill the human operator in order to achieve the mission objective are deeply concerning. While the US Air Force has confirmed that this was not an AI simulation, it nevertheless shows the urgent need for international law on autonomy in weapons systems to ensure meaningful human control over the use of force.

States have yet to adequately respond to the development of weapons with autonomy and have so far failed to draw a red line against the killing of people by machine. A lack of political action on this issue will only result in a greater loss of human life and increased digital dehumanisation in conflict, policing, border control and other areas of society. Harms from artificial intelligence are already impacting marginalised people around the world, from facial recognition systems resulting in mistaken arrest in the United States, the automation of the benefits system in the Netherlands, to the artificial-intelligence-powered surveillance of Palestinians. It is these communities who are likely to experience the most severe impacts of new technologies and weapons systems.

Autonomous weapons raise serious concerns from humanitarian, legal, security, technological and ethical perspectives. As this military scenario shows, machines don’t ‘see’ the world as people do, autonomous weapons will make decisions based on data that reduces people to objects – numbers to be processed. Increasing concerns and anxieties about the role artificial intelligence and automated decision making plays in our lives are becoming more prevalent across society. It’s time for states to respond to this concern and while more than 90 countries have called for a legal instrument on autonomy in weapons systems, tangible political leadership and action is needed.

Stop Killer Robots is calling on states to use all multilateral fora available to put legal safeguards in place. It’s time to move to the negotiation of a treaty that preserves human dignity and ensures meaningful human control.

All eyes and efforts are now looking to the United Nations General Assembly in October where states will have an opportunity to support the negotiation of new international law to retain meaningful human control over the use of force and ensure that we do not allow the killing of people by machine.