Company founders demand UN action on killer robots

The Campaign to Stop Killer Robots urges governments to heed an open letter signed by 126 founders and directors of more than 100 robotics and artificial intelligence companies from 28 countries demanding urgent action to address fully autonomous weapons concerns.

Released at the opening of a major international conference on artificial intelligence (AI) in Melbourne on August 21, the letter lists numerous concerns with fully autonomous weapons, also called lethal autonomous weapons systems or killer robots. It warns that these weapons could be used by “despots and terrorists” and “hacked to behave in undesirable ways.”

The company founders state that they “feel especially responsible in raising this alarm” and caution that “we do not have long to act” as “once this Pandora’s box is opened, it will be hard to close.” They call on states at the Convention on Conventional Weapons (CCW) to “protect all of us” from the dangers posed by fully autonomous weapons and offer their researchers and engineers to provide technical support to the faltering CCW process on fully autonomous weapons.

The signatories include Elon Musk, founder of Tesla, SpaceX, OpenAI (USA), Mustafa Suleyman, founder of Google’s DeepMind (UK), and Jüergen Schmidhuber, leading deep learning expert and founder of Nnaisense (Switzerland).

AI expert Toby Walsh of the University of New South Wales collected the signatures on the letter from the founders of companies that “employ tens of thousands of researchers, roboticists and engineers” in Australia, Canada, China, Czech Republic, Denmark, Estonia, Finland, France, Germany, Iceland, India, Ireland, Italy, Japan, Latvia, Mexico, Netherlands, New Zealand, Norway, Poland, Russia, Singapore, South Africa, Spain, Switzerland, UK, United Arab Emirates and USA.

According to Walsh, the letter “demonstrates clear concern and strong support for this from the robotics & AI industry.”

Leading deep learning expert Yoshua Bengio, founder of Element AI (Canada), said he endorsed the letter because, “the use of AI in autonomous weapons hurts my sense of ethics” and “would hurt the further development of AI’s good applications.” Bengio urged the international community to approach autonomous weapons the same way as “other morally wrong weapons” such as biological, chemical, nuclear weapons.

Another signatory is one of the world’s preeminent AI researchers, Stuart Russell, founder of Bayesian Logic Inc., who described a treaty banning lethal autonomous weapons as “vital for national and international security.” Russell warned against the development and proliferation of “new weapons of mass destruction – in the form of vast swarms of lethal microdrones.”

The signatories include Ryan Gariepy, founder & chief technical officer at Clearpath Robotics (Canada), which was the first company to endorse a ban on fully autonomous weapons three years ago. According to Gariepy, “autonomous weapons systems are on the cusp of development right now and have a very real potential to cause significant harm to innocent people along with global instability.”

A previous open letter issued in July 2015 at the last International Joint Conference on Artificial Intelligence (IJCAI) called “a military AI arms race” a “bad idea” and urged ban on weapons systems that “lack meaningful human control.” The 2015 letter has been signed by over 20,000 people to date, including over 3,100 AI/robotics researchers.

In December 2016, states agreed to formalize their deliberations on lethal autonomous weapons systems at the Convention on Conventional Weapons (CCW). The Group of Governmental Experts (GGE) was supposed to meet twice in 2017, but financial challenges have resulted in the cancellation of the first meeting of the GGE that was supposed to open on 21 August.

The Campaign to Stop Killer Robots urges states to use the CCW Group of Governmental Experts to begin negotiations on a new CCW protocol by the end of 2018 that preemptively bans fully autonomous weapons. More than 90 countries are expected to participate in the GGE meeting at the UN in Geneva on 13-17 November as well as key UN agencies, the International Committee of the Red Cross, and Campaign to Stop Killer Robots.

UPDATE: This post was updated on 25 August to reflect latest number of signatories to the open letter, which was originally 116 company founders in 26 countries.

For more information see:

• 2017 open letter, PDF version (as of August 24), press release, and 23 August Melbourne event with key signatories
• 2015 open letter and web post
• 2017 International Joint Conference on Artificial Intelligence (IJCAI)
• Interview with signatories Toby Walsh and Ryan Gariepy

Photo: Pepper the robot is made by Aldebaran Robotics (France), whose founder Jerome Monceaux has signed the 2017 letter. Pepper is shown here at an event convened by the Association for Aid and Relief Japan in Tokyo in March 2016 to discuss the Campaign to Stop Killer Robots.