Drones are a common sight on the battlefields of Ukraine, but they are normally controlled by human pilots
Frank Herrmann/Getty Images
Fully autonomous drones with no human oversight have killed soldiers on the battlefield for the first time. This is according to a senior figure in the Ukrainian defence industry, marking a watershed moment in warfare.
The one-off test involved 10 AI-controlled “Terminator” drones on the front line of the Ukraine war. Russian soldiers were killed.
“We tried it,” says drone-maker Alexander Kokhanovskyy, who supplied the technology and spoke to New Scientist at a press event hosted by the Ukrainian embassy. “It’s a test. We never implemented it [more widely].”
The test took place two years ago and involved quadcopter drones that were programmed to fly towards the front line, cover between 3 and 5 kilometres over around 10 minutes and then engage “Terminator mode”, in which an AI model searches for and intercepts targets.
“We just launch it and we know everything will be dead – everything that will be found there in this particular area will be dead,” says Kokhanovskyy. “There is no connection to the drone at all, you cannot see the video, nothing… Everything it sees will be killed.”
With no way to tell what the automated drones had seen or targeted, human-piloted drones were sent into the area after the test to manually check results. Victims included “a couple of soldiers, one truck”, says Kokhanovskyy. While there is no recording of the automated drones attacking these targets, it was concluded that the drones had killed them.
Kokhanovskyy says that he was not at the test personally but that it was carried out by an unnamed military unit near the cities of Bakhmut and Chasiv Yar as part of a Ukrainian counteroffensive push. The Ukrainian Ministry of Defence did not respond to questions about the test or the current legal position on the use of fully autonomous weapons.
The use of AI is common in militaries around the world, helping to pick targets among overwhelming piles of intelligence data and automating certain functions of weapons, but humans are always in the loop at some point. Kokhanovskyy’s admission is the most categorical evidence yet that a death has occurred in battle solely at the hands of AI.
The Ukrainian government currently bans the use of AI at the final stage of intercepting targets, according to defence company sources speaking at the embassy press conference, although AI is used for many parts of the process by many devices up to that point. Kokhanovskyy says that the government is aware of the growing capabilities of AI and that it is in talks with defence companies about whether or not rules should be made more lenient.
Reports in 2023 suggested that Ukrainian attack drones equipped with artificial intelligence were finding and attacking targets without human assistance – but were being deployed against vehicles such as tanks, rather than infantry. At the time, no human casualties were confirmed.
While there is no official international ban on autonomous weapons that can kill without human intervention, United Nations Secretary-General António Guterres has called for one, saying last year that “there is no place for lethal autonomous weapon systems in our world”.
The UN has said that there are concerns that such weapons could violate international humanitarian and human rights laws by removing human judgement from warfare. There is also a risk that autonomous systems could make mistakes, either attacking soldiers or equipment from the same side or striking civilians.
Most militaries are developing devices that automate at least some part of the process of attacking targets. The US has software that accumulates and analyses vast amounts of disparate data and selects targets on the battlefield that can then be struck by drones, but, in theory, this requires human confirmation. There have been claims that the US is also developing so-called Goalkeeper flying drones and Whiplash naval drones, which are capable of finding their own targets and taking them out.
A UN report from 2021 even suggested that a Kargu-2 quadcopter produced by a Turkish firm may have been used to autonomously attack humans the previous year. The report gave no specific detail on the source of the claims or whether any humans had been injured or killed, but suggested that Libya’s Government of National Accord (GNA) had used the drones against retreating Haftar forces.
Major Danylo Polozhukhno, a senior figure in Ukraine’s 21st Separate Unmanned Systems Regiment of the 3rd Army Corps who was not aware of or involved with the test, told New Scientist that his soldiers use semi-autonomous control systems but that there is always a human in the loop.
“These drone systems and platforms are capable of automatically acquiring and tracking targets, as well as autonomously guiding themselves during the final metres of the approach, which helps simplify the operators’ work. However, we do not use fully autonomous drone systems that independently select and engage targets without any operator involvement,” says Polozhukhno. “Ukraine adheres to international humanitarian law and takes seriously its responsibility to uphold the rights of all combatants. It also exercises great care in decision-making in order to prevent civilian casualties.”
Mariarosaria Taddeo at the University of Oxford says killing with AI steals the dignity of the soldier, removes responsibility from the attacker and must be banned. “It’s not just problematic, it’s horrendous,” she says. “Do we want to be the society who kills other people, who allows their government to kill other people, without humans being involved?”
Anthony King at the University of Exeter, UK, says that though fully autonomous attacks without humans in the loop are technologically possible, they may be less of a decisive tool than many think.
“It is certainly possible governments would allow this if it gave them any military advantage,” he says. “However, the fact remains that very few if any of the millions of drones which have been used in the Ukrainian war by Russian and Ukrainian forces have been [fully] autonomous.”
“So it’s not just that it’s ethically right to keep humans in the loop, at this point, it’s more militarily effective,” says King.
Kokhanovskyy says that the Terminator project has not progressed since the test because of Ukraine’s rules. He is now CEO of drone-maker Aero Center, which he says was not involved in the test as it had not been created at the time, a Ukranian firm working on autonomous interceptor drones. These are designed to target incoming Russian Shahed kamikaze drones and take them out before they can reach towns and cities full of civilians or important infrastructure.
The company’s ALITA system will consist of 16 launch pads, equipped with 64 drones. It will be ready by October and capable of watching for incoming drones, automatically launching and travelling towards the target at 450 kilometres per hour before taking out everything from small drones to helicopters.
But Ukraine’s current rules will forbid fully autonomous operation and demand humans verify targets in the final stages of interception. Even in that mode, the entire battery of 64 drones will require just two human operators, meaning it will dramatically reduce personnel.
“Every step of this one can be either manual or automatic. We’re not allowed to do the final stage automatically,” says Kokhanovskyy, who believes that the rules should change. “I would love to,” he says.
Topics:
