A US teenager was handcuffed by armed police after an artificial intelligence (AI) system mistakenly said he was carrying a gun – when really he was holding a packet of crisps.
“Police showed up, like eight cop cars, and then they all came out with guns pointed at me talking about getting on the ground,” 16-year-old Baltimore pupil Taki Allen told local outlet WMAR-2 News.
Baltimore County Police Department said their officers “responded appropriately and proportionally based on the information provided at the time”.
It said the AI alert was sent to human reviewers who found no threat – but the principal missed this and contacted the school’s safety team, who ultimately called the police.
But the incident has prompted calls by some for the schools’ procedures around the use of such technology to be reviewed.
Mr Allen told local news he had finished a bag of Doritos after football practice, and put the empty packet in his pocket.
He said 20 minutes later, armed police arrived.
“He told me to get on my knees, arrested me and put me in cuffs,” he said.
Baltimore County Police Department told BBC News Mr Allen was handcuffed but not arrested.
“The incident was safely resolved after it was determined there was no threat,” they said in a statement.
Mr Allen said he now waits inside after football practice, as he does not think it is “safe enough to go outside, especially eating a bag of chips or drinking something”.
In a letter to parents, school principal Kate Smith said the school’s safety team “quickly reviewed and cancelled the initial alert after confirming there was no weapon”.
“I contacted our school resource officer (SRO) and reported the matter to him, and he contacted the local precinct for additional support,” she said.
“Police officers responded to the school, searched the individual and quickly confirmed that they were not in possession of any weapons.”
However, local politicians have called for further investigation into the incident.
“I am calling on Baltimore County Public Schools to review procedures around its AI-powered weapon detection system,” Baltimore County local councilman Izzy Pakota wrote on Facebook.
The BBC has approached Omnilert, the reported provider of the AI tool, for comment.
Omnilert says it is a “leading provider” of AI gun detection – citing a number of US schools among its case studies on its website.
The company claims its tech uses real, diverse data, leading to “more reliable detection, fewer false positives, and a system that actually works where it matters most”.
“Real-world gun detection is messy,” it states. “Lighting varies, weapons come in all shapes, and environments are full of noise and movement.
“Our data-centric methodology trains AI to succeed in these exact scenarios — because we use real data from real conditions, not simulations.”
But Mr Allen said: “I don’t think no chip bag should be mistaken for a gun at all.”
The adequacy of AI to accurately identify weapons has been subject to scrutiny.
Last year, a US weapons scanning company Evolv Technology was banned from making unsupported claims about its products after saying its AI scanner, used in thousands of US schools, hospitals and stadiums entrances, could detect all weapons.
BBC News investigations showed these claims to be false.
