AI models go head-to-head in Chatbot Arena
Andriy Onufriyenko/Getty Images
An industry-standard league table for ranking artificial intelligence models is being deliberately distorted by technology giants, researchers have claimed, leading to a misleading picture of which AIs are the best.
Sara Hooker at Cohere Labs, a US non-profit, and her colleagues claim to have found that the popular Chatbot Arena benchmark is a “distorted playing field”, with policies that end up giving an advantage to large companies like Meta, Amazon and Google by allowing them to discard models that score poorly.
